Archive for July 6, 2006

The Grizzly Comet or why space shuttle Discovery launch was delayed.

Space shuttle Discovery was delayed recently, and the real reason was kept secret. Something strange was observed by the Hubble Space Telescope. The Hubble Ultra Deep Field(HUDF) image was showing a new star coming extremely fast to earth. Even after washing the main mirror with AJAX, the HUDF was clear: the Grizzly Comet is entering our atmosphere….


This time I will discuss the new Comet support in Grizzly (sometimes called request polling, http streaming or continuation), build on top of Grizzly’s Asynchronous Request Processing(ARP). From Wikipedia:

Comet is a programming technique that enables web servers
to send data to the client without having any need for the client
to request for it. It allows creation of event-driven web 
applications which are hosted in the browser.

After I’ve blogged about ARP, I’ve started defining some Comet APIs on top of it. I was waiting for free time to define the perfect API. I think I was dreaming….I will never have free time with Grizzly! Since Comet Request processing is more and more popular (Jetty supports it for a while, Tomcat just have a fresh implementation…and a new NIO based connector (finally!!!), and GlassFish have ARP), I’ve decided to make available my own implementation. I didn’t update my implementation based on Greg Wilkins’ proposal, but hey, this is build on top of ARP and once the Servlet EG approves Comet support, it will be easy to implement it.

The next couple of paragraphs will introduce the API. I did the famous Chat implementation using jMaki, and will soon publish it once Greg reviewed my modifications to his jMaki application :-)

Comet API

A component (Servlet, JSP, JSF or a java class) that wants to support Comet requests first need to register to the CometEngine:

        CometEngine cometEngine = CometEngine.getEngine();
        CometContext context = cometEngine.register(contextPath);    

Mainly, you first get an instance of CometEngine, then register the context path on which Comet requests will be allowed. The CometContext is the main object a component will use to interact with others Comet requests, implemented as CometHandler.

         // Class that implement CometHandler interface
         CometResponseHandler handler = new CometResponseHandler();
         CometContext cometContext = 
         cometContext.notify("User X just entered the room");

The CometHandler interface is quite simple:

     * Attach an instance of E to this class.
    public void attach(E attachment);
     * Receive CometEvent notification.
    public void onEvent(CometEvent event) throws IOException;   
     * Receive CometEvent notification when the underlying 
     * tcp communication is started by the client
    public void onInitialize(CometEvent event) throws IOException;   
     * Receive CometEvent notification when the underlying 
     * tcp communication is closed by the CometHandler
    public void onTerminate(CometEvent event) throws IOException;    
     * Receive CometEvent notification when the underlying 
     * tcp communication is interrupted by the Grizzly ARP.
    public void onInterrupt(CometEvent event) throws IOException;

Below is an example of a CometHandler implementation

    public class CometResponseHandler implements CometHandler{
        private HttpServletResponse httpServletResponse;
        public void attach(HttpServletResponse httpServletResponse){
            this.httpServletResponse = httpServletResponse;
        public void onEvent(CometEvent event) throws IOException{   
            System.out.println("==== onEvent =====");
                PrintWriter printWriter = httpServletResponse.getWriter();
                // We just received a new chat message from another user.
                // Flush it to the browser.
            } catch (Throwable t){

        public void onInitialize(CometEvent event) throws IOException{  

        public void onTerminate(CometEvent event) throws IOException{

        public void onInterrupt(CometEvent event) throws IOException{
            CometContext cometContext = event.getCometContext();  

Once added to a CometContext, the CometHandler will be invoked everytime the CometContext.notify(Object) is invoked (ex: A new message is added to a Chat forum):

        cometContext.notify(new Message
          ("Grizzly Comet", userId + " has entered our atmosphere."));

The CometHandler can be invoked by the Grizzly ARP via the CometEngine and by others CometHandler. The CometEngine will notify CometHandler when the request is received (onInitialize) and when the request is interrupted (onInterrupt). The polled request will be resumed when the delay set on the CometContext expires:

         context.setExpirationDelay(60 * 1000); //60 seconds

Using the Chat example, a Servlet will use the CometContext to notify the CometHandler when a new chat user is added or when a new message is received:

    public void doPost(HttpServletRequest request, 
                       HttpServletResponse  response)
            throws IOException, ServletException {                      
         try {
            String action = request.getParameter("action");
            // negotiate a userid
            if ("valid-register".equals(action)) {
            } else if ("register".equals(action)) {
                CometEngine cometEngine = CometEngine.getEngine();
                CometContext cometContext = 
                   new Message("Chatter", userId + " has joined."));               
            } else if ("add-message".equals(action)) {
                CometEngine cometEngine = CometEngine.getEngine();
                CometContext cometContext = 
                cometContext.notify(new Message(userId,msg);

From an AJAX application, an http connection will be openned and a CometHandler will be created using the HttpServletResponse object created for that connection. The CometHandler.onEvent(..) will wait for CometEvent, and based on the CometEvent.attachment(), will proceed by pushing back data to the AJAX application. The CometEvent.attachment() can be any type. For a Chat application, you will most probably attach a String containing the new message.

Technical details

In Grizzly ARP, the Comet request isn’t holding a Thread, but instead polled using NIO Selector (unfortunately using the SelectionKey.attach(..) for now ;-)). For scalability, it is very important to avoid holding a Thread during the request polling. Another important detail is the CometHandler doesn’t need to manage any Threads when notifying others CometHandler. This is implemented by the Grizzly ARP directly and not exposed to the CometHandler API. Of cource CometHandler can do whatever they want, hence they can decide to implement different strategies using Threads to notify others CometHandler.

The time where the polling happens can be configured. In the chat example, the polling will occurs after the Servlet.service() has been invoked. In the GMail/JavaMail example, the polling happens before Servlet.service(). You can decide when polling will happens using the CometEngine.register(..) method:

        CometEngine cometEngine = CometEngine.getEngine();
        CometContext context = 

see CometEngine implementation for more details.

When notifying CometHandler, you can specify the type of CometEvent you want to push:

         CometEngine cometEngine = CometEngine.getEngine();
         CometContext cometContext = 
         cometContext.notify("Closing the chat room",

see CometEvent for more details.

I’m still thinking about how the CometContext is retrieved from a component:

         cometContext = cometEngine.getCometContext(contextPath);

A Servlet from another application can easily compute the contextPath and retrieve the “external” CometContext. Although I can envision very interesting applications who are sharing CometContext and CometHandler, a security mechanism needs to be added to allow configuring the shareability of CometContext.

Finally, Grizzly ARP works with clean and SSL connection (both supported by NIO non blocking socket), thus Comet support can be used for secure and non secure communication.


La suite des choses

The Comet support is fairly new and available only starting in GlassFish 9.1 ea-b10. That means the implementation most probably need improvements. As usual, any feedback is welcomed. I will soon add to the Grizzly workspace a couple examples on how to use the API with AJAX client. More important, if you don’t like the current implementation, it is very easy to extend Grizzly ARP with a completely different Comet support approach. I saw some implementation at JavaOne :-)

P.S To enable Comet Support, just add, in GlassFish domain.xml

        <http-listener acceptor-threads="1" address="" 
           blocking-enabled="false" default-virtual-server="server"
           enabled="true" family="inet" id="http-listener-1" port="8080"
           security-enabled="false" server-name="" xpowered-by="true">
                <property name="cometSupport" value="true"/>

and make sure, in case you are using a Servlet|JSP, that it is initialized when GlassFish starts up by adding in your web.xml:


UPDATED: A second blog that describe an example can be read here.

_uacct = “UA-3111670-1″;


Categories: Uncategorized

Tricks and Tips with NIO part II: Why SelectionKey.attach() is evil

First, thanks for all the good feedback on part I. Please use the thread instead of sending me private email, as everybody can contribute to the answer (and I will not forgot to respond :-) ). Now If I can have the same kind of feedback for Grizzly code, I will be very happy (subliminal marketing here :-) )

OK this time I want to discuss the java.nio.channels.SelectionKey.attach(). I recommend you read about SelectionKey and the way you handle them before reading this blog. As usual, my first try might contains unclear parts (I should really start blogging in French instead :-) ).

The Java documentation for SelectionKey.attach() states:

Attaches the given object to this key.

An attached object may later be retrieved via the attachment method. Only one 
object may be attached at a time; invoking this method causes any previous 
attachment to be discarded. The current attachment may be discarded by 
attaching null.

Wow…the devil exists, and he lives inside the NIO API!

Why? Well, let takes a simple example. Usually, you handle SelectionKey by doing:

            selectorState = 0;

                selectorState =;
            } catch (CancelledKeyException ex){
            readyKeys = selector.selectedKeys();
            iterator = readyKeys.iterator();
            while (iterator.hasNext()) {
                key =;
                if (key.isValid()) {
                } else {

The will always return the set of SelectionKey whose ready-operation sets were updated. Then the handleConnection implementation will most likely looks like:

        if ((key.readyOps() & SelectionKey.OP_ACCEPT) == 
        } else if ((key.readyOps() & SelectionKey.OP_READ) == 
                 SelectionKey.OP_READ) {

Next in handleRead(key), you will do:

            socketChannel = (SocketChannel);
            while ( socketChannel.isOpen() && 
                    (count => -1)){
                // do something

Well, the scary part is the // do something.

Gold Candidate for a Memory Leak (GCML)

At this stage, socketChannel is ready to read bytes. Hence you invoke, and you find that you haven’t read all the bytes from the socket (or you are ready to handle the next request), so you decide to register the SelectionKey back to the Selector by doing:

                    selectionKey.interestOps() | SelectionKey.OP_READ);

and…and…and do something like:


Boum…the little is where the devil is hiding! What you are attaching to the SelectionKey is very dangerous, because there is some probability that your SelectionKey might never return to a ready-operation state, leaving the SelectionKey and its evil attachment forever inside the Selector keys set. Does it sound like a GC…ML (GCML)? But what’s the point, nobody will ever do that, because we are all very talented engineers, and we always take care of cleaning our SelectionKey from the Selector keys set, right?

The problem might comes when your framework needs to handle thousand of connections, and you need to keep-alive those connections for a very long time (from 60 seconds to 5 minutes). Most framework (and unfortunately a lot of tutorials and talks) will usually attach their framework object to the SelectionKey (ex: The Reactor pattern). Those framework objects will most probably include:

  • A ByteBuffer
  • Some keep-alive object (let’s assume a Long)
  • A SocketChannel
  • A Framework Handler (like the Reactor pattern)
  • etc.

So you can ends up with thousand of objects taking vacations, enjoying idle time inside the Selector keys set. If you didn’t implement any mechanism to make periodical look inside the Selector keys set, then you will most probably ends up with a memory leak (or your framework performance will be impacted). Worse, you might never notice the problem….


Recommended solutions?

My first experimentation of NIO was using that kind of approach (like Mina, like EmberIO and like our ORB NIO implementation). Then Scott started working with me on Grizzly and pointed the problems after a couple of benchmarks. Under HTTP stress, you might ends up with 10 000 connections (so 10 000 actives SelectionKey). If they all have as an attachment a ByteBuffer or an Handler, then a lot of memory will be consumed, reducing your scalability and having fun eating all your memory.

Even if most Virtual Machine are very good those days, I consider this as a very bad design anyway, unless you have a very good reason. But don’t get me wrong, I’m not saying the framework listed above are bad, I’m just pointing some problems.

The next couple of paragraphs will describe some solutions

How do I retrieve the SocketChannel if I don’t attach it to my framework object.

Most existing framework include, inside their framework object, the SocketChannel associated with the SelectionKey. This is wrong, because the SocketChannel can always be retrieved using

How do I deal with incomplete socketChannel read.

When you do, you can never predict when all bytes are read from the socket buffer. Most of the time, you will have to register the SelectionKey back to the Selector, waiting for more bytes to be available. In that case, you will most probably attach the incomplete ByteBuffer to the SelectionKey, and continue adding bytes to it once the SelectionKey is ready. Instead, I would recommend you register the SelectionKey to a temporary Selector (I will blog about this trick in more details):

            SocketChannel socketChannel = (SocketChannel);
            while (count > 0){
                count =;

            if ( byteRead == 0 ){
                readSelector = SelectorFactory.getSelector();
                tmpKey = socketChannel
                tmpKey.interestOps(tmpKey.interestOps() | SelectionKey.OP_READ);
                int code =;
                    tmpKey.interestOps() & (~SelectionKey.OP_READ));

                if ( code == 0 ){
                    return 0; 

                while (count > 0){
                    count =;
        } catch (Throwable t){

In this example, you try to read more bytes using a temporay Selector (on the same Thread, without having to return to your original Selector, which most of the time run on another thread). With this trick, you don’t need to attach anything to the SelectionKey.

But there is a drawback. If the temporary blocks (because the SelectionKey isn’t ready, most probably because the client isn’t sending all the bytes), you will block a processing Thread for “readTimeout” seconds, ending up in a similar well known situation called blocking socket ;-) (one thread per connection). That wouldn’t have been the case if you had registered the SelectionKey back the original Selector with a ByteBuffer attached. So here you gonna need to decide based on your use of NIO: do you want a dormant ByteBuffer attached to a SelectionKey or a Thread blocking for readTimeout.

In Grizzly, both approaches can be configured, but by default the thread will block for 15 seconds and cancel the SelectionKey if the client isn’t doing anything. You can configure Grizzly to attach the ByteBuffer if you really like to use memory :-) . We did try on slow network, with broken client, etc., and blocking a Thread scale better than having a dormant ByteBuffer,

Have one ByteBuffer per Thread, not per object framework.

The good news about not using SelectionKey.attach() is you only need to create a ByteBuffer per Thread, instead of a ByteBuffer per SelectionKey. So for 10 000 connections, instead of having 10 000 ByteBuffer, you will only have X ByteBuffer, where X = the number of active Threads. This significantly improve scalability by not overloading the VM with dormant ByteBuffer. As an example, in Grizzly, the keep-alive mechanism is implemented as follow:

  • Thread-1:
  • Thread-2: Do until the HTTP 1.1 request is fully read
  • Thread-2: Process the request and send the response
  • Thread-2: register the SelectionKey back to the Selector (without SelectionKey.attach())
  • Thread-1: If the SelectionKey ready, then
  • Thread-2: Do until the HTTP 1.1 request is fully read
  • etc.

As you can see, Thread-2 is not blocked between keep-alive requests. Theorically, you would probably be able to serve hundreds of requests with only two threads. Not to say no dormant ByteBuffer, no pending framework objects, etc.

Wow that one was very long. Agree, disagree?…..

Next time I will discuss when its appropriate to spawn a thread to handle OP_ACCEPT, OP_READ and OP_WRITE, and when its not.

_uacct = “UA-3111670-1″;


Categories: Uncategorized

Get every new post delivered to your Inbox.

Join 50 other followers