Home > Uncategorized > Tricks and Tips with NIO part II: Why SelectionKey.attach() is evil

Tricks and Tips with NIO part II: Why SelectionKey.attach() is evil

First, thanks for all the good feedback on part I. Please use the thread instead of sending me private email, as everybody can contribute to the answer (and I will not forgot to respond :-) ). Now If I can have the same kind of feedback for Grizzly code, I will be very happy (subliminal marketing here :-) )

OK this time I want to discuss the java.nio.channels.SelectionKey.attach(). I recommend you read about SelectionKey and the way you handle them before reading this blog. As usual, my first try might contains unclear parts (I should really start blogging in French instead :-) ).

The Java documentation for SelectionKey.attach() states:

Attaches the given object to this key.

An attached object may later be retrieved via the attachment method. Only one 
object may be attached at a time; invoking this method causes any previous 
attachment to be discarded. The current attachment may be discarded by 
attaching null.

Wow…the devil exists, and he lives inside the NIO API!


Why? Well, let takes a simple example. Usually, you handle SelectionKey by doing:


            selectorState = 0;
            enableSelectionKeys();                

            try{
                selectorState = selector.select(selectorTimeout);
            } catch (CancelledKeyException ex){
                ;
            }
            readyKeys = selector.selectedKeys();
            iterator = readyKeys.iterator();
            while (iterator.hasNext()) {
                key = iterator.next();
                iterator.remove();
                if (key.isValid()) {
                    handleConnection(key);
                } else {
                    cancelKey(key);
                }
            }

The Selector.select() will always return the set of SelectionKey whose ready-operation sets were updated. Then the handleConnection implementation will most likely looks like:


        if ((key.readyOps() & SelectionKey.OP_ACCEPT) == 
                SelectionKey.OP_ACCEPT){
            handleAccept(key);
        } else if ((key.readyOps() & SelectionKey.OP_READ) == 
                 SelectionKey.OP_READ) {
            handleRead(key);
        }

Next in handleRead(key), you will do:


            socketChannel = (SocketChannel)key.channel();
            while ( socketChannel.isOpen() && 
                    (count = socketChannel.read(byteBuffer))> -1)){
                // do something
            }

Well, the scary part is the // do something.

Gold Candidate for a Memory Leak (GCML)

At this stage, socketChannel is ready to read bytes. Hence you invoke socketChannel.read(byteBuffer), and you find that you haven’t read all the bytes from the socket (or you are ready to handle the next request), so you decide to register the SelectionKey back to the Selector by doing:


            selectionKey.interestOps(
                    selectionKey.interestOps() | SelectionKey.OP_READ);

and…and…and do something like:


            selectionKey.attach(...)

Boum…the little is where the devil is hiding! What you are attaching to the SelectionKey is very dangerous, because there is some probability that your SelectionKey might never return to a ready-operation state, leaving the SelectionKey and its evil attachment forever inside the Selector keys set. Does it sound like a GC…ML (GCML)? But what’s the point, nobody will ever do that, because we are all very talented engineers, and we always take care of cleaning our SelectionKey from the Selector keys set, right?

The problem might comes when your framework needs to handle thousand of connections, and you need to keep-alive those connections for a very long time (from 60 seconds to 5 minutes). Most framework (and unfortunately a lot of tutorials and talks) will usually attach their framework object to the SelectionKey (ex: The Reactor pattern). Those framework objects will most probably include:

  • A ByteBuffer
  • Some keep-alive object (let’s assume a Long)
  • A SocketChannel
  • A Framework Handler (like the Reactor pattern)
  • etc.

So you can ends up with thousand of objects taking vacations, enjoying idle time inside the Selector keys set. If you didn’t implement any mechanism to make periodical look inside the Selector keys set, then you will most probably ends up with a memory leak (or your framework performance will be impacted). Worse, you might never notice the problem….

BelleIsle-2.jpg

Recommended solutions?

My first experimentation of NIO was using that kind of approach (like Mina, like EmberIO and like our ORB NIO implementation). Then Scott started working with me on Grizzly and pointed the problems after a couple of benchmarks. Under HTTP stress, you might ends up with 10 000 connections (so 10 000 actives SelectionKey). If they all have as an attachment a ByteBuffer or an Handler, then a lot of memory will be consumed, reducing your scalability and having fun eating all your memory.

Even if most Virtual Machine are very good those days, I consider this as a very bad design anyway, unless you have a very good reason. But don’t get me wrong, I’m not saying the framework listed above are bad, I’m just pointing some problems.

The next couple of paragraphs will describe some solutions

How do I retrieve the SocketChannel if I don’t attach it to my framework object.

Most existing framework include, inside their framework object, the SocketChannel associated with the SelectionKey. This is wrong, because the SocketChannel can always be retrieved using SelectionKey.channel().

How do I deal with incomplete socketChannel read.

When you do socketChannel.read(), you can never predict when all bytes are read from the socket buffer. Most of the time, you will have to register the SelectionKey back to the Selector, waiting for more bytes to be available. In that case, you will most probably attach the incomplete ByteBuffer to the SelectionKey, and continue adding bytes to it once the SelectionKey is ready. Instead, I would recommend you register the SelectionKey to a temporary Selector (I will blog about this trick in more details):


        try{
            SocketChannel socketChannel = (SocketChannel)key.channel();
            while (count > 0){
                count = socketChannel.read(byteBuffer);
            }

            if ( byteRead == 0 ){
                readSelector = SelectorFactory.getSelector();
                tmpKey = socketChannel
                        .register(readSelector,SelectionKey.OP_READ);
                tmpKey.interestOps(tmpKey.interestOps() | SelectionKey.OP_READ);
                int code = readSelector.select(readTimeout);
                tmpKey.interestOps(
                    tmpKey.interestOps() & (~SelectionKey.OP_READ));

                if ( code == 0 ){
                    return 0; 
                }

                while (count > 0){
                    count = socketChannel.read(byteBuffer);
                }
            }
        } catch (Throwable t){

In this example, you try to read more bytes using a temporay Selector (on the same Thread, without having to return to your original Selector, which most of the time run on another thread). With this trick, you don’t need to attach anything to the SelectionKey.

But there is a drawback. If the temporary Selector.select() blocks (because the SelectionKey isn’t ready, most probably because the client isn’t sending all the bytes), you will block a processing Thread for “readTimeout” seconds, ending up in a similar well known situation called blocking socket ;-) (one thread per connection). That wouldn’t have been the case if you had registered the SelectionKey back the original Selector with a ByteBuffer attached. So here you gonna need to decide based on your use of NIO: do you want a dormant ByteBuffer attached to a SelectionKey or a Thread blocking for readTimeout.

In Grizzly, both approaches can be configured, but by default the thread will block for 15 seconds and cancel the SelectionKey if the client isn’t doing anything. You can configure Grizzly to attach the ByteBuffer if you really like to use memory :-) . We did try on slow network, with broken client, etc., and blocking a Thread scale better than having a dormant ByteBuffer,

Have one ByteBuffer per Thread, not per object framework.

The good news about not using SelectionKey.attach() is you only need to create a ByteBuffer per Thread, instead of a ByteBuffer per SelectionKey. So for 10 000 connections, instead of having 10 000 ByteBuffer, you will only have X ByteBuffer, where X = the number of active Threads. This significantly improve scalability by not overloading the VM with dormant ByteBuffer. As an example, in Grizzly, the keep-alive mechanism is implemented as follow:

  • Thread-1: Selector.select()
  • Thread-2: Do socketChannel.read() until the HTTP 1.1 request is fully read
  • Thread-2: Process the request and send the response
  • Thread-2: register the SelectionKey back to the Selector (without SelectionKey.attach())
  • Thread-1: Selector.select(). If the SelectionKey ready, then
  • Thread-2: Do socketChannel.read() until the HTTP 1.1 request is fully read
  • etc.

As you can see, Thread-2 is not blocked between keep-alive requests. Theorically, you would probably be able to serve hundreds of requests with only two threads. Not to say no dormant ByteBuffer, no pending framework objects, etc.

Wow that one was very long. Agree, disagree?…..

Next time I will discuss when its appropriate to spawn a thread to handle OP_ACCEPT, OP_READ and OP_WRITE, and when its not.

_uacct = “UA-3111670-1″;
urchinTracker();

technorati:

About these ads
Categories: Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 51 other followers

%d bloggers like this: