Home > Uncategorized > Tricks and Tips with NIO part III: To Thread or Not to Thread

Tricks and Tips with NIO part III: To Thread or Not to Thread

partIII.jpg

This time I will share some observations I’ve experimented when handling OP_ACCEPT, OP_READ and OP_WRITE using Threads. When I started working on Grizzly, I’ve designed the framework open enough so I can easily add thread pool mostly everywhere during the request processing. At that time there weren’t a lot of NIO framework available neither clear recommendations about what to do and what to avoid. To avoid having to redesign Grizzly every weeks, I’ve decided to make OP_READ, OP_ACCEPT and OP_WRITE processing configurable. By configurable, I mean being able to execute different strategies, e.g. being able to execute the processing of those operations on their own thread or using the same thread as the Selector:


            if ( myExecutor == null ){
                myExecutor = Executors.newFixedThreadPool(maxThreads);
            }

            try{
                selectorState = selector.select(selectorTimeout);
            } catch (CancelledKeyException ex){
                ;
            }

            readyKeys = selector.selectedKeys();
            iterator = readyKeys.iterator();
            while (iterator.hasNext()) {
                key = iterator.next();
                iterator.remove();
                if (key.isValid()) {                  
                   if ((key.readyOps() & SelectionKey.OP_ACCEPT) 
                         == SelectionKey.OP_ACCEPT){
                      myExecutor.execute(getAcceptHandler(key));
                   } else if ((key.readyOps() & SelectionKey.OP_READ) 
                         == SelectionKey.OP_READ) {
                      myExecutor.execute(getReadHandler(key));
                   } 
                   ....
                } else {
                    cancelKey(key);
                }
            }

From the code above, the getAcceptHandler(key) will return a Runnable object which most usually does:


        ServerSocketChannel server = (ServerSocketChannel) key.channel();
        SocketChannel channel = server.accept();

        if (channel != null) {
           channel.configureBlocking(false);
           SelectionKey readKey = 
                channel.register(selector, SelectionKey.OP_READ);
           setSocketOptions(((SocketChannel)readKey.channel()).socket());
        }

where getReadHandler(key) will do:


        key.interestOps(key.interestOps() & (~SelectionKey.OP_READ));
        socketChannel = (SocketChannel)key.channel();
        while ( socketChannel.isOpen() && (
          ((count = socketChannel.read(byteBuffer))> -1)){
              // Do something

An alternative is to execute the getAcceptHandler(key)and getReadHandler(key) on the same thread as the Selector.select(..):


            try{
                selectorState = selector.select(selectorTimeout);
            } catch (CancelledKeyException ex){
                ;
            }

            readyKeys = selector.selectedKeys();
            iterator = readyKeys.iterator();
            while (iterator.hasNext()) {
                key = iterator.next();
                iterator.remove();
                if (key.isValid()) {                  
                   if ((key.readyOps() & SelectionKey.OP_ACCEPT) 
                         == SelectionKey.OP_ACCEPT){
                      getAcceptHandler(key);
                   } else if ((key.readyOps() & SelectionKey.OP_READ) 
                         == SelectionKey.OP_READ) {
                      getReadHandler(key);
                   } 
                   ....
                } else {
                    cancelKey(key);
                }
            }

and of course, without having to create a Runnable object. Finally, the other alternative is a mix of the first two strategy:


            try{
                selectorState = selector.select(selectorTimeout);
            } catch (CancelledKeyException ex){
                ;
            }

            readyKeys = selector.selectedKeys();
            iterator = readyKeys.iterator();
            while (iterator.hasNext()) {
                key = iterator.next();
                iterator.remove();
                if (key.isValid()) {                  
                   if ((key.readyOps() & SelectionKey.OP_ACCEPT) 
                         == SelectionKey.OP_ACCEPT){
                      getAcceptHandler(key);
                   } else if ((key.readyOps() & SelectionKey.OP_READ) 
                         == SelectionKey.OP_READ) {
                      myExecutor.execute(getReadHandler(key));
                   } 
                   ....
                } else {
                    cancelKey(key);
                }
            }

or executing the getAcceptHandler(key) using myExecutor and
getReadHandler on the same thread as the Selector.select(..). Like I said earlier, Grizzly can be configured to support all strategies.

Which strategy perform the best

I’ve benchmarked all of the above strategies and find that the one that perform the best is:


            try{
                selectorState = selector.select(selectorTimeout);
            } catch (CancelledKeyException ex){
                ;
            }

            readyKeys = selector.selectedKeys();
            iterator = readyKeys.iterator();
            while (iterator.hasNext()) {
                key = iterator.next();
                iterator.remove();
                if (key.isValid()) {                  
                   if ((key.readyOps() & SelectionKey.OP_ACCEPT) 
                         == SelectionKey.OP_ACCEPT){
                      getAcceptHandler(key);
                   } else if ((key.readyOps() & SelectionKey.OP_READ) 
                         == SelectionKey.OP_READ) {
                      myExecutor.execute(getReadHandler(key));
                   } 
                   ....
                } else {
                    cancelKey(key);
                }
            }

c’est a dire executing the OP_ACCEPT on the same thread as the Selector.select(..), and using a Thread for executing the OP_READ. I’ve shared my observations with my colleagues and they also came to the same conclusion.

Well, what about OP_WRITE

I didn’t forget the OP_WRITE. I also tested the strategies described above and came to the conclusion than OP_WRITE should be handled using the same Thread as the one handling OP_READ. One thing that might explain why I’m getting such results is the use of temporary Selector when the main Selection is not able to flush the socket outgoing buffer (see part I for more details), or when socketChannel.read() return 0. Another important observation is all the tests I’ve ran are either HTTP or IIOP based protocol. Other protocols might perform differently, although I suspect it will not make such a difference.

Are you getting different results? This is not easy to measure because you have to make sure the framework itself is not the bottleneck.

As usual, feedback is more than welcome. Next time I will discuss using more than one Selector under high load. Merci!!

technorati:

_uacct = “UA-3111670-1”;
urchinTracker();

Categories: Uncategorized
  1. Simeon
    June 18, 2012 at 8:19 pm

    In my server I use the same technic but I do experience a strange behavior. Once the READ inserts is disabled by calling

    key.interestOps(key.interestOps() & (~SelectionKey.OP_READ));

    Of course, I do not forget to set ip back on at the end of the read worker thread:

    key.interestOps(key.interestOps() | SelectionKey.OP_READ);

    key.selector().wakeup();

    The problem is, the key get never selected again for READ even there is waiting incoming data on the socket. Do you have any idea, why?

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: