Archive

Archive for July 7, 2006

Tricks and Tips with NIO part III: To Thread or Not to Thread

partIII.jpg

This time I will share some observations I’ve experimented when handling OP_ACCEPT, OP_READ and OP_WRITE using Threads. When I started working on Grizzly, I’ve designed the framework open enough so I can easily add thread pool mostly everywhere during the request processing. At that time there weren’t a lot of NIO framework available neither clear recommendations about what to do and what to avoid. To avoid having to redesign Grizzly every weeks, I’ve decided to make OP_READ, OP_ACCEPT and OP_WRITE processing configurable. By configurable, I mean being able to execute different strategies, e.g. being able to execute the processing of those operations on their own thread or using the same thread as the Selector:


            if ( myExecutor == null ){
                myExecutor = Executors.newFixedThreadPool(maxThreads);
            }

            try{
                selectorState = selector.select(selectorTimeout);
            } catch (CancelledKeyException ex){
                ;
            }

            readyKeys = selector.selectedKeys();
            iterator = readyKeys.iterator();
            while (iterator.hasNext()) {
                key = iterator.next();
                iterator.remove();
                if (key.isValid()) {                  
                   if ((key.readyOps() & SelectionKey.OP_ACCEPT) 
                         == SelectionKey.OP_ACCEPT){
                      myExecutor.execute(getAcceptHandler(key));
                   } else if ((key.readyOps() & SelectionKey.OP_READ) 
                         == SelectionKey.OP_READ) {
                      myExecutor.execute(getReadHandler(key));
                   } 
                   ....
                } else {
                    cancelKey(key);
                }
            }

From the code above, the getAcceptHandler(key) will return a Runnable object which most usually does:


        ServerSocketChannel server = (ServerSocketChannel) key.channel();
        SocketChannel channel = server.accept();

        if (channel != null) {
           channel.configureBlocking(false);
           SelectionKey readKey = 
                channel.register(selector, SelectionKey.OP_READ);
           setSocketOptions(((SocketChannel)readKey.channel()).socket());
        }

where getReadHandler(key) will do:


        key.interestOps(key.interestOps() & (~SelectionKey.OP_READ));
        socketChannel = (SocketChannel)key.channel();
        while ( socketChannel.isOpen() && (
          ((count = socketChannel.read(byteBuffer))> -1)){
              // Do something

An alternative is to execute the getAcceptHandler(key)and getReadHandler(key) on the same thread as the Selector.select(..):


            try{
                selectorState = selector.select(selectorTimeout);
            } catch (CancelledKeyException ex){
                ;
            }

            readyKeys = selector.selectedKeys();
            iterator = readyKeys.iterator();
            while (iterator.hasNext()) {
                key = iterator.next();
                iterator.remove();
                if (key.isValid()) {                  
                   if ((key.readyOps() & SelectionKey.OP_ACCEPT) 
                         == SelectionKey.OP_ACCEPT){
                      getAcceptHandler(key);
                   } else if ((key.readyOps() & SelectionKey.OP_READ) 
                         == SelectionKey.OP_READ) {
                      getReadHandler(key);
                   } 
                   ....
                } else {
                    cancelKey(key);
                }
            }

and of course, without having to create a Runnable object. Finally, the other alternative is a mix of the first two strategy:


            try{
                selectorState = selector.select(selectorTimeout);
            } catch (CancelledKeyException ex){
                ;
            }

            readyKeys = selector.selectedKeys();
            iterator = readyKeys.iterator();
            while (iterator.hasNext()) {
                key = iterator.next();
                iterator.remove();
                if (key.isValid()) {                  
                   if ((key.readyOps() & SelectionKey.OP_ACCEPT) 
                         == SelectionKey.OP_ACCEPT){
                      getAcceptHandler(key);
                   } else if ((key.readyOps() & SelectionKey.OP_READ) 
                         == SelectionKey.OP_READ) {
                      myExecutor.execute(getReadHandler(key));
                   } 
                   ....
                } else {
                    cancelKey(key);
                }
            }

or executing the getAcceptHandler(key) using myExecutor and
getReadHandler on the same thread as the Selector.select(..). Like I said earlier, Grizzly can be configured to support all strategies.

Which strategy perform the best

I’ve benchmarked all of the above strategies and find that the one that perform the best is:


            try{
                selectorState = selector.select(selectorTimeout);
            } catch (CancelledKeyException ex){
                ;
            }

            readyKeys = selector.selectedKeys();
            iterator = readyKeys.iterator();
            while (iterator.hasNext()) {
                key = iterator.next();
                iterator.remove();
                if (key.isValid()) {                  
                   if ((key.readyOps() & SelectionKey.OP_ACCEPT) 
                         == SelectionKey.OP_ACCEPT){
                      getAcceptHandler(key);
                   } else if ((key.readyOps() & SelectionKey.OP_READ) 
                         == SelectionKey.OP_READ) {
                      myExecutor.execute(getReadHandler(key));
                   } 
                   ....
                } else {
                    cancelKey(key);
                }
            }

c’est a dire executing the OP_ACCEPT on the same thread as the Selector.select(..), and using a Thread for executing the OP_READ. I’ve shared my observations with my colleagues and they also came to the same conclusion.

Well, what about OP_WRITE

I didn’t forget the OP_WRITE. I also tested the strategies described above and came to the conclusion than OP_WRITE should be handled using the same Thread as the one handling OP_READ. One thing that might explain why I’m getting such results is the use of temporary Selector when the main Selection is not able to flush the socket outgoing buffer (see part I for more details), or when socketChannel.read() return 0. Another important observation is all the tests I’ve ran are either HTTP or IIOP based protocol. Other protocols might perform differently, although I suspect it will not make such a difference.

Are you getting different results? This is not easy to measure because you have to make sure the framework itself is not the bottleneck.

As usual, feedback is more than welcome. Next time I will discuss using more than one Selector under high load. Merci!!

technorati:

_uacct = “UA-3111670-1”;
urchinTracker();

Advertisements
Categories: Uncategorized