Archive for September, 2006

Tricks and Tips with NIO part V: SSL and NIO, friend or foe?

Over the last couple of months, I was investigating how I can add SSL support in GlassFish without having to redesign its HTTP Connector called Grizzly. I have to admit that everytime I’ve looked at the example included with the JDK, I was feeling like trying to bike on a frozen lake.


This is doable, but very hard (I’m the one near the person using skis (one scary person :-)!)

Maybe that’s only me, but I’ve found it very difficult to implement it properly, mostly because it didn’t fit well in the Grizzly framework. Of course I can implements it as a special case in Grizzly, but I was trying to integrate SSL support and be able to use all the tricks I’ve previously discussed (OP_WRITE, SelectionKey, Temporay Selectors). I was aware that other NIO based framework sucessfully implemented it, but wanted to come with something before looking at their implementation. The excellent MINA framework have a very interesting way of support this, but I wasn’t able to figure out how to re-initiate an handshake when the endpoint (most probably a Servlet) wants to look at the client certificate (CLIENT_CERT). If someone figured how to do it, please let me know! I would have liked to get their implementation into Grizzly.

Anyway, I’ve finally decided to implement it from scratch and be able to re-use the tricks I’ve already described. The good new is you can see the implementation here.
Now the details. The entry point when using SSL is the SSLEngine. The SSLEngine is associated with the lifetime of the SocketChannel, so you need to take care of re-using the same SSLEngine between registration of the SelectionKey. Rrrr, for HTTP, it means you will most probably use SelectionKey.attach() for doing it. I don’t like that SelectionKey.attach(..) (see here why). My problem with this is when you want to implement your SelectionKey registering code (ex: the HTTP keep-alive support), you need to create a data structure that will contains the SSLEngine and a long (or worse, a ByteBuffer), and then attach it to the SelectionKey. Your data structure will most likely looks like:

public class SSLAttachment{
    protected SSLEngine sslEngine;
    protedted long keepAliveTime;
    protected ByteBuffer byteByffer;

and you will pool them to avoid creating one everytime you need to register the SelectionKey. Naaaaa I don’t like that, mostly because Grizzly, by default, doesn’t attach data structures to the SelectionKey. Not to say that we did a lot of benchmarks and having to pool the data structure (or worse, create a new one everytime you need to register) is not optimal. OK I understand some protocols realy needs to do that (see all the comments here about this). Fortunalty, I digged the SSLEngine API and was able to use the SSLSession attached to an SSLEngine (SSLEngine.getSession()). Hence no needs for a data structure, just need to do something like:

                            (EXPIRE_TIME, System.currentTimeMillis());

So, when using NIO + SSL, I recommend you use the SSLEngine.getSession() to store the data structure. This way you don’t have to synchronize on a pool and/or create your own data structure.

Another things I wanted to support with Grizzly SSL is the temporary Selectors tricks I’ve described here. The way Grizzly handles an HTTP request by default is by trying to read all the header bytes without having to register the SelectionKey back to the main Selector. When the main Selector cannot reads more bytes, I’m always registering the SocketChannel to a temporary Selector and try to do more reads. This approach gives very good result, and I did implement the same approach for SSL. The way I did it is by:

  • Read available bytes
  • Invoke SSLEngine.unwrap(..)
  • If all the required bytes needed for the handshake are read, flush all the response bytes (using a temporary Selector trick to make sure all the bytes are flushed to the client)
  • Once the handshake is done and successful, reads more bytes
  • If the client is still alive (might have closed the connection), then call SSLEngine.unwrap().
  • Try to parse the headers. If bytes are required, read more by using a pool of temporary Selectors
  • Once the headers has been read, execute the request, attach the SSLEngine to the SelectionKey and return to the main Selector

But wait…One thing I don’t get with the SSLEngine is when you complete the handshake operation:

        Runnable runnable;
        while ((runnable = sslEngine.getDelegatedTask()) != null) {

In which situation will you execute the delegated task on a thread? It might be protocol specific, but with HTTP, that doesn’t make any sense to execute it using a Thread (maybe I should try and see what I’m getting). Another observation is when the Servlet wants to see the certificate during its execution, you need a way to re-initiate the handshake. Something like:

    protected Object[] doPeerCertificateChain(boolean force)
        throws IOException {
        Logger logger = SSLSelectorThread.logger();
        sslEngine.setNeedClientAuth(true);[] jsseCerts = null;
        try {
            jsseCerts = sslEngine.getSession().getPeerCertificateChain();
        } catch(Exception ex) {
        if (jsseCerts == null)
            jsseCerts = new[0];
         * We need to initiate a new handshake.
        if(jsseCerts.length <= 0 && force) {
            handshake= true;
            if (!doHandshake()){
                throw new IOException("Handshake failed");

        Certificate[] certs=null;
        try {
            certs = sslEngine.getSession().getPeerCertificates();
        } catch( Throwable t ) {
            if ( logger.isLoggable(Level.FINE))
                logger.log(Level.FINE,"Error getting client certs",t);
            return null;

Here the doHandshake() implementation is the same used when doing the initial handshake. This is where it gets complicated in term of design, because the endpoint (here Servlet) needs to have a way to retrieve the SSLEngine and execute the handshake. Since the doHanshake() code is far from simple, then you don’t want it duplicated in several classes. For that, I’ve created the SSLUtil class. If you planning to use SSL with NIO, take a look at it (and find bugs :-).

The good news is we have benchmarked this implementation and the performance is much better than when using blocking socket. Now be aware that when you work with SSLEngine, you cannot uses interface X509KeyManager because SSLEngine instead require the use of X509ExtendedKeyManager. This fact is hidden in the documentation and the exception you are getting is quite confusing if you fail to implement it properly:

Caused by: no cipher suites in common

I was puzzled because the blocking case was working perfectly. Anyway just make sure you aren’t using X509KeyManager. I’ve filled a bug against the JDK as I suspect I will not be the only one that face this problem.

OK, this is it! As usual, feedback are more than welcome, and many thanks for all the great feedback I got from the previous blogs

_uacct = “UA-3111670-1”;


Categories: Uncategorized

Writing a Comet web application using GlassFish

This blog describes how to write Comet enabled web application using GlassFish’s Comet Engine.

A couple of months ago, I’ve blogged about the technical details of the GlassFish‘s Comet support. Since then, I’ve got a lot of feedbacks on the blog and also privately. Surprisingly, a lot of peoples have started using the API and an asked for a blog describing a basic example. So here it comes … a basic Chat Servlet 🙂


First, to enable Comet Support in GlassFish, add the following in ${glassfish.home}/domains/domain1/config/domain.xml

        <http-listener acceptor-threads="1" address="" 
           blocking-enabled="false" default-virtual-server="server"
           enabled="true" family="inet" id="http-listener-1" port="8080"
           security-enabled="false" server-name="" xpowered-by="true">
                <property name="cometSupport" value="true"/>

Next, add in you web.xml:


OK now the interesting parts. The first things to decide when writing a Comet enabled web app is the component that will get polled. For this example, I will use a Servlet. First, the Servlet needs to register to the CometEngine:

   48     public void init(ServletConfig config) throws ServletException {
   49         super.init(config);
   50         contextPath = config.getServletContext().getContextPath() + "/chat";
   51         CometEngine cometEngine = CometEngine.getEngine();
   52         CometContext context = cometEngine.register(contextPath);
   53         context.setExpirationDelay(60 * 1000);
   54     }

The important part to define first is the context path that will be considered for Comet processing (or polling). All requests that takes the form of http://:/context/chat will be considered for polling. The context.setExpirationDelay() will determine how long a request will be polled. For this example, I’ve set the expiration delay to 60 seconds. After 60 seconds, the polled connection will be closed.

Next, you need to define a Comet request Handler which will get invoked every time the CometContext is updated. For the Chat, the handler will be created after the user has entered its user name (by issuing http://…/login.jsp)

   71                 if ("login".equals(action)) {
   72                     String username = request.getParameter("username");
   73                     request.getSession(true).setAttribute("username", username);
   75                     if (firstServlet != -1){
   76                          cometContext.notify("User " + username
   77                           + " from " + request.getRemoteAddr()
   78                           + " is joinning the chat.",CometEvent.NOTIFY,
   79                                  firstServlet);
   80                     }
   82                     response.sendRedirect("chat.jsp");
   83                     return;
   84                 } else if ("post".equals(action)){
   85                     String username = (String) request.getSession(true)
   86                         .getAttribute("username");
   87                     String message = request.getParameter("message");
   88                     cometContext.notify("[ " + username + " ]  "
   89                             + message + "<br/>");
   90                     response.sendRedirect("post.jsp");
   91                     return;
   92                 } else if ("openchat".equals(action)) {
   93                     response.setContentType("text/html");
   94                     CometRequestHandler handler = new CometRequestHandler();
   95                     handler.clientIP = request.getRemoteAddr();
   96                     handler.attach(response.getWriter());
   97                     cometContext.addCometHandler(handler);
   98                     String username = (String) request.getSession(true)
   99                         .getAttribute("username");
  100                     response.getWriter().println("<h2>Welcome "
  101                             + username + " </h2>");
  102                     return;

After the user has logged in, the browser will be redirected to the chat.jsp page, which will sent the action=”openchat”. The CometHandler (the class that will update the chat message box) implementation looks like:

  134         public void onEvent(CometEvent event) throws IOException{
  135             try{
  137                 if (firstServlet != -1 && this.hashCode() != firstServlet){
  138                      event.getCometContext().notify("User " + clientIP
  139                       + " is getting a new message.",CometEvent.NOTIFY,
  140                              firstServlet);
  141                 }
  142                 if ( event.getType() != CometEvent.READ ){
  143                     printWriter.println(event.attachment());
  144                     printWriter.flush();
  145                 }
  146             } catch (Throwable t){
  147                t.printStackTrace();
  148             }
  149         }
  152         public void onInitialize(CometEvent event) throws IOException{
  156         }

Every time the user will post a new message, the CometHandler.onEvent(…) will be invoked and the Chat message pushed back to the browser.

On the client side, the chat.jsp page looks like

     26 <frameset>
     27   <iframe name="chat" src ="/comet/chat?action=openchat" width="100%" scrolling="auto"></iframe>
     28   <iframe name="post" src="post.jsp" width="100%" scrolling="no"/>
     29 </frameset>

You can download the application (which include the src) here.

Note that the application described here is really to give an example. I would never recommend the use of static variables like I did in the example.


Before I forgot, one interesting feature I’ve recently added (was requested on first blog on Grizzly’s Comet) is the ability to update a single CometHandler (or a single polled request). When calling cometContext.addCometHandler(..), the returned value can be later re-used to push datas only to that cometHandler by doing:

     cometContext.notify(String message, int type, String cometListenerID);

See the API for more info. For the Chat example, I’ve added a pop up window where a chat moderator receives all the chat messages, who is connected and from where:

    138   event.getCometContext().notify("User " + clientIP
    139      + " is getting a new message.",CometEvent.NOTIFY,
    140        firstServlet);

That’s it. Very simple, is it? No needs to spawn a thread anywhere on the Servlet side, no special Servlet operations, etc.

Once I’ve a chance, I will try to use AJAX and improve the client. Any help is appreciated on that side 🙂 As usual, thanks for all the feedbacks sent by emails!

_uacct = “UA-3111670-1”;


Categories: GlassFish, Grizzly

Extending GlassFish’s WebContainer

The GlassFish WebContainer is based on the Tomcat Servlet implementation called Catalina. The Catalina architecture, introduced in Tomcat 4.x, was designed to easily allow developer to extends it. There is several interception point in Catalina that can be extended:

  • Valve: A Valve is a request processing component associated with a particular virtual-server(host) or web-module (servlet). A series of Valves are generally associated with each other into a Pipeline. Developer will usually inject their valves in order to have access to the Catalina internal objects, and manipulate the request/response object before filters or servlets are invoked. As an example, the access logging mechanism in GlassFish is implemented as a Valve.
  • ContainerListener: A listener for significant Container generated events. As an example, every time a WAR is deployed, a ContainerListener implementation will be notified for every new created Servlet, TagLib or Filter.
  • InstanceListener: A listener for significant events related to a specific servlet instance. An implementation of this interface will get notified everytime a Servlet is about to be invoked or when some operations are made on the Servlet (like when calling its ServletContextListeners).
  • LifecycleListener: A listener for significant events (including “virtual server start” and “web module stop”) generated by a component that implements the Lifecycle interface. An implementation of this interface will get notified by mostly all internal objects when an it is started|stopped (like when a new virtual-server is created). As an example, we are using this interface internally to notify the other GlassFish’s containers.

Virtual server supports the injection of Valve, ContainerListener and LifecycleListener. Web module supports the same as the virtual server with the addition of the InstanceListener.

Once you have your extension implemented, you need to install it in GlassFish. To install an extension:

For virtual-server

  • Add your classes into a jar file (ex: glassfish-ext.jar)
  • Place your jar under ${glassfish.home}/lib
  • Edit ${glassfish.home}/domains/domain1/config/domain.xml, locate the virtual-server element, and add <property name=”type” value=”fully qualified class name“> where type can be valve_ or listener_, with a unique number allowing you to define more than one type.

    Ex: <property name=”valve_1″ value=”org.apache.catalina.valves.RequestDumperValve”/>

For web-module

  • Same as for virtual-server, but under the web-module element of domain.xml. Except you can bundle your implementation under WEB-INF/lib or WEB-INF/classes (easier to deploy to several GlassFish installations)

    Ex: <property name=”listener_1″ value=”org.glassfish.MyServletListener”/>

Very easy, is it :-). This is supported starting in GlassFish v2 build 17.

_uacct = “UA-3111670-1”;


Categories: GlassFish