Archive

Archive for the ‘Grizzly’ Category

GlassFish Vee(gri)zzly(v3): Unofficial benchmarks results

We are still working hard on GlassFish v3 and will soon release a new technology preview (JavaOne is coming :-)). What about its performance? As you may know, in v3 we have cleaned up the noise by using a modular approach (thanks Jerome!). In Quebecois (French peoples are probably using some English world to say the same ;-), we say “Faire le Grand Menage”

mathieu 071.jpg

What about Grizzly-In-V3? In v3, the monster is now integrated using its framework module (no http pollution like in v2, just NIO:-)). So when v3 starts, part of its startup logic is done on top of Grizzly (using 1.7.3.1 runtime). Does it make a difference at startup and more important, at runtime? It make a huge one because now, with the modular nature of v3 there is no longer modules that add noise in Grizzly. So let’s benchmark the static resources (.html) performance using Faban (see Scott’s blog about it what is Faban) by using

java -classpath fabandriver.jar:fabancommon.jar -server 
-Xmx1500m -Xms1500m com.sun.faban.driver.ab -c 2000 http://xxx:8080/index.html

This command runs 2000 separate clients (each in its own thread), each of which continually requests index.html with no think time. The common driver reports three pieces of information: the number of requests served per second, the average response time per request, and the 90th percentile for requests: 90% of requests were served with that particular response time or less. Let’s focus on the number of operations per seconds for now by running the test against Grizzly Http 1.0.20, Grizzly Http 1.7.3, v2 and v3:

Grizzly_1.0 Grizzly_1.7 GlassFish_v2 GlassFish_v3
 5739.025    5979.917    5432.300     5882.808

Hey hey v3 is almost as fast as Grizzly 1.7.3. Why? It really because v3 run on top of Grizzly directly, without any noise in between. Now let’s compare the dynamic resource performance using a very simple Servlet


    public void doPost(HttpServletRequest request, HttpServletResponse response){
        response.setContentType("text/plain");
        response.setStatus(HttpServletResponse.SC_OK);
        PrintWriter wrt = response.getWriter();
        for (int i = 0; i < 1 ; i++) {
            wrt.write("Hello world");
        }
    }

Yes, its a dummy Servlet!


Grizzly_1.7 GlassFish_v2 GlassFish_v3
 3621.600    3256.775      2998.225

v3 is a little slower here……most probably because of the container’s mapping algorithm. What is that? With v3, a tcp/udp port can handle any type of web applications (jruby, groovy, servlet, phobos, etc. (all build on top of Grizzly http :-)). So when a request comes in, v3 inspects and dispatch it to the proper container. That code introduces overhead and I suspect it is part of the regression. But a lot of changes has been made in v3, so I might be wrong…still, I consider the number very impressive as we have now a modular architecture which required a lot of changes.

But wait a minute. Grizzly 1.7 has a Servlet Container? Naaa not a real one. I’m just experimenting a tiny Servlet container, with great help from the Grizzly community. But forget Grizzly for now and look at the number :-) The goal for our v3 official release is set: We want to be better than Grizzly, which I’m sure we will :-). We need to improve the Grizzly extension in v3…in case you want to learn how to write such extension, just stop by us at JavaOne!

_uacct = “UA-3111670-1″;
urchinTracker();

technorati:

Categories: GlassFish, Grizzly

Writing a Comet web application using GlassFish

This blog describes how to write Comet enabled web application using GlassFish’s Comet Engine.

A couple of months ago, I’ve blogged about the technical details of the GlassFish‘s Comet support. Since then, I’ve got a lot of feedbacks on the blog and also privately. Surprisingly, a lot of peoples have started using the API and an asked for a blog describing a basic example. So here it comes … a basic Chat Servlet :-)

werixc.jpg

First, to enable Comet Support in GlassFish, add the following in ${glassfish.home}/domains/domain1/config/domain.xml


        <http-listener acceptor-threads="1" address="0.0.0.0" 
           blocking-enabled="false" default-virtual-server="server"
           enabled="true" family="inet" id="http-listener-1" port="8080"
           security-enabled="false" server-name="" xpowered-by="true">
                <property name="cometSupport" value="true"/>
        </http-listener>

Next, add in you web.xml:


        <load-on-startup>0</load-on-startup>

OK now the interesting parts. The first things to decide when writing a Comet enabled web app is the component that will get polled. For this example, I will use a Servlet. First, the Servlet needs to register to the CometEngine:


   48     public void init(ServletConfig config) throws ServletException {
   49         super.init(config);
   50         contextPath = config.getServletContext().getContextPath() + "/chat";
   51         CometEngine cometEngine = CometEngine.getEngine();
   52         CometContext context = cometEngine.register(contextPath);
   53         context.setExpirationDelay(60 * 1000);
   54     }

The important part to define first is the context path that will be considered for Comet processing (or polling). All requests that takes the form of http://:/context/chat will be considered for polling. The context.setExpirationDelay() will determine how long a request will be polled. For this example, I’ve set the expiration delay to 60 seconds. After 60 seconds, the polled connection will be closed.

Next, you need to define a Comet request Handler which will get invoked every time the CometContext is updated. For the Chat, the handler will be created after the user has entered its user name (by issuing http://…/login.jsp)


   71                 if ("login".equals(action)) {
   72                     String username = request.getParameter("username");
   73                     request.getSession(true).setAttribute("username", username);
   74
   75                     if (firstServlet != -1){
   76                          cometContext.notify("User " + username
   77                           + " from " + request.getRemoteAddr()
   78                           + " is joinning the chat.",CometEvent.NOTIFY,
   79                                  firstServlet);
   80                     }
   81
   82                     response.sendRedirect("chat.jsp");
   83                     return;
   84                 } else if ("post".equals(action)){
   85                     String username = (String) request.getSession(true)
   86                         .getAttribute("username");
   87                     String message = request.getParameter("message");
   88                     cometContext.notify("[ " + username + " ]  "
   89                             + message + "<br/>");
   90                     response.sendRedirect("post.jsp");
   91                     return;
   92                 } else if ("openchat".equals(action)) {
   93                     response.setContentType("text/html");
   94                     CometRequestHandler handler = new CometRequestHandler();
   95                     handler.clientIP = request.getRemoteAddr();
   96                     handler.attach(response.getWriter());
   97                     cometContext.addCometHandler(handler);
   98                     String username = (String) request.getSession(true)
   99                         .getAttribute("username");
  100                     response.getWriter().println("<h2>Welcome "
  101                             + username + " </h2>");
  102                     return;

After the user has logged in, the browser will be redirected to the chat.jsp page, which will sent the action=”openchat”. The CometHandler (the class that will update the chat message box) implementation looks like:


  134         public void onEvent(CometEvent event) throws IOException{
  135             try{
  136
  137                 if (firstServlet != -1 && this.hashCode() != firstServlet){
  138                      event.getCometContext().notify("User " + clientIP
  139                       + " is getting a new message.",CometEvent.NOTIFY,
  140                              firstServlet);
  141                 }
  142                 if ( event.getType() != CometEvent.READ ){
  143                     printWriter.println(event.attachment());
  144                     printWriter.flush();
  145                 }
  146             } catch (Throwable t){
  147                t.printStackTrace();
  148             }
  149         }
  150
  151
  152         public void onInitialize(CometEvent event) throws IOException{
                      ....
  156         }

Every time the user will post a new message, the CometHandler.onEvent(…) will be invoked and the Chat message pushed back to the browser.

On the client side, the chat.jsp page looks like


     26 <frameset>
     27   <iframe name="chat" src ="/comet/chat?action=openchat" width="100%" scrolling="auto"></iframe>
     28   <iframe name="post" src="post.jsp" width="100%" scrolling="no"/>
     29 </frameset>

You can download the application (which include the src) here.

Note that the application described here is really to give an example. I would never recommend the use of static variables like I did in the example.

werixc96.jpg

Before I forgot, one interesting feature I’ve recently added (was requested on first blog on Grizzly’s Comet) is the ability to update a single CometHandler (or a single polled request). When calling cometContext.addCometHandler(..), the returned value can be later re-used to push datas only to that cometHandler by doing:


     cometContext.notify(String message, int type, String cometListenerID);

See the API for more info. For the Chat example, I’ve added a pop up window where a chat moderator receives all the chat messages, who is connected and from where:


    138   event.getCometContext().notify("User " + clientIP
    139      + " is getting a new message.",CometEvent.NOTIFY,
    140        firstServlet);

That’s it. Very simple, is it? No needs to spawn a thread anywhere on the Servlet side, no special Servlet operations, etc.

Once I’ve a chance, I will try to use AJAX and improve the client. Any help is appreciated on that side :-) As usual, thanks for all the feedbacks sent by emails!

_uacct = “UA-3111670-1″;
urchinTracker();

technorati:

Categories: GlassFish, Grizzly

Can a Grizzly run faster than a Coyote?

Un coyote court t-il plus vite qu’un grizzly? Or can an NIO based HTTP connector be as fast as a traditional IO HTTP Connector or a C HTTP Connector? The next couple of lines will compare Tomcat 5.5.16, both the Coyote HTTP11 and the Tomcat Native (version 1.1.2) connector (aka APR) with the GlassFish Grizzly NIO powered HTTP Connector. Grizzly is an NIO extension of the HTTP11 implementation.

But don’t be fooled by the numbers I’m gonna publish. My goal here is to clarify the myth that NIO non-blocking sockets cannot be used along with the HTTP protocol. This blog is not against Tomcat, and I’m still part of the Tomcat community (although I’m not helping a lot those days, but very interested by Costin works on NIO). OK enough rant…..

First, if my numbers aren’t matching your real life application, I will be interested to hear about it. If you like APR/OpenSSL functionalities and think they should be included in GlassFish, I will be more than happy to port them in GlassFish. But let waits for the numbers before saying yes :-)

Oh…BTW some Grizzly numbers has already been published as part of the SJSAS PE 8.2 specJ2004 results. All results can be found here.

Passons maintenant aux choses serieuses….

Differences between Tomcat and GlassFish
First, in order to compare the two, let’s explore the differences between the products. Since GlassFish is a J2EE Container, the bundled WebContainer has to support more extensions than Tomcat. Those extensions mainly consist of supporting EJB and the JavaTM Authorization Contract for Containers. Both extensions have an impact on performance because internaly, it means extra events notification needs to happen (In Catalina, in means LifecycleEvent).

A perfect integration will means no performance regressions when extensions are added, but unfortunaly having to support EJB is adding a small performance hit. Fortunalty, JavaTM Authorization Contract for Containers isn’t impacting performance. Hence the best comparison would have been to compare JBoss and GlassFish, or Tomcat with Grizzly in front of it. But I’m too lazy to install JBoss….


Out-of-the-box difference

The main difference are:
+ GlassFish has Single Sign On enabled, Tomcat doesn’t
+ GlassFish has Access Logging enabled, Tomcat doesn’t
+ GlassFish starts using the java -client, Tomcat doesn’t set any.

Let’s turn off the differences in GlassFish, by adding, in domain.xml:


<property name="accessLoggingEnabled" value="false" />
<property name="sso-enabled" value="false" />
</http-service>

and starts both product using java -server. For all the benchmarks I’m using Mustang:


Java(TM) SE Runtime Environment (build 1.6.0-beta2-b75)
Java HotSpot(TM) Server VM (build 1.6.0-beta2-b75, mixed mode)

Also, Grizzly has a different cache mechanism based on NIO, where Tomcat cache static file in memory using its naming implementation. Grizzly Cache is configured using:


<http-file-cache file-caching-enabled="true" file-transmission-enabled="false" globally-enabled="true" hash-init-size="0" max-age-in-seconds="600" max-files-count="1024" medium-file-size-limit-in-bytes="9537600" medium-file-space-in-bytes="90485760" small-file-size-limit-in-bytes="1048" small-file-space-in-bytes="1048576"/>

And I’ve turned off the onDemand mechanism:

-Dcom.sun.enterprise.server.ss.ASQuickStartup=false

to fully starts GlassFish.


Envoyons de l’avant nos gens, envoyons de l’avant!

ApacheBench (ab)
Let start with a well know stress tool called ab (google ApacheBench if you don’t know it). I will use ab to compare performance of:

+ small, medium, and very large gif files (large & medium gives similar results)
+ basic Servlet
+ basic JSP (the default index.jsp page from Tomcat)

I’m gonna compare Tomcat HTTP11, Tomcat APR and GlassFish. I have tried to comes with the best configuration possible for Tomcat. I got the best number with:


<Connector useSendFile="true" sendfileSize="500" port="8080" maxHttpHeaderSize="8192" pollerSize="500" maxThreads="500" minSpareThreads="25" maxSpareThreads="250" enableLookups="false" redirectPort="8443" acceptCount="5000" connectionTimeout="20000" disableUploadTimeout="true" />

I’m putting more threads than the expected number of simultaneous connections (300) here so HTTP11 can be tested properly (since HTTP11 is one connection per thread). I’ve also try adding:

firstReadTimeout="0" pollTime="0"

but the result wasn’t as good as with the default.

For Grizzly, I’ve used:


<http-listener id="http-listener-1" ... acceptor-thread="2" .../>
<request-processing header-buffer-length-in-bytes="4096" initial-thread-count="5" request-timeout-in-seconds="20" thread-count="10" thread-increment="10"/>

Yes, you read it it correctly. Tomcat needs 500 threads where Grizzly needs only 10. NIO non-blocking is facinating, is it?

The ab command I’ve used is:

% ab -q -n1000 -c300 -k http://perf-v4.sfbay.sun.com:8080/XXX

Ready to see numbers? Not yet, here is the machine setup:

server

OS : Red Hat Enterprise Linux AS release 4 (Nahant Update 2)
ARCH : x86
Type : x86
CPU : 2x3.2GHz
Memory : 4 GB

client

OS : RedHat Enterprise Linux AS 4.0
ARCH : x86
ype : x86
CPU : 2x1.4GHz
Memory : 4GB

OK, now the numbers. For each test, I’ve run 50 times the ab command and took the means (for large static resource, I’ve ran between 10 and 13 because it takes time). The number aren’t changing if I remove the ramp up time for every Connector, so I decided to not remove them.

Small static file (2k)

% ab -q -n1000 -c300 -k http://perf-v4.sfbay.sun.com:8080/tomcat.gif

4k.jpg
Grizzly: 4104.32 APR: 4377.2 HTTP11: 4448.08
Here HTTP11 is doing a very good job, where Grizzly seems to be hunting and maybe servicing requests. Why? My next blog will explain a problem I’m seeing with MappedByteBuffer, small files and FileChannel.transferTo.

Medium static file (14k)

% ab -q -n1000 -c300 -k http://perf-v4.sfbay.sun.com:8080/grizzly2.gif

small.jpg
Grizzly: 746.27 APR: 749.63 HTTP11: 745.65
OK here Grizzly is better (thanks MappedByteBuffer).

Very large static file (954k)

% ab -q -n1000 -c300 -k http://perf-v4.sfbay.sun.com:8080/images.jar

large.jpg
Grizzly: 11.88 APR: 10.5 HTTP11 10.6.
Hum… here APR has connections errors (means 10) as well as HTTP11 (means 514) and keep-alive hasn’t been honored for all connections. Grizzly is fine on that run.

Simple Servlet

% ab -q -n1000 -c300 -k http://perf-v4.sfbay.sun.com:8080/tomcat-test/ServletTest

Servlet.jpg
Grizzly: 10929.93 APR: 10600.71 HTTP11: 10764.67
Interesting numbers…but can’t say if it’s the Connector or Catalina (Servlet Container). GlassFish Catalina is based on Tomcat 5.0.x (plus severals performance improvement)).

Simple JSP

% ab -q -n1000 -c300 -k http://perf-v4.sfbay.sun.com:8080/index.jsp

JSP.jpg
Grizzly: 1210.49 APR: 1201.09 HTTP11: 1191.57
The result here is amazing because Tomcat supports JSP 2.0, where GlassFish supports JSP 2.1. Kin-Man, Jacob and Jan (to name a few) doesn’t seems to have introduced performance regressions :-).

One thing I don’t like with ab is it doesn’t measure the outliers, meaning some requests might takes 0.5 seconds, some 5 seconds. All requests are counted, whatever they took to be serviced. I would prefer a Connector that avoid outlier (or at least I don’t want to be the outlier when I log on to my bank account!). Let see what the second benchmarks can tell:

Real world benchmark
The purpose of my second benchmark is to stress the server with a real world application that contains complex Servlet, JSP and Database transaction. I think ab is good indicator of performance but focus more on throughput than scalability. The next benchmark simulates an e-commerce site. Customers can browse through a large inventory of items, put those items into shopping carts, purchase them, open new accounts, get account status: all the basic things you’d expect. There is also an admin interface for updating prices, adding inventory, and so on. Each customer will execute a typical scenario at random; each hit on the website is counted as an operation.

The benchmark measures the maximum number of users that the website can handle assuming that 90% of the responses must come back within 2 seconds and that the average think time of the users is 8 seconds. The result are:

j-tpcw.jpg
Grizzly: 2850 APR: 2110 HTTP11: 1610

Note: I have try APR with 10 threads, 1000 threads and the best results was obtainned using 500 threads.

Conclusion
Do your own conclusion ;-) My goal here was to demonstrate that NIO (using non blocking sockets) HTTP Connector are ready for prime time. I hope the myth is over….

Once I have a chance, I would like to compare Grizzly with Jetty (since Jetty has an NIO implementation). I just need to find someone who knows Jetty and can help me configuring it properly :-)

Finally, during this exercise I’ve found a bug with the 2.4 kernel and the performance was really bad. Also, it is quite possible you run benchmarks where Tomcat perform better. I would be interested to learn about the way Tomcat was configured…..

// <![CDATA[// technorati:



Categories: GlassFish, Grizzly

Grizzly part III: Asynchronous Request Processing (ARP)

As described in part II, it is possible to extend the Grizzly Http Engine by writting Pipeline, Algorithm, Handler and AsyncHandler. By default, every HTTP connection are executed synchronously, e.g the request is parsed, the servlet executed and the response flushed to the browser. There is situation where this model doesn’t work, e.g. when a business process is calling out to another one over a slow protocol, or if there is a work flow interruption, like the “manager approval email” case. Fortunately, it is possible in Grizzly to implement asynchronous request processing (ARP) using the AsyncHandler interface. But not only in Grizzly ;-)

Next couple of paragraphs will discuss the set of interfaces available, describe the default implementation and conclude with a Servlet which execute only when a new message is available inside a Gmail account. The Servlet can be seen as HTTP Gmail notifier. But first, what’s the goal or supporting ARP?

Goal
The goal is to be able to build, on top of Grizzly and NIO, a scalable ARP implementation that doesn’t hold one thread per connection, and achieve as closer as possible the performance of synchronous request processing (SRP).

Available Interfaces
Grizzly currently expose three interfaces that can be used to implement ARP. They are:

AsyncHandler: This interface is the main entry point. When ARP is enabled, Grizzly will delegate the request processing execution to this interface. Instead of executing a Task, the Task execution will be delegated to the AsyncHandler implementation. This interface is mandatory.

AsyncExecutor: This interface implementation will usually decide when a Task needs to be interrupted, under which conditions (using AsyncFilter), and when the Task needs to be resumed. A Task is interrupted when the conditions required for its execution aren’t meet. An AsyncExecutor must have one or more AsyncFilter. If no AsyncFilter are defined, the SRP model will happens. This interface isn’t mandatory, but recommended.

AsyncFilter: Implementation of this interface will determine if the current request meet its execution conditions, e.g. does the request need to be executed or be interrupted. An AsyncFilter who decide to allow all requests to be executed without being interrupted will simulate the SRP mode.

The Default implementation
GlassFish ship with a default ARP implementation. The default implementation consist of a DefaultAsyncHandler, a DefaultAsyncExecutor and a new Task called AsyncProcessorTask. The new AsyncProcessorTask is a wrapper around a ProcessorTask, which contains all the code for executing an HTTP request and used with the SRP mode. Mainly, ARP happens when:

1. The request execution is delegated to the DefaultAsyncHandler
2. In DefaultAsyncHandler, the Task is wrapped with an AsyncProcessorTask
3. The AsyncProcessorTask is executed. Its execution consist of delegating the request processing to an DefaultAsyncExecutor.
4. AsyncProcessorTask will first invoke DefaultAsyncExecutor.preExecute(). preExecute() will parse the request line and its headers.
5. Second, AsyncProcessorTask will invoke DefaultAsyncExecutor.interrupt(). interrupt() will execute all defined AsyncFilter(s).
6. An AsyncFilter will validate its execution conditions. If the conditions aren’t meet, the Task will be interrupted and added to a Scheduler that will re-execute it once the conditions are meet. It is important to note here that the AsyncFilter thread will not be locked or interrupted, but will be returned to the Pipeline (Grizzly Thread Pool Wrapper). This is the beauty of non blocking NIO, where the one thread per socket paradigm doesn’t apply.
7. Once the AsyncFilter determine the execution conditions are meet, the Task will be removed from the interrupted queue and executed. This is usually where a Servlet/JSP are executed.
8. DefaultAsyncExecutor.postExecute() will flush the response to the client.

Note that DefaultAsyncExecutor.preExecute/interrupt/postExecute always delegate the execution to a set of ProcessorTask methods, thus requests are always executed the same way, independently of ARP or SRP.

It is important to note that the default implementation isn’t locking the thread when a Task is interrupted. The default Scheduler only have one thread to hold all SelectionKey associated with all interrupted requests.

OK that’s it for the theory. Let’s work on a real application using GlassFish.


The Gmail Servlet Notifier

Would it be nice if we can have a way to have a Gmail Notifier similar functionalities using HTTP? Not to say there is no Unix version. Of course you can always log in your account and refresh the page, but I don’t want to refresh every minutes or so. You can probably do something similar using AJAX…but I want to do it in Grizzly :-)

What we need is a Servlet that execute only when new emails are available in a Gmail account. Not only to Gmail, but to any POP3 account. In this case, the execution conditions would consist of connecting to the account, look for new emails. If new emails are available, then execute a Servlet that list all new emails (or display a flashing flag). Whatever the Servlet does, the goal here to to execute it only when its execution conditions are meet.

The solution is simple. We just need to define an AsyncFilter who does:

1. Connect to our Gmail account
2. Look for new emails. If emails are available, execute the Task.
3. If no news emails, interrupt the Task (ARP), wait a couple of seconds (here 10 seconds), then re-evaluate if new emails are in. If new emails has come, then execute the Servlet.

Fortunately we can use JavaMail for (1), and Grizzly default ARP implementation for (2) and (3), and by adding a new AsyncFilter implementation. Then, we just have to:

1. Starts Grizzly using ARP mode.
2. Create a JavaMail Session for Gmail to be used by the AsyncFilter.
2. Deploy the Gmail war file. All the account information (username, password, server url, port, etc.) are included inside web.xml (I agree the password shouldn’t be there).
3. Use a browser to invoke the Servlet. The browser will keep the connection openned until you receive a new email.
4. When new emails are read, the Servlet will be executed and your email information (headers, text) will be displayed.

The implementation just consist of:

+ JavaMailAsyncFilter: this is the AsyncFilter than will determine if ARP or SRP is used.
+ JavaMailAsyncFilterHandler: This interface need to be implemented by the Servlet. Via this interface, the JavaMailAsyncFilter will be able to determine if the execution conditions are meet or not.
+ JavaMailAsyncFilterEvent: this object contains context information about the current execution conditions. This object is shared between the JavaMailAsyncFilter and the Servlet implementation.
+ EmailNotifierServlet: the Servlet that will be interrupted unless new emails are available.

You can find the basic implementation here. Make sure you read the README.TXT which explain how to build and deploy in GlassFish. Since ARP is not a default feature, it is a little complicated to setup (feel free to post here for help). I’ve tested with my local POP3 provider and my Gmail account and it worked fine.

When testing with Gmail, I selected Settings > Forwarding and POP. In the POP Download, I’ve picked “Enable POP only for mail that arrives now on”. Thus my first request to the Servlet will not meet the execution conditions, and the Servlet will be interrupted until I send an email to myself.

Of course the Gmail implementation is far from perfect, and can be greatly improved (improvement are more that welcome!). My goal here was to demonstrate how you can extend Grizzly and start supporting ARP. As always, contributions are welcome.

That’s it. Next time I will try to give some NIO tricks and tips (specially when it time to deal with OP_WRITE).

P.S Thanks to Andrea Egloff for his useful feedback on the default ARP implementation.

_uacct = “UA-3111670-1″;
urchinTracker();

technorati:

Categories: GlassFish, Grizzly

Grizzly NIO Architecture: part II

First, see part I here.

Second, Sun published specJ2004 using the upcoming SJSAS 8.2 PE, which contains a similar Grizzly version than the current one available in GlassFish. I think it is the first ever NIO implementation that got benchmarked :-) I suspect it’s our best score ever, but I didn’t look at all the results.

OK enough marketing, and let see how Grizzly looks like under the hood. The next couple of lines will describe the main components in Grizzly, why they exists and how they can be customized. All the classes decribed in the text are available here

Pipelines
The main entry point is the Pipeline. I don’t know why I’ve picked that name from Tomcat’s Catalina, since everybody confuse Catalina Pipeline with Grizzly Pipeline. I guess I should have picked ThreadPoolWrapper or something along that line, because a Grizzly Pipeline component is really a Thread Pool Wrapper. A Pipeline is reponsible of the execution of a Task implementation. There is currently three Pipeline implementation included in the code base:

(1) LinkedListPipeline: thread pool based on a linked list
(2) ThreadPoolExecutorPipeline: thread pool based on java.util.concurrent.ThreadPoolExecutor
(3) ExecutorServicePipeline: based on java.util.concurrent.Executors

Surprisingly, all benchmarks perform better when (1) is used, hence the default. It seems the park()/unpark() uses in (2)(3) are slower that a simple lock.

SelectorThread
The main entry point in Grizzly is the SelectorThread. The SelectorThread is where the NIO selector is/are created. It is possible to use more than one Selector, based on the number of cpu GlassFish is installed on. When processing a request, the SelectorThread always create instances of Task, and pass the instance to the Pipeline. The SelectorThread can pass three type of Task:

AcceptTask to handle NIO event OP_ACCEPT
ReadTask|AsyncReadTask|ReadBlockingTask to handle OP_READ
ProcessorTask to handle the request processing, and OP_WRITE

The SelectorThread is configurable and can create one Pipeline per Task, or share a Pipeline amongs Tasks. The best performance I’ve measured as of now is when OP_ACCEPT is executed on the same thread as the SelectorThread (so outside a Pipeline), OR_READ and OP_WRITE on the same thread using a Pipeline. Note that during that time, I’ve faced that strange bug, which is now fixed in 5.0 ur7 and 6.0. Make sure you are using the right VM version if you plan to play with Grizzly.

Tasks
The ReadTask is mainly responsible for pre-processing the request bytes. By pre-processing, I means to decide when we think we have enough bytes to start the processing of the request. Different strategy can be used to pre-process the request, and a strategy can be implemented using the StreamAlgorithm interface.

The ReadTask always do the first read on a socketChannel, and then delegate the work to a StreamAlgorithm implementation. Grizzly has three StreamAlgorithm implementation:

(1) StateMachineAlgorithm: this algorithm reads the request bytes, and seek for the HTTP content-length header. The strategy here is to find the content-length and reads bytes until the requests headers and the body is fully read (because with NIO, you can’t predict if you have read all the bytes or not). Once all the bytes are read, the algorithm tells the ReadTask it can pass the processing to the ProcessorTask. The algorithm only supports HTTP 1.1, and GET/POST/HEAD/OPTIONS (but not PUT). But performance is quite impressive, but you must always cache the request in memory, which is bad for PUT or large POST body.

(2) ContentLengthAlgorithm: same as (1), except supports all HTTP method and HTT 0.9/1.0/1.1. This Algorithm is based on Coyote HTTP/11 connector way of parsing the request, and perform very well.

(3) NoParsingAlgorithm: this algorithm doesn’t pre process the requests bytes. The processing of the bytes will be delayed until the ProcessorTask is executed. The strategy here is to assume we have read all the bytes, and if bytes weren’t all read, let the ProcessorTask decide when/how to read the missing bytes. This is achieved by the ByteBufferInputStream class, which internally use a pool of Selectors to register the socketChannel and waits for more bytes.

(3) is the current default startegy.

The ReadTask also manage the life cycle of the SelectionKey (if you aren’t familiar with NIO, consider a SelectionKey == Socket). To avoid holding a thread when processing a persistent connection (keep-alive), like most of the current Java HTTP Server implementation, the ReadTask implement a strategy where threads aren’t used in between request. The strategy used is:

(1) If the request is completed, but keep-alived, release the thread and register the SelectionKey for IO event
(2) if the request isn’t completed, release the thread but keep the ReadTask attached to the SelectionKey, to keep the connection state/history.
(3) if the request is completed and not keep-alived, release the thread and the SelectionKey.

This strategy prevent one thread per request, and enable Grizzly to server more that 10 000 concurrent users with only 30 threads. Note that (2) happens when a StreamAlgorithm fail to properly parse the request (ex: by not predicting properly the end of the stream). By default, (2) is never executed, and shouldn’t. But I’m still exploring strategy, so the functionality is still available.

ReadTask also implement the missing socket.setSoTimeout() functionality which was dropped in J2SE when a socketChannel is non blocking. I don’t know what was the rational behind not supporting this API, but it is pretty bad because every NIO implementation will have to implement that mechanism.

Currently, the implementation reside in KeepAlivePipeline, which internally uses a java.util.concurrent scheduler to simulate socket.setSoTimeout. Performance wise, I have to admit Grizzly perform better when this mechanism is turned off. But since setSoTimeout() is crucial when parsing HTTP requests (to detect DOS, attack, etc.), the implementation is turned on by default. I’m still trying to find a better way to implement it, as I really don’t like the current implementation, event if the Rule architecture looks promising, specialy when application resource allocation (ARA) is used. I will soon discuss what ARA does, but you can take a look at the code here

Next, the ProcessorTask mainly operates on the request bytes, parsing the request line and headers. Once parsed, it pass the request to an adapter, which is configurable (the current adapter is the entry point inside the Servlet Container). The ProcessorTask can execute synchronously the request, or delegate the request processing to an AsyncHandler implementation (I will discuss asynchronous request processing in my next blog, which I hope isn’t in 6 months ;-))

Handlers
ReadTask and ProcessorTask can be configured with Handler, which can be used to intercept the state of the processing. Current Handler implementation include a new static resource cache, which serve static resources (html, gif, etc.) using NIO MappedByteBuffer, without delegating the call to the Catalina Default Servlet. Only static resources that aren’t protected by a filter or a security constraint are candidate to the FileCache.

Handler are always associated with a StreamAlgorithm, and execute at the ByteBuffer level.

[…]

Ouf…I should blog more often, so my blogs are shorter :-). To recap, here is the current interfaces available in Grizzly that can be customized:

1. Pipeline: Thread Pool Wrapper, entry point for Task
2. StreamAlgorithm: The parsing strategy used.
3. Handler: Manipulating the requests before it get parsed.
4. AsyncHandler: The entry point for implementing asynchronous request processing system.

In my next blog, I would like to cover the OP_WRITE headache we had in Grizzly, specially when executing on a win32 platform. I would also like to introduce the Asynchronous Request Processing hooks available in Grizzly, e.g. the ability to serve a request over a long period of time without forcing the entire state to be kept around (e.g. when a business process is calling out to another one over a slow protocol, or if there is a work flow interruption, like the “manager approval email” case).

That’s it for now. Let me know what you think. I’m very interested about improving the setSoTimeout implementation, and to explore more performant StreamAlgorithm implementation.

_uacct = “UA-3111670-1″;
urchinTracker();

technorati:

Categories: GlassFish, Grizzly

Grizzly: An HTTP Listener Using Java Technology NIO

Writing scalable server applications in Java technology has always been difficult. Before the advent of NIO, thread management issues made it impossible for an HTTP server to scale to thousands of users. I’m gonna start blogging on Grizzly, the HTTP Connector based on NIO shipped in GlassFish.

First, the truth: This is my first ever blog, I’m Quebecois and do a lot of typos (and speak with an ugly accent)..and I’m tempted to write in french…so there is still time to hit the back button of your browser!

For my first blog ever (and hopefully not the last), I will describe a new HTTP Connector based on NIO, called Grizzly on which I’m working on. Grizzly is currently the HTTP front-end for SJSAS PE 8.1 (throttled version), and included in the GlassFish project.

Grizzly has been designed to work on top of the Apache Tomcat Coyote HTTP Connector. The Coyote Connector is used in Tomcat 3/4/5 and has proven to be a highly performant HTTP Connector when it is time to measure raw throughput. But as other Java based HTTP Connector, scalability is always limited to the number of available threads, and when keep-alive is required, suffer the one thread per connection paradigm. Because of this, scalability is most of the time limited by the platform’s maximum thread number. To solve this problem, people usually put Apache in front of Java, or use a cluster to distribute requests among multiple Java server.

With NIO available in 1.4, I’ve started thinking about exploring a new HTTP connector based on NIO. Grizzly started as a side project and based on throughput performance number I surprisingly got (I wasn’t scare about scalability), I’ve replaced Coyote with Grizzly. At the time, I didn’t find any open source framework I can rely on to start my work, so I started to write my own framework (couple of weeks after I started, Mike Spille blogged about EmberIO). EmberIO looked promising but seems little activities occurred since then.

Another reason I started the work was because I was tired of hearing that NIO wasn’t appropriate for HTTP, without seeing a single real implementation. I wanted to see real stuff, not judgments peoples have. If performance wasn’t there, I would have never bloged about Grizzly…

Grizzly differ from Coyote in two areas. First, Grizzly allow the pluggability of any kind of thread pool (three are currently available in the workspace). I will not give details now, but will blog later with the result I’m getting with those threads pools, and why you may want to change the default one based on what you are doing with GlassFish. Second, Grizzly supports two modes: traditional IO and non blocking IO. Why? the main reason is SSL. At the time I’ve started, SSL non blocking wasn’t available, so I needed to use Coyote SSL implementation (which is quite good). I should soon add support for SSL non blocking.

OK enough words. Let me try to explain Grizzly architecture. The problem with NIO non blocking and HTTP is your never know when all the bytes have been read (this is how non blocking works). With blocking, you just wait on the socket’s input stream. With non-blocking, you cannot wait on the socket input stream because it will return as soon as you have read all the available data. But that doesn’t means you have received all the request bytes. Technically, you have no guarantee than all the bytes has been read from the socket channel:

count = socketChannel.read(byteBuffer));

This may not read the entire stream bytes, and may require several extra read operations. Unfortunately, this situation occurs frequently when reading HTTP requests. I’ve explored several strategies (which I will details in my next blog) and kept the most performant one, which consist of a state machine used to parse the content-length header and predict the end of the stream.

Having to support several strategies and several IO modes, I did design Grizzly in such a way it is easy to integrate new strategy within the framework. A Task based architecture has been used, each task representing an operation when manipulating the request:

+ AcceptTask for managing OP_ACCEPT
+ ReadTask for managing OP_READ
+ ReadBlockingTask for managing blocking IO operation.
+ ProcessorTask for parsing/processing the byte stream.

Every task can execute on their own thread pool or use a shared one. By default, one thread handle OP_ACCEPT and the Selector.select() operation, and a thread pool all others operations. This is how I’m getting the best performance number when running internal benchmarks. Designed to measure scalablity and throughput simultaneously, our benchmarks look for how many concurrent clients can we support, with the following criteria:

+ Avg. client think time is 8 seconds
+ 90% response time is 3 seconds
+ Error rate < .1%

Plug : But I will not discuss the numbers here, because I keep the numbers for my JavaOne session on June 28 :-)

Well, that’s it for today. Next time I will gives more details about the strategies I have explored (with code, not words so less typos) and the characteristics of every threads pools available in Grizzly. And sorry if it was boring…at least I’m exercising my english writting skills :-)

Jeanfrancois

P.S BTW, even if I use “I”, I’ve developed Grizzly with Charlie Hunt(Netbeans team), Scott Oaks (Performance team) and Harold Carr (Corba team).

P.P.S Grizzly is far from perfect, so patches and strategies are more than welcome!

_uacct = “UA-3111670-1″;
urchinTracker();

technorati:

Categories: GlassFish, Grizzly
Follow

Get every new post delivered to your Inbox.

Join 51 other followers