Archive

Archive for March, 2007

New Adventures in Comet: Throttling the Server push

Web Applications that uses Comet Request processing are more an more used with asynchronous clients (AJAX, SOA, GWT, etc.). At AjaxWorld this week, I was surprised to see how many developers attended the two scheduled Comet talks and how many companies already have Comet based application in production. But like with every new technologies, no formal design recommendations have been widely described and discussed (unless I’m missing something :-)). Hence two Comet applications doing essentially the same thing might significantly differ in their design. Well, let me share what I’ve learned over the last year, having see multiples real Comet based applications. Oh no! Another series of blogs from me!

comet.JPG

So far, the biggest problem I’ve seen is related to the frequency data are pushed to clients. When a server push data back to the client, it can easily flood the subscribed clients by sending too many data in a short period of times. To demonstrate the problem, let use the most famous Comet application after the Chat which is the real time stock option application. As an example, take a look at the Lighstreamer demo. In that demo, stock options are updated real time. I don’t know the technical details about the application, but let’s assume it use Comet. Depending on the source you use, stock options value can be updated very frequently, mostly every second. Thus if the server have to push data every seconds, it is easy to understand how bad your application will scale if 10 000 Comet clients are connected, waiting for a server push. In that situation, you need a way to throttle the stock options updates so the subscribed Comet clients aren’t getting flooded by server pushes. There is two solutions:

  • On the client side, you decide to read ant update the page, let’s say every 5 seconds, and discard all other pushes from the server or
  • throttle the pushes on the server side.

Most of the time, I would recommend the latter approach as pushing data over a connection is not free, not to say that if 10 000 users are connected, updating them with data they will not use might eat all both client and server CPU. I know this is trivial, but still scalability problems might be visible only when your application goes into production, which is unfortunately late in the development cycle. Now the big question is how to do that, because this is not as simple as it looks. I cannot speak for other Comet implementation, but in Grizzly Comet, a database (assuming the stock options data are stored/updated there) will invoke the CometContext.notify() method, passing the updated data. If we want to throttle the data to avoid flooding the Comet clients, the logic will have to happens on the database side. Well, I don’t like that approach for two reasons:

  • First, if your application uses several sources (a database, a web service, etc.) to push data, it will means all those sources will have to implement the ‘throttling logic’.
  • Second, if one of your Comet client start flooding your server with data, the ‘throttling logic’ will have to be implemented either on the client or server side.

Naaa…I don’t like that because in my opinion, the ‘throttling logic’ should be part of the Comet implementation API itself. Fortunately for GlassFish users using Grizzly Comet, there is an API you can use to customize the ‘throttling logic’:

public interface NotificationHandler {

    /**
     * Return true if the invoker of notify() should block when
     * notifying Comet Handlers.
     */
    public boolean isBlockingNotification();

    /**
     * Set to true if the invoker of notify() should block when
     * notifying Comet Handlers.
     */
    public void setBlockingNotification(boolean blockingNotification);


    /**
     * Notify all CometHandler.
     * @param cometEvent the CometEvent used to notify CometHandler
     * @param iteratorHandlers An iterator over a list of CometHandler
     */
    public void notify(CometEvent cometEvent,Iterator<CometHandler> iteratorHandlers)
        throws IOException;


    /**
     * Notify a single CometHandler
     * @param cometEvent the CometEvent used to notify CometHandler
     * @param iteratorHandlers An iterator over a list of CometHandler
     */
    public void notify(CometEvent cometEvent,CometHandler cometHandler) 

An implementation of this interface will most of the time contains the logic to reduce server pushes, which resources to use when updating clients, etc. Your application just have to call CometContext.setNotificationHandler(myHandler)…that’s it! Thus, when writing (or porting) a Comet application, you can always customize (or re-use the available ones) NotificationHandler and make sure that when under load, neither clients nor the server flood the subscribed clients when invoking CometContext.notify(). Better, you might develop with an un-throttled NotificationHandler and latter change with one that only push data every 10 seconds, only once you are production ready.

To recap, when writing (or porting) a Comet application, you must make sure a server push doesn’t bring down the server because the server is pushing data too frequently, flooding clients and the server itself. Being able to customize the way data are pushed back to client is something I consider extremely important. Under load, your customized NotificationHandler can make a huge difference and make browser experience much better.

Next time I will discuss another commons problem, which is when a client start sending tons of data to the server. Should the data be throttled on the client or server side?

_uacct = “UA-3111670-1″;
urchinTracker();

technorati:

Categories: Comet

Configuring Grizzly for performance part II: Setting the proper values in domain.xml

This time I will describe how to properly configure Grizzly in GlassFish. The out-of-the-box values are really not appropriate when GlassFish is used in production and can gives extremely bad results. Here is a couple of recommendations to make in domain.xml:

First, make sure the -server VM is used instead of client:

<jvm-options>-server</jvm-options>

Disable AS Quick Startup:

<jvm-options>-Dcom.sun.enterprise.server.ss.ASQuickStartup=false</jvm-options>

Increase the number of worker threads:

<request-processing header-buffer-length-in-bytes="8192" initial-thread-count="10"
request-timeout-in-seconds="30" thread-count="130" thread-increment="10"/>

Increase the worker thread queue:

<connection-pool max-pending-count="-1" queue-size-in-bytes="-1" 
receive-buffer-size-in-bytes="4096" send-buffer-size-in-bytes="8192"/>/code>

Here -1 will configure the queue to indefinitely accept connection. You might want to increase the default value instead.

Enable the http file cache:

<http-file-cache globally-enabled="true" file-caching-enabled="true" ... >

Make sure the acceptor-threads value is large enough:

<http-listener id="http-listener-1" address="0.0.0.0" port="8080" acceptor-threads="2" ... 

I usually match the value based on how many CPU are available.

If you aren't planning to use WSIT, then turn off port unification by removing all occurrence of proxiedProtocols:

<property name="proxiedProtocols" value="ws/tcp"/>

Those modifications combined with part I will always make a difference :-)

_uacct = "UA-3111670-1";
urchinTracker();

technorati:

Categories: Uncategorized

Asynchronous Comet Processing

Recently I’ve had the opportunity to visit companies that build their product on top of GlassFish Grizzly’s Comet API and brainstorm about what improvements we should add to the current Comet API, what limitation, etc. The most interesting idea I got from what I’ve seen is what I call Asynchronous Comet Processing. But wait, since Comet is already asynchronous, what does that means? Well let’s take an example to illustrate the problem most if not all existing Server that support Comet implementation “suffer”. In case this is the first time you read about Grizzly’s Comet, you might want to read this Comet example before continuing.

acp.jpg

In Grizzly, a request will be marked as a Comet request (or long polled request) when its response object is attached to a CometHandler:

 
                   CometRequestHandler handler = new CometRequestHandler();
                   handler.attach(response.getWriter());
                   cometContext.addCometHandler(handler);

After the CometHandler has been added, the request will be polled, waiting for server data push coming from other http clients, EJBs or databases. The server will push data as soon as they come, e.g. when the CometContext’s notify method is invoked (a small AJAX example):


                    cometContext.notify("<script id='" + username + "_" 
                                 counter++ + "'>" + "window.parent." + callback
                                 + "(" +  message + ");</script>")

Internally, the cometContext will notify every CometHandler (Comet request or long polled connection) by invoking their:


                    public void onEvent(CometEvent);

For a limited number of CometHandler, the component that invoke the notify method will block a few seconds and then continue. But let say (as I’ve learned last week by seeing real world Comet apps) you have 5000 CometHandler (ya, 5000 Comet requests). Invoking the CometComet.notify method will block until all CometHandler has been notified. This is what I call synchronous Comet processing. For some application it will be desired to block on the notify, but I suspect for the majority of Comet application, blocking might not have the desired effect. Hence I’ve implemented in Grizzly Comet what I call asynchronous Comet processing (ACP). I’ve also enabled the feature by default, meaning invoking:


                    cometContext.notify(...)

will no longer block. For some application (if not all) it will significantly improve the performance (or at least the user experience). The important thing to remember is you don’t have to write any thread management code inside your application because this is handled internally by the Grizzly default pipeline. Of course you can always spawn a thread from your Servlet to do the same, but I always prefer to delegate thread management stuff to the container. With Grizzly, you just need to decide if the notification process needs to be blocking or not. Event better, an application might decide that in some case the notify method needs to block (like when you know which CometHandler to call), which is easy to do by just invoking:


                    cometContext.setBlockingNotification(true);
                    cometContext.notify("ACP is really nice");

The drawback of asynchronous Comet processing is now you must make sure your CometHandler is thread safe because Grizzly internal threads may call the CometHandler.onEvent() concurrently as multiple http requests could fires cometContext.notify() operations. The simplest way (but not the fastest way..read/write lock might be better) is to do something like:


                    public synchronized void onEvent(CometEvent event){
                          ....
                    }

OK I’ve learned something during my last California travel!

_uacct = “UA-3111670-1″;
urchinTracker();

technorati:

Categories: Uncategorized
Follow

Get every new post delivered to your Inbox.

Join 51 other followers