Archive

Archive for the ‘Async Http client’ Category

Writing Portable WebSockets Application using Java

Writing Portable WebSockets’ applications using the Java Language can be challenging. Since there is no API standardization yet, several “native” API exists (Play!, Vert.x, Kaazing), you can easily get locked with their API, changes, single vendor etc. The same “problem” happens with well known WebServer: Jetty, Tomcat or GlassFish all have native API making your application not portable. Worse, the same problem happens on the client side: Grizzly, Netty, JWebSocket all have their own API. Not better on the browser side as well: Firefox went with its own API instead of the recommended one (MozWebSocket, fixed in Firefox 12).

So, already giving up? The solution is simple: use the AHC library(client) and the Atmosphere Framework (browser and server). The AHC library adds a thin layer on top of existing WebSockets’ client libraries, using the Netty Framework as default. The Atmosphere Framework allow portable WebSockets across Java based WebServers.

Part I — Writing the Server Component

The Atmosphere Framework works on top of existing native WebSocket implementation and currently supports Jetty, Tomcat, GlassFish and the Netty Framework (the NettoSphere). The Atmosphere Framework allows writing pure WebSockets applications as well as applications supporting the HTML 5 Server Side Events, Long-Polling or HTTP Streaming. Today this blog will only focus on writing WebSockets-only applications. So, no fallback on another transport in case WebSockets aren’t supported, but it is important to note that Atmosphere transparently support that transport’s fallback.

The portable API and easiest way to write pure WebSockets with Atmosphere is called WebSocketHandler, and can be defined as:

public abstract class WebSocketHandler implements WebSocketProtocol {

    public void onByteMessage(WebSocket webSocket, 
                              byte[] data, int offset, int length) {
    }

    public void onTextMessage(WebSocket webSocket, String data) {
    }

    public void onOpen(WebSocket webSocket) {
    }

    public void onClose(WebSocket webSocket) {
    }

    @Override
    public void onError(WebSocket webSocket, 
                        WebSocketProcessor.WebSocketException t) {
    }

For example, let’s say we want to write a simple echo server that broadcast received messages to all connected WebSockets, all we need to do is to extends that class with:

    @Override
    public void onTextMessage(WebSocket webSocket, String message) {
        MetaBroadcaster.getDefault().broadcastTo("/*", messages);
    }

All this code is doing is to broadcast the received messages to all connected client using the MetaBroadcaster utility classes. Since, by default, all connected WebSockets are registered at ‘/*’, no extra code is required. Of course we could have wrote something more complicated (like writing our own WebSocket Protocol), but that’s not the goal of this blog.

Part II — Writing the Browser Component

For the Browser component, let’s use the Atmosphere JQuery Plugin, which supports all browsers and could fallback to another transport in case WebSocket aren’t supported. In the easiest form, all we need to do to receive WebSockets messages is:

   var socket = $.atmosphere;
   var req = new $.atmosphere.AtmosphereRequest();
   req.url = document.location.toString() + '/echo';
   req.transport = "websocket';

   req.onOpen = function(response) {
       alert("WebSocket opened");
   }

   req.onTransportFailure = function(request) {
       alert("This browser or the remote Server doesn't support WebSocket");
   }

   req.onMessage(response) {
      var message = response.responseBody;
      alert("WebSocket Message received: " + message);
   }

   var subSocket = socket.subscribe(request);

Of course a real application will not use the alert call. An interesting callback here is the onTransportFailure, which is called in case the browser or the server isn’t supporting WebSockets. For sending WebSocket’s message:

    subSocket.push("Hello World");

Part III — Writing a Java Client

Now let’s write a portable Java client using the AHC library. As simple as

    AsyncHttpClient c = new AsyncHttpClient();
    WebSocket w = c.prepareGet("ws://127.0.0.1:8080")
                   .execute(new WebSocketUpgradeHandler.Builder().build())
                  .get();
    w.addWebSocketListener(new WebSocketTextListener() {

       public void onMessage(String message) {
         System.out.println("Message Received: " + message);
       }

       public void onOpen(WebSocket websocket) {
         System.out.println("WebSocket Opened");
       }
   }).sendTextMessage("Hello World");

By default the AHC library use the Netty Framework for WebSockets support, but other framework (like the Grizzly Framework) can easily be replaced.

For a more complex, portable WebSockets application, take a look at the Atmosphere Chat sample, which transparently support WebSocket, Server Sides Events, Long-Polling and Streaming. If you are planning to use WebSockets and Java, I strongly recommend you look at Atmosphere instead of using private native API and get stuck on a server/framework forever. For more information, ping me on Twitter or follow the Atmosphere Framework!

Transparently adding WebSockets to your application using SwaggerSocket

Today at Wordnik we announced the immediate availability of SwaggerSocket, a REST over WebSockets protocol that build on of Atmosphere and AsyncHttpClient. In this blog I will give some details about how the protocol works and how we implemented it.

First let’s gives some details about how SwaggerSocket works. SwaggerSocket build on top of the Atmosphere WebSocketProtocol API. The Atmosphere Runtime dispatchs WebSocket events to WebSocketProtocol API, allowing the manipulation of WebSockets events. It is important to note that Atmosphere Runtime takes care of all the WebServer WebSocket details (remember there is no standard on the server side with WebSocket). That why writing a WebSocket application with Atmosphere is portable and can be run on top of Netty, Jetty, GlassFish and Tomcat WebSocket implementation. By default, Atmosphere uses the SimpleHttpProtocol for dispatching WebSocket events to framework or application.

The SimpleHttpProtocol is “simple”: when a WebSocket message is received, it is dispatched as a POST to applications and frameworks. As an example, the Jersey Framework works without any modification inside Atmosphere and transparently work with WebSocket. But you can’t dynamically pass information like form/query params and headers unless you add those inside the WebSocket message itself, by writing your own “sub protocol”. Worse, why waiting for a response to arrive before sending another one? Why not sending an array of requests and get responses as they come, asynchronously? This is where SwaggerSocket can saves our life!

The SwaggerSocket Protocol is exactly that, a Protocol on top of WebSocket. The First version of the Protocol is using JSON to pass information between the client and server. As an example, a WebSocket message looks like:

    "requests" : [
        {
            "uuid" : 0,
            "method" : "POST",
            "path" : "/foo",
            "headers" : [
                {
                    "name" : "Content-Type",
                    "value" : "application/json"
                }
            ],
            "queryStrings" : [
                {
                    "name" : "foo2",
                    "value" : "bar2"
                }
            ],
            "messageBody" : "SwaggerSocket Protocol is cool"
        }
    ]

A Response looks like

    "responses" : [
        {
            "uuid" : 0,
            "status" : "status",
            "path" : "/foo",
            "headers" : [
                {
                    "name" : "name",
                    "value" : "value"
                }
            ],
            "messageBody" : "You are right! SwaggerSocket is coool!"
        }
    ]

It is important to note that any existing applications will WORKS AS IT IS, without any modifications server side. That means your Wicket/GWT/Vaadin/Resis/Hazelcast/Jersey/Spring/Guice/RichFaces/etc. can take advantage of the protocol right now. As an example, a simple Jersey resource:

@Path("/helloworld")
class HelloWorld {

  @Path("/sayHello")
  @GET
  def get(): String = {
    "Swagger Socket Hello World"
  }
}

will run unmodified. For the client, you have choice:

  1. Open a WebSocket and use the swaggersocket-protocol library to create requests and responses
  2. Use the swaggersocket.js library
  3. Use the SwaggerSocket Scala Library, which build on top of AHC (AsyncHttpClient) WebSocket

I strongly recommends the use of 2 and 3, but would like to see contribution for 1 :-). Our goal is to integrate SwaggerSocket with Swagger and ships other client language as well.

With Javascript, all you need to do is:

        $(document).ready(function() {
  new jQuery.swaggersocket.SwaggerSocket()
     .open(document.location.toString() + "ss",
          function(swaggerSocket, r) {
              if (r.getStatus() == 200) {
              var ss = new jQuery.swaggersocket.SwaggerSocketListener();
              ss.onResponse = function(r) {
               // Write the response
              };

              var request = new jQuery.swaggersocket.Request()
                       .method("GET")
                       .path('/helloworld')
                        listener(ss);

               swaggerSocket.send(request);
          } 
   ....

All you need to do is to create a SwaggerSocket, invoke the open method and pass a function that will be invoked when the initial handshake as succeeded. Then you can call swaggerSocket.sendto send a single or an array of requests. You can do the same using the SwaggerSocket Scala Library:

  val ss = SwaggerSocket().open("http://127.0.0.1:8080/helloworld")
  ss.send(new Request.Builder()
   .path("/sayHello")
   .build(), new SwaggerSocketListener() {
  	override def message(r: Request, s: Response) {
        // print the response using response.getData()
     }
   })

The fun just begin! We will soon add support for other transport like Server Side Events, Long-Polling, HTTP Streaming and JSONP so you can deploy your SwaggerSocket application in your current production infrastructure which most probably isn’t supporting WebSocket yet. But you don’t have to wait for WebSocket, Atmosphere always use the best available one and the framework makes sure to switch to WebSockets when available.

I strongly encourage you to take a look at our demos, specially the Twitter one that bring real time search to Twitter, as fast as what Google Search recently started to support.

For any questions or to download SwaggerSocket, go to our main site, use our Google Group forumfollow the team or myself and tweet your questions there!

Writing WebSocket Clients using AsyncHttpClient

December 21, 2011 3 comments

The AsyncHttpClient version newly released 1.7.0 now supports WebSocket. Both Netty and Grizzly provider supports the latest version of the specification.

Like HTTP support with AHC, WebSoket is quite simple. Starting with AHC 1.7, a new interface called UpgradeHandler is available to the client so any kind of protocol can be used. You can event replace the actual WebSocket implementation with your in case you don’t like mine :-)

public interface UpgradeHandler<T> {

    /**
     * If the HTTP Upgrade succeed (response's status code equals 101), 
     * the {@link AsyncHttpProvider} will invoke that
     * method
     *
     * @param t an Upgradable entity
     */
    void onSuccess(T t);

    /**
     * If the upgrade fail.
     * @param t a {@link Throwable}
     */
    void onFailure(Throwable t);

For WebSocket, I wrote a simple one called WebSocketUpgradeHandler which extends the usual AsyncHandler (always required with AHC) and UpgradeHandler. The interesting part is the WebSocketUpgradeHandler.Builder, which allow you to add listeners and WebSocket properties:

        /**
         * Add a {@link WebSocketListener} that
         * will be added to the {@link WebSocket}
         *
         * @param listener a {@link WebSocketListener}
         * @return this
         */
        public Builder addWebSocketListener(WebSocketListener listener) {
            l.add(listener);
            return this;
        }

        /**
         * Remove a {@link WebSocketListener}
         *
         * @param listener a {@link WebSocketListener}
         * @return this
         */
        public Builder removeWebSocketListener(WebSocketListener listener) {
            l.remove(listener);
            return this;
        }

        /**
         * Set the WebSocket protocol.
         *
         * @param protocol the WebSocket protocol.
         * @return this
         */
        public Builder setProtocol(String protocol) {
            this.protocol = protocol;
            return this;
        }

        /**
         * Set the max size of the WebSocket byte message that will be sent.
         *
         * @param maxByteSize max size of the WebSocket byte message
         * @return this
         */
        public Builder setMaxByteSize(long maxByteSize) {
            this.maxByteSize = maxByteSize;
            return this;
        }

        /**
         * Set the max size of the WebSocket text message that will be sent.
         *
         * @param maxTextSize max size of the WebSocket byte message
         * @return this
         */
        public Builder setMaxTextSize(long maxTextSize) {
            this.maxTextSize = maxTextSize;
            return this;
        }

        /**
         * Build a {@link WebSocketUpgradeHandler}
         * @return a {@link WebSocketUpgradeHandler}
         */
        public WebSocketUpgradeHandler build() {
            return new WebSocketUpgradeHandler(this);
        }

You can add several type of listeners like WebSocketTextListener, WebSocketByteListener, etc. To create a WebSocket, you just need to do:

        AsyncHttpClient c = new AsyncHttpClient(); 
           // or new AsyncHttpClient(new GrizzlyAsyncHttpprovider(config))

        WebSocket websocket = c.prepareGet("ws://something:port)
                .execute(
                  new WebSocketUpgradeHandler.Builder().addWebSocketListener(
                     new WebSocketByteListener() {

                    @Override
                    public void onOpen(WebSocket websocket) {
                    }

                    @Override
                    public void onClose(WebSocket websocket) {
                    }

                    @Override
                    public void onError(Throwable t) {
                    }

                    @Override
                    public void onMessage(byte[] message) {
                    }

                    @Override
                    public void onFragment(byte[] fragment, boolean last) {
                    }
                }).build()).get();

The AHC Future returned is a WebSocket, and it is returned as soon as the handshake is successful. The WebSocket API is also simple:

public interface WebSocket {

    /**
     * Sen a byte message.
     * @param message a byte message
     * @return this
     */
    WebSocket sendMessage(byte[] message);

    /**
     * Send a text message
     * @param message a text message
     * @return this.
     */
    WebSocket sendTextMessage(String message);

    /**
     * Add a {@link WebSocketListener}
     * @param l a {@link WebSocketListener}
     * @return this
     */
    WebSocket addMessageListener(WebSocketListener l);

    /**
     * Close the WebSocket.
     */
    void close();
}

Conclusion, writing WebSocket is super simple!!! Here is a fully asynchronous echo client

        WebSocket websocket = c.prepareGet("ws://localhost:80")
                .execute(
                  new WebSocketUpgradeHandler.Builder()
                    .addWebSocketListener(new WebSocketTextListener() {

                    @Override
                    public void onMessage(String message) {
                        System.out.println(message);
                    }

                    @Override
                    public void onFragment(String fragment, boolean last) {
                    }

                    @Override
                    public void onOpen(WebSocket websocket) {
                        websocket
                          .sendTextMessage("ECHO")
                          .sendTextMessage("ECHO");
                    }

                    @Override
                    public void onClose(WebSocket websocket) {
                    }

                    @Override
                    public void onError(Throwable t) {
                        t.printStackTrace();

                    }
                }).build()).get();

For any questions you can use our Google Group, irc.freenode.net #asynchttpclient or use Twitter to reach me! You can checkout the code on Github as well!

Writing powerful REST Client using the AsyncHttpClient Library and Jersey

The Jersey Client Framework is a powerful library that easily allow the writing of client based REST client.  Recently I’ve implemented a new transport for the framework based on the AsyncHttpClient library. You can now get the best of both worlds: an easy REST Client API powered by a poweful http client library.

Currently the Jersey Client Framework ships with two transport, one based on the infamous URLConnection, and one based on Apache Http Client.  Recently I needed to replace Wink with Jersey in one of my Sonatype project (guess one?) and  at the same time realized the Jersey Client features was quite limited in terms of what you can do and what is exposed to the client. I understand the original design was to abstract the underlying transport, but unfortunately all http library aren’t supporting the same set of features. Now if you are familiar with the Jersey client, creating a resource is as simple as (see here for the complete tutorial):

    Client c = Client.create(); 

Now if you want to customize the client a little, you can do (complete info here):

    ClientConfig cc = new DefaultClientConfig();
    cc.getProperties().put(
        ClientConfig.PROPERTY_FOLLOW_REDIRECTS, true);
    Client c = Client.create(cc); 

Next step is as simple as:

     WebResource r = c.resource(“http://localhost:8080/xyz”);
     ClientResponse response = r.get(ClientResponse.class);
     EntityTag e = response.getEntityTag();
     String entity = response.getEntity(String.class);

Very simple. Now in order to take advantage of AHC (AsyncHttpClient) features, all you need to do is to first get the library from here, and then just do get a reference to the AsyncHttpClient’s Config class (if you aren’t familiar with AHC, take a look at this introduction)

     DefaultAhcConfig config = new DefaultAhcConfig();
     config.getAsyncHttpClientConfigBuilder().setRealm(
           new Realm.RealmBuilder()
               .setScheme(Realm.AuthScheme.SPNEGO)
               .setUsePreemptiveAuth(false)
               .build());
     Client c = Client.create(config);
     WebResource r = c.resource(getUri().build());

You just added Kerberos support to Jersey Client! The trick here consists of getting a reference to the AsyncHttpClientConfig and configure all the cool stuff offered by AHC. As an example, you can add resumable support, Request/Response or IOException Filter by just doing:

     config.getAsyncHttpClientConfigBuilder()
        .addIOExceptionFilter(new ResumableIOExceptionFilter());

For any questions, use Twitter to reach me! You can checkout the code on Github, download the jars from Maven Central or use Maven:

<dependency>
    <groupId>org.sonatype.spice</groupId>
    <artifactId>jersey-ahc-client</artifactId>
    <version>1.0.0</version>
</dependency>

Categories: Async Http client

AsyncHttpClient 1.5.0 Released

January 31, 2011 2 comments

We are pleased to announce the immediate availability of the Asynchronous Http Client (AHC) library version 1.5.0.

This version includes:

For any questions you can use our Google Group, irc.freenode.net #asynchttpclient or use Twitter to reach me! You can checkout the code on Github, download the jars from Maven Central or use Maven:

<dependency>
    <groupId>com.ning</groupId>
    <artifactId>async-http-client</artifactId>
    <version>1.5.0</version>
</dependency>

Categories: Async Http client

Going Asynchronous using AsyncHttpClient: For Dummies

January 12, 2011 10 comments

Concluding on my “Going Asynchronous using AsyncHttpClient (Basic and Complex)” blog series, this week I will explains how to use the new SimpleAsynHttpClient API to write powerful async client applications.

The SimpleAsynchHttpClient is an implementation on top of the AsyncHttpClient and its builders class like AsyncHttpClientConfigBuilder, RealmBuilder etc. With SimpleAsynchHttpClient, all you need to do is to create a single builder, and configure your request from that builder:

 SimpleAsyncHttpClient client = new SimpleAsyncHttpClient.Builder()
                .setUrl("http://..")
                .setProxyHost("http://...")
                .addHeader("foo","bar")
                .setRealmPrincipal("me")
                .setRealmPassword("pwd")
                .build();

Once you have configured your SimpleAsyncHttpClient, you can invoke your HTTP method:

 client.get();
   or
 client.head();

Now the biggest difference with the AsyncHttpClient API is the support for BodyConsumer and BodyGenerator. The BodyGenerator API allows an application to customize how bytes are sent. This is quite useful when you need to stream large fie and you don’t want to load it in memory. As an example, the library contains a FileBodyGenerator defined as

public class FileBodyGenerator implements BodyGenerator {

    private final File file;

    public FileBodyGenerator(File file) {
        this.file = file;
    }

    public RandomAccessBody createBody() throws IOException {
        return new FileBody(file);
    }

    protected static class FileBody
            implements RandomAccessBody {

        private final RandomAccessFile file;

        private final FileChannel channel;

        private final long length;

        public FileBody(File file)
                throws IOException {
            this.file = new RandomAccessFile(file, "r");
            channel = this.file.getChannel();
            length = this.file.length();
        }

        public long getContentLength() {
            return length;
        }

        public long read(ByteBuffer buffer)
                throws IOException {
            return channel.read(buffer);
        }

        public long transferTo(long position,
                               long count, WritableByteChannel target)
                throws IOException {
            return channel.transferTo(position, count, target);
        }

        public void close()
                throws IOException {
            file.close();
        }
    }
}

The library ships with InpuStreamBodyGenerator, ByteArrayBodyGenerator and the one above. Those BodyGenerator never buffers bytes in memory, reducing the probability to fill the heap and get an OOM. For the response, the SimpleAsyncHttpClient support a new API called BodyConsumer, which is simply defined as

public interface BodyConsumer {

    void consume(ByteBuffer byteBuffer) throws IOException;

    void close() throws IOException;

}

Simple callback that are invoked as soon as the response’s bytes are available. As an example, the library ships with

public class AppendableBodyConsumer implements BodyConsumer {

    private final Appendable appendable;
    private final String encoding;

    public AppendableBodyConsumer(Appendable appendable, String encoding) {
        this.appendable = appendable;
        this.encoding = encoding;
    }

    public AppendableBodyConsumer(Appendable appendable) {
        this.appendable = appendable;
        this.encoding = "UTF-8";
    }

    public void consume(ByteBuffer byteBuffer) throws IOException {
        appendable.append(new String(byteBuffer.array(), encoding));
    }

    public void close() throws IOException {
        if (Closeable.class.isAssignableFrom(appendable.getClass())) {
            Closeable.class.cast(appendable).close();
        }
    }
}

The library ships with ByteBufferBodyConsumer, FileBodyConsumer and OutputStreamBodyConsumer. Hence normally you shouldn’t have to write BodyConsumer and BodyGenerator…just use the one available! Simply do:

 SimpleAsyncHttpClient client = new SimpleAsyncHttpClient.Builder()
                .setIdleConnectionInPoolTimeoutInMs(100)
                .setUrl("http://.....)
                .setHeader("Content-Type", "text/html")
                .build();

  Future future = client.post(
                new InputStreamBodyGenerator(myInputStream),
                new OutputStreamBodyConsumer(myOutputStream));

  Response response = future.get();

Finally, if you need to re-use your instance of SimpleAsyncHttpClient, you an use the Request API directly like when you work with AsyncHttpClient:

 SimpleAsyncHttpClient client = new SimpleAsyncHttpClient.Builder()
                .setIdleConnectionInPoolTimeoutInMs(100)
                .setUrl("http://.....)
                .setHeader("Content-Type", "text/html")
                .build();

  Future future = client.post(
                new InputStreamBodyGenerator(myInputStream),
                new OutputStreamBodyConsumer(myOutputStream));

  Response response = future.get();

  client.get(new RequestBuilder().setUrl("http://...").build(),
             new OutputStreamBodyConsumer(myOutputStream));
  response = future.get();  

The SimpleAsyncHttpClient will be included in 1.5.0, which is targeted to ship weeks of January 18. For any questions you can use our Google Group, irc.freenode.net #asynchttpclient or use Twitter to reach me! You can checkout the code on Github, download the jars from Maven Central or use Maven:

<dependency>
    <groupId>com.ning</groupId>
    <artifactId>async-http-client</artifactId>
    <version>1.5.0-SNAPSHOT</version>
</dependency>

Categories: Async Http client

Going Asynchronous using AsyncHttpClient: The Complex

The Async Http Client library purpose is to allow Java applications to easily execute HTTP requests and asynchronously process the HTTP responses. In this second part on the topic, I will describe more complex operations that can be done with the AsyncHttpClient like resumable download, zero in memory bytes copy, oAuth calculation, optimal transfer listener and performance tricks.

Resumable Donwload

The AsyncHttpClient supports resumable download in two differents scenarios:

  • IOException: If an IOException occurs (for whatever reason), you can configure the library to restart the download automatically without having to restart the download from the beginning.
  • JVM crashes: If your application or the JVM goes down during a file download, the library can also restart the download automatically when the same download is requested.

You can configure the AsyncHttpClient Library to survive IOException using the IOException Filter:

  AsyncHttpClient c = new AsyncHttpClient(
          new AsyncHttpClientConfig.Builder()
            .addIOExceptionFilter(
               new ResumableIOExceptionFilter()).build());

  ResumableAsyncHandler a =
      new ResumableAsyncHandler(
         new ResumableRandomAccessFileListener());

  a.setResumableListener(
       new ResumableRandomAccessFileListener(
           new RandomAccessFile( "file.avi", "rw" ) ) );

 Response r = c.prepareGet("http://host:port/file.avi")
       .execute(a).get();

If you need something more high level and configurable, you can use a ResumableAsyncHandler, and or implement a ResumableProcessor:

 AsyncHttpClient c = new AsyncHttpClient();
 ResumableAsyncHandler a =
     new ResumableAsyncHandler( new PropertiesBasedResumableProcessor() );
 a.setResumableListener(
     new ResumableRandomAccessFileListener(
        new RandomAccessFile( "file.avi", "rw" ) ) );

 Response r = c.prepareGet( "http://localhost:8081/file.AVI" )
        .execute( a ).get();

You can also simply use a ResumableListener (or use the ResumableRandomAccessFileListener, which does what’s described below):

public interface ResumableListener {

    public void onBytesReceived(ByteBuffer byteBuffer) throws IOException;

    public void onAllBytesReceived();

    public long length();
}

As simple as:

 AsyncHttpClient c = new AsyncHttpClient();
 final RandomAccessFile file = new RandomAccessFile( "file.avi", "rw" );        
 ResumableAsyncHandler a = new ResumableAsyncHandler();
 a.setResumableListener( new ResumableListener() {

    public void onBytesReceived(ByteBuffer byteBuffer) throws IOException {
       file.seek( file.length() );
       file.write( byteBuffer.array() );
    }

    public void onAllBytesReceived() {
       file.close();
    }

    public long length() {
       return file.length();
    }
} );
Response r = c.prepareGet( "http://localhost:8081/file.AVI" )
        .execute( a ).get();

Make it simple: TransferListener

In some scenario an application may need to manipulate the received bytes in more than one place, e.g. saves the bytes on disk but also accumulate it for checksum checking later. In that case, instead of using an AsyncHandler and mixes logic inside an AsyncHandler.onBodyPartReceived, it is recommended to use the TransferListener simple API:

public interface TransferListener {

    public void onRequestHeadersSent
       (FluentCaseInsensitiveStringsMap headers);

    public void onResponseHeadersReceived
        (FluentCaseInsensitiveStringsMap headers);

    public void onBytesReceived(ByteBuffer buffer)
         throws IOException;

    public void onBytesSent(ByteBuffer buffer);

    public void onRequestResponseCompleted();

    public void onThrowable(Throwable t);
}

All you need to do in that case is to create a TransferCompletionHandler and add as many TransferListener as you need:

 AsyncHttpClient client = new AsyncHttpClient();
 TransferCompletionHandler tl = new TransferCompletionHandler();
 tl.addTransferListener(new TransferListener(){...});

 Response response = httpClient.prepareGet("http://...").execute(tl).get();

Zero Bytes Copy

When uploading or downloading bytes, it is important to try to avoid buffering bytes in memory.

Upload

On the upload side, the mechanism is enabled by default when setting the Request’s body to a File:

 AsyncHttpClient client = new AsyncHttpClient();

 File file = new File("file.avi");
 Future f = client.preparePut("http://localhost").setBody(file).execute();

If you can’t use a File, the recommended way (as described in part I) is to use a BodyGenerator. It is strongly recommended to avoid using InputStream as the library will unfortunately buffer the entire content in memory in order to set the content-lenght, which can cause out of memory error.

Download

On the download side, you can use the HttpResponseBodyPart.writeTo to avoid loading bytes in memory and unnecessary copy:

 AsyncHttpClient client = new AsyncHttpClient();

 File tmp = new File("zeroCopy.txt");
 final FileOutputStream stream = new FileOutputStream(tmp);
 Future f = client.prepareGet("http://localhost/largefile.avi")
       .execute(new AsyncHandler() {
     public void onThrowable(Throwable t) {
     }

     public STATE onBodyPartReceived(HttpResponseBodyPart bodyPart)
         throws Exception {
        bodyPart.writeTo(stream);
        return STATE.CONTINUE;
     }

     { .... }
});
Response resp = f.get();

Limiting the number of connections to improve raw performance

By default the library uses a connection pool and re-use connections as needed. It is important to not let the connection pool grow too large as it takes resources in memory. One way consist of setting the maximum number of connection per host or in total:

 AsyncHttpClientConfig config = new AsyncHttpClientConfig.Builder()
     .setMaximumConnectionsPerHost(10)
     .setMaximumConnectionsTotal(100)
     .build();
 AsyncHttpClient c = new AsyncHttpClient(config);

There is no magic number, so you will need to try it and decide which one gives the best result.

Using OAuth

You can use the library to pull data from any OAuth site (like Twitter). This is as simple as:

 private static final String CONSUMER_KEY = "dpf43f3p2l4k3l03";
 private static final String CONSUMER_SECRET = "kd94hf93k423kf44";
 public static final String TOKEN_KEY = "nnch734d00sl2jdk";
 public static final String TOKEN_SECRET = "pfkkdhi9sl3r4s00";
 public static final String NONCE = "kllo9940pd9333jh";
 final static long TIMESTAMP = 1191242096;

 public void oAuth() {
   ConsumerKey consumer =
     new ConsumerKey(CONSUMER_KEY, CONSUMER_SECRET);
   RequestToken user =
     new RequestToken(TOKEN_KEY, TOKEN_SECRET);
   OAuthSignatureCalculator calc =
      new OAuthSignatureCalculator(consumer, user);
   AsyncHttpClient client = new AsyncHttpClient();

   Response response = client.prepareGet("http://...")
      .setSignatureCalculator(calc).execute().get();

What’s Next

In the next part of this series I will explain more complex operations that can be done with the AsyncHttpClient library like:

  • Supporting the WebSocket protocol.
  • Concurrent use of AsyncHandler

For any questions you can use our Google Group, irc.freenode.net #asynchttpclient or use Twitter to reach me! You can checkout the code on Github, download the jars from Maven Central or use Maven:

<dependency>
    <groupId>com.ning</groupId>
    <artifactId>async-http-client</artifactId>
    <version>1.4.1</version>
</dependency>

Categories: Async Http client
Follow

Get every new post delivered to your Inbox.

Join 51 other followers