Configuring HAProxy for WebSocket
A lot of peoples (including myself at Wordnik) needed to configure HAProxy in order to make WebSocket working. For my Atmosphere Framework project, I’m using:
$ cat /etc/haproxy/haproxy.cfg
global
maxconn 4096 # Total Max Connections. This is dependent on ulimit
nbproc 1
defaults
mode http
frontend all 0.0.0.0:80
timeout client 86400000
default_backend www_backend
acl is_websocket hdr(Upgrade) -i WebSocket
acl is_websocket hdr_beg(Host) -i ws
use_backend socket_backend if is_websocket
backend www_backend
balance roundrobin
option forwardfor # This sets X-Forwarded-For
timeout server 30000
timeout connect 4000
server apiserver 127.0.0.1:8080 weight 1 maxconn 1024 check
backend socket_backend
balance roundrobin
option forwardfor # This sets X-Forwarded-For
timeout queue 5000
timeout server 86400000
timeout connect 86400000
server apiserver targetserver:7777 weight 1 maxconn 1024 check
Thanks to Matthias L. Jugel for sharing … see his use of Atmosphere at twimpact.com.
For any questions or to download Atmosphere Client and Server Framework, go to our main site, use our Google Group forum, follow the team or myself and tweet your questions there! You can also checkout the code on Github.
Categories: Atmosphere, Comet, Websocket
4096 connections is not many; what might a typical upper limit be?
Just as a reference for anyone else who finds this as a way to get Atmosphere/Nettosphere working behind a HAproxy load balancer, this HAproxy config only works with HAproxy versions before 1.5. I tried to get this working with HAproxy 1.5 (dev17) to no avail. Nettosphere by itself works great, but as soon as you turn on HAproxy (pointing at the Nettosphere instance as a “backend”, Nettosphere starts spewing a continuous stream of exceptions like this:
18:41:44.029 DEBUG (New I/O worker #1) [NettyAtmosphereHandler:403] Exception
java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.SocketDispatcher.write0(Native Method) ~[na:1.6.0_30]
at sun.nio.ch.SocketDispatcher.write(Unknown Source) ~[na:1.6.0_30]
at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source) ~[na:1.6.0_30]
at sun.nio.ch.IOUtil.write(Unknown Source) ~[na:1.6.0_30]
at sun.nio.ch.SocketChannelImpl.write(Unknown Source) ~[na:1.6.0_30]
at org.jboss.netty.channel.socket.nio.SocketSendBufferPool$UnpooledSendBuffer.transferTo(SocketSendBufferPool.java:205) ~[netty-3.5.5.Final.jar:na]
…..
But as soon as downgrade HAproxy to latest version of 1.3 (haven’t tried 1.4 yet), all is fine. No more exceptions. Nettosphere works as expected behind HAproxy. So something that was changed in HAproxy 1.5 is causing this.
Anyways, I hope this helps somebody else who also finds this as an example of getting Nettosphere working behind HAproxy. Stick to HAproxy 1.3 and you’ll be fine.