Chloe and the Realtime Web

Chloe and the RTW
Trotter Cashion
Strange Loop 2011

For those
looking only
at slides!

Go Get It!
http://github.com/mashion/chloe

We can Haz?
http://utterinsanity.files.wordpress.com/2008/09/funny-pictures-cat-drives-an-invisible-racecar.jpg%3Fw%3D497%26h%3D318

Chloe
A realtime web-server... that doesn’t suck.

Cool History, Right?

Usage...
A Simple Chat Service

<html>
  <head>

</script>
  </head> Include Chloe JS
  <body>
    <form>
    </form>
  </body>
</html>

<html>
  <head>

</script>
  </head>
  <body>
    <form> Include Your JS
    </form>
  </body>
</html>

$(function () {
  var form = $('form'),
      container = $('#messages');
      chloe = new Chloe({host: 'localhost', port: 8901});

  chloe.connect(function () {
   form.submit(function () {
      chloe.send(form.serialize());
      $('#message').val('');
      return false;
    });

    chloe.onmessage(function (message) {
      container.prepend("<li>" + message + "</li>");
    });
  });

});

$(function () {

   form.submit(function Instantiate
() { Chloe
      return false;
    });

    });
  });

});

$(function () {

      $('#message').val(''); Server
Connect to the
      return false;
    });

    });
  });

});

$(function () {

      return false;
    }); Send data to the server
    });
  });

});

$(function () {

      return false; this function when
Run a message arrives
    });

    });
  });

});

get '/' do
  erubis :index
end

post '/updates' do
  data = URI.decode_www_form(request.body.read)
params = Hash[data]
uri = URI.parse("http://localhost:8901/send")
  Net::HTTP.post_form(uri,
                {"data" => params["message"]})
  'ok'
end

get '/' do
  erubis :index
end

post '/updates' html from earlier
Get the do
params = Hash[data]
  'ok'
end

get '/' do
  erubis :index
end
post '/updates' do
params = Hash[data]
  'ok'
end

get '/' do
  erubis :index
end

post '/updates' do Ruby trick!
params = Hash[data]
  'ok'
end

get '/' do
  erubis :index
end

post '/updates' do
params = Hash[data]
Chloe broadcast/multicast url
  'ok'
end

get '/' do
  erubis :index
end

post '/updates' do
params = Hash[data] data to Chloe
Send the
  'ok'
end

Chrome Server *

*could be Java, Ruby, Scala, Clojure, Node...

Chrome Server * Chloe


GET /


GET /
index.html


GET /
index.html
/chloe.js


GET /
index.html
/chloe.js
“Hey Everyone”


GET /
index.html
/chloe.js
“Hey Everyone”
POST /updates “Hey Everyone”


GET /
index.html
/chloe.js
“Hey Everyone”
POST /send “Hey Everyone”


GET /
index.html
/chloe.js
“Hey Everyone”
POST /send “Hey Everyone”
“Hey Everyone”


Fallbacks

Websockets
Long polling (jsonp)
Cross domain xhr - SOON!
This seems to be enough for now...

  chloe.subscribe('chat-room-5', function (message) {
    $("#messages").prepend("<li>" + message + "</li>")
  });
});

post '/updates' do
       { "data" => "Hello room!",
"channel" => "chat-room-5"})
  "ok"
end

Channel Name
  });
});

post '/updates' do
  "ok"
end

  });
});
Called when message arrives

post '/updates' do
  "ok"
end

  });
});

post '/updates' do
  "ok"
end
Channel for broadcast

What’s left for Channels

Bi-directionality
Per Channel callbacks

Performance
Difficult to determine...

50
50

40

33.5
Memory (MB)

30

20
17

10

0
0 500 1000
Sessions

Tune Your TCP Stack
# ulimit -n 65536
# ifconfig eth0 txqueuelen 8192
# /sbin/sysctl -w net.core.somaxconn=4096
# /sbin/sysctl -w net.core.netdev_max_backlog=16384
# /sbin/sysctl -w net.core.rmem_max=16777216
# /sbin/sysctl -w net.core.wmem_max=16777216
# /sbin/sysctl -w net.ipv4.tcp_max_syn_backlog=8192
# /sbin/sysctl -w net.ipv4.tcp_syncookies=1

From cometd: http://cometd.org/documentation/2.x/howtos/loadtesting

Tune Your TCP Stack
# ulimit -n 65536
# Increase number of file descriptors
/sbin/sysctl -w net.core.netdev_max_backlog=16384


Tune Your TCP Stack
# ulimit -n 65536
# Increase transaction queue length
/sbin/sysctl -w net.core.rmem_max=16777216


Tune Your TCP Stack
# ulimit -n 65536
# Increase size of listen queue
/sbin/sysctl -w net.core.wmem_max=16777216


Tune Your TCP Stack
# ulimit -n 65536
# Increase incoming packet queue
/sbin/sysctl -w net.ipv4.tcp_max_syn_backlog=8192


Tune Your TCP Stack
# ulimit -n 65536
# /sbin/sysctl Increase maximum receive window
-w net.ipv4.tcp_syncookies=1


Tune Your TCP Stack
# ulimit -n 65536
Increase send window


Tune Your TCP Stack
# ulimit -n 65536

How many SYN requests to keep in memory


Tune Your TCP Stack
# ulimit -n 65536

Makes tcp_max_syn_backlog work


Binaries Available

Mac, Ubuntu 32bit, Ubuntu 64bit
Fully self-contained (Erlang included)
.deb and .rpm will be coming soon...

It’s Easy!
# wget http://bit.ly/oqyQfL
# tar xzvf chloe-0.0.3-osx.tgz
# cd chloe-0.0.3
# ./bin/chloe start
# ./bin/chloe stop

./etc/app.config

{chloe,
[
{application_server, "http://localhost:4567"},
{application_server_url, "http://localhost:4567/updates"},
{port, 8901},
{doc_root, "./public"},
{log_dir, "/var/log/chloe"},
{secret, “SEKRET PASSFRASE”}
]}

./etc/app.config

{chloe, For XHR
[
{port, 8901},
]}

./etc/app.config

{chloe,
[
Callback url on your application
{port, 8901},
]}

./etc/app.config

{chloe,
[
Chloe port
{port, 8901},
]}

./etc/app.config

{chloe,
[
Where Chloe’s JavaScript Lives
{port, 8901},
]}

./etc/app.config

{chloe,
[
Directory for Logs
{port, 8901},
]}

./etc/app.config

{chloe,
[
{port, 8901},
]}

For security (not required)...

Signing Requests
SECRET = "SEKRET PASSFRASE”

post '/updates' do
message = Hash[data]["message"]
sig = Digest::MD5.hexdigest(message + SECRET)
             {"data" => message, "sig" => sig})
  'ok'
end

Signing Requests

post '/updates' do
From ./etc/app.config
  'ok'
end

Signing Requests

post '/updates' do
Concat t wo strings
  'ok'
end

Signing Requests

post '/updates' do
Hash them
  'ok'
end

Signing Requests

post '/updates' do
  'ok'
end
Add signature to request

If `secret` is in your
config, all requests to
`/send` must be signed

Coming Soon
Improved Performance
Bi-directional channels
Per-channel callback urls
Runtime configuration
SSL Support

Coming Later

Improved monitoring
Explore switch to mochiweb internally
Client side authentication

Internals
Erlang Nerdery Time!

Session Manager
Transports
Yaws
Callback
Sessions

Channel Store

Session
Yaws Websocket
Manager Session

Session
Yaws Websocket
Manager Session
Browser

Session
Yaws Websocket
Manager Session
Browser
New Conn

Session
Yaws Websocket
Manager Session
Browser
New Conn
Session?

Session
Yaws Websocket
Manager Session
Browser
New Conn
Session?
Session Pid

Session
Yaws Websocket
Manager Session
Browser
New Conn
Session?
Session Pid

Message!

Session
Yaws Websocket
Manager Session
Browser
New Conn
Session?
Session Pid

Message!
Server

Channel
Yaws Store Session Websocket

Channel

Server

Channel

Server
Sessions?

Channel

Server
Sessions?
Session Pid

Channel

Server
Sessions?
Session Pid
Message!

Channel

Server
Sessions?
Session Pid
Message!
Message!

Channel

Server
Sessions?
Session Pid
Message!
Message!
Browser

Message Structure
{ data,
version,
type,
channel,
id,
sessionId
}

Message Structure
{ data,
version,
type,The message itself
channel,
id,
sessionId
}

Message Structure
{ data,
version,
type,
Version of the message protocol
channel,
id,
sessionId
}

Message Structure
{ data,
version,
type,
channel,
Four types: connect, message,
id, channel-subscribe, poll
sessionId
}

Message Structure
{ data,
version,
type,
channel,
id,
sessionId the message is going
Where
}

Message Structure
{ data,
version,
type,
channel,
id,
sessionId
} No Idea...

Message Structure
{ data,
version,
type,
channel,
id,
sessionId
}
Session id for connection

Please Go Play With It

http://mashion.net
http://trottercashion.com
@cashion

Moved to Mountain View
Be my friend?

Thank You!
Good luck w/ Chloe

Chloe and the Realtime Web

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Chloe and the Realtime Web (20)

Recently uploaded (20)

Chloe and the Realtime Web