Jul 8, 2013

Server Sent Events with Aura and Node.js

You can view the demo app or the source on Github.

Recently I uncovered a relatively fresh and unique HTML5 feature; Server Sent Events.

Note: I will digress from the onset that if you are familiar with Websockets, then you should understand that they are the superior implementation for scalable bidirectional push communication, and not a competiting technology for SSE.

SSE on the other hand can be viewed as more of a "lightweight" alternative, for when using websockets is not a viable option, most likely due to server limitations.

To form a base understanding of SSE, here are the main design characteristics:

Everything happens over the HTTP, no special protocol required
Keeps open a HTTP/S connection to the server
Communication is unidirectional (read only by the client)
Different event sources/channels can be fed down the same connection
Auto reconnection and event IDs come for free as part of the implementation.i The auto-reconnection is ideal for flaky connections, like mobile devices.

There are some pros and cons of using SSE over websockets. Let's go over the cons first:

Cons:

The data source is read only. While this lends itself to actually being more scalable for RO type applications, for RW type app/pages you'll have to revert to POST'ing or long polling type server writes.
Depending on the server configuration, the connection may be timed out frequently or free'd up for other users accessing the server, but mitigated by the auto-reconnection as per the EventSource spec.
Each SSE source maintains an open connection to the server. This takes away server side resources from other users, such as memory, CPU, NOFILE and lower max_connections type limits of the HTTP server (as opposed to available kernel connections/ports).

Pros:

With that said SSE does hold advantages over Websockets in some scenarios:

Doesn't require any special protocol to be supported in the browser or server
SSE support can be polyfilled (backported) for older browsers that don't yet implement it.
Integrated Cross-site/origin request support (via CORS)
SSE can ingeniously benefit from HTTPS with no extra configuration, unlike websockets/WSS, which require certificate setup and implementation on top of TLS.

Enough, shut up and show me the glory of SSE!

OK then, let us wield this technology for the betterment of the internets! If you want to dive straight into the demo app then go here: https://github.com/jdpaton/sse-dash

As prescribed, we'll look at implementing this in Node.js. Here's the first excerpt you should get familiar with:

module.exports.writeSSEHead = function (req, res, cb) {
    res.writeHead(200, {
        "Content-Type": "text/event-stream",
        "Cache-Control": "no-cache",
        "Connection": "keep-alive"
    });
    res.write("retry: 10000\n");
    return cb(req, res);
}

Nothing so unusual, but the first part of the SSE spec is here, specifically the retry line, telling the client, that in the event a disconnect, it should retry after ten seconds. Note the header is text/event-stream so the browser knows it's SSE, we tell the browser not to cache any of the responses (you don't want stale real time data do you? ) and we make sure the connections is kept open by the browser/client via Connection: keep-alive. Straight forward stuff.

Next up is writing a custom event with data:

module.exports.writeSSEData = function (req, res, event, data, cb) {
    var id = (new Date()).toLocaleTimeString();
    res.write("id: " + id + '\n');
    res.write("event: " + event + "\n");
    res.write("data: " + JSON.stringify(data) + "\n\n");
    if (cb) {
        return cb(req, res);
    }
};

The first part here is making sure we send a unique ID with each request, we just use a simple date based one here. The ID is useful so that if a client reconnects later, it will send the Last-Event-ID header in the response, so the server can decide on a delta of which to resume push messages (not implemented in the demo app though).

Note in Apache, Nginx or as here for Node.js you should consider upping the timeout for SSE requests. This is easily set in Node 0.10.x, and for per-request instead of a global socket setting you can do the following (20 min idle):

res.connection.setTimeout(1000 * 60 * 20)

Using these functions in an express.js app to echo the servers date every second might look like the following:

app.get('/stream/date', function(req, res) {

    var event = 'date';

    sse.writeSSEHead(req, res, function(req, res) {
      sse.writeSSEData(req, res, event, new Date(), function(req, res) {

        intervalx = setInterval(function() {
          sse.writeSSEData(req, res, event, new Date());
        }, 1000);

        req.connection.addListener("close", function() {
            clearInterval(intervalx);
        });
      });
    });
});

We then use a new EventSource ( MDN ) native browser object to connect to this endpoint ( full code: here ):

function connect() {
        source = new EventSource("stream/date");

        source.addEventListener("date", function(event) {
          self.updateDate(event.data);
        }, false);

        source.addEventListener("open", function(event) {
          button.text("Disconnect");
          button.click( function(event) {
            source.close();
            button.text("Connect");
            button.click(connect);
            status.text("Connection closed!");
          });
          button_cont.removeClass("warning");
          status.text("Connection open!");
        }, false);

 [...]

The EventSource parameter is the endpoint resource relative the page path, or a full URI. Note that SSE supports cross-domain requests out of the box, via CORS. The date event listener is listening for lines from the server like:

event: data
data: '"2013-07-07T07:04:55.162Z"'

and on each update we simply update our HTML element with the new element - simples!

Note the client code examples here are all using Aura, a pretty cool project mentored by @addyosmani. We can write a Aura widget per SSE endpoint, and use it anywhere on the website without having to do anything - just add a <div>! :

<div data-aura-widget="sse-date"></div>

"Aura is an event-driven architecture for wrapping your code into reusable widgets and extensions that can easily communicate with each other" - aurajs.com

I won't cover Aura in depth here, but the code in the demo app and on material on their site should be enough to get you started. In pursuit of this project you could combine widgets to create customizable dashboards whose components can talk to each other with realtime SSE data, so lots of avenues to explore! I'd highly recommending using Aura for projects like this, so check it out!

Jun 30, 2013

Disabling SSLv2 with Node.js

It recently came to light that SSLv2 support in the TLS/HTTPS modules in node.js is enabled by default if supported by the system OpenSSL library. Now, it may or may not be a suprise to some that SSLv2 is sufficiently broken and has in fact has been deprecated (since 1996) that it warrants disabling in any serious production environment.

As of OpenSSL v1.0+ it is diabled by default, as evident in newer versions of Debian and Ubuntu since 10.04.

Now, for node.js depending on the SSL library used, it will enable SSLv2 by default , which as you know by know is not a good idea.

To verify that you cannot connect with SSLv2 to your node server, fire off the following openssl command:

$ openssl s_client -connect hostname:443 -ssl2

And the response should look something akin to:

CONNECTED(00000003)
83295:error:1407F0E5:SSL routines:SSL2_WRITE:ssl handshake failure:/SourceCache/OpenSSL098/OpenSSL098-47.1/src/ssl/s2_pkt.c:428:

If you see something like the following, then you should consider disabling SSLv2:

CONNECTED(00000003)
..

SSL handshake has read 736 bytes and written 434 bytes

One (seemingly undocumented on the server side) is to pass one of the available OpenSSL CTX methods such as SSLv3_method to the httpsOptions object in the TLS/HTTPS .createServer(...)methods in Node.js.

Setting this option will force a single protocol to the SSL handshake, meaning that for some older clients such as the default OpenSSL on OSX this will cause an issue for the handshake, where it tries to negotiate a SSLv23_method, which with the above change is an unsupported fallback method for SSLv2 ( note the SSL23_WRITE ), as per OSX 10.7:

○ openssl version
OpenSSL 0.9.8x 10 May 2012

○ openssl s_client -connect hostname:443
CONNECTED(00000003)
60349:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:/SourceCache/OpenSSL098/OpenSSL098-47.1/src/ssl/s23_lib.c:182:

A solution for this particular problem which may affect applications that use the system OpenSSL on is to recompile node with the --no-ssl2 option.

After passing that option as ./configure --no-sslv2, you should find that older clients are now able to connect directly using TLSv1/SSLv3, and avoid the instant handshake fail as before:

○ openssl s_client -connect stacka.to:443

CONNECTED(00000003)
depth=2 /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA

..snip..

SSL-Session:
    Protocol  : TLSv1
    Cipher    : RC4-SHA
    Session-ID: 7B78B50B595C19AF3DA7BE845D0724C7F969505CEDDB3EA799E09C18A83433AB
    Session-ID-ctx: 
    Master-Key: EA61C4401AEA8057690E36D45B5101CBA970820A89F8A1D1AFF159578D14C39935B844BC8DA5A65E6970250F9D43D270
    Key-Arg   : None
    Start Time: 1372659259
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

If you have Nginx/Apache in front of Node.js then since v1.0 then make sure you're disabling it there too: http://www.evanhoffman.com/evan/2011/09/20/making-sure-sslv2-is-disabled-in-apache/

Note that on some popular SSL strength/vulnerability analysers, SSlv2 support is an automatically failing grade. By disabling it, you can go from an 'F' to a 'A' with a minor tweak!

Nov 5, 2012

Setting up a Github Pages blog with Hexo

Setup

You'll need the following installed before you can pop the champagne:

Node.js
Hexo npm install -g hexo

Setup Hexo

Once you've got Hexo installed, you can simply run:

1 2	$ hexo init username.github.com $ cd username.github.com

Next, we need to configure a couple of things so open up the config file:

1	$ vim _config.yml

Some items you'll want to immediately change are:

Title
Subtitle
Url
Author

To make Github deployments work with hexo deploy you'll want to make sure that the # Deployment section looks something like:

deploy:
      type: github
      repository: https://github.com/username/username.github.com.git
      branch: master

Setup Github

Github provides hosting for static websites via Github Pages. As long as you have an index.html in the root of your repo, Github will serve up the static files for you.

Create a new repository for your blog

You can get this setup by creating the required repository for your new blog, simply create a new repo with the name format as <github username>.github.com, for example: jdpaton.github.com.

Once the repo has been created you can setup the deployment process in hexo with:

1	$ hexo setup_deploy

Generate some content

Now that everything is assembled in place, it's time to flesh out the blog. To generate a new post in Hexo simply issue:

1	$ hexo add_post <title>

That command will create a new post in source/_posts/<title>.md. More info about article structure here

Now you can go ahead and add your content, and then when finished, simply run:

1 2	$ hexo generate $ hexo deploy

And your new blog will be live at http://username.github.com

Hexo: Docs

Here's one I made earlier ;)

Jamie Paton

My Notations