Rails in Realtime, Part 2

After publishing “Rails in Realtime,” many questions popped up. Most folks asked to see more depth of how we handle realtime events at LayerVault along with more detailed code samples.

This post will expand upon the previous post and talk about how to take a vanilla Rails application to a super-charged, realtime application.

Introduction

There are a few crucial technologies that we use to make all of this work at LayerVault. It should also be said that we target the Webkit browsers, although things work well enough all the way down to IE7.

These are the most important ones for the purposes of this post:

• HTML5 data-* attributes.
• Socket.IO v0.9. Particularly the rooms feature.
• localStorage for some local caching.
• jQuery 1.6+. We use 1.8.1 in production.

We make a few bold assumptions that many might disagree with:

• No HTML is rendered using client-side templates.
• No data payloads are passed through a realtime server. Only events such as “updated,” “created,” or “deleted.”

At the high level

Normally, a vanilla Rails app will look roughly like the following. It’s your standard request-response loop.

It’s our familiar request-response loop. The browser makes an HTTP request at a certain address (1) and gets a response (2).

To augment the application and bring it into the super-charged realm of realtime, the process looks a bit different.

Step by step:

1. The browser makes the HTTP request for initial page in its entirety.
2. The browser receives the request and renders the page.
3. Next, Socket.IO goes to work and establishes a Websocket connection (or gracefully falls back.) Once a connection is established, we join all of the rooms that we care about. In the context of LayerVault, we join rooms for users, projects and files. We can join anywhere between one to dozens of rooms. We idle here until…
4. Another browser or app changes the state of a model we care about. The web application server queues up a call to the report_updated method.
5. Our report_updated method runs, connecting to the Realtime Server and issuing a onetime request, e.g. we publish a file_updated event to the room file/42.
6. The Socket.IO server publishes this information back to all of the web browsers that are listening in the appropriate room.
7. Upon receiving the event, each piece of the page that cares about file/42 issues off an AJAX request for the partial to refresh its state.
8. We receive the partial and update the information to the page appropriately. This can either be through a straight .html() replacement or through a few of the jQuery plugins found in our Cosmos projects.

Whew. It might seem like quite a bit, but it’s not too bad once you start breaking down the individual pieces. Let’s go one level deeper.

Structuring your HTML

When structuring our HTML, we need to include 2 pieces of information per element on the page: (1) information on how to listen and (2) information on where to retrieve its updated state.

With LayerVault, we do this by making liberal use of data-* attributes. We never store any sort of state in memory unless required for performance reasons. Let’s look at an example:

<div 
  class="Folder" 
  data-folder-id="23"
  data-updated-at="..."
  data-url="/folder/23"
>

  <article
    class="File"
    data-file-id="42"
    data-updated-at="..."
    data-url="/file/42"
  >
    <!-- file data -->
  </article>

  <article
    class="File"
    data-file-id="81"
    data-updated-at="..."
    data-url="/file/81"
  >
    <!-- file data -->
  </article>

</div>

Here, we have a folder that contains 2 files. Each element here has its object’s ID and the updated_at attribute. This will come in handy when building cache keys. Other relevant child elements have been omitted for this example.

It’s important to keep all page rendering in an HTML template. For LayerVault, we use ERB templates and make use of the content_tag method for increased readability. Each element is contained within a partial for reusability. The Rails-way of generating this HTML is as follows:

<% cache(folder) do %>
  <%= content_tag(:div, {
    class: 'Folder',
    data: {
      folder_id: folder.id,
      updated_at: folder.updated_at,
      url: url_for(folder)
    }
  }) do %>

    <%- folder.files.each do |file| -%>
      <% cache(file) do %>
        <%= content_tag(:article, {
          class: 'File',
          file_id: file.id,
          updated_at: file.updated_at,
          url: url_for(file)
        }) do %>
          <!-- file data-->
        <%- end -%>
      <%- end-%>
    <%- end -%>

  <%- end -%>
<%- end -%>

Connecting your JavaScript

When it comes time to listen in to the appropriate rooms, that becomes pretty easy. Here’s what the JavaScript will look like in its entirety.

var 
  subscribeToRooms,
  messenger;

subscribeToRooms = function () {
  $('.Folder').each(function () {
    messenger.emit(
      'subscribe', 
      'folder/' + $(this).attr('data-folder-id')
    );
  });

  $('.File').each(function () {
    messenger.emit(
      'subscribe', 
      'file/' + $(this).attr('data-folder-id')
    );
  });
};

messenger = io.connect("/socketio.js");
messenger.on('did_connect', function (socket) {
  subscribeToRooms();
});

// Continued in the section "Listening for Events"

This code connects to the Socket.IO central server. Once it’s connected, it calls the method subscribeToRooms. The subscribeToRooms lives up to its namesake, it goes through and subscribes to each room the page cares about.

In the actual LayerVault application, we have multiple disparate controllers that hook into these events.

Publishing events

It’s up to the Rails server to publish events back out when things change. LayerVault accomplishes this through ActiveRecord::Observer. Here’s a code snippet:

class FileObserver < ActiveRecord::Observer
  observer :lv_file

  def after_commit(record)
    record.delay.report_updated
  end
end

Pretty simple stuff. After the database transaction has gone through, we use delayed_job to queue up a call to report_updated.

Let’s see what report_updated looks like.

class LVFile < ActiveRecord::Base
  def report_updated
    Messenger.publish_message('file_updated', "file/#{self.id}")
  end
end

The report_updated method makes a class method call to a separate Messenger class (our Socket.IO interface on the web app side) and reports that a file has changed to the appropriate room.

It’s important to execute the report_updated outside of the HTTP request loop for speed purposes. We like the simplicity of delayed_job to get this done.

Listening to events

Now comes the time to listen to events. Let’s go straight to code.

// Continued from "Connecting your JavaScript"

messenger.on('file_changed', function (room) {
  fireEvent('file_changed', room);
});

When a file changes, we bubble that event up to the page. We’ve got a fireEvent method which broadcasts the change to everything on the page. Similar results can be achieved with jQuery’s on and trigger.

So let’s say we had an LVFile class in our JavaScript. It will contain the following code:

// … inside our LVFile class on the page

addEventListener('file_changed', function (room) {
  var fileId = room.replace("file/", "");

  $('.File').dfilter('file-id', fileId).each(function () {
    var $file = $(this);

    $.get($file.attr('data-url'))
    .success(function (response) {
      $file.html(response);

      // Or we could use a Cosmos plugin to transition the
      // info to a new state gracefully. 
    });
  });
});

We are leaving out one major case here, which is the “created” case. The web browser doesn’t know anything about a file before it’s created. In the case of LayerVault, we are lucky: we can listen for a file created event at the folder level because all file creation happens inside of a folder. Hopefully, you can find analogies for your own application. In the worst case scenario, you can always publish a user-level event.

Building your cache keys

Until cache_digests arrive in Rails 4, building Rails-style cache keys on the client-side remains easy. One of the few things we build both server-side and client-side are the cache keys. We do this so we can store some HTML snippets client-side, like Activity Feed stories, using localStorage. Fetching new records should always reach through localStorage.

Here’s the method we use to generate cache keys:

var cacheKey = function ($element) {
    return [
      $element.attr('data-table-name'),
      "/",
      $element.attr('data-id'),
      "-",
      $element.attr('data-updated-at')
    ].join();
};

This assumes each HTML snippet has the necessary information contained within its data-* attributes. Here’s some example HTML:

<article 
  class="ActivityStory" 
  data-table-name="activity_items"
  data-id="321"
  data-updated-at="20120827134910"
>
  <!-- some other data that will be cached -->
</article>

You can also make an optimization to cacheKey(), which is to not include the table-name on every element. However, this seemingly duplicated information compresses extremely well if one uses GZIP’d responses.

Conclusion

As I wrote in the first post, it’s possible to get instantaneous realtime performance out of Rails while keeping a nice separation of responsibilities. We deal with huge pieces of data (PSD files) and manage to deliver a realtime experience.

One of the most important parts of this approach is that without the realtime updating, you still have a fully functioning application. This means it’s also easy to take an existing, well-structured Rails application and bring it into the realm of realtime.

If you enjoyed this post, you should follow us on Tumblr and Twitter.

I’ll be happy to answer any questions on Hacker News.

— Kelly