Converting a Rails application to Node.js

CONVERTING TO NODE
How I converted a medium sized Rails project to Node.js

BACKGROUND
•

CTO of Ideal Candidate

•

MVP existed when I joined

•

Author of Haraka and a few
other Node.js libraries

•

Background in Perl and
Anti-Spam

THE RAILS APP
•

Unicorn + Thin + Rails

•

CoffeeScript with Angular.JS

•

SASS+Compass for stylesheets

•

HAML for templates

•

MySQL + MongoDB

•

Sidekiq for background tasks

APP SIZE
•

5700 Lines of Ruby (170 files)

•

2500 Lines of CoffeeScript (119 files)

•

3800 Lines of SASS (42 files)

•

1500 Lines of HAML (100 files)

FINAL GOAL
•

Express + minimal middleware

•

PostgreSQL hand coded with referential integrity

•

Jade templates

•

SCSS for CSS

•

Javascript + Angular.js frontend

HOW?
It’s all about the information, Marty

RAILS GIVES YOU
•

Routes

•

Middleware

•

DB abstraction layer (aka Model)

•

Views

•

Controllers

•

Conﬁguration

•

Plugins

•

A directory structure

•

Unit tests

EXPRESS GIVES YOU
•

Routes

•

Middleware

•

That’s it

•

(This scares a lot of people)

DIRECTORY STRUCTURE
lib
- general JS libraries

⌞ model - database accessors

public
- files that get served to the front end

⌞ assets - files that are considered part of the app

⌞ javascripts

⌞ stylesheets

⌞ images

routes
- files that provide express routes for HTML

⌞ api/v1 - files providing REST API endpoints (JSON)

views
- Jade templates

app.js
- entry point

RAILS ASSET PIPELINE
•

Rails does a lot for you, but it’s f ’ing confusing

•

HAML Template Example:

•

= javascript_include_tag :application

•

You’d think this adds <script
src=“application.js”>, but no.

RAILS ASSET PIPELINE
•

Easy if you understand it

•

Too much magic for my liking

•

But… the overall effect is good. So I copied some
of it.

JADE EQUIVALENT

each script in javascripts
script(src=script, type=“text/javascript”)

APP.JS:
var js = find.fileSync(/.(coffee|js)$/, __dirname +
‘/public/assets/javascripts/controllers')
.map(function (i) {
return i.replace(/^.*/assets/, ‘/assets')
.replace(/.coffee$/, ‘.js')
});
app.all('*', function (req, res, next) {
res.locals.javascripts = js;
next();
});

SERVING COFFEESCRIPT
// development
app.use(require('coffee-middleware')({
src: __dirname + '/public',
compress: false,
}));

!

// Production
var all_js = coffee_compiler.generate(js, true);

!

var shasum = crypto.createHash('sha256');
shasum.update(all_js);
var js_sha = shasum.digest('hex');

!

js = ['/assets/javascripts/application-' + js_sha + '.js'];

!

app.get('/assets/javascripts/application-' + js_sha + '.js', function (req, res) {
res.type('js');
res.setHeader('Cache-Control', 'public, max-age=31557600');
res.end(all_js);
});

SERVING SCSS
•

Node-sass module serves .scss ﬁles

•

Doesn’t serve up .sass ﬁles (weird, huh?)

!
# mass-convert.bash
for F in `find . -name *.sass`; do
O=“${F/.sass/.scss}”
sass-convert -F sass -T scss “$F” “$O”
git rm -f “$F”
git add “$O”
done

SCSS APP.JS
function compile_css () {
console.log("Recompiling CSS");
resources.watchers.forEach(function (w) {
w.close();
});

resources.watchers = [];

!

!
!
!
}

resources.css = sass.renderSync({
file: __dirname + '/public/assets/stylesheets/application.scss',
includePaths: [__dirname + '/public/assets/stylesheets'],
});
var shasum = crypto.createHash('sha256');
shasum.update(resources.css);
var css_sha = shasum.digest('hex');
resources.stylesheets[0] = '/assets/stylesheets/application-' + css_sha + '.css';
resources.stylesheets[1] = '//fonts.googleapis.com/css?family=Open+Sans:400,700,800,300,600';
find.fileSync(cssregexp, __dirname + '/public/assets/stylesheets').forEach(function (f) {
resources.watchers.push(fs.watch(f, compile_css));
})

SCSS APP.JS

compile_css();

!
// Dev and Production the same
app.get(//assets/stylesheets/application-(w+).css/, function (req, res) {
res.type('css');
res.setHeader('Cache-Control', 'public, max-age=31557600');
res.end(resources.css);
});

ADDING COMPASS
•

Ruby CSS framework

•

Luckily only the CSS3 part used

•

CSS3 code is just SASS ﬁles

•

Once I ﬁgured this out, copied them into my
project, et voila!

CONVERTING HAML TO JADE
•

Both indent based

•

HAML:

•

•

%tag{attr: “Value”}

Some Text
.person

.name

Bob

JADE:

tag(attr=“Value”) Some Text
.person

.name Bob

Other subtle differences too

CONVERSION TOOL: SUBLIME
TEXT
•

Afterthought: I should have written something in Perl

•

Regexp Find/Replace

•

^(s*)%

=> $1 (ﬁx tags)

•

(w){(.*)} => $1($2) (attribute curlys)

•

(w):s*([“‘]) => $1=$2 (attribute key: value)

THINGS LEFT TO FIX
•

Helpers: = some_helper

•

Text on a line on its own - Jade treats these as tags

•

Nested attributes:

•

->

%tag{ng:{style:’…’,click:’…’},class:’foo’}
tag(ng-style=‘…’, ng-click=‘…’, class=‘foo’)

Making sure the output matched the Ruby/HAML
version was HARD - HTML Diff tools suck

HELPERS BECAME MIXINS
•

Standard Rails Helpers:

= form_for @person do |f|
f.label :first_name
f.text_field :first_name
%br

!
f.label :last_name
f.text_field :last_name
%br

!
f.submit

•

Custom Rails helpers stored in app/helpers/ folder

•

http://api.rubyonrails.org/classes/ActionView/Helpers/FormHelper.html

JADE MIXINS
•

Very powerful. Poorly documented.

•

Standalone or can have block contents

// implementation
mixin form(action)
form(accept-charset="UTF-8", action=action, method="POST")
input(name="utf8" type="hidden" value="✓")
block
// usage:
+form(‘/post/to/here’)
input(type=“text”,name=“hello”,value=“world”)

•

Supports js code

mixin app_error(field)
if (errors && errors[field])
div(class="err_msg " + field)
each error in errors[field]
span= error

JADE: NO DYNAMIC
INCLUDES
•

HAML/Helpers can do:

= popup ‘roles/new'

•

Jade needed:

+popup(‘roles/new’)

include ../roles/new

CREATING THE MODEL
•

Hand coded db.js layer developed over time from previous
projects

•

One ﬁle per table (generally) in lib/model/*.js

"use strict";

var db = require('./db');

exports.get = function get (id, cb) {

db.get_one_row("SELECT * FROM Answers WHERE id=$1", [id], cb);
}

exports.get_by_submission_id = function (submission_id, cb) {

db.query("SELECT * FROM Answers

WHERE submission_id=$1

ORDER BY question_id", [submission_id], cb);
}

WHY CONTROL THE SQL?
exports.get_avg_team_size = function (company_id, role_id, months, cb) {
var role_sql = role_id ? ' AND role_id=$2' : ' AND $2=$2';

!
!

}

if (months == 0 || months == '0') {
months = '9000'; // OVER NINE THOUSAND
}
var sql = "SELECT avg(c) as value
FROM (
SELECT g.month, count(e.*)
FROM generate_series(
date_trunc('month', now() - CAST($3 AS INTERVAL)),
now(),
INTERVAL '1 month') g(month)
LEFT JOIN employees e ON e.company_id=$1" + role_sql + "
AND (start_date, COALESCE(end_date, 'infinity'))
OVERLAPS
(g.month, INTERVAL '1 month')
GROUP BY g.month
HAVING count(e.*) > 0
) av(month,c)";
db.get_one_row(sql, [company_id, role_id, months + " months"], cb);

AND FINALLY…
•

Routes

•

Run Rails version of App

•

Open Chrome Dev Console Network tools

•

Hit record

•

Find all routes and implement them

CREATING ROUTES/MODELS
•

While I glossed over this, it was the bulk of the
work

•

Each endpoint was painstakingly re-created

•

This allowed me to get a view of the DB layout

•

And ﬁx design bugs in the DB layout

find.fileSync(/.js$/, __dirname + '/routes').forEach(function (route_file) {
require(route_file);
});

BACKGROUND TASKS
•

Currently using Sidekiq, which uses Redis as a queue

•

Used for downloading slow data feeds

•

Node.js doesn’t care if downloads are slow

•

So I punted on background tasks for now

•

If I need them later I will use Kue (see npmjs.org)

DEPLOYMENT
•

Linode + Nginx + Postgres 9.3.1 + Runit +
Memcached

•

/var/apps and deploy_to_runit for github autodeploy

•

Monitoring via Zabbix

•

60M used vs 130M for Rails

NEXT STEPS
•

Convert coffeescript code to plain JS - I ﬁnd
coffeescript too much of a pain

•

Implement graceful restarts using cluster

•

Consider porting CSS to Bootstrap so we get
mobile support

Converting a Rails application to Node.js

More Related Content

What's hot (11)

Similar to Converting a Rails application to Node.js (20)

Recently uploaded (20)

Converting a Rails application to Node.js

Editor's Notes