Coming from ruby and rails managing application dependencies and deployment is quite easy and straightforward thanks to tools like capistrano, rvm and bundler. A relatively young technology like node.js seems not to have any best practice for these common tasks.

Time to show how adeven manages it.

As mentioned we come from ruby and rails so there is no reason why not to use capistrano for deploying node applications. Although capistrano was originally build for rails application @hollow made quite a good job in generalizing capistrano recipes. Thats why we added some recipes to capper for node.

When deploying node application you basically have to think about the five main tasks:

Node version management
Handling Dependencies
Application Startup
Working with different Environments and stages
Deploy your app to different servers

Node version management

node 0.8 is around the corner and we have the need to run our application with different node versions. The ruby community uses rvm to manage different ruby versions. Although there no silver bullet in the node.js world to manage different node version nave is adevens weapon of choice for this task.

It’s just a shell script that installs it’s node versions into ~/.nave by default and can be used pretty much like rvm.

Usage: nave <cmd>

Commands:

install <version>    Install the version passed (ex: 0.1.103)
use <version>        Enter a subshell where <version> is being used
use <ver> <program>  Enter a subshell, and run "<program>", then exit
use <name> <ver>     Create a named env, using the specified version.
                     If the name already exists, but the version differs,
                     then it will update the link.
usemain <version>    Install in /usr/local/bin (ie, use as your main nodejs)
clean <version>      Delete the source code for <version>
uninstall <version>  Delete the install for <version>
ls                   List versions currently installed
ls-remote            List remote node versions
ls-all               List remote and local node versions
latest               Show the most recent dist version
help                 Output help information

<version> can be the string "latest" to get the latest distribution.
<version> can be the string "stable" to get the latest stable version.

So no root access needed for installing different node versions - awesome.

Handling Dependencies

With npm managing third party dependencies becomes easy. Simply define a package.json file that looks something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

{
    "private": true
  , "name":         "my awesome app"
  , "description":  "Just a sample"
  , "version":      "0.0.1"
  , "main":         "./index"
  , "keywords": [
        "Postgres", "Redis","RabitMQ"
    ]
  , "authors": [
        "your Name <[email protected]>"
    ]
  , "repository": {
        "type": "git"
      , "url":  "https://github.com/user/app.git"
    }
  , "engine": {
      "node": ">=0.8"
    }
  , "scripts": {
        "test":   "cake test"
    }
  , "dependencies": {
      "pg"          : "*"
    , "redis"       : "*"
    , "redis-lua"   : "*"
    , "forever"     : "git://github.com/nodejitsu/forever.git#node-0.8"
    , "winston"     : "*"
    , "amqp": "git://github.com/adeven/node-amqp.git#master"
    , "async"       : "*"
  }
  , "devDependencies": {
        "coffee-script":  "1.3.x"
      , "docco"     :    "*"
      , "colors"    :    "*"
      , "mocha"     :    "*"
      , "should"    :    "*"
      , "uglify-js" :    "*"
    }
}

And a simple

npm install

from your apps root will install all the modules under a node_modules folder.

However there are quite a lot discussions whether you should include you node_modules into version control or just rebuild binaries for deployment.

For us dealing with quite complex application and thus complex dependencies both does not work due to different platforms (MacOs for development, gentoo for production) and environments. The ruby community uses Gemfile and Gemfile.lock for managing dependency and locking them down.

With npm you can

npm shrinkwrap

your package.json file and will get something similar to Gemfile.lock a npm-shrinkwrap.json file. Read Managing Node.js Dependencies with Shrinkwrap about the motivation.

So you should definitely run

npm shrinkwrap

and add node_module to your .gitignore file

Application Startup

You want your app to run in the background and automatically start on error? Forever deamonizes your app and let your easily start and stop it.

usage: forever [options] [action] SCRIPT [script-options]

Monitors the script specified in the current process or as a daemon

actions:
start               Start SCRIPT as a daemon
stop                Stop the daemon SCRIPT
stopall             Stop all running forever scripts
restart             Restart the daemon SCRIPT
restartall          Restart all running forever scripts
list                List all running forever scripts
config              Lists all forever user configuration
set <key> <val>     Sets the specified forever config <key>
clear <key>         Clears the specified forever config <key>
logs                Lists log files for all forever processes
logs <script|index> Tails the logs for <script|index>
columns add <col>   Adds the specified column to the output in `forever list`
columns rm <col>    Removed the specified column from the output in `forever list`
columns set <cols>  Set all columns for the output in `forever list`
cleanlogs           [CAREFUL] Deletes all historical forever log files

options:
-m  MAX          Only run the specified script MAX times
-l  LOGFILE      Logs the forever output to LOGFILE
-o  OUTFILE      Logs stdout from child script to OUTFILE
-e  ERRFILE      Logs stderr from child script to ERRFILE
-p  PATH         Base path for all forever related files (pid files, etc.)
-c  COMMAND      COMMAND to execute (defaults to node)
-a, --append     Append logs
--pidFile        The pid file
--sourceDir      The source directory for which SCRIPT is relative to
--minUptime      Minimum uptime (millis) for a script to not be considered "spinning"
--spinSleepTime  Time to wait (millis) between launches of a spinning script.
--plain          Disable command line colors
-d, --debug      Forces forever to log debug output
-v, --verbose    Turns on the verbose messages from Forever
-s, --silent     Run the child script silencing stdout and stderr
-w, --watch      Watch for file changes
--watchDirectory Top-level directory to watch from
-h, --help       You're staring at it

[Long Running Process]
The forever process will continue to run outputting log messages to the console.
ex. forever -o out.log -e err.log my-script.js

[Daemon]
The forever process will run as a daemon which will make the target process start
in the background. This is extremely useful for remote starting simple node.js scripts
without using nohup. It is recommended to run start with -o -l, & -e.
ex. forever start -l forever.log -o out.log -e err.log my-daemon.js
    forever stop my-daemon.js    

Working with different Environments and stages

In the rails world different environments for development, test and production are quite common. Especially when dealing with databases it is quite usefull to have different work environments. With node.js manage different configurations with environment variables and modules

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

//config.js
(function() {
  var config, environment, redis;
      redis = require("redis");
      environment = process.env.NODE_ENV || 'development';

  config = {
    development: {
      pg: "postgres://postgres:@localhost:5432/app_development",
      redis: redis.createClient(),
      rabbitmq: {
        host: 'localhost',
        port: 5672,
        login: 'guest',
        password: 'guest',
        vhost: '/'
      }
    },
    test: {
      pg: "postgres://postgres:@localhost:5432/app_test",
      redis: redis.createClient(),
      rabbitmq: {
        host: 'localhost',
        port: 5672,
        login: 'guest',
        password: 'guest',
        vhost: '/'
      }
    },
    production: {
      pg: "postgres://user:[email protected]:5432/app_production",
      redis: redis.createClient(6379,redis.example.com),
      rabbitmq: {
        host: 'rabbit.example.com',
        port: 5672,
        login: 'user',
        password: 'secret',
        vhost: '/myapp'
      }
    },
  };

  exports.redis = config[environment]['redis'];
  exports.postgres = config[environment]['pg'];
  exports.rabbitmq = config[environment]['rabbitmq'];

}).call(this);

Starting your app with the right NODE_ENV environment variable will give you the right configuration when required:

1
2
3
4

var config      = require('config'),
    postgres    = config.postgres,
    redis       = config.redis,
    rabbitmq    = config.rabbitmq;

Putting it all together deployment with capistrano and capper:

I don’t want to go into the details about capistrano there are tons of very good documentations about this. With our capper fork things are pretty straight forward.

To add capper to your app, create a Gemfile with capper dependency in your app’s root:

1
2
3

# Gemfile
source :rubygems
gem 'capper', :git => "[email protected]:adeven/capper.git"

You can then simply gem install bundler and run bundle install

Next creates two files:

The Capfile specifies which recipes you want to use. It should look something like this:

1
2
3
4
5
6
7
8
9
10

# Capfile
require 'rubygems'
require "capper"

load "capper/npm"
load "capper/forever"
load "capper/nave"

# load your custom deploy script
load "config/deploy"

The deployment configuration goes into config/deploy.rb

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

#config/deploy.rb

set :application, "your_app"

# see https://help.github.com/articles/deploying-with-capistrano
# on how to deploy with github and capistrano

set :repository, "[email protected]:yourName/your_app.git"
ssh_options[:forward_agent] = true
set :scm, :git                                      #capper default

set :use_sudo, false                                #capper default
set :keep_releases, 5                               #capper default
set :deploy_via, :remote_cache                      #capper default
set :main_js, "build/your_app.js"

# your log folder to share between different releases
# you can add more folders here...
set :symlinks, {"log" => "log"}

# We use two different stages here production / staging
desc "production stage"
task :production do
# skip using nave on production server
  set :use_nave, false
  set :branch, 'master'                                     #default
  set :user, 'your_app_user'

  set :deploy_to, , "/home/#{user}/deploy/#{application}"   #capper defaults to "/var/app/#{application}"
  set :node_env, 'production'
  server 'your.app.server.com', :app                        #add more / different roles
  set :forever_cmd, "./node_modules/.bin/forever"           #use the forever that is installed along with the app
end

desc "staging stage"
task :staging do
# use node 0.8.1. together with nave
  set :node_ver, '0.8.1'

# test a different branch on staging  
  set :branch, 'node-0.8'
  set :user, 'your_app_user'
  set :deploy_to, "/home/#{user}/deploy/#{application}"
  set :node_env, 'staging'
  server 'your.stage.server', :app
  set :forever_cmd, "./node_modules/.bin/forever"
end

desc "tail the application logfile"
task :log do
  log = "#{application_dir}/current/log/#{node_env}.log"
  run "tail -f #{log}"
end

Having the tasks production and staging gives us the ability to use different settings for our deploy scenarios.

run bundle exec cap -T to see whats in the box:

cap deploy               # Deploys your project.
cap deploy:cleanup       # Clean up old releases.
cap deploy:cold          # Deploys and starts a `cold' application.
cap deploy:migrate       # Blank task exists as a hook into which to install ...
cap deploy:migrations    # Deploy and run pending migrations.
cap deploy:pending       # Displays the commits since your last deploy.
cap deploy:pending:diff  # Displays the `diff' since your last deploy.
cap deploy:restart       # Blank task exists as a hook into which to install ...
cap deploy:rollback      # Rolls back to a previous version and restarts.
cap deploy:rollback:code # Rolls back to the previously deployed version.
cap deploy:setup         # Prepares one or more servers for deployment.
cap deploy:start         # Blank task exists as a hook into which to install ...
cap deploy:stop          # Blank task exists as a hook into which to install ...
cap deploy:symlink       # Updates the symlink to the most recently deployed ...
cap deploy:update        # Copies your project and updates the symlink.
cap deploy:update_code   # Copies your project to the remote servers.
cap deploy:web:disable   # Present a maintenance page to visitors.
cap deploy:web:enable    # Makes the application web-accessible again.
cap forever:restart      # Restart the servers using the forever cmd If the f...
cap forever:start        # Start the servers using the forever cmd If the for...
cap forever:stop         # Stop the servers using the forever cmd If the fore...
cap forever:tail         # tail the forever logfile using the forever cmd If ...
cap invoke               # Invoke a single command on the remote servers.
cap log                  # tail the application logfile
cap nave:install         # set :node_ver, "stable" # e.g.
cap nave:setup           # Install nave into nave target dir.
cap npm:install          # Install the current npm environment.
cap production           # production stage
cap shell                # Begin an interactive Capistrano session.
cap staging              # staging stage

To get more detailed informations enter

bundle exec cap -e taskname

To run a dry test for your deployment run:

cap staging deploy -n

And enjoy the output.