Transcript

Howdy and welcome back. Like most of episodes this one is going to move quickly. I do this so they don’t go on forever, however, I highly recommend stopping and rewatching parts that might go to fast. So, I thought it would be useful to go over how to deploy activityOverlord into the wild. This episode will cover deployment to heroku, a self-described cloud application platform.

Heroku makes it really easy to deploy node apps without a lot of the overhead typically associated with deploymnet. Although this episode will concentrate on heroku, I plan on covering other platforms in future episodes.

First, let’s look at the current stack of technologies used by activityOverlord.

I’m running OS X Mountain Lion locally on a MacBook Pro. I’ve got node installed and sails of course runs on top of that. Our models, currently the user model, uses a mongodb database also running locally on mountain lion. And finally, we store our sessions and sockets in memory.

To do this deployment the stack of technologies is going to change.

For example, instead of running locally on OS X Mountain Lion, node and sails will run on an instance of the hardware and software provided by heroku. Looking at the heroku docs, node.js runs on instances of Ubuntu 10.04.

Next, instead of our current local mongodb database, we’ll create a new instance of the database on a hosted solution provided by mongohq. Finally, we’ll move our in memory socket and session store to redis and specifically redis-to-go hosted by heroku as well.

Moving from local mongodb to hosted mongohq So you’ll first need to create an account on mongohq. Once you have an account and are logged in, create a new hosted mongodb instance using the sandbox option. Although this option isn’t for production it works for the purposes our project. Next, I created a user for the database that we’ll use to authenticate in our app.

So I want to incorporate this hosted database in my local instance of activityOverlord before we move it to heroku and we do that by changing the local.js file. First let’s do a bit of review.

Our local configuration of mongoDB uses local.js within the config folder while adapters.js is completely commented out at the moment. The adapters.js file is one place sails looks to for connections to databases. The local.js file overrides anything contained in adapters.js. Also recall that local.js is part of .gitignore so the file will not be saved to git or our repo.

The impact of this is that anything sails relies upon in terms of configuration in local.js will not be part of the deployment unless we provide for it somewhere else in the code or by using environment variables prior to deployment.

Before we address the deployment issues lets first go into the code and make a few changes to local.js to use our new mongohq instance.

We no longer need the host, user, password, and database attributes because they are combined in a new attribute called url. I’ll copy and paste the url from the mongohq interface and then insert the username and password I created earlier for the database.

/config/local.js

1
2
3
4
5
6
7
8
9
10

adapters: {

   'default': 'mongo',

    mongo: {
      module   : 'sails-mongo',
      url: "mongodb://admin:1234@paulo.mongohq.com:10099/activityoverlord",
      schema: true
    }
  }

Let’s see if this worked. I’ll go into the terminal and lift activityoverlord. Next, I’ll create a new user and great it looks like things are working. So, I’ll go back into mongohq and into the user collection and there’s my new user. To start things off, I’ll change my admin attribute to true. Now, I’ll log-in to the account and…great I have my admin user set-up.

Okay, now, let’s deploy to heroku. If you don’t already have an account on heroku, go ahead and create one now. Next, you’ll want to install the heroku toolbelt which can be found here. Finally, you’ll want to login to heroku from the terminal. So let’s go to the terminal and type heroku login, this is going to ask us for our heroku credentials and the first time you run it, it’s also going to set-up your public SSH key. Don’t worry If you don’t already have one, heroku will walk you through setting one up.

So now that we have an account, got the toolbelt installed, and we’ve logged in from the terminal, we want to back to the heroku dashboard and select create a new app. If you want to enter in an app name it will need to be something other than activityoverlord as app names must be unique. If you don’t put in an app name, heroku will create one for you and regardless, you can always go back and rename your app later.

Now you’ll recall that we have a local.js file pointing to the mongodb database hosted on mongohq. Since local.js will not be part of our repo because of .gitignore we need some way of letting the heroku server instance know about the mongodb configuration and we do this with environment variables. So let’s go back into our adapters.js file and I’m going to copy and paste the mongodb configuration information from our local.js file into this adapters.js file. However, I’m going to replace where we have a string that contains the Username, password,path, and port of our mongodb instance on mongohq with an environment variable called DB_URL and that’s going to be prefaced by process.env.

/config/adapters.js

1
2
3
4
5
6
7

'default': 'mongo',

    mongo: {
      module   : 'sails-mongo',
      url: process.env.DB_URL,
      schema: true
    }

So, process.env is a node object that contains our environment which also includes the environment variables that we’ll add to it. So to add the environment variable to our heroku instance, let’s go back to the terminal and we’ll type heroku config:set DB_URL= and to get our path I’ll look in our local.js and copy the path of our mongohq instance and paste it here, I’ll also add —app activitityoverlord1 to point specify which app to associate with.

So we’ve set DB_URL on the remote heroku instance using heroku config:set and used that environment variable in our adapters.js file to point to our mongoHQ instance.

So how is heroku going to start ativityoverlord? We do that by creating a Procfile in the root of our app. A Procfile is a mechanism for declaring what commands are run by our app’s dynos on the Heroku platform. More on dynos in a second. Let’s go back into the code and add a new file named Procfile with no extension. The file will have one line: web: node app.js

Next, make sure you have sails-mongo in your package.json file and that it’s installed in node_modules. In fact it’s probably best to do an npm install to make sure you have all of the depencies installed. Now we need to link up the heroku end point with our project. Let’s go back to the heroku dashboard and look under the settings tab. Go ahead and c opy the git url and then go back to the terminal and enter: git remote add heroku <and then paste the git url here> and press enter. Add all of your changes to git using: git add . and then commit them using git commit -am “my very descriptive change log”. Finally push the project by entering: git push heroku master.

The last step before we fire up our browser and look at activityOverlord is to set up a dyno for our app. Heroku suggests thinking of a dyno asa virtualized Unix container. In sum, it’s the place where our app will run. To assign one dyno to our app, type: heroku ps:scale web=1.

So let’s go back into the browser, refresh the app, and log-in. Everything looks to be working, however, open up the console and you’ll see an error. Socket.io is doing its job. Websockets isn’t working so its failing over to long polling so we still have a connection. As it turns out, heroku has just started to support web sockets and you must enable it on the application instance. To do that, we’ll go back to the console and type heroku labs:enable websockets. It can take a second before websockets starts working. There we go. Also, I have had issues with it periodically failing and going back to web polling, but it is in beta so we’ll see how it improves over the coming weeks.

Next, I want to take a look at moving our session and socket store to redis. But first, why would we want to do this in the first place? Let’s take the following example. I have three instances of heroku running activityOverlord on each instance. I use a load balancer to distribute the incoming requests across the three instances.

Suppose we store sessions in memory, and on the first request the load balancer sends us to instance A where we authenticate and the session cookie is set for that server instance. On the next request we’re sent to instance B, where we haven’t yet authenticated and therefore won’t be able to access the resources we would have had access on instance A. Therefore we need some way for the multiple instances of our application to share the same session store.

This is why we’re moving our session and socket store to redis.

So let’s set-up redis. We’ll head back to the heroku dashboard and our activityoverlord instance. Select add-ons and redis-to-go. Under Plans, select Nano, or the free plan. I had to put in a credit card despite picking the free option. Select add nano for free. Go back to you instance of activityoverlord and select redis-to-go nano under add-ons.

Here you’ll see the configuration path to our redis instance. First, let’s go into activityoverlord and our session.js file. As the documentation suggests, I’m going to uncomment, the adapter, host, port, db, and password keys. We can then go back to our redis-to-go configuration file and copy and paste each value into the keys in session.js.

/config/session.js

1
2
3
4
5
6
7

adapter: 'redis',

  host: 'soldierfish.redistogo.com',
  port: 9599,
  // ttl: <redis session TTL in seconds>,
  db: 'redistogo',
  pass: 'd5d68502e87bf36e5d6d25d9c0f37b5a'

Okay, let’s see if this worked. I’ll go back into the terminal and commit my changes and then push them to our heroku instance. Now let’s go back to the browser and try to log in. Even though that worked the true test, is whether our session id is in the redis database. To determine this, I’m going to use the redis-cli command line tool. To use this tool we need to again use the host, port and password to authenticate to the database. Once connected I’ll use the keys command passing in an * as an argument to get all keys. And there’s our session key, great. The redis website has good documentation on other useful commands.

You might be asking yourself, I don’t really want to put my redis database credentials in my github repo, and you know you would be right, that would be a very bad idea! So instead we can use environment variables to set these credentials to our heroku instance. Let’s go back into session.js and change the values for the host, port, db, and pass keys to environment variables.

/config/session.js

1
2
3
4
5
6
7

adapter: 'redis',

  host: process.env.REDIS_HOST,
  port: process.env.REDIS_PORT,
  // ttl: <redis session TTL in seconds>,
  db: process.env.REDIS_DB,
  pass: process.env.REDIS_PASS

Now the server instance will be looking for those environment variables for the values. We’ll set them on the heroku instance the same way we did for DB_URL using heroku config:set.

Okay, now let’s do the same for sockets. We’ll go back to sockets.js. Similar to session.js we’ll uncomment the host, port, db, and pass keys and then insert the environment variables for the values.

/config/sockets.js

1
2
3
4
5
6

adapter: 'redis',

  host: process.env.REDIS_HOST,
  port: process.env.REDIS_PORT,
  db: process.env.REDIS_DB,
  pass: process.env.REDIS_PASS,

Now, I’m going to go back to the terminal and commit my changes again and push them to our heroku instance. Now I’ll go back to the browser, notice that I don’t have to login as my session is now maintained by redis whereas before we were doing things in memory which required us to login each time the server was lifted. I’ll manually log out and log back in. And great it looks like everything is working.

Okay, the last thing I want to address with deployment is changing the value of the NODE_ENV variable from development to production. For sails one of the biggest outwardly facing changes as a result of using production instead of development is that all of our css files will be combined and minified into one file. Javascript files will also be combined and minified as well. In addition many modules utilize NODE_ENV as a hook to determine whether to make changes based upon its value. We’re going to actually set the environment variable in our Procfile. So let’s go over to the Procfile and add web: NODE_ENV=production node app.js. I’ll commit my changes and push them to heroku. Back in the browser I’ll refresh the page and then look at the source to confirm that all of my css is minified in one file and all of my javascript is minified in one file.

So now that we’ve successfully deployed activityOverlord to Heroku I want to address the work-flow for moving forward with development. The repo for activityOverlord will have the following set-up for local.js, adapter.js, session.js, and sockets.js. The local.js file will default to our local instance of mongodb.

/config/local.js

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

module.exports = {

  port: process.env.PORT || 1337,



  // The runtime "environment" of your Sails app is either 'development' or 'production'.
  //
  // In development, your Sails app will go out of its way to help you
  // (for instance you will receive more descriptive error and debugging output)
  //
  // In production, Sails configures itself (and its dependencies) to optimize performance.
  // You should always put your app in production mode before you deploy it to a server-
  // This helps ensure that your Sails app remains stable, performant, and scalable.
  // 
  // By default, Sails sets its environment using the `NODE_ENV` environment variable.
  // If NODE_ENV is not set, Sails will run in the 'development' environment.

  environment: process.env.NODE_ENV || 'development',

  // LOCAL MONGO DB
  adapters: {

   'default': 'mongo',

    mongo: {
      module   : 'sails-mongo',
      host     : 'localhost',
      user     : '',
      password : '',
      database : 'activityoverlord',

      schema   : true
    }
  }

  // // // HOSTED MONGO HQ
  // adapters: {

  //  'default': 'mongo',

  //   mongo: {
  //     module   : 'sails-mongo',
  //     url: "mongodb://admin:1234@paulo.mongohq.com:10099/activityoverlord",
  //     schema: true
  //   }
  // }

};

If you want the local version of activityOverlord to use hosted mongohq instance, just uncomment and comment the following lines. Since local.js will overwrite adapters.js we can leave the existing code in it.

/config/adapters.js

1
2
3
4
5
6
7
8
9
10

module.exports.adapters = {

  'default': 'mongo',

  mongo: {
    module   : 'sails-mongo',
    url: process.env.DB_URL,
    schema: true
  }
};

Session.js will use the in memory session store, but when you want to deploy just uncomment these lines and comment your in memory config.

/config/session.js

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

module.exports.session = {

  // Session secret is automatically generated when your new app is created
  // Replace at your own risk in production-- you will invalidate the cookies of your users,
  // forcing them to log in again. 
  secret: 'd494d185735d00432bc3485d32bd5ca8',


  // In production, uncomment the following lines to set up a shared redis session store
  // that can be shared across multiple Sails.js servers

  // HOSTED REDIS INSTANCE
  adapter: 'redis',

  host: process.env.REDIS_HOST,
  port: process.env.REDIS_PORT,
  // ttl: <redis session TTL in seconds>,
  db: process.env.REDIS_DB,
  pass: process.env.REDIS_PASS
  // prefix: 'sess:'

  // // USE IN MEMORY
  //   adapter: 'memory'

  // Uncomment the following lines to use your Mongo adapter as a session store
  // adapter: 'mongo',
  //
  // host: 'localhost',
  // port: 27017,
  // db: 'sails',
  // collection: 'sessions',
  //
  // Optional Values:
  //
  // # Note: url will override other connection settings
  // url: 'mongodb://user:pass@host:port/database/collection',
  //
  // username: '',
  // password: '',
  // auto_reconnect: false,
  // ssl: false,
  // stringify: true

};

The same holds true for the Sockets.js configuration file.

/config/sockets.js

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194

/**
 * Socket Configuration
 *
 * These configuration options provide transparent access to Sails' encapsulated
 * pubsub/socket server for complete customizability.
 *
 * For more information on using Sails with Sockets, check out:
 * http://sailsjs.org/#documentation
 */

module.exports.sockets = {


  // `transports`
  //
  // A array of allowed transport methods which the clients will try to use.
  // The flashsocket transport is disabled by default
  // You can enable flashsockets by adding 'flashsocket' to this list:
  transports: [
  'websocket',
  'htmlfile',
  'xhr-polling',
  'jsonp-polling'
 ],




  // Use this option to set the datastore socket.io will use to manage rooms/sockets/subscriptions:
  // default: memory

  // HOSTED REDIS INSTANCE
  adapter: 'redis',

  host: process.env.REDIS_HOST,
  port: process.env.REDIS_PORT,
  db: process.env.REDIS_DB,
  pass: process.env.REDIS_PASS,

  // IN MEMORY
     // adapter: 'memory',

  // Worth mentioning is that, if `adapter` config is `redis`, 
  // but host/port is left unset, Sails will try to connect to redis 
  // running on localhost via port 6379  
  // `authorization`
  //
  // Global authorization for Socket.IO access, 
  // this is called when the initial handshake is performed with the server.
  // 
  // By default (`authorization: true`), when a socket tries to connect, Sails verifies
  // that a valid cookie was sent with the upgrade request.  If the cookie doesn't match
  // any known user session, a new user session is created for it.
  //
  // However, in the case of cross-domain requests, it is possible to receive a connection
  // upgrade request WITHOUT A COOKIE (for certain transports)
  // In this case, there is no way to keep track of the requesting user between requests,
  // since there is no identifying information to link him/her with a session.
  //
  // If you don't care about keeping track of your socket users between requests,
  // you can bypass this cookie check by setting `authorization: false`
  // which will disable the session for socket requests (req.session is still accessible 
  // in each request, but it will be empty, and any changes to it will not be persisted)
  //
  // On the other hand, if you DO need to keep track of user sessions, 
  // you can pass along a ?cookie query parameter to the upgrade url, 
  // which Sails will use in the absense of a proper cookie
  // e.g. (when connection from the client):
  // io.connect('http://localhost:1337?cookie=smokeybear')
  //
  // (Un)fortunately, the user's cookie is (should!) not accessible in client-side js.
  // Using HTTP-only cookies is crucial for your app's security.
  // Primarily because of this situation, as well as a handful of other advanced
  // use cases, Sails allows you to override the authorization behavior 
  // with your own custom logic by specifying a function, e.g:
  /*
    authorization: function authorizeAttemptedSocketConnection(reqObj, cb) {

        // Any data saved in `handshake` is available in subsequent requests
        // from this as `req.socket.handshake.*`

        //
        // to allow the connection, call `cb(null, true)`
        // to prevent the connection, call `cb(null, false)`
        // to report an error, call `cb(err)`
    }
  */
  authorization: true,




  // Match string representing the origins that are allowed to connect to the Socket.IO server
  origins: '*:*',

  // Should we use heartbeats to check the health of Socket.IO connections?
  heartbeats: true,

  // When client closes connection, the # of seconds to wait before attempting a reconnect.
  // This value is sent to the client after a successful handshake.
  'close timeout': 60,

  // The # of seconds between heartbeats sent from the client to the server
  // This value is sent to the client after a successful handshake.
  'heartbeat timeout': 60,

  // The max # of seconds to wait for an expcted heartbeat before declaring the pipe broken
  // This number should be less than the `heartbeat timeout`
  'heartbeat interval': 25,

  // The maximum duration of one HTTP poll-
  // if it exceeds this limit it will be closed.
  'polling duration': 20,

  // Enable the flash policy server if the flashsocket transport is enabled
  // 'flash policy server': true,

  // By default the Socket.IO client will check port 10843 on your server 
  // to see if flashsocket connections are allowed.
  // The Adobe Flash Player normally uses 843 as default port, 
  // but Socket.io defaults to a non root port (10843) by default
  //
  // If you are using a hosting provider that doesn't allow you to start servers
  // other than on port 80 or the provided port, and you still want to support flashsockets 
  // you can set the `flash policy port` to -1
  'flash policy port': 10843,

  // Used by the HTTP transports. The Socket.IO server buffers HTTP request bodies up to this limit. 
  // This limit is not applied to websocket or flashsockets.
  'destroy buffer size': '10E7',

  // Do we need to destroy non-socket.io upgrade requests?
  'destroy upgrade': true,

  // Should Sails/Socket.io serve the `socket.io.js` client? 
  // (as well as WebSocketMain.swf for Flash sockets, etc.)
  'browser client': true,

  // Cache the Socket.IO file generation in the memory of the process
  // to speed up the serving of the static files.
  'browser client cache': true,

  // Does Socket.IO need to send a minified build of the static client script?
  'browser client minification': false,

  // Does Socket.IO need to send an ETag header for the static requests?
  'browser client etag': false,

  // Adds a Cache-Control: private, x-gzip-ok="", max-age=31536000 header to static requests, 
  // but only if the file is requested with a version number like /socket.io/socket.io.v0.9.9.js.
  'browser client expires': 315360000,

  // Does Socket.IO need to GZIP the static files?
  // This process is only done once and the computed output is stored in memory. 
  // So we don't have to spawn a gzip process for each request.
  'browser client gzip': false,

  // Optional override function to serve all static files, 
  // including socket.io.js et al.
  // Of the form :: function (req, res) { /* serve files */ }
  'browser client handler': false,

  // Meant to be used when running socket.io behind a proxy. 
  // Should be set to true when you want the location handshake to match the protocol of the origin. 
  // This fixes issues with terminating the SSL in front of Node 
  // and forcing location to think it's wss instead of ws.
  'match origin protocol': false,

  // Direct access to the socket.io MQ store config
  // The 'adapter' property is the preferred method
  // (`undefined` indicates that Sails should defer to the 'adapter' config)
  store: undefined,

  // A logger instance that is used to output log information.
  // (`undefined` indicates deferment to the main Sails log config)
  logger: undefined,

  // The amount of detail that the server should output to the logger.
  // (`undefined` indicates deferment to the main Sails log config)
  'log level': undefined,

  // Whether to color the log type when output to the logger.
  // (`undefined` indicates deferment to the main Sails log config)
  'log colors': undefined,

  // A Static instance that is used to serve the socket.io client and its dependencies.
  // (`undefined` indicates use default)
  'static': undefined,

  // The entry point where Socket.IO starts looking for incoming connections. 
  // This should be the same between the client and the server.
  resource: '/socket.io'

};

We’ve covered a bunch of material in this episode. I hope you found it helpful and as always thanks for watching.