Screen Shot 2014-12-04 at 10.55.39 AM

in Projects, Programming, Technology

Top 6 Gotchas with Node.JS

Node.js has many advantages. Many developers know Javascript at least a little bit. Node offers great performance for applications that wait on I/O (like web services). There’s a vibrant open source community creating reusable node modules for almost every task (and a handy package manager to bundle them together). Node is cross platform – the same code runs on Windows, OSX and Linux. And of course, there’s the promise of re-using Javascript patterns on both the client side and the server side.

I love node, but I’ve run into several gotchas with node in production the last few years. Here are the top 6.

1. The website (server process) is fragile. With node, the process restarts on every unhandled exception. This is different from Ruby on Rails, where you have a separate server process like Unicorn, or PHP where it runs as an Apache module.

With one client that was transitioning from PHP to node, our first deployments felt like we were “plugging the dike” – there were so many server crashes. Until we stabilized things, this sort of gave node a bad name (the perception was that PHP was more stable – but in fact the exceptions were just hidden).

Handling all potential errors, as well as unplanned exceptions is impossible. Most deployments have another process to restart the primary node server process when it bombs. If you’re using a PAAS provider, this will be provided for you, but if you want fine grain control of your server (most web application companies will), hosting and scaling (keeping processes up and enough process around) is a challenge.

Common server-restarting libraries for node are: Upstart, Forever, or God.

2. Writing and maintaining asynchronous code. Callbacks are a simple mechanism starting out, but as your server code grows in complexity, they become a hairball. Crazy callback chains (AKA “callback hell”) make code unreadable. Even simple code that could’ve been procedural is forced to use callbacks due to the libraries you’re using.

Mixing synchronous and asynchronous code makes it difficult to handle every error. Furthermore, you have to understand the event loop to know when execution actually switches contexts. As you’re running your code – if you forget to call a callback, you’ll be left waiting to find out what happened. And, asynchronicity makes even your tests become more complicated.

The Async library exists to handle common asynchronous patterns. Personally, I prefer the “promise” (or “deferred”) pattern. It seems like a hack to send a function as a parameter, instead the promise pattern returns an object representing the state of the asynchronous call instead. jQuery has a promise library for the client side. When is the promise library we selected for a recent node-based web site (comprehensive API and more standards based).

3. Too much choice.  The flip side of all those great NPM packages is that there is no clear winning web framework (think Ruby on Rails) for node – in fact, there are several competing server side frameworks (express, hapi, sails, kraken). Each with their own conventions for:

a) File structure
b) Deploying code
c) Building assets
d) Automated testing
e) ORM

And of course, the best choices for each of these changes monthly. This, combined with #2 is why Node is not yet a great choice for simple human facing web sites.

Of course, it’s not necessary to use node on the server.  On one team we used it to great success for building thousands of lines of Javascript and templates to be hosted inside a Python Django website. The problem of too much choice exists for client side frameworks (and build tools) as for the server side (you’ve heard of Backbone, Angular, Ember). Node (with the help of tools like grunt or gulp) is essential to building a modern front-end, just prepared for the evaluation time and conflicting opinions on frameworks.

Screen Shot 2014-12-04 at 10.55.39 AM

Visualization of NPM Packages

 

4. Javascript the language. Many developers think they “know” javascript from having used it in client side code. In fact, they haven’t learned about things like prototypal inheritance, function binding (what is this pointing to?), or variable scoping that are unique to Javascript. Even once you’ve chosen Javascript, there are still a lot of decisions to be hammered out amongst your team. Code conventions (4 spaces or two?) Camelcase or snake case? Jshint is a huge help with this. But, be ready for…

5. Functional versus object oriented programming. There are two very distinct paths any Javascript project can take. Functional style favors dependency injection for testability and clear interfaces (that are theoretically more re-usable). Object oriented favors composition and inheritance for more readable code. The best choice is probably a mixture, as both are silly at the extremes. My current team is finding compromises, but please let me know if you’ve got any guidelines for when to be functional and when to be OO.

6. What about Coffeescript? I hate semicolons. Compared to other languages that give white space meaning (Python and Ruby), Javascript feels heavy and hard to read. I love Coffeescript because it lets you write clear (information dense) code while avoiding common Javascript pitfalls (see #4 above). Node supports Coffeescript natively, but unfortunately, transpiling from Coffeescript means you lose a debugger (and sometimes line numbers).

You also lose the benefit of going with a “lowest common denominator” language (everyone knows Javascript, C, or Java). And, you still need to know javascript deeply (#4). Worse, it can create division on your team. Make sure if you switch to coffeescript, you switch all the way.

Thanks to Scott Nonnenberg, author of thehelp-cluster (a solution to #1) for feedback on this post.

Please leave a comment and tell me: what problems are you having with Node.js?

 

 

Write a Comment

Comment

  1. +1 on Coffeescript. Even if you end up going back, understanding exactly how/why it gets translated into specific javascript will increase your understanding of a lot of tricky issues in js.

  2. Haven’t used Node because I haven’t identified a situation where the benefits of non-blocking I/O are worth the overhead of learning Node. Not sure why you would use it for any other reason.

    Coffeescript is awesome, would love anything that abstracts away javascript and I’m curious to see more like this.

    Put my eggs into the Angularjs basket and so far it’s a mixed bag. GOOD: Unit testing and E2E testing are very good, I like bower, and grunt, html directives are cool once you get the hang of them. BAD: Steep and long learning curve, could use more structure and conventions, jury is still out about whether the user experience of an SPA (Single Page App) justifies the 3 -5X dev time hit you are going to take over just doing a plain ole html app. Oh and then there is this Angularjs 2.0 ticking time bomb…

  3. Same here. Haven’t used node in production. Only to support building client side AngularJS apps I agree with the pros/cons of angular, but embrace SPAs especially since more and more they just consume a wide range of JSON services and do a fairly good job of it.

  4. I’ve worked out 3 broad scenarios in my head about where we are right now with regards to SPAs/Traditional app.

    1) First scenario is a traditional server side app using jquery for ajax/and UI fanciness,and I have to concede that this is still the optimum choice for the vast majority of web apps considering time to market and user experience as constraints, so in this case you are sacrificing user experience for time to market. Also I think there are a wide swath of sites for which SPA is just totally unnecessary, like I don’t think Craiglist would be better if it was redone as an SPA.

    2) Second scenario is SPA consuming JSON services, here you get the benefits of a more desktop or native like user experience so the decision to go this route really hinges on whether or not you need a very responsive application user experience where jQuery just wouldn’t be enough. In my particular case I’m building a web version of a native app so I decided SPA was the way to go. Is there any other reason to use an SPA besides better user experience?

    3) Third scenario is hybrid client/server-side app where some parts are done as SPA and other parts are done as server side with the two different pieces redirecting to one another. I haven’t tried this yet and I’m still working it out in my head, and it means deciding when something is better off done server side and when it should be done client side. So for instance if in a part of the app, I’m just serving up static html, wouldn’t it be better to do this server side only? BTW What SPA frameworks do you like?

  5. Single Page Application, it refers to the type of app that passes data back and forth to a server via async calls (generally JSON), pages tend to not be refreshed but rather repainted as data or states are changed, state is managed separately on the client, and it’s this last thing that is really different from a dev point of view.

  6. I find that SPA is a better split of concerns. Server side is basic data management with any cross user / organization logic. Since its simpler, it can be written in safer languages like golang (my new fave) or boring java. It basically just marshals json around from various data providers. The SPA can then be completely removed from data storage concerns, as it only consumes services. This lets things evolve more naturally in my experience, and makes it hard for the front end to blow up the back end (like making horrendous SQL calls that destroy the db etc). It also lets us use better distributed data stores like cassandra or partition out more traditional RDBMS in a transparent way to the client. As for frameworks, I worked mostly with Angular building my own stuff on top. I used backbone and knockout a fair amount as well. I haven’t tried any of the higher level ones like meteor etc.

  7. It certainly is a thing to work with many dependencies but Javascript is flexible enough to write yours. As someone commented earlier, some of the things are not related to just Node.js. A couple of gotchas here would be resolved by following JS patterns. By the way I would recommed you to look at PM2 Instead of Forever or others. If you use something like React you can get the benefit of isomorphic apps but should always understand well whatever you want to use before take a decision.