SprintStack is a little over 3 months old at this point. While I’ve managed to hack out something that’s satisfyingly useful (for my purposes at least!), I’ve not committed any significant changes for the last couple of weeks. Now’s probably a good time to take a step back and take a look at where it’s going and what I want it to achieve in the longer term.

A bit of background might clarify the method in my madness: when node.js entered the world, I was pretty sceptical of it as a concept. I saw the requisite glut of sycophantic HN posts, read a couple of them, wrote a few toy applications and promptly forgot about it. It’d seem that in hindsight my pessimism regarding its success was certainly misplaced; today node is cool, a virtue which brings with it omnipresence among the armies of web application developers looking to move onto whatever’s currently hot. In short, node, and by implication server-side JavaScript as a concept, ain’t going to fade back into the ether any time soon. To cement the idea that JS development on the server is more than just a cult, the CommonJS project appeared- an attempt at agreeing on a unified API for non-DOM applications that can be programmed using JavaScript. Writing as many tiers in your system as possible in a common language with the same API is undoubtedly desirable, and on that point I recognize node is a useful tool to have at your disposal.

That said, my core concerns with node.js still remain, and they really boil down to:

  • A questionable concurrency model
  • A peverse amount of wheel reinvention

OK, concurrency: node.js espouses the doctrine of the single-threaded reactor. Given an application that is heavily I/O bound and has not yet reached a baroque level of complexity* then there’s a good chance that this is a great choice. However it’s either naivety or willful ignorance to assume that this is a good fit for all scenarios (c.f. the inflamatory node is cancer diatribe). Rigidly adhering to a dogma which in essence says “no parallelism thanks, we’re JavaScripters” is a massive decision with deep consequences. It’s a glaring omission considering the rate at which CPUs are sprouting processing cores and the proliferation of languages and frameworks which do a good job of telling all those processors what to do. Whatever type of language you’re working with, there’s a reasonable liklihood that its designers are looking into adding some parallel language constructs. For me, the changes to the cluster module in v0.8 were frustrating in this respect: forking OS processes isn’t a sound way to manage concurrency in the long term and re-working the API doesn’t fix a fundamentally flawed model. Sure you can append a third-party module that might manage concerns like threading, IPC, etc, but something as fundamental as this should come as a dictation from Joyent rather than leaving a void to be filled ad-hoc by individual developers. This was the key motivation behind building SprintStack.

Wheel reinvention? I’ll qualify that somewhat: it probably won’t have passed you by that there’s a marked trend going on in PL design- namely the creation of languages that target commonplace VMs rather than relying on their own compiler / interpreter infrastructure. Typically this gives the language designer speed, platform independence and garbage collection out of the box. Thanks to V8 node has all of these attributes. There are a few problems though. First of all, this choice denies node access to a set of large, mature libraries which users of the JDK, ObjC+Cocoa, .NET e.t.c. have come to expect. It’s correct that Joyent have “drawn our boundaries” [sic], aiming for a small core API. But what happens when you need to do something that this core can’t cater for? Hopefully you could implement it in JavaScript without too much hassle. Failing that you might end up wrapping a C/C++ library and possibly stick it on npm and become famous. That’s an awful lot of work that’s almost certainly been done before though, no? In Java or C# you google a relevant implementation, download a library and carry on with what you were doing. Clojure, JRuby, F# et al sit on VMs that allow this kind of code reuse, even when the third-party library is written in a completely different language. This makes it much easier to develop a heterogeneous system consisting of numerous different languages/programming paradigms, all interoperating, without having to ‘bet the house’ on any single one. It’s a strategy which appears to be gaining some traction, but currently isn’t a luxury afforded to node.js developers. I want JS developers to be able to leverage the vast number of libraries available to the JVM so they can get on with writing applications. I also want your applications to be able to interoperate with any other language available on the Java platform. SprintStack was born out of a need to fix this, too.

So… a platform with a huge user base that’s fast, has an enormous class library, housing a large ecosystem of dynamic languages that run on [almost] any platform. Oh, and a JS interpreter. Yes, SprintStack sits on the JVM. And at the time of writing it uses Rhino, although it is not married to it in the way node is to V8. The next 12 months will see several Java-based script interpreters appear on the scene and it’s quite possible that Rhino may get the boot if I find something else that does the job better. I/O is handled for the most part by Netty and concurrency is dealt with by the fine Akka library. Like node, most of the core library functions are written in JavaScript itself.

None of this really sweetens the deal if SprintStack’s API diverges significantly from node’s. I’ve been doing a lot of thinking about how it should shape up, what its relationship with node.js should be, and so on. In the short term, compliance with the node API (at least, portions where the stability level is at least 4) should be a reasonable target. Many of the stream-based modules are at an advanced state of development despite the wildly varying differences in under-the-hood behaviour. Node’s core API is small and ultimately most of it should be plug-compatible: write your app once, run it on node.js or SprintStack without making any changes. At some point this contract will break down because of the changes I’ve explained above; then a call has to be made about what the hell SprintStack actually is. The GitHub page describes it as “Concurrent + Evented I/O for Rhino”, a corruption of node’s “evented I/O for v8 javascript”. For a start, it’s not tied to its interpreter. Secondly, is a mature SprintStack framework just a node implementation for Java? My gut feeling is that it’ll maintain a high level of compatibility for a long time to come but I can’t make any more than hand-wavy guarantees about that. It’d be possible to go off on a developmental tangent and be ‘just another’ CommonJS framework but the reality is that node.js is the elephant in the JavaScript room. As all dualist politicians love to point out, “You’re either with us or against us”. SprintStack is ‘with’ node, but it’s a long-distance relationship:)

Got any issues or suggestions? I’d love to hear them.