debuggable

The last two days at JSConf.eu can only be described as epic. Thanks to holger, jan and malte the whole event was perfectly planned and executed.

Node was in the air and managed to infiltrate a big part of the conference. But I also got a kick out of some of the fantastic frontend talks.

Microsoft was hard at work trying to convince developers that IE9 is going to be the real deal, but Douglas Crockford seemed skeptical, getting into a bit of a fight with one of the Microsoft representative during his talk. The efforts they are putting into making canvas fast seem good, but I'm skeptical if they can finally deliver an all-around decent browser.

Palm was showing of their new node-powered WebOS stuff, and if all works out I'll soon have a device which I'm looking forward to play with.

Ryan gave a presentation on a few of the problems the node projects needs to overcome in the near future, and Jed pulled of one of the best presentations I've seen at a conference so far - I'm very much looking forward to playing with his new version of fab.

I gave a talk about dirty, which is a tiny key-value store I've written in node.js, basically to test the waters for writing more serious databases with node. If you're interested in it, you can checkout the slides or download them directly.

Another fantastic part of the conference was getting the chance to meet some fantastic new people such as:

• Bryan Cantrill - He works with ryan at joyent and is one of the dtrace authors
• Philip Hofstetter - He is using node-paperboy to run tempalias.com
• Erik Corry - A member of the v8 team
• Mark Mayo - The node hosting stuff they do at Joyent is really fantastic
• Nicole Sullivan - She does object oriented css and is working on a cool CSS lint tool
• Tobias Schneider - The crazy guy who ports flash to JS
• Blaine Cook - Former twitter hacker and co-creator of OAuth

My brain is still somewhat damaged from partying until 5am last night, so the above list is missing a bunch of other people I really enjoyed meeting as well.

And of course we were also spreading plenty of transloadit propaganda, resulting in tons of good feedback - some people really seemed to dig our little project : ).

Anyway, if you weren't here - you should really keep your eyes open for the next JSConf.us which, depending on the volcano situation, I'll hopefully be at as well!

--fg

PS: Please let me know if you have any questions about dirty, I'd be happy to answer them.

This was a fantastic weekend for Transloadit. Kevin came from Amsterdam to Berlin to join Tim and myself for two days of intense hacking on our startup.

The main user-facing result is our new pricing model.

You can now use transloadit without any fixed costs. In fact, if you sign up right now you even get $5 of free credit to get started.

And there is even more good news. While our previous usage fees were ranging from $5.00 to $3.00, we have now lowered our pricing to range from $4.00 to $1.80.

We hope that those two changes will make Transloadit much more attractive for small projects that like to avoid fixed costs, as well as larger applications that need lower pricing on high usage.

Existing customers are affected by this right away, expect your next bill to be significantly lower.

When we launched Transloadit initially, we were hesitant to commit to pure pay-as-you-go pricing. After all, it leaves a lot of money on the table for small CMS systems that may never exceed $1 of usage / month.

But at the same time, we believe even more strongly in listening to our customers. We talked to many of our users, as well as surveyed close to 100 people about our service. 85% said that a pay-as-you-go would make a big difference in their decision to purchase.

And this is not the end of things. We're still working on many exciting new features that we hope to introduce soon. One of them might even be an industry-first : ).

We are also still hungry for more feedback, so if you'd like to share your thoughts, let us know - we'll listen!

--fg

I just ran into a really nasty bug related to node's process.on('uncaughtException') handler and thought I should share my experience.

Basically you need to be very careful with handling uncaught exceptions in node. Ideally you should just send out an error email and perform a graceful restart of your service whenever you hit an uncaught exception.

Why? Well, if you think you can use "uncaughtException" to just resume your service as if nothing had happened, you may be in for a nice suprise. Consider the following example:

Unless you are very familar with the inner workings of fs.ReadStream, the assumed output sequence would be:

data event fired
{ message: 'oh no', stack: [Getter/Setter] }  
end event fired

However, the actual output is:

data event fired
{ message: 'oh no', stack: [Getter/Setter] }

You might already be able to guess what happens. The exception caused in the callback to the 'data' event causes node to trigger the "uncaughtException" event, and drop the whole call stack after that. Unfortunately however, "fs.ReadStream" had plans to execute more code after firing this "data" callback, which doesn't get executed anymore, causing the "end" event to never fire.

This is a fairly harmless example. The bug I actually ran into caused my database queries to get out of order, so I was looking at the wrong end of my code base for quite a while.

So, unless you really know what you are doing, you should perform a graceful restart of your service after receiving an "uncaughtException" exception event. Otherwise you risk the state of your application, or that of 3rd party libraries to become inconsistent, leading to all kinds of crazy bugs.

Let me know if you have any questions, or other suggestions for dealing with uncaught exceptions!

--fg

PS: This bug was especially painful for, since I'm also to blame for "uncaughtException" being in the node core to begin with : ).

With JSConf.eu coming closer, I slowly have to start preparing my talk, which mostly means hacking on node-dirty.

I have a few goals for the project. My main interest is challenging a few assumptions people have about the performance and complexity of database systems. Most of that is material for another post and my talk itself, but today I'd like to talk about hidden classes in v8.

One of the things that is really fast in v8 is property lookups. This is due to an optimization that creates hidden classes for an object.

I could go into a lengthy explanation of how that works, but instead I'll invite you to see for yourself.

Consider the following two examples and guess which runs faster, and by how much:

Example 1:

Example 2:

If you have guessed example 2, congratulations! Bonus points if you have also guessed that example 2 is a nifty 10x faster than example 1.

For those familiar with v8, you probably already know what is going. For those who don't, let me explain.

In example 1, every time you are setting a property on the o object, v8 is creating a new hidden class that defines the "data structure" of the o object to optimize property lookup performance.

In example 2, we are initializing these "hidden classes" the first time we create our o object. So when we are overwriting these properties later on, it is blazing fast, because v8 already knows how to efficiently lookup those properties.

So if you're writing node.js code, the lesson learned here is that it is much faster to work with existing properties in v8, than to add new ones.

In my next version of dirty I am planning to take advantage of this behavior by pre-allocating properties before they are actually used. This will probably require a little trickery to map user-defined keys to pre-allocated properties, but it should result in an overall 10x performance boost for setting new keys.

Let me know what you think / if you have other clever v8 hacks to speed stuff up : ).

--fg

PS: You can read more about hidden classes in the v8 docs.

Sorry for the lack of post, some stuff came up this week, and the hour / day to write this series was needed to deal with it. Anyway, it's overall good news, so I hope to continue this series either next week or the week after.

--fg