Peterbe.com

The advantage with WebSockets (over AJAX) is basically that there's less HTTP overhead. Once the connection has been established, all future message passing is over a socket rather than new HTTP request/response calls. So, you'd assume that WebSockets can send and receive much more messages per unit time. Turns out that that's true. But there's a very bitter reality once you add latency into the mix.

So, I created a simple app that uses SockJS and an app that uses jQuery AJAX to see how they would perform under stress. Code is here. All it does is basically, send a simple data structure to the server which echos it back. As soon as the response comes back, it starts over. Over and over till it's done X number of iterations.

Here's the output when I ran this on localhost here on my laptop:

# /ajaxtest (localhost)
start!
Finished
10 iterations in 0.128 seconds meaning 78.125 messages/second
start!
Finished
100 iterations in 0.335 seconds meaning 298.507 messages/second
start!
Finished
1000 iterations in 2.934 seconds meaning 340.832 messages/second

# /socktest (localhost)
Finished
10 iterations in 0.071 seconds meaning 140.845 messages/second
start!
Finished
100 iterations in 0.071 seconds meaning 1408.451 messages/second
start!
Finished
1000 iterations in 0.466 seconds meaning 2145.923 messages/second

Wow! It's so fast that the rate doesn't even settle down. Back-of-an-envelope calculation tells me the WebSocket version is 5 times faster roughly. Again; wow!

Now reality kicks in! It's obviously unrealistic to test against localhost because it doesn't take latency into account. I.e. it doesn't take into account the long distance the data has to travel from the client to the server.

So, I deployed this test application on my server in London, England and hit it from my Firefox here in California, USA. Same number of iterations and I ran it a number of times to make sure I don't get hit by sporadic hickups on the line. Here are the results:

# /ajaxtest (sockshootout.peterbe.com)
start!
Finished
10 iterations in 2.241 seconds meaning 4.462 messages/second
start!
Finished
100 iterations in 28.006 seconds meaning 3.571 messages/second
start!
Finished
1000 iterations in 263.785 seconds meaning 3.791 messages/second

# /socktest (sockshootout.peterbe.com) 
start!
Finished
10 iterations in 5.705 seconds meaning 1.752 messages/second
start!
Finished
100 iterations in 23.283 seconds meaning 4.295 messages/second
start!
Finished
1000 iterations in 227.728 seconds meaning 4.391 messages/second

Hmm... Not so cool. WebSockets are still slightly faster but the difference is negligable. WebSockets are roughly 10-20% faster than AJAX. With that small a difference I'm sure the benchmark is going to vastly effected by other factors that make it unfair for one or the the other such as quirks in my particular browser or the slightest hickup on the line.

What can we learn from this? Well, latency kills all the fun. Also, it means that you don't necessarily need to re-write your already working AJAX heavy app just to gain speed because even though it's ever so slightly faster, the switch from AJAX to WebSocket comes with other risks and challenges such as authentication cookies, having to deal with channel concurrency, load balancing on the server etc.

Before you say it, yes I'm aware than WebSocket web apps comes with other advantages such as being able to hold on to sockets and push data at will from the server. Those are juicy benefits but massive performance boosts ain't one.

Also, I bet that writing this means that peeps will come along and punch hole in my code and my argument. Something I welcome with open arms!

DoneCal users are to 80+% Chrome and Firefox users. Both Firefox and Chrome support the HTML <audio> element without any weird plugins and they both support the Ogg Vorbis (.ogg) file format. change log here

So, I used use the rather enterprisey plugin called SoundManager2 which attempts to abstract away all hacks into one single API. It uses a mix of browser sniffing, HTML5 and Flash. Although very promising, it is quite cumbersome. It doesn't work flawlessly despite their hard efforts. Unfortunately, using it also means a 30kb (optimized) Javascript file and a 3kb .swf file (if needed). So, instead of worrying about my very few Internet Explorer users I decided to go really dumb and simple on this.

The solution basically looks like this:

// somewhere.js
var SOUND_URLS = {
  foo: 'path/to/foo.ogg',
  egg: 'path/to/egg.ogg'
};

// play-sounds.js

/* Call to create and partially download the audo element.
 * You can all this as much as you like. */
function preload_sound(key) {
 var id = 'sound-' + key;
 if (!document.getElementById(id)) {
   if (!SOUND_URLS[key]) {
     throw "Sound for '" + key + "' not defined";
   } else if (SOUND_URLS[key].search(/\.ogg/i) == -1) {
     throw "Sound for '" + key + "' must be .ogg URL";
   }
   var a = document.createElement('audio');
   a.setAttribute('id', id);
   a.setAttribute('src', SOUND_URLS[key]);
   document.body.appendChild(a);
 }
 return id;
}

function play_sound(key) {
  document.getElementById(preload_sound(key)).play();
}

// elsewhere.js
$.lightbox.open({
   onComplete: function() {
      preload_sound('foo');
   }
});
$('#lightbox button').click(function() {
   play_sound('foo');
});

Basically, only Firefox, Chrome and Opera support .ogg but it's a good and open source encoding so I don't mind being a bit of an asshole about it. This little script could be slightly extended with some browser sniffing to work with Safari people but right now it doesn't feel like it's worth the effort.

This make me happy and I feel lean and light. A good feeling!

This is a piece of Javascript code I use on "Kwissle" to make the document title change temporarily. Other people might find it useful too.

Code looks like this:

var Title = (function() {
 var current_title = document.title
   , timer;

 return {
    showTemporarily: function (msg, msec) {
      msec = typeof(msec) !== 'undefined' ? msec : 3000;
      if (msec < 100) msec *= 1000;
      if (timer) {
        clearTimeout(timer);
      }
      document.title = msg;
      timer = setTimeout(function() {
        document.title = current_title;
      }, msec);
    }
 }
})();

Demo here

On Kwissle I'm using Google Closure Compiler to minify all Javascript files. It's fine but because it's java and because I'm running this on a struggling EC2 micro instance the CPU goes up to 99% for about 10 seconds when it does the closure compilation. Sucks!

So, I threw UglifyJS into the mix and instead of replacing the Closure compiler I added it so it runs alongside but I obviously only keep one of the outputs.

Here is the log output when I run it on my MacbookPro:

MAKING ./static/js/account.js
UglifyJS took 0.0866 seconds to compress 3066 bytes into 1304 (42.5%)
Closure took 1.2365 seconds to compress 3066 bytes into 1225 (40.0%)
MAKING ./static/js/ext/jquery.cookie.js
UglifyJS took 0.0843 seconds to compress 3655 bytes into 3009 (82.3%)
Closure took 1.3472 seconds to compress 3655 bytes into 4086 (111.8%)
MAKING ./static/js/ext/jquery.tipsy.js
UglifyJS took 0.1029 seconds to compress 7527 bytes into 3581 (47.6%)
Closure took 1.3062 seconds to compress 7527 bytes into 3425 (45.5%)
MAKING ./static/js/maxlength_countdown.js
UglifyJS took 0.082 seconds to compress 1502 bytes into 1033 (68.8%)
Closure took 1.2159 seconds to compress 1502 bytes into 853 (56.8%)
MAKING ./static/js/ext/socket.io-0.6.3.js
UglifyJS took 0.299 seconds to compress 76870 bytes into 30787 (40.1%)
Closure took 2.4817 seconds to compress 76870 bytes into 30628 (39.8%)
MAKING ./static/js/scoreboard.js
UglifyJS took 0.084 seconds to compress 2768 bytes into 1239 (44.8%)
Closure took 1.2512 seconds to compress 2768 bytes into 1167 (42.2%)
MAKING ./static/js/rumbler.js
UglifyJS took 0.0872 seconds to compress 3087 bytes into 1384 (44.8%)
Closure took 1.2587 seconds to compress 3087 bytes into 1235 (40.0%)
MAKING ./static/js/ext/shortcut.js
UglifyJS took 0.0987 seconds to compress 5796 bytes into 2537 (43.8%)
Closure took 1.3231 seconds to compress 5796 bytes into 2410 (41.6%)
MAKING ./static/js/play.js
UglifyJS took 0.1483 seconds to compress 18473 bytes into 10592 (57.3%)
Closure took 1.4497 seconds to compress 18473 bytes into 10703 (57.9%)
MAKING ./static/js/playsound.js
UglifyJS took 0.0824 seconds to compress 1205 bytes into 869 (72.1%)
Closure took 1.2335 seconds to compress 1205 bytes into 873 (72.4%)  

(Note here that for the file ./static/js/ext/jquery.cookie.js Closure failed and when it fails it leaves the code as is and prepends it with a copy about the error from the stdout. that's why it's greater than 100% on that file)

Here are the averages of those numbers:

AVERAGE TIME: (lower is better)
  * UglifyJS: 0.11554 seconds
  * Closure: 1.41037 seconds

AVERAGE REDUCTION: (higher is better)
  * UglifyJS: 45.6% 
  * Closure: 51.5%

(I'm skipping the file that Closure failed to minify)

So, what does that mean in bytes? These are the source Javascript files for two pages but the total is 123949.0 bytes. With Closure that saves me 63833.7 bytes (62 kbytes) whereas UglifyJS only saves me 57760.2 bytes (56 kbytes) of bandwidth.

Discussion...

The fact that Closure fails on one file is a real bummer. I'm not even using the advanced options here.

UglifyJS doesn't save as many bytes as Closure does. This is potentially important because after all, minification process happens only once per revision of the original file but might be served hundreds or millions of times.

Because I run my minifications on-the-fly it does matter to me that UglifyJS is 1220% faster.

It's rare but I have observed twice that the minified Javascript from Closure has actually broken my code. I default to suspect that's my fault for not making the code "minifyable" enough (e.g. too many global variables).

I've just noticed that I'm relying on files that almost never change (e.g. jquery.tipsy.js). I might as well create a ...min.js versions of them and add them to the repository.

In conclusion...

Because of the convenience of UglifyJS being so much faster and that it doesn't choke on that jquery.cookie.js file I'm going switch to UglifyJS for the moment. The remaining bytes that I don't save become insignificant if you add the gzip effect and compared to images the bandwidth total is quite insignificant.

UPDATE (JAN 2016)

I wrote a follow-up post comparing UglifyJS2 with Closure Compiler with the advanced option.

If you look at html5boilerplate.com they never use any type on their <script> tags. Hmm... but is there more to it?

If you don't specify a type, the default becomes "text/javascript" but according to RFC4329 the "text/javascript" MIME type is obsolete in favor of "application/javascript".

If the default MIME type for a <script> tag thus becomes either "text/javascript" or "application/javascript" is there any sensible browser on this green earth that would not translate a piece of inline Javascript code as, exactly that; Javascript? Probably not.

What about <script> tags with a src attribute? Does the type matter?

I read the spec a couple of times and it feels like reading legalese but it ultimately says: the value of the type tag must be that of the body of the script tag.

So, what happens if you embed a javascript file with a mismatching type? Let's see:

If people find this useful I might turn it into a proper jQuery plugin and publish it.

Without further ado; here's the demo

What it does is that for all input fields that have a maxlength="nnn" it shows a counter to the right that increases in opacity as it reaches the maximum. You can generally start it like this:

$('input[maxlength]').maxlength_countdown();

Since the plugin "hard codes" the count down expression in English you can override it easily like this:

$('input[name="message"]').maxlength_countdown(function (count, max) {
   return count + " (max: " + max + ")";
});

What do you think? Is it useful? Does it make sense?

Because this took me a long time to figure out I thought I'd share it with people in case other people get stuck on the same problem.

The problem is that Mongoose doesn't support DBRefs. A DBRef is just a little sub structure with a two keys: $ref and $id where $id is an ObjectId instance. Here's what it might look like on the mongodb shell:

> db.questions.findOne();
{
       "_id" : ObjectId("4d64322a6da68156b8000001"),
       "author" : {
               "$ref" : "users",
               "$id" : ObjectId("4d584fb86da681668b000000")
       },
       "text" : "Foo?",
       ...
       "answer" : "Bar"
       "genre" : {
               "$ref" : "question_genres",
               "$id" : ObjectId("4d64322a6da68156b8000000")
       }
}

DBRefs are very convenient because various wrappers on drivers can do automatic cross-fetching based on this. For example, with MongoKit I can do this:

for question in db.Question.find():
   print question.author.first_name

If we didn't have DBRefs you'd have to do this:

for question in db.Question.find():
   author = db.Authors.findOne({'_id': question.author})
   print author.first_name

Yay! I've figured out how to properly unit tests my jQuery Mobile app that I'm working on. This app uses localStorage, localSession, lots of AJAX and works both offline and online. Through various hacks and tricks I've managed to mock things so that I can easily test the otherwise asynchronous AJAX calls and I can also test the little quirks of jQuery Mobile such as re-rendering <ul> tags after having changed the DOM tree.

I'm using the QUnit testing framework and I like it. The app isn't launched yet and the code is currently protected but once I get it nailed a bit more I'll blog about it more fully so other people can jump in unit testing their Javascript heavy jQuery Mobile sites too. For now I'm up to 75 tests and it's growing steadily.

Here's a little taster for how I mock the $.mobile, AJAX and the localStorage stuff:

module("Storage and AJAX", {
  setup: function() {
     localStorage.clear();
  },
  teardown: function() {
     localStorage.clear();
  }
});

var MockMobile = function() {
  this.current_page;
};
MockMobile.prototype.changePage = function(location) {
  this.current_page = location;
};

test("test Auth", function() {
  $.mobile = new MockMobile();
  var _last_alert;
  alert = function(msg) {
     _last_alert = msg;
  };
  $.ajax = function(options) {
     switch (options.url) {
      case '/smartphone/checkguid/':
        options.success({ok:true});
        break;
      case '/smartphone/auth/login/':
        if (options.data.password == 'secret')
          options.success({guid:'10001'});
        else
          options.success({error:"Wrong credentials"});
        break;
      default:
        console.log(options.url);
        throw new Error("Mock not prepared (" + options.url + ")");
     }
  };
  var result = Auth.is_logged_in(false);
  equals(result, false);

  Auth.ajax_login('peterbe@example.com', 'other junk');
  ok(_last_alert);
  ...

(Note: this is copied slightly out of context but hopefully it reveals some of the hacks and tricks I use)

What a lovely title for a book! I wanted to read a book about the proper way to write Javascript but I couldn't wait any longer for John Resig's Secrets of the JavaScript Ninja which isn't out in print yet (also a great title by the way).

Eloquent Javascript begins very lightly with the basics of Javascript programming. Variables, scope, data structures and control flow. To be perfectly honest I didn't read it very carefully but I believe I did pick up a thing or two at least. The chapter on error handling was useful but the really interesting chapters were "Functional Programming" and "Object-Oriented Programming". What I love about Marijn's style of writing is that he starts very very simple and builds up the code to be better and better. Not based on what you can do but instead why you should do it. As you go along you can then immediately snap up what the benefits are for yourself. Sometimes it's brevity and sometimes it's for faster performance. My only criticism if I'm allowed is that the jargon is quite a lot keep up with. Especially around "constructors" and "prototypes" which is sometimes easy to forget (especially if you're from another language where these things mean different things).

I'm not great at it but I already knew how to write modules and "classes" so ultimately there wasn't a whole lot to take away from it to be honest. Some tricks such as the inheritance function which Marijn introduced was neat and that might be something I'll copy. Nevertheless, this book showed and educated me in why we do things as modules and stuff which I genuinely appreciated.

Thanks for a great book Marijn! Keep up the good work!

I needed a very basic spell correction string matcher in my current NodeJS project so I wrote a simple class called EditDistanceMatcher that compares a string against another string and matches if it's 1 edit distance away. With it you can do things like Google search's "Did you mean: poop?" when you search for pop.

Note, this code doesn't check popularity of correct words (e.g. "pop" might appear much more often than "poop" so it'll suggest "pop" if you enter "poup"). Anyway this simple snippet from the unit tests will reveal how it works:

     /* The match() method */
     var edm = new EditDistanceMatcher(["peter"]);
     // edm.match returns an array and remember,
     // in javascript ['peter'] == ['peter'] => false
     test.equal(edm.match("petter").length, 1);
     test.equal(edm.match("petter")[0], 'peter');
     test.equal(edm.match("junk").length, 0);

     /* the is_matched() method */
     var edm = new EditDistanceMatcher(["peter"]);
     test.equal(typeof edm.is_matched('petter'), 'boolean');
     test.equal(typeof edm.is_matched('junk'), 'boolean');
     test.ok(edm.is_matched("petter"));
     test.ok(!edm.is_matched("junk"));

The most basic use case is if you have a quiz and you want to accept some spelling mistakes. "What's the capital of Sweden?; STOKHOLM; Correct!"

For the unlazy this NodeJS code can very easily be used in a browser by simply removing the exports stuff.

edit_distance.js

tests/test_edit_distance.js

Note! I wrote this in an airport lounge so I'm sure it can be improved lots more.