Peterbe.com

I've spent a lot of time trying to figure out which Javascript script loading framework to use. RequireJS or HeadJS. I still don't have an answer. Neither website refers to each other.

In general

• To me, it's important to be able to load and execute some Javascript before downloading Javascript modules that aren't needed to render the initial screen. Makes for a more responsive behaviour and gets pixels drawn quicker for Javascript-heavy sites.
• An understated, massive, benefit to combining multiple .js files into one is sporadic network bottlenecks. Fewer files to download and fewer things can go wrong. These bottlenecks can make a few Kb of a Javascript file take 10 seconds to download.
• Public CDNs (e.g. jQuery from Google's CDN) is an extremely powerful optimization technique. Not only are they extremely fast, it's very likely they're preloaded because some other site uses the exact same URL.
• Where does it say that Javascript has to be loaded in the head? Even html5-boilerplate loads Javascript just before the </body> tag.
• Realistically, in the real world, it's not uncommon that you can't combine all .js files into one. This is not true for web apps that consists of just one HTML file. One page might require A.js, B.js and C.js but another page requires A.js, B.js and D.js. Requires manual thinking whether you should combine A,B,C,D.js or A,B.js + C|D.js. No framework can predict this.
• All loading and browser incompatibility hacks will eventually become obsolete as browsers catch up. Again, requires manual thinking because supporting and ultra-boosting performance might have a different cost today compared to a year from now. The most guilty of this appears to be ControlJS
• I'm confident that optimization in terms of file concatenation and white space optimization does not belong to the framework.
• Apparently iPhone 3.x series can't cache individual files larger than 15Kb (25Kb for iPhone 4.x). That's a very small about if you combine several large modules.
• Accepting the fact of life that sporadic network bottlenecks can kill your page, think hard about asynchronous loading and preserved order. Perhaps ideal is a mix of both. What framework allows that? (both RequireJS and HeadJS it seems)
• Loading frameworks are not for everything and everyone. If you're building something "simple" or landing page like Google's search page frameworks might just get in your way.

RequireJS

• Author well known but his Dojoesque style shines through in RequireJS's syntax and patterns.
• Is only about Javascript. No CSS hacks or other html5ish boilerplates.
• Gets into the realm of module definitions. Neat but do you want the loading framework to get involved in how you prefer to write your code or do you just want it to load your files?
• All the module definition stuff feels excessive for every single project I can imagine but we're entering an era of "web apps" (as opposed to "web sites") so this might need to change.
• What you learn in using RequireJS you can reuse when building NodeJS (a server-side framework). It's also possible to use RequireJS in Rhino (server-side Javascript engine) but personally I haven't reached that level yet.

HeadJS

• Author relatively unknown. quite well known too. Author also of Flowplayer and jQuery Tools.
• Contains a kitchen sink (CSS tricks, modernizer.js) but perhaps they're really quite useful. After all, you don't write your web site in Assembly.
• There's a fork of HeadJS that does just the Javascript stuff. But will it be maintained? And does that defeat the whole point of using HeadJS?
• With its CSS hacks (aka. kitchen sink) HeadJS seems great if you really care about combining HTML5 techniques with Internet Explorer.
• This awesome experiment shows that HeadJS really works and that asynchronous loading can be really powerful. But ask yourself, are you ready to build in an asynchronous way?
• With HeadJS I can label a combined and optimized bundle and load my code once that bundle is loaded. Can I do that with RequireJS? It seems to depend on the filename (minus the .js suffix).
• Makes the assumption that just because a file is loaded the order of execution is a non-issue. This means you might have trouble controlling dependencies during execution. This is a grey area that might or might not matter depending on the complexity of your app.
• A feeling I get is that HeadJS without the CSS kitchen sink stuff reduces to become LabJS or EnhanceJS.

Other alternatives

The ones I can think of are: ControlJS (feels too "hacky" for my taste), CommonJS (not sufficiently "in-browser specific" for my taste) and EnhanceJS (like HeadJS and LabJS but with less power/features)

The one I haven't studied as much is LabJS. It seems more similar to HeadJS in style. Perhaps it deserves more attention but the reason HeadJS got my attention is because it's got a better looking website.

In conclusion

You mileage will vary. The deeper I look into this I feel personal taste comes into play. It's hard enough for a single framework other to write realistic benchmarks; even harder for "evalutators" like myself to benchmark them all. It gets incrementally harder when you take into account the effects of http latency, sporadic network bottlenecks, browser garbage collection and user experience.

Personally I think HeadJS is a smoother transition for general web sites. RequireJS might be more appropriate when write web apps with virtually no HTML and a single URL.

With the risk of starting a war... If you're a Rails/Django/Plone head, consider HeadJS. If you're a mobile web app/NodeJS head consider RequireJS.

UPDATE

Sorry, I now realise that Tero Piirainen actually has built a fair amount of powerful Javascript libraries.

This feels like a bit of a face-plant moment but I've never understood why anyone would use the code module when you can use the pdb when the pdb is like the code module but with less.

What you use it for is to create you own custom shell. Django does this nicely with it's shell management command. I often find myself doing this:

$ python
Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from this import that
>>> from module import something
>>> from db import connection
>>> con = connection('0.0.0.0', bla=True)
>>> con.insert(something(that))

And there's certain things I almost always import (depending on the project). So use code to write your own little custom shell loader that imports all the stuff you need. Here's one I wrote real quick. Ultra useful time-saver:

#!/usr/bin/env python
import code, re
if __name__ == '__main__':
   from apps.main.models import *
   from mongokit import Connection
   from pymongo.objectid import InvalidId, ObjectId
   con = Connection()
   db = con.worklog
   print "AVAILABLE:"
   print '\n'.join(['\t%s'%x for x in locals().keys()
                    if re.findall('[A-Z]\w+|db|con', x)])
   print "Database available as 'db'"
   code.interact(local=locals())

This is working really well for me and saving me lots of time. Hopefully someone else finds it useful.

After about two months of evening hacking I'm finally ready to release my latest project: DoneCal.com

It's a simple calendar that doesn't get in your way. You just click on a day and type what you did that day. DoneCal can be an ideal replacement to boring spreadsheet-like timesheets. And unlike regular timesheets/timetrackers with tags you immediately get statistics about how you've spent your time.

I'm personally excited about the Bookmarklet because I practically live in my webbrowser and now I can quickly type what I've just done (could be a piece of support work for a client) with one single click.

If you're a project manager trying to track what your developers are working on, ask them to start tracking time on DoneCal and then ask them to share their calendar with you. They can set up their share so that it only shares on relevant tags.

I'm going to improving it more and more as feedback comes in. Hopefully later this week I'm going to be writing about the technical side of this since this is my first web app built with the uber-fast Tornado framework

This isn't rocket science but it might help someone else.

I often do testing of my various restful HTTP APIs on the command line with curl but often the format the server spits out is very compact and not easy to read. So I pipe it to a little script I've written. Used like this:

$ curl http://worklog/api/events.json?u=1234 | jsonpprint
{'events': [{'allDay': True,
            'end': 1290211200.0,
            'id': '4ce6a2096da6814e5b000000',
            'start': 1290211200.0,
            'title': '@DoneCal test sample'},
           {'allDay': True,
            'end': 1290729600.0,
            'id': '4ce6a22b6da6814e5b000001',
...

A classic blunder in Javascript client side apps is heavy use of the incredibly useful Firebug feature that is console.log() and then forgetting to remove any such debugging and thus causing Javascript errors for people without Firebug. To remedy this use this little nifty function:

function log() {
  if (window.console && window.console.log)
  for (var i = 0, l = arguments.length; i < l; i++)
    console.log(arguments[i]);
}

That way you can do plenty of debugging and if you accidentally leave a logging line in your code it won't break anything even if they don't have Firebug installed/enabled. Also you can easily "annotate" your debugging by writing two things in one line. Like this:

function foo(bar) {
   log("bar is:", bar);
   return bar * 2;
}

MongoKit is a Python wrapper on top of pymongo that adds structure and validation and some other bits and pieces. It's like an ORM but not for an SQL database but for a document store like MongoDB. It's a great piece of code because it's thin. It's very careful not to molly cuddle you and your access to the source. What I discovered was that I was doing an advanced query and with the results they we instantiated as class instances and later turned into JSON for the HTTP response. Bad idea. I don't need them to be objects really so with MongoKit it's possible to go straight to the source and that's what I did.

With few very simple changes I managed to make my restful API app almost 10 times faster!!

Read the whole story here

I was happily using Jörn Zaefferer's jQuery autocomplete plugin but then I found out that it had been deprecated and I should instead use the jQuery UI's autocomplete plugin instead.

After "upgrading" I started getting suspicious. Is it actually worse than the old one?

I had a piece of code that looked like this:

function add_to_form(f, all_day) {
  console.log(all_day);
  if (all_day)
    $('input[name="all_day"]', f).val('1');
  else;
    $('input[name="all_day"]', f).val('');
  console.log($('input[name="all_day"]', f).val(''));
  return f; 
}

When I ran it, the console output was this:

true
(an empty string)

What had happened was that I had accidentally put a semi-colon after the else statement. Accidentally as in stumbled on the keyboard. I didn't spot it because semi-colons are so common in JavaScript that you sort of go blind to them.

The wart was that it didn't cause a syntax error. IMHO it should have because you'd expect there to always be something happening after the else.

So instead of using the shortcut notation for if statements I've decided to write it out in full instead:

function add_to_form(f, all_day) {
  if (all_day) {
     $('input[name="all_day"]', f).val('1');
  } else {
     $('input[name="all_day"]', f).val('');
  }
  return f; 
}

Optimizers like Google Closure will do a much better job optimizing the code than I ever will anyway.

I've been doing more and more web development with Tornado recently. It's got an awesome class for running client HTTP calls in your integration tests. To run a normal GET it looks something like this:

from tornado.testing import AsyncHTTPTestCase
class ApplicationTestCase(AsyncHTTPTestCase):
   def get_app(self):
       return app.Application(database_name='test', xsrf_cookies=False)

   def test_homepage(self):
       url = '/'
       self.http_client.fetch(self.get_url(url), self.stop)
       response = self.wait()
       self.assertTrue('Click here to login' in response.body)

Now, to run a POST request you can use the same client. It looks something like this:

   def test_post_entry(self):
       url = '/entries'
       data = dict(comment='Test comment')
       from urllib import urlencode
       self.http_client.fetch(self.get_url(url), self.stop, 
                              method="POST",
                              data=urlencode(data))
       response = self.wait()
       self.assertEqual(response.code, 302)

And by "dangerous" I mean super slow to the point of making your browser shake of over exhaustion.

I have a big fat table where on the left hand side of each row there's a little toggle to open up an initially hidden sub-table. And there are toggles for those sub-tables to open up further sub-tables. It might sound complicated but it works great. The code for each toggle looks something like this:

$('a.toggle-order-on').click(function() {
   var tbody_parent = $(this).parents('tbody');

   $('tr.printdisplay:hidden', tbody_parent).show();
   $('tr.outdoor-marketing:hidden', tbody_parent).show();
   $('tr.digital:hidden', tbody_parent).show();

   // expand the little table too
   $('a.toggle-printdisplay-on', tbody_parent).click();

   var td_parent = $(this).parents('td');
   $(this).hide();
   $('a.toggle-order-off:hidden', td_parent).show();

   return false;
});

Note the heavy use of the super useful :hidden selector which is basically a reversing wrapper on the :visisble selector

What I then needed was a way to open up every single row in the whole table with one click. Here was the code I wrote:

$('a.toggle-order-on:visible').trigger('click');

See? Makes sense does it?

Problem with this was that when the table was big sometimes clicking this would make my otherwise fast browser (Chrome or Firefox) stutter and sometimes stall or at worst the alert pop-up about "a script is slowing this page down" would appear.

So I started the Firebug Profiler and clicked a couple of times and collected some numbers. On average it took 3-4.5 seconds!! and about 20,000-35,000 calls to complete the full expansion. Yikes! About 90% of the time spent by jQuery was on the visisble() function.

Solution: Instead of using the click trigger I simply just called the .show() effect on all things manually without using any :visible or :hidden operators. Here's the new code:

$('tr.printdisplay').show();
$('tr.outdoor-marketing').show();
$('tr.digital').show();
$('tr.printdisplaydetail').show();

$('a.toggle-order-off').show();
$('a.toggle-order-on').hide();

$('a.toggle-printdisplay-off').show();
$('a.toggle-printdisplay-on').hide();

This time, with the profile again, I it took on average 0.2-0.3 seconds and required about 2000-4000 calls. HUGE difference.

So, remember that next time. Don't just re-use working code en mass if it's using a much of :visible or :hidden selectors somewhere in there.