Mocking os.stat in Python

November 8, 2009
5 comments Python

I have some code that checks that a file is treated differently if some time has passed. In other words, it reacts if the modification time is more than it was before. The code uses os.stat(filename)[stat.ST_MTIME] to do this. The challenge was to mock os.stat so that I can pretend some time has passed without having to wait. Here was my first solution which made running many tests sloooow:


def test_file_changed(self):
    filename = 'foo.txt'
    expect_timestamp = int(time.time())
    ...run the unit test with 'expect_timestamp'...
    time.sleep(1)
    expect_timestamp += 1
    ...run the unit test with 'expect_timestamp'...

So, here's how I mock os.stat to avoid having to do the time.sleep(1) in the test:


def test_file_changed(self):
    filename = 'foo.txt'
    expect_timestamp = int(time.time())
    ...run the unit test with 'expect_timestamp'...
    import os
    from posix import stat_result
    def fake_stat(arg):
        if arg == filename:
            faked = list(orig_os_stat(arg))
            faked[stat.ST_MTIME] = faked[stat.ST_MTIME] + 1
            return stat_result(faked)
        else:
            return orig_os_stat(arg)
    orig_os_stat = os.stat
    os.stat = fake_stat

    expect_timestamp += 1
    ...run the unit test with 'expect_timestamp'...

I hope this helps someone else who's trying to do the same. It took me some time to figure out that os.stat is used by lots of various sub routines in the os module so it's important to only mock the relevant argument otherwise you might get unexpected problems.

"Frank Zappa: The Biography" by Barry Miles

October 30, 2009
7 comments Books

Available on Amazon.com I've just read Frank Zappa: The Biography by Barry Miles. It's a detailed book on 380 pages about virtually every year of Frank Zappa's life. From his parents to his death.

I'm a huge Frank Zappa fan and have been for more than a decade. It's probably the most listened to artist ever in my life in terms of number of listened to songs. Actually not probably; definitely. I adore his music and his personality and this is the second book I read about him. The other book I read was Real Frank Zappa Book written by Peter Occhiogrosso based on biography interviews with Frank for the purpose of writing this book. That book was much more bland and emphasized particularly his early political work and also very much emphasizes on his work as a orchestral conductor/business man.

The detail work in this book is really fantastic. It's thanks to Barry's in-depth understanding of music and the music industry that you get deep down to the nitty-gritty details of Zappa's work. As always with books like this, it's not till you read about the lyrics that you fully understand the lyrics even if you have listened to them many a times. Some of these lyrics I'm actually kind of sad to have understood now as of reading about them in this book. For example, I now understand that the song We're Turning Again which is an up-yours to his old band members Mothers of Invention.

If anything bad can be said about the book it's that it sort of ends on a bad note (no pun intended), as it ends on the sad last few years when Frank was really sick and up to the point of his death. And also, I would have liked to find out more about Barry's own personal relationship with Frank because he couldn't possibly have written this book had he not admired the guy too.

There is no doubt in my mind that Frank Zappa is one of the most innovative and inspirational characters in twentieth-century music history. And probably show-biz too for that matter. Even though the book reveals some truths about Frank as a bit of "douche bag" I'm still firmly one of his biggest fans. If you wanna find out more about Frank Zappa this is most like the book to get.

Following is an extract from a quoted interview by Gail Sloatman who later became Gail Zappa:

"And I remember thinking, Oh my God! Here's this guy, I think he's extraordinary, it's such a different sensation! I know he hasn't taken a bath in four months and his moustache smells like peanut butter..."

She sums it up nicely in her own very personal words so well. There is something amazing about this guy beyond the less appealing facade.

iPhone push notifications for Twitter with Prowl

October 25, 2009
1 comment Web development

iPhone push notifications for Twitter with Prowl Bruno Renié has written a nifty app:iPhone push notifications for Twitter with Prowl

With the power of Prowl it pushes a notification to your iPhone when someone mentions you on Twitter. You first need to install the Prowl app on your iPhone and then go to their website to get your notification key. Then, go to Bruno's site, sign in with your Twitter account (OAuth, so no password give-away) and badabing! you get instant notifications on your phone, for free, when someone mentions you.

If you're one of those people who use Twitter instead of instant messaging and have lots of mentions this might be excessive for you but if not it can be very useful if you are like me who is very slow to spot that someone has replied or mentioned your name.

Great work Bruno!

What makes my website slow? DNS

October 23, 2009
14 comments This site, Linux

Pagetest web page performance test is a great tool for doing what Firebug does but not in your browser. Pagetest can do repeated tests to iron out any outliers. An alternative is Pingdom tools which has some nifty sorting functions but is generally the same thing.

So I ran the homepage of my website on it and concluded that: Wow! Half the time is spent on DNS lookup!

First Second Third

The server it sits on is located here in London, UK and the Pagetest test was made from a server also here in the UK. Needless to say, I was disappointed. Is there anything I can do about that? I've spent so much time configuring Squid, Varnish and Nginx and yet the biggest chunk is DNS lookup.

In a pseudo-optimistic fashion I'm hoping it's because I've made the site so fast that this is what's left when you've done all you can do. I'm hoping to learn some more about this "dilemma" without having to read any lengthy manuals. Pointers welcomed.

What I hate about PIL and Image in Python

October 19, 2009
6 comments Python

One really annoying thing about PIL is that it's importable as Image and PIL. It leads me and other newbies to think if it's different. I don't want choices:


Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import PIL
>>> import Image
>>> ?

When PIL/Image is put into standard lib, can we call the module: imaging?

Messed up columns in Django Admin

October 16, 2009
1 comment Django

In case this bites someone else like it bit more and chewed off many many minutes of debugging time.

If you ever get weird columns in your Django Administration interface, I now know why that happens. See this screenshot example:

Messed up columns in Django Admin

This happens when you've defined a TEMPLATE_STRING_IF_INVALID in your settings.py. I always put in my local_settings.py this line:


TEMPLATE_STRING_IF_INVALID = '{{ %s }}'

So that I can quickly see which variable references in template code is potential typos. I'm not a big fan of the implicit magic of equating absence to False/None so I try to avoid the confusion altogether.

Automatically strip whitespace in Django forms

October 12, 2009
2 comments Django

The current project I'm working has at the time of writing 20 different forms (90% model forms) instantiated in different scenarios. Django doesn't automatically strip whitespace in text based fields. So instead of doing this:


class ContactMarketingForm(forms.ModelForm):
   class Meta:
       model = ContactMarketing
       exclude = ('contact',)

   def clean_notes(self):
       return self.cleaned_data['notes'].strip()

   def clean_name(self):
       return self.cleaned_data['name'].strip()

Instead I wrote a common class for all of my form classes to use:


class _BaseForm(object):
   def clean(self):
       for field in self.cleaned_data:
           if isinstance(self.cleaned_data[field], basestring):
               self.cleaned_data[field] = self.cleaned_data[field].strip()
       return self.cleaned_data

class BaseModelForm(_BaseForm, forms.ModelForm):
   pass

class ContactMarketingForm(BaseModelForm):
   class Meta:
       model = ContactMarketing
       exclude = ('contact',)

Now all text inputs and textareas are automatically whitespace stripped. Perhaps useful for other Djangonauts.

A user-friendly TinyMCE config

October 8, 2009
4 comments Web development

A user-friendly TinyMCE config When you enable TinyMCE you can either choose theme="simple" or theme="advanced". If you go for the simple you get one lovely little bar of buttons but it's missing the link button which is more important than most other buttons altogether. When you enable "advanced" you get three rows of buttons that makes you dizzy. I mean, should really be editing advanced tables in a WYSIWYG editor or mathematical equations?

Here's a config that I think works great. It's all in one row and it's got the bare minimum in terms of additional plugins (no extra downloads required). It's in Python but translates quite easily into Javascript:


TINYMCE_DEFAULT_CONFIG = {
   'plugins': "fullscreen,paste,autoresize",
   'theme': "advanced",
   'theme_advanced_buttons1' : "bold,italic,strikethrough,bullist,numlist,"\
                               "separator,undo,redo,separator,link,unlink,image"\
                               ",separator,cleanup,code,removeformat,charmap,"\
                               "fullscreen,paste",
   'theme_advanced_buttons2' : "",
   'theme_advanced_buttons3' : "",
}

London Frock Exchange launched

September 26, 2009
0 comments Work, Django

London Frock Exchange launched Today we launched The London Frock Exchange which is a joint project between Fry-IT, Charlotte Davies and Sarah Caverhill

Elevator sales pitch: Unlike other clothes swapping sites, with Charlotte, Sarah and Rani as an expert hub in the middle you don't swap straight across; no you swap one frock in and can choose a frock (of equal value) from the pool of frocks.

Fry-IT is co-founding this venture and hope it'll make us billionaires by the end of the year (They take a small admin fee of £25 for sending you a frock back but sending it in is free with freepost). It's been great fun to work on it over the last couple of months as it means we (Fry-IT is a all-male highly technical company) have had to learn about sizes, body shapes and trying to learn how a female web audience thinks. The ladies have done a great job of seeding it with lots and lots of frocks all of which you can wear in a matter of days if you just swap one of equal value in first. Enjoy!