Peterbe.com

About 5 years ago Martijn Faasen wrote the wonderful blog article What is Pythonic. One thing that I feel is extremely Pythonic is to not compare certain thing to other things when Python has built-in understanding of what false or true means.

Having reviewed/read a lot of beginner code or senior code but of people coming from lower-level languages I often see this:

if variable == False:
...
if variable == 0:
...
if variable == None:
...
if len(variable) == 0:
...
if variable == []:
...
if variable == {}:
...
if ORM.filter(user=variable).count == 0:
...
if not bool(variable):
... 

To be Pythonic is to understand that Python evaluates all of these to false. All built in types have a perfectly sensible boolean operator which is automatically used in an if statement or an embedded if statement in a list comprehension. Keep it clean a pure just like this to check for true:

if not variable:
...
if not ORM.filter(user=variable):
...

And if you have your custom class such as the example just above with the pseudo "ORM" it's easy to extend it by writing your own custom __bool__ like this:

class MyCustomType(somebuiltintype):
   ...
   def __bool__(self):
       return self.somedate and self.somecondition

By playing along with Python just the way Guido indented it you can abstract yourself from being overly dependent of types. By doing the shorthand notation a variable that is otherwise a list can be None if it's not set and your code will continue to work.

All the above might not be true for more explicit lower-level languages like C++ but it sure is Pythonic in Python and that's a good thing.

The lovely people of Packt Publishing asked me to review Karen Tracey's latest book Django 1.1 Testing and Debugging.

I didn't actually read the book but rather skimmed it, apart from some selected parts and from what I read it's obvious that Karen has an ability to write to people who are not experts on the subject. Years of being a top contributor on the Django users mailing list must have something to do with it.

But here's the cracker. I didn't learn anything from this book (actually, I wasn't aware of the pp command in the pdb debugger). Is that a complaint about the book? No! It just means that the book was aimed at beginners and apparently I'm not a beginner any more. Great!

One thing I would have liked to see is more about testing strategy since this is something beginners often have problems with. I don't know if there even is such a word as "testing strategy" but I'm referring to the thinking behind what to test and more importantly sometimes what not to test. Beginners have a tendency to write tests for the most specific things and thus spending all their time assuring the most unrealistic scenarios are covered. Also, a lot of beginner tests I see check basic things like types which the semi-compiler will just automatically cover for you. Perhaps for a beginner, just getting some tests up and running this is a big step forward.

I'm a little bit disappointed that my lovely gorun wasn't mentioned in the book :) Perhaps the next version Karen?

This is fairly obvious stuff I guess but it has troubled me for a long time. Some programs on Linux don't spit out their results to stdout. Instead they start a little program similar to less. So what is a console nerd to do?

Pipe it cat! I don't know why I've never thought of this before:

$ psql -l | cat

So I upgraded to the latest Ubuntu Lucid Lynx 10.04 the other day and to my horror it removed Python 2.4 and Python 2.5. I rely more on those programs than I do on some silly Facebook connecting social widget crap. On my laptop I have lots of Zopes requiring Python 2.4 and I have about 10 active Django projects that rely on Python2.5. This fuckup by Ubuntu caused me to write this complaint.

So my estimeed colleague and Linux wiz Jan Kokoska helped me set things straight by showing me how to downgrade these packages to Karmic version and how to pin them in the apt preferences. First of all, make your /etc/apt/source.list look like this:

deb http://gb.archive.ubuntu.com/ubuntu/ karmic main restricted universe multiverse
deb-src http://gb.archive.ubuntu.com/ubuntu/ karmic main restricted universe multiverse

deb http://gb.archive.ubuntu.com/ubuntu/ karmic-updates main restricted universe multiverse
deb-src http://gb.archive.ubuntu.com/ubuntu/ karmic-updates main restricted universe multiverse

deb http://gb.archive.ubuntu.com/ubuntu/ karmic-backports main restricted universe multiverse
deb-src http://gb.archive.ubuntu.com/ubuntu/ karmic-backports main restricted universe multiverse

deb http://security.ubuntu.com/ubuntu karmic-security main restricted universe multiverse
deb-src http://security.ubuntu.com/ubuntu karmic-security main restricted universe multiverse

deb http://gb.archive.ubuntu.com/ubuntu/ lucid main restricted universe multiverse
deb-src http://gb.archive.ubuntu.com/ubuntu/ lucid main restricted universe multiverse

deb http://gb.archive.ubuntu.com/ubuntu/ lucid-updates main restricted universe multiverse
deb-src http://gb.archive.ubuntu.com/ubuntu/ lucid-updates main restricted universe multiverse

deb http://gb.archive.ubuntu.com/ubuntu/ lucid-backports main restricted universe multiverse
deb-src http://gb.archive.ubuntu.com/ubuntu/ lucid-backports main restricted universe multiverse

deb http://security.ubuntu.com/ubuntu lucid-security main restricted universe multiverse
deb-src http://security.ubuntu.com/ubuntu lucid-security main restricted universe multiverse

If you know what you're doing you might have other additional sources in there then keep those as is. Next thing to do is to update and upgrade:

# apt-get update
# apt-get dist-upgrade

You should now see that it's intending to upgrade a bunch of juicy packages like python2.4-dev for example. To check that python2.4 is now getting in from Karmic run this:

$ apt-cache madison python2.4

Now for the trick that really makes the difference:

# apt-get install python2.4=2.4.6-1ubuntu3.2.9.10.1 python2.4-dbg=2.4.6-1ubuntu3.2.9.10.1 \
python2.4-dev=2.4.6-1ubuntu3.2.9.10.1 python2.4-doc=2.4.6-1ubuntu3.2.9.10.1 \
python2.4-minimal=2.4.6-1ubuntu3.2.9.10.1

The command is quite self-explanatory. You use the equal sign to basically say what version you want to install. If you now for example want to install something like python-profiler for your Python 2.4 since this isn't available as a PyPi package. First, find out what version you have to install:

$ apt-cache madison python-profiler | grep karmic

From that list you'll get a bunch of versions. Chose the one from karmic-updates or karmic-security. Then install it:

# apt-get install python-profiler=2.6.4-0ubuntu1

Now, to avoid this causing a conflict and thus be removed the next time you do an upgrade you need to pin it. Create a file called /etc/apt/preferences and put the following into it:

Package: python-profiler
Pin: version 2.6.4-0ubuntu1
Pin-Priority: 999

And that concludes it. A word of warning from Jan:

"he slight problem is that with this setup, suppose a big security flaw was found in python-imaging and got patched in karmic that is still supported... you wouldn't get the package update. That is because it's pinned and while asterisks can be used in the version number, we don't know in advance what the version will match and what the Lucid version that we don't want will match"

"so you basically lose security upgrades for affected packages"

"minor annoyance when you have one or two packages on a laptop, but a big deal if you have a dozen packages on 100 VMs on server"

Having written about this helps me remember it myself for the next time I need it. Also, hopefully it will help other people who get bitten by this. Hopefully this will shame the Canonical guys into action so that the next time they don't haste their deprecation process and actually think about who's using their products. I bet a majority of Ubuntu's users care more about programming or something like that than they do about the ability to buy music on Ubuntu One or whatever it's called.

Ludvig Ericson ("Sweden-based backend-centric super-programmer.") yesterday released the most important Python package you'll ever see this year. Sort of. It animates a little fish on your terminal that goes back and forth across the screen.

Maybe I'm exaggerating a bit. This is the kind of superficial hype that made Rails successful at least. What the package is really useful for is a great start for those who want to do those fancy writes to the terminal without linebreaks. Spoiler alert:

sys.stderr.write("\x1b[2K\r" + fish + "\r")

It works. Sort of.

The screenshot attached here doesn't really show it but the nav at the bottom is supposed to stay at the bottom as you scroll but that doesn't seem to work on the iPad. Shows me up for not using solid basic (X)HTML.

Ever since we started building SnapExpense we ask people we meet how they do their business/travel expenses at their company. Almost always they say their software is annoyingly bad and it's a right pain in the ass to process their expenses. Today I heard another such horror story from a friend about: GlobalExpense (www.globalexpense.com). Apparently it's dreadfully slow to use and it never remembers or adapts to what you have entered previously.

So my friend went to the website on my computer to show me how crap it was but that didn't work because GlobalExpense doesn't work in Firefox! What?! On SnapExpense, about 30% of our visitors use Firefox (with 31% using Internet Explorer).

I'm glad to see that the competition is lagging behind. Gives me heart. Especially as I have recently added Google OpenID log in to SnapExpense so you can log in or register with your Google (or Yahoo! or any OpenID provider) account straight away.

This is going to be useful to keep handy when I move to America.

I've learned a couple of things this week on deploying my first site to use a user friendly OpenID.

My first revelation was when I realized that Google and Yahoo! have solved the usability stumbling block that you can use them as providers without having to know a personally unique URL. For example, for Yahoo! it's just http://yahoo.com which means that you don't need to offer a cryptic URL form and you can just show it as a logo image.

The second thing is that Google's hybrid OpenID + OAuth isn't as complicated as it sounds. It's basically a light extension to the OpenID "protocol" whereby you say, "while you're at it, also give me a OAuth token please so that I can connect back into Google's services later". What's important to understand though is that if you use this you need to know the "scope". scope is a URL to a service. Google Docs is a service for example and you need to search the web to figure out what the scope URL is for that service.

The third revelation was when I understood the difference between Simple Registration Extension (SREG) and Attribute Exchange (AX). Basically, AX is a newer more modern alternative and SREG was the first one. AX is better but some OpenID providers don't yet support it. Google for example, only supports AX. Key to be able to support not just Google's OpenID but any OpenID is that you can request both AX and SREG and whichever one works will be returned.

The fourth thing that helped a lot to understand was the Google's OpenID has a bug in its implementation of Attribute Exchange. Actually, perhaps it's a deliberate design choice they've made but in my opinion a bad one. Unless you say you require email, firstname, lastname, country etc. it won't return it. If you use the if_available directive you won't get it. Another bug/bad design choice is that Google seems to not forward the country attribute. It can happily do first- and last name but not country even if the documentation claims so.

The fifth thing is that python-openid is a lot easier to work with than you think. You don't need to do any crazy network checks or callbacks. For initiating the challenge all you're effectively doing is creating a long URL. If you don't like the API methods python openid offers, just add your own with:

redirect_url += '&openid.ax.mode=fetch_request' # etc.

After so many years since OpenID arrived, I'm only now excited about it. It's tonnes easier to implement than OAuth and now it's actually really pleasant to use as an end user.

According to my analytics the most popular Google search for getting to Crosstips.org is exactly crosstips.org which clearly proves that a lot of people type in the domain name in the search field these days. I do it too.

The second most popular search is "word whomp" which Crosstips has a dedicated page for. What's cute about that is that it was just a little side project I threw in yet it has grown to become one of the most popular features. You can never predict these kinds of things. I think the next thing I'm going to add is a Hangman solver which shouldn't be too hard.

So fellow Word Whomp cheaters, go for it!