Peterbe.com

I think a lot of Python people have seen Peter Novig's beautiful article about How to Write a Spelling Corrector. So have I and couldn't wait to write my own little version of it to fit my needs.

The changes I added were:

• Python 2.4 compatible
• Uses a pickleable dict instead of a collection
• Compiled a huge list of Swedish words
• Skipped edit distances 2 of words longer than 10 characters
• Added a function suggestions()
• All Unicode instead
• A class instead of a function
• Ability to train on your own words and to save that training persistently

Don't do the silly misstake that I did today. I improved my code to better support unicode by replacing all plain strings with unicode strings. In there I had code that looked like this:

if type_ is 'textarea':
   do something

This was changed to:

if type_ is u'textarea':
   do something

And it no longer matched since type_ was a normal ascii string. The correct wat to do these things is like this:

if type_ == u'textarea':
    do something
elif type_ is None:
    do something else

Remember:

>>> "peter" is u"peter"
False
>>> "peter" == u"peter"
True
>>> None is None
True
>>> None == None
True

SEE UPDATE BELOW

--

Suppose you have a list in python that looks like this:

['a','b','a']
# or like this:
[1,2,2,2,3,4,5,6,6,6,6]

and you want to remove all duplicates so you get this result:

['a','b']
# or
[1,2,3,4,5,6]

How do you do that? ...the fastest way? I wrote a couple of alternative implementations and did a quick benchmark loop on the various implementations to find out which way was the fastest. (I haven't looked at memory usage). The slowest function was 78 times slower than the fastest function.

This has been a problem for a long time for me. Whenever someone enters a title in my CMS the id of the document is derived from the title. Spaces are replaced with '- and &' is replaced with and etc. The final thing I wanted to do was to make sure the Id is ASCII encoded when it's saved. My original attempt looked like this:

>>> title = u"Klüft skräms inför på fédéral électoral große"
>>> print title.encode('ascii','ignore')
Klft skrms infr p fdral lectoral groe

But as you can see, a lot of the characters are gone. I'd much rather that a word like "Klüft" is converted to "Kluft" which will be more human readable and still correct. My second attempt was to write a big table of unicode to ascii replacements.

It looked something like this:

u'\xe4': u'a',
u'\xc4': u'A',
etc...

If you have some code that you need to optimise, like some Javascript code that is well commented but costs too many bytes of download for your users then you might want to use my slimmer web service. I'll let a Python example speak for itself:

>>> import xmlrpclib
>>> s=xmlrpclib.Server('https://www.peterbe.com/')
>>> css='h1 { font-family: Arial, Verdana; }'
>>> s.slim(css)
'h1{font-family:Arial,Verdana}'

I've just added some nice stats to RememberYourFriends.com. It's basically two line charts. One of how many reminders have been sent that week and one of how many reminders have been sent, ever up to a particular week from the very start.

To do this I use the wonderful CharDirector package for Python which is really fast and very easy to implement. These graphs are created on the fly and apart from generating the image it obviously needs to generate the data. Pretty fast I'd say. Will be interesting to see how it will fair when the load starts to get interesting.

Once I get a bit more users I'll start thinking of other funky charts to draw. It's fun.

I've just quickly put together a little script that computes the difference between two texts in a human readable format. The result when you run diff is a bit difficult to understand for a human being and I wanted something more "humane" that quickly summarises what's different on one simple line. Eg. "Added 2 lines, change 1 line".

This little script is going to be part an undo function in our new CMS that I'm working on. Instead of just pinpointing which revision date you want to go back to you'll also be able to see what the differences were between each revision in the undo history for the CMS.

A little while ago I wrote about how I got Jed + TAGS to work thanks the ntags library. I've been using it now for a while and I love it! I doubt there are any IDEs that beats a swift combination of Ctrl+2 followed by Alt+. and you get the definition of a function or variable without losing any focus.

If you're not into programming stop reading now because it's going to get even more technical.

In Python you indicate whether a function/method is private by naming it so that it starts with an _ underscore like this:

def _thisIsPrivate():
    return 'a'

def thisIsPublic():
    return 'b'

It means that if you have these two functions in a file called dummy.py and you do something like this:

>>> from dummy import *
>>> thisIsPublic()
'a'
>>> _thisIsPrivate()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
NameError: name '_thisIsPrivate' is not defined

Seems fair doesn't it. Now is there such similar naming convention and functionality in Javascript?

Casting is when you convert a variable value from one type to another. This is, in Python, done with functions such as int() or float() or str(). A very common pattern is that you convert a number, currently as a string into a proper number.

Let me exemplify:

>>> x = '100'
>>> y = '-90'
>>> print x + y
100-90
>>> print int(x) + int(y)
10

That was the int() function. There's also another very common one which is float() which does basically the same thing:

>>> print float(x) + float(y)
10.0