Peterbe.com

I've learned something today. The cPickle module in Python can be boosted with very little effort. I've also learnt that there's something even faster than a hotted 'cPickle': marshal.

The code in question is the CheckoutableTemplates which saves information about the state of templates in Zope to a file on the file system. The first thing I did was to insert a little timer which looked something like this:

def write2config(...):
    t0=time()
    result = _write2configPickle(...)
    t1=time()-t0
    debug("_write2configPickle() took %s seconds"%t1)
    return result

In recent code in the IssueTrackerProduct I needed to have a cgi parameter called remember-filterlogic so the URL might look like this: ListIssues?rememeber-filterlogic=no&Filterlogic=show. Because I want the cgi parameters to look human I had to write the following little utility function:

def niceboolean(value):
   falseness = ('','no','off','false','none','0', 'f')
   return str(value).lower().strip() not in falseness

It basically converts what you say to what you mean. In Python "f" is a one letter string and would normally mean True, but since humans are involved here it from means something else for a moment. What do you think?

I needed to write this little function because I need to add some parameters to a URL that I was going to open with urllib2. The benefit with this script is that it can combine a any URL with some structured parameters. The URL could potentially already contain a query string (aka CGI parameters). Here's how to use it if it was placed in a file called 'urlfixer.py':

>>> from urlfixer import parametrize_url
>>> parametrize_url('https://www.peterbe.com?some=thing',
                    any='one', tv="b b c")
'https://www.peterbe.com?some=thing&tv=b+b+c&any=one'
>>> 

In my Zope usage I use a lot of SQL stored on the filesystem. These files contain DTML syntax and parameters like ZSQL Method objects inside the ZODB. Sometimes, to test the SQL code I have in these files I want to run it on the command line. To be able to do that I've written this little Python script which is placed in my ~/bin directory in Linux. It only works with PostgreSQL at the moment but people who like the idea and prefer Oracle or MySQL could probably find where to make their changes. (The psql -f command is the ticket)

Example foo.sql:

<params>id=1
name</params>
SELECT *
FROM sometable
WHERE id = <dtml-sqlvar id type="int">
AND   name = <dtml-sqlvar name type="string">

Which is run like this:

peterbe@trillian:~ $ rundotsql.py -U peterbe testdb foo.sql

I stumbled across a problem today with running a python package from a different directory than where the python files are. The package looked like this (in principle):

/home/someone/mypackage/
   __init__.py
   template.html
   foobar.py

The foobar.py script contains this code:

def foo():
   tmpl = open('template.html').read()
   return tmpl.replace('X','Y')

That's fine and works when you run the file directly. The problem comes when you run the foobar.py script from somewhere completely different (e.g. /home/mrbloggs/otherpackage/). The solution to this comes from Zope's package_home.

Here's a little program I wrote recently to fix incorrectly defined characters into HTML entities. For example, this is incorrect:

<b>Bärs &amp; Öl</b>

But this is correct:

<b>B&amp;auml;rs &amp;amp; &amp;Ouml;l</b>

To demonstrate I have set up a little test page here so that you can test to convert your impure HTML content.
Run test program

Just wanted to make you aware of my little recipe on the ASPN Python Cookbook

This is used in the next release of the IssueTrackerProduct for the email notifications.

If you missed the entry about Add links to text with URLs I suggest you read that first. This script has now been improved with bug fixes thanks to David Otton and Flump Cakes.

Download: addhrefs-0.4.tgz
It does not come with a nice installer because I don't think it belongs to the generic Python library anyway. If you want to use it, copy the file addhrefs.py to somewhere useful.

"Urwid is a curses-based UI/widget library for Python. It features fluid interface resizing, multiple text layout options, simple markup for attributes, powerful scrolling list boxes and flexible edit boxes."

I've been looking for something like this for a long time. See the screenshots to get an idea of what a curses-based UI is.

I had the problem today that several files in a directory started with an _ underscore character. (e.g. _red.gif). Instead of manually renaming each file used the power of shell and python to solve it. (and some help from my collegue of course). Fortunately none of the files had an underscore in the middle of the name so I could keep the command quiet simple:

$ find -iname '_*' \
| xargs python -c \
'import sys;print "\n".join(["mv %s %s"%(x, x.replace("_","")) for x in sys.argv[1:]])'\
| sh -s