Peterbe.com

Do you, like me, use the powerful print function to debug your Python code? Do you, like me, sometimes forget to remove lines with print and ship this out to less development-like environments? Do you, like I did, need a way to check for forgotten print statements?

I've written a dumb-simple script that scans through a directory for print statements. It uses a recursive grep command so this won't work for you hopeless Windows people :)

Download it and put it in your ~/bin (you might want to read it through) then all you have to do is to run it against a directory where there are lots of .py files:

$ find_print_statements.py ~/dev/mypythonmodule

If it doesn't work perhaps you can patch it and let me know. This might not be for the faint-hearted. There might be serious security concerns unless you have full control over your users and su account. I take no responsibility if this script rm -fr / your hd.

Did you see my mentioning about addhrefs last week? addhrefs is a python module that turns a text full of URLs and email addresses into links. For example:

>>> from addhrefs import addhrefs
>>> print addhrefs("""Visit www.peterbe.com 
and email mail@peterbe.com""")
Visit <a href="https://www.peterbe.com">www.peterbe.com</a>
and email <a href="mailto:mail@peterbe.com">mail@peterbe.com</a>

Then I saw this instruction on google.com about Preventing comment spam They suggest you add a rel="nofollow" to links in your comments on your blog and with a bit of luck this will reduce the amount of comment spam you get on your blogs. So, how to do that?

Python gurus are going to laugh at me for this one but I learnt it today; how to check if an object is callable. Before this I thought I was clever with this:

>>> def foo(): return "s"
>>> hasattr(foo, '__call__')
True
>>> hasattr(foo(), '__call__')
False

But there was an easier way. I discovered this by accident and looked it up in the documentation after. Use the builtin function called 'callable()':

>>> def foo(): return "s"
>>> callable(foo)
True
>>> callable(foo())
False

D'oh!

I've now made some improvements to my little addhrefs script. If you don't know what this is, read Add links to a text (take II) and Add links to a text with URLs which explains what this does.

The latest changes are that you can now pass your own function for turning an email or a URL into a link. For example:

>>> from addhrefs import addhrefs
>>> def emailify(e):return '<a href="/sendemail?to=%s">%s</a>'%(e,e)
>>> print addhrefs("Hello foo@bar.com", emaillinkfunction=emailify)
Hello <a href="sendemail?to=foo@bar.com">foo@bar.com</a>

You can also do the same with URLs by filling the urllinkfunction parameter.

Download: addhrefs-0.6.tgz

UPDATE Bug fixed version: addhrefs-0.7.tgz

If you do a speed report on Plone.org you notice that the three biggest files it services are plone.css, plone_javascripts.js and index.html. If you run these through my little slimmer whitespace optimizing program in Python you get the following results:

$ python slimmer.py --test http://plone.org/plone.css
Took 0.016 seconds
Bytes before: 29355
Bytes after:  20265
Bytes saved:  9090  (69.0% of original size)

$ python slimmer.py --test http://plone.org/plone_javascripts.js
Took 0.057 seconds
Bytes before: 27251
Bytes after:  19974
Bytes saved:  7277  (73.0% of original size)

$ python slimmer.py --test http://plone.org/ 
Took 0.029 seconds
Bytes before: 37186
Bytes after:  26466
Bytes saved:  10720 (10K)  (71.0% of original size)

I've learned something today. The cPickle module in Python can be boosted with very little effort. I've also learnt that there's something even faster than a hotted 'cPickle': marshal.

The code in question is the CheckoutableTemplates which saves information about the state of templates in Zope to a file on the file system. The first thing I did was to insert a little timer which looked something like this:

def write2config(...):
    t0=time()
    result = _write2configPickle(...)
    t1=time()-t0
    debug("_write2configPickle() took %s seconds"%t1)
    return result

In recent code in the IssueTrackerProduct I needed to have a cgi parameter called remember-filterlogic so the URL might look like this: ListIssues?rememeber-filterlogic=no&Filterlogic=show. Because I want the cgi parameters to look human I had to write the following little utility function:

def niceboolean(value):
   falseness = ('','no','off','false','none','0', 'f')
   return str(value).lower().strip() not in falseness

It basically converts what you say to what you mean. In Python "f" is a one letter string and would normally mean True, but since humans are involved here it from means something else for a moment. What do you think?

I needed to write this little function because I need to add some parameters to a URL that I was going to open with urllib2. The benefit with this script is that it can combine a any URL with some structured parameters. The URL could potentially already contain a query string (aka CGI parameters). Here's how to use it if it was placed in a file called 'urlfixer.py':

>>> from urlfixer import parametrize_url
>>> parametrize_url('https://www.peterbe.com?some=thing',
                    any='one', tv="b b c")
'https://www.peterbe.com?some=thing&tv=b+b+c&any=one'
>>> 

In my Zope usage I use a lot of SQL stored on the filesystem. These files contain DTML syntax and parameters like ZSQL Method objects inside the ZODB. Sometimes, to test the SQL code I have in these files I want to run it on the command line. To be able to do that I've written this little Python script which is placed in my ~/bin directory in Linux. It only works with PostgreSQL at the moment but people who like the idea and prefer Oracle or MySQL could probably find where to make their changes. (The psql -f command is the ticket)

Example foo.sql:

<params>id=1
name</params>
SELECT *
FROM sometable
WHERE id = <dtml-sqlvar id type="int">
AND   name = <dtml-sqlvar name type="string">

Which is run like this:

peterbe@trillian:~ $ rundotsql.py -U peterbe testdb foo.sql

I stumbled across a problem today with running a python package from a different directory than where the python files are. The package looked like this (in principle):

/home/someone/mypackage/
   __init__.py
   template.html
   foobar.py

The foobar.py script contains this code:

def foo():
   tmpl = open('template.html').read()
   return tmpl.replace('X','Y')

That's fine and works when you run the file directly. The problem comes when you run the foobar.py script from somewhere completely different (e.g. /home/mrbloggs/otherpackage/). The solution to this comes from Zope's package_home.