PyGolf - Winner of competition 1

Well the deadline has come and passed, and we have a clear winner. That winner is Katie. Her solution comes in at a tiny 121 characters, with the closest solution coming from Tim with 187 characters.

The main difference came down to the fact that Katie decided to avoid using the regular expression that was given in the sample, and just parse the lines with str.split.
So without much more talking, here is the winning solution.

import sys
s=sum(([l[0]]*int(l[-1])for l in map(str.split,sys.stdin)if l[-1]!='-'),[])
for x in set(s):print x,s.count(x)

It is also interesting to note how she uses the sum function to append a set of lists together.

Python golf competition

I have written a few entries on python golf before, but I have now decided to make an official competition from it. The rules are fairly simple: I pose a problem simple problem which must be solved (in python) in the lowest number of characters (where a new line counts as one byte). Solutions may be written for any 2.x version of python (ie, 2.3, 2.4, 2.5 or 2.6), and may use any library found on default install on a Debian machine.

Solutions must be emailed to me. In the case where two people have the same character count, the solution that arrived in my inbox first will be declared the winner (this is to stop people from playing with the date header in the email ^^). The winner will receive a chocolate bar or coffee -- their choice.

The problem this week is one of parsing log files. You must parse a log file in the common log format that will be given on to your program on stdin. You must then print on stdout the amount of data that was sent to each ip address (and the ip address). A sample program has been provided:

#!/usr/bin/env python2.5
# Copyright 2008 Greg Darke <greg+laptop@tsukasa.net.au>
# Licensed for distribution under the GPL version 2, check COPYING for details
# Sample program to show the output format for the python golf problem posed at
# http://blag.tsukasa.net.au/2008/11/23/python-golf-competitionpython-golf-competition/

import re, sys

def main():
    clf_regexp = re.compile(r'''^(\S+)\s(\S+)\s(\S+)\s\[([^\]]*)\]\s"([^"]*)"\s(\d*)\s(\d*)$''')
    mapping = {}

    for line in sys.stdin:
        m = clf_regexp.match(line)
        if not m:
            continue
        ip, _, _, _, _, _, size = m.groups()
        size = int(size)

        if ip not in mapping:
            mapping[ip] = 0

        mapping[ip] += size

    for ip in mapping:
        print '%s %d' % (ip, mapping[ip])

if __name__ == "__main__":
    main()

Solutions will be accepted until 11:59:59pm(EDT) on Friday 28th November 2008.

Moving away from Mail.app

As most people know, I have moved away from using Apple's Mail.app as my email client. I now use mutt to perform all of my mail duties... or, so I thought.

The other day I decided to use iCal to send an event invitation to a friend. When I clicked on the send button, I was greeted with a copy of Mail.app, not exactly I wanted. At this point I changed the default Mail application to mutt (via iTerm).

Now you would expect iCal to listen to this setting, but alas, the mail was still (automatically) sent with Mail.app.

After a lot of investigating, I discovered there is a apple script within the iCal app bundle. The script of interest is /Applications/iCal.app/Contents/Resources/Mail.applescript. After spending a few hours learning applescript, I came up with a solution. You will need to replace the file /Applications/iCal.app/Contents/Resources/Mail.applescript with the one available from my repositories.

Once you have replaced this file, you must then recompile the script: osacompile -o /Applications/iCal.app/Contents/Resources/Scripts/Mail.scpt /Applications/iCal.app/Contents/Resources/Mail.applescript

Note: You will need to have at the mail script and modify it so that it points to the correct version of mutt on your system.

Not so smooth transition to xubuntu-intrepid

So James and I decided to take the plunge and install the intrepid update on a newly installed machine we had here (it had had xubuntu-hardy installed only a week earlier).

This machine was using the nvidia binary package, but other than that did not have anything that was from a restricted repo.

Upon the upgrade we found that the nvidia package no longer works. In fact, it no longer compiles the kernel module (even though it claims that the package is installed correctly). This left us without an X11 server. Even when trying to use the nv driver we could not get it to work.

On a side note: We found that the nv driver is now no-longer installed by default.

After about an hour of trying to get things to work, we decided that we will just have to live with the vesa driver. Although this makes the system slow, at least we can use X11.

Another thing I noticed, is that it now spams me with an annoying motd, I mean sure system stats on login is may be nice for some people, but I already have most of that coded in my scripts. The easiest way to get rid of this is to remove the package landscape-client with the following command:
sudo aptitude purge landscape-client

Simple cgi script to grab images from your Mac's built in iSight camera

While I was meant to be working on my thesis, I decided to update my little script to grab images from the built in iSight camera. The older version depended upon a instance of procfs to be started from within a login shell (which is not always possible).

To get around this, I stared looking for a way to inject a process into a particular mach bootstrap session (After reading the OSX internals book, I knew this was where I should be looking). Now, I will not claim that this script is the best thing I have written... It requires you to set up a rule in the sudoers file to allow the _www user to execute the specified sudo command (as root). I will leave writing this line as an exercise to the user (I have written one version, it is just not completely secure). You will also have to grab the isightcapture binary from off the net, and update the script with the correct location.

#!/bin/bash

function pgrep {
    ps -A -o pid=,command= | grep "$1" | awk '{ print $1; }' | grep -v $$
}

function cleanup {
    if [ ! -z "$MYTEMP" ]; then
        rm -rf "$MYTEMP"
    fi
}

MYTEMP="$(mktemp -d)"
trap "cleanup" 15 0

LOGIN_WINDOW_PID="$(pgrep loginwindow.app)"
OUTPUT_FILENAME="${MYTEMP}/isightcapture.jpg"
sudo launchctl bsexec "${LOGIN_WINDOW_PID}" /Users/gregdarke/bin/isightcapture -t jpg "${OUTPUT_FILENAME}"

echo -en 'Content-type: image/jpeg\r\n\r\n'
cat "${OUTPUT_FILENAME}"

Of databases and repositories

One thing that most programmers will cringe at, is the thought of placing a database into a version control repository such as subversion or mercurial. Now I know that many of have done this for various reasons (I know I am guilty of it myself).

The point of this post is to show how this can be made a little nicer under mercurial using encode/decode filters. With a carefully constructed set of filters, you are able to actually perform text diffs and make sane merges between repositories. All you have to do is drop the following into your hgrc file (either the one in your project or ~/.hgrc):

[encode]
data.db = tempfile: sqlite3 INFILE .dump > OUTFILE

[decode]
data.db = tempfile: sqlite3 OUTFILE '.read INFILE'

This requires that you have the sqlite3 binary installed, otherwise you will end up with a data.db file containing the raw sql used to generate the database.

Recursive tuples

Now,
I have been thinking about this for a few days now... Is it possible to create a tuple in python that refers to itself. I don't mean via some other object, so the following does not count:

def recursive():
    l = []
    t = (l,)
    l.append(t)
    return t

I believe it is possible to do directly from C, but I can not think of a way to do it from within python.

The idea of creating a recursive tuple was spawned from this little comment I found in the pickle source code: "... recursive tuples are a rare thing". At first I thought it was talking about a tuple that directly refers to itself, but then figured that it must be talking about tuple's that indirectly refer to themselves.