Using Multiple Python Environments With Gentoo

It’s been some time since Python 2.5 became stable and released. Version 2.5 has plenty of new features that have helped me in deciding that it was time to go ahead and start using it for primary development of all my new projects. One of the reasons I was still using version 2.4 is that Gentoo hadn’t upgraded 2.5 to the stable package system.

I decided to go ahead and unmask version 2.5 anyway. Installing Python 2.5 isn’t very complicated and I’ll leave the details out. I’ll just mention that after unmasking Python with the ~x86 keyword and installing the package, you’ll need to run the python-updater. Python-updater had it’s own problems with not being able to find some packages it thought needed to be re-emerged but I found pretty much all of those were unneeded old dependencies and I simply un-emerged them.

When you update Python, you can still get to your old version of Python by tacking on the version number to the python command, e.g., /usr/bin/python2.4. Since the Python updater uses emerge to install your python dependencies in the site-packages of your Python installation and emerge unmerges the old versions, your old Python probably doesn’t have all the site-packages any longer. This is only an issue if you find you need the old Python.

For me, I have a couple applications that didn’t quite want to work with Python 2.5 for some reason. I decided to use VirtualEnv to work on those applications.

The Steps

  1. To install virtualenv, you need setuptools. That package was one of the packages transfered to the 2.5 site-packages install and was no longer available with python2.4.

    To get around the issue, use ez_setup.py to install a 2.4 version of setuptools instead of using emerge.
    cd <working dir>
    wget http://peak.telecommunity.com/dist/ez_setup.py
    /usr/bin/python2.4 ez_setup.py setuptools

  2. I simply used the virtualenv command that came with python2.5 but changed the interpreter to by python2.4 instead of python.

    cp /usr/bin/virtualenv .
    # edit virtualenv to have the correct interpreter line
    vim < or whatever editor > virtualenv
    ------ snip local virtualenv ------
    +#!/usr/bin/python2.4
    -#!/usr/bin/python
    ------ snip ----------------------
    ./virtualenv --no-site-packages <virtual env install dir>
    cd <virtual env install dir>/bin/
    ln -s python2.4 python
    cd <working dir>

  3. Use the old python in it’s virtual environment:

    source <virtual env install dir>/bin/activate
    python
    Python 2.4.4 (#1, Mar 5 2008, 10:47:15)
    [GCC 4.1.2 (Gentoo 4.1.2)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>>

Anyway, that’s it. You can have a virtual environment that uses any version of Python you like on your system. I imagine this procedure would be somewhat similar on pretty much any Linux Distribution.

Virtual Hosting TurboGears Applications on Mac OS X Leopard

For a couple years now, I’ve been learning and applying various tricks for developing and hosting multiple Python web sites on my development machines. During that time, I made a migration to Mac OS X. Most setup files for python applications and libraries work out of the box on the Linux distributions I’ve tried. For OS X, you can find a lot of prebuilt packages for the necessary dependencies but those packages want to be installed in the OS X system library location for Python.

If you want to host or develop more than one site with Python, you’ll need to use virtual environments. This applies equally to any OS, not just Mac OS X. There seem to be a few more tricks to getting things up and running on OS X however. Here is the process I went through to get TurboGears applications hosted in virtual environments on OS X.

Continue reading “Virtual Hosting TurboGears Applications on Mac OS X Leopard”

Keeping a process running

Have you ever had a process that dies on occasion? For me, I hate that situation and prefer to fix the software as opposed to have a monitor that restarts the process when it dies. I’ve run into a case lately however, that has defied me for a solution to my dying process. I think it may be a hardware related issue but haven’t tracked down the cause yet. Anyhow, I read an email on the Provo Linux User Group in which the poster referred to PS-Watcher. I thought I’d give it a try for kicks.

After installing the program and reading through the documentation, I found that PS-Watcher is really quite nice. In addition to monitoring the results of the ps command, you can add custom actions that occur at the beginning or ending of the monitor cycle ($PROLOG and $EPILOG). You can also customize actions to be taken based on the number of processes, memory size, and a few other useful metrics.

For most situations where you want to monitor a process and take action, I think PS-Watcher will probably do the job nicely. After all this however, I decided what I really wanted was a little script that did a custom restart of my particular web server when the test URL wasn’t functioning properly. I decided to simply run it on a scheduled interval with cron. I’ve placed the script below for all to glean information from or make fun of as appropriate. Feel free to provide some additional tips as I don’t claim to be a “Bash Jedi Master”. The following script sends a request to the web server and parses the response for a string that lets us know the server is working properly.

#!/bin/bash

user="<the user my process is running under>"
port="<the port>"
okresp="^OK$" # I configured a test URL that returns OK if the server is up and running right.

# make a simple HTTP request to send
req="GET /lbuptest HTPP/1.0

"
# send it using netcat
resp=$(echo "$req" | nc localhost $port)
# test for the ok string
ok=0
echo "$resp" | grep $okresp 2>&1 >> /dev/null && ok="1"

# you could really place whatever actions you want here.
if [[ $ok != "1" ]]; then
/etc/init.d/<my process init script> restart
fi

The process I’m having trouble with is a TurboGears web application. I don’t think this is a Python problem however. Like I mentioned before, it only happens on this one server so I think I’ve got a hardware problem. Either way, if you found this page searching for TurboGears information, you might as well be interested in my TurboGears Init Scripts.

YUI Image Uploader Example with TurboGears

After completing the YUI Image Uploader, I received a lot of requests for a working example. I didn’t originally create a working example, because that requires server functionality that this server didn’t have. I’ve remedied the situation and have completed an example with TurboGears. Of course, any server side language or framework will do as long as you have the ability to upload and store an image.

Continue reading “YUI Image Uploader Example with TurboGears”

Similarity of texts: The Vector Space Model with Python

I’m working on a little task that compares the similarity of text documents. One of the most common methods of doing this is called the Vector Space Model. In short, you map words from the documents you want to compare onto a vector that is based on the words found in all documents. Then, you find the cosine of the angle between the vectors of the documents that you want to compare. This is called the cosine measure. When the cosine measure is 0, the documents have no similarity. A value of 1 is yielded when the documents are equal.

I found an example implementation of a basic document search engine by Maciej Ceglowski, written in Perl, here. I thought I’d find the equivalent libraries in Python and code me up an implementation.

Continue reading “Similarity of texts: The Vector Space Model with Python”

Programming a client for the WHOIS protocol

I have a little task that involves programmatically determining whether DNS servers are set correctly for a domain. Since this project is written in Python, I first set out to see if there were any “whois” clients already available for Python. I eventually found rwhois.py, which is a whois client with recursive ability. I noticed it hasn’t changed since 2003, but thought that if it works, that shouldn’t be much of a problem.

My first run of the program resulted in an error. The client successfully found the registrar information for my domain, but failed to parse and display it. There was a “NoParser for: whois.godaddy.com” exception. I set out to analyze the rwhois.py client and the whois protocol and see if I couldn’t either fix it or come up with something for a replacement.

Continue reading “Programming a client for the WHOIS protocol”