Virtual Hosting TurboGears Applications on Mac OS X Leopard

For a couple years now, I’ve been learning and applying various tricks for developing and hosting multiple Python web sites on my development machines. During that time, I made a migration to Mac OS X. Most setup files for python applications and libraries work out of the box on the Linux distributions I’ve tried. For OS X, you can find a lot of prebuilt packages for the necessary dependencies but those packages want to be installed in the OS X system library location for Python.

If you want to host or develop more than one site with Python, you’ll need to use virtual environments. This applies equally to any OS, not just Mac OS X. There seem to be a few more tricks to getting things up and running on OS X however. Here is the process I went through to get TurboGears applications hosted in virtual environments on OS X.

1. Install Apple Development Tools

   To do most of these steps, including using VirtualEnv, you’ll need Apple’s development tools installed. In my opinion, if you’re developing anything, you’re bound to need them anyway so if you don’t have them, go on and get them here.

2. Upgrade to VirtualEnv

   Virtual Env is a nifty utility that creates a copy of your Python program and libraries, each modified appropriately to run in their own environment. You can create as many virtual environments as you like. This enables you to install different versions of the same library for different sites.

   You can install VirtualEnv in your system Python installation. At the terminal, type:
   sudo easy_install virtualenv


   Once VirtualEnv is installed, you’re ready to create an environment to work with:
   virtualenv --no-site-packages <virual env dir>


   Specify whatever directory you’d like the virtual environment to exist in. I prefer to start with a clean installation and install the necessary dependencies for my applications. You could however eliminate the –no-site-packages argument if you’d like copies of anything already installed on your system to be copied into your virtual environment.

   To start using your virtual environment, you need to source the activate script into your shell.
   source /path/to/your/virtualenvironment/bin/activate


3. Install TurboGears

   When I tried to install TurboGears with the tgsetup.py file provided on TurboGears.org, I received permission denied errors because the script attempts to copy tg-admin and other scripts to /usr/local/bin instead of the bin directory inside the virtual python environment. You can get it installed correctly simply by using easy install. For this however, you’ll have to tell easy_install where to find some of the eggs files built specifically for OS 10.5. I encountered a problem with one of the eggs however and had to install SimpleJson from the pypi directory and NOT from the TurboGears provided eggs.

   easy_install simplejson
easy_install -Zf http://files.turbogears.org/eggs TurboGears


   Just a side note, I don’t know if it is specific to my applications or not, but I always get a FormEncode TypeError when trying to run TurboGears applications after a new TurboGears install. I edited formencode/api.py as follows to fix the problem:

   # around line 193
try:
 return trans(self._messages[msgName], **self.gettextargs) % kw
except TypeError, e: # add this and the next line
 return trans(self._messages[msgName]) % kw
except KeyError, e:
 raise KeyError(
  "Key not found (%s) for %r=%r %% %r (from: %s)"
  % (e, msgName, self._messages.get(msgName), kw,
  ', '.join(self._messages.keys())))


4. Install Application Dependencies

   For me, I find it easiest to create setup.py files and use easy_install to install my application dependencies. Here are a few of my personal dependencies and challenges I had installing them in my virtual environment on OS 10.5.

   • psycopg2

     I prefer Postgresql for my database solution. The library that provides database connections to Postgresql is psycopg2. Unfortunately, the install files for this Python package want to install it into your system Python library. I couldn’t find a way to modify that. To use easy_install to set it up, you have to be able to compile it against libpq, which requires a local copy of the Postgresql development libraries.

     I found the easiest way to get the correct libraries installed is through Fink. Fink is basically an installer, written for OS X, that brings open source libraries and programs to the Mac. Installer’s have been written for earlier versions of OS X, but you’ll need to use the source release instructions for OS 10.5. I’m sure that will become untrue after a short period of time so you’ll want to check their site.

     Anyway, follow their instructions and install Fink. I noted the steps below. You may need to replace the SourceForge mirror for the download:

     curl -O http://easynews.dl.sourceforge.net/sourceforge/fink/fink-0.27.8.tar.gz
tar -xzf fink-0.27.8.tar.gz
cd fink-0.27.8
./bootstrap /sw
# from here, complete the setup questions according to your system and needs.
# Fink will take a while to install and compile.
source /sw/bin/init.sh # this enables fink commands in your shell
fink selfupdate # this updates the library package
fink index # rebuild the library index, I don't know if you really have to do this
fink install postgresql82-dev
easy_install psycopg2


   • PIL

     I use PIL to process images. I haven’t ever figured out why the PythonWare guys haven’t uploaded PIL to the pypi database for easy installation. Their setup.py file does work ok, you just have to download the source manually.

     In order to compile PIL on OS 10.5, you’ll need to have libjpeg. If you don’t require any JPEG support, I guess you could skip that step. I found an article about getting PIL to work on Leopard. I didn’t have the problem they specifically addressed though. Also, since I used Fink to install the Postgresql libraries, I thought I might as well use it to install libjpeg as well. You can reference the article for instructions on installing libjpeg manually if you didn’t install Fink.

     Here is my method:
     fink install libjpeg
curl -O http://effbot.org/downloads/Imaging-1.1.6.tar.gz
tar -xzf Imaging-1.1.6.tar.gz
cd Imaging-1.1.6
python setup.py install


   • PyXML

     I still use PyXML, even though it has been deprecated with Python 2.5. The Python 2.5 implementation of the xml package does not contain an xpath library. Eventually, I’ll have to rewrite my XML dependencies but in the mean time, PyXML still works fine.

     easy_install PyXML

     After installing this library, I noticed that I still couldn’t import xpath. I did a little debugging and found that the system xml library was being imported instead of the PyXML library. I don’t know why but the package for my virtual environments was named _xmlplus instead of xml. I just moved the package and that fixed the problem:

     cd <virtual python dir>/lib/python2.5/site-packages/PyXML...egg/
mv _xmlplus xml
# edit EGG-INFO/top-level.txt
vim EGG-INFO/top-level.txt
# change _xmlplus to xml

     Now, my PyXML took preference and my XML processing worked as it had on other platforms.

After getting TurboGears and all my application dependencies installed, I’m up and running just fine with Leopards native Python 2.5. I can now develop multiple sites without them overriding each other’s libraries. If you have a better way of doing some of this, or any of it becomes outdated, feel free to leave a comment.