Printing on one line with Python

You might be used to using sys.stdout when you want to print multiple items on one line with Python. Did you know you can do the same thing with the print statement?

Ok so I’ve been using Python for a long time and have written a lot of lines of Python code. But this one is pretty simple and I’m kind of embarrassed that somehow I didn’t know about it.

In the past, I’ve used sys.stdout when I wanted to print multiple things on one line, e.g., you want to print periods to show progress.

import sys
for i in range (10):
 sys.stdout.write ( "%d.." % i )

I also knew you could use print with more than one item.

print 1,2,3, "Hi"

But the other day I inadvertently found that you can continue to print on one line if you just leave the trailing comma….

for i in range(10):
  print "%d.." % i, # no newline appended
print # add a newline after the loop.

And who cares you say??? Well now I don’t have to import sys and use sys.stdout when a simple print can do just fine 🙂

Quick convert raw g711 uLaw audio to a .au file

I had reason to playback some raw g.711 audio data. I made the following script to convert the data to a .au file that is playable.

From the AU file spec.. which is way simple by the way:
You just need to add 6 32-bit header flags.

raw2au.py:

import struct

header = [ 0x2e736e64, 24, 0xffffffff, 1, 8000, 1 ]
o=open('out.au','wb')
o.write ( struct.pack ( ">IIIIII", *header ) )

raw = open('in.raw','rb').read()
o.write(raw)
o.close()

You can probably see that this would be extremely simple to use with other audio formats too. Modify as you need.
Header Bytes:

  1. Const AU identifier: 0x2e736e64
  2. Size of Header: 24.
  3. Size of File: use 0xffffffff for unknown, or you could replace with the actual size.
  4. Audio format. 1 is uLaw. From the spec choose whatever you need if it isn’t uLaw.
  5. Sample rate. uLaw is always 8000.
  6. Channels. 1 channel in this case.

The Python install system needs an overhaul

Perhaps this is more of a rant than a useful blog post. I do plan on posting something useful though, so bear with me. First, a little background.

I’ve been working on installing a website based on Django for the last while. As part of the process, I wanted to bring in all the dependencies of my project with a setup.py file so I can easily move the site around to different development machines or server environments as needed. I wanted to make sure I didn’t install different versions on a different machine, so I’ve provided a packages directory and install the file with the dependencies locked at specific versions and the packages directory noted. Easy install then just uses the package I place in the packages directory and every place I install this website, I get the same setup.

My setup.py snippets

setup (
 # ...
 install_requires=[
  'Django=1.1',
  'MySQL-Python==1.2.3c1',
  # etc 
 ],
 dependency_links = [ 'packages' ],
 # ...
)

So far so good. I should point out that some have suggested pip as a replacement to easy_install. For my rant portion of this post, both pip and easy_install don’t offer any help. If I’m wrong about pip, somebody please point out the proper way to do this..

My problem begins when I find a package with no setup.py file. The package I’m attempting to use is the forum module from Sphene Community Tools. As I download and go through their documentation, I find that they pretty much expect you to just add the sphene package to your python path. That doesn’t fit with my installable website model though so I want to add a simple setup.py file and install it with the rest of my packages.

Issues:

  1. Package Data

    Turbogears contributers worked on a find_package_data function that can recursively add your package data. The fact that this isn’t included yet in the distutils api is quite annoying. I copied the file from another project [1]. I also added ‘.svn’ to the ignore directory list since I was building from the svn view.

  2. Extra Data

    Just as annoying as the package data problem is the static data. I originally thought I could just include the static directory in data files like this:

     data_files=[ ('static', findall('static') ]
    

    That just puts every file recursively found in the static directory all in one static directory for the install. Instead, I modified the result returned by the find_package_data function to use the data_files format. Here is my completed setup.py file:

    import os
    from setuptools import setup, find_packages
    from finddata import find_package_data
    
    packages=find_packages('sphenecoll')
    package_data=find_package_data('sphenecoll')
    static = find_package_data('static','sphene') 
    # not in correct format
    static_dict={} # dir -> files
    for path in static['sphene']:
       dir, file = os.path.split(path)
       dir = os.path.join('static', dir )
       files = static_dict.setdefault( dir, [] )
       files.append(os.path.join('static',path))
    
    setup(
     name='sphene',
     version='0.6dev',
     packages=packages,
     package_data=package_data,
     package_dir={'sphene':'sphenecoll/sphene'},
     scripts=['dist/scripts/make-messages.py', 'dist/scripts/compile-all-sph-messages.py' ],
     data_files=static_dict.items()
    )
    

    If someone else is interested, you can put finddata.py and this setup.py in the sphene/communitytools directory and build your own egg. Not really the point of this post, but perhaps someone will find it useful. Actually, the Sphene tools should be installable I think. The expectation to just put the directory in your PYTHONPATH isn’t very flexible.

  3. Install from source

    Both easy_install and pip didn’t actually copy any of the package data or data files into the installed site-packages. I had to use setuptools bdist_egg and then installing the egg caused the files to be copied correctly.

  4. No uninstall

    This one isn’t really an issue with this install in particular, but it is always annoying.

Maybe one of these days some Python developer will come up with a good solution to the install mess. I’m too busy working on the projects I’ve got to get done aside form the occasional sidetrack that installing Python modules causes.

1: I copied paste.util.finddata http://trac.pythonpaste.org/pythonpaste/browser/Paste/trunk/paste/util/finddata.py

Launching wxPython apps with an iPython shell

Suppose you want to run your fancy wxPython application but have a shell in the background to peek and poke at certains settings, help debug, and possibly even use an API that your program provides to automate tasks. iPython has built in wx support (as well as support for other GUIs and frontends). So anyway, I’ve been playing around with this lately and have noticed an inconsistency that needs addressed.

Launching other GUIs with iPython does not block. You instantly get a shell and your application loads. The “-wthread” option for wx applications executes any passed in script before dropping to the shell though.

Problem:

> ipython -wthread myapp.py
> # application runs here
> # no shell yet
> # you exit the application
"Welcome to iPython"
>>> 

If you happened to do things like this, you’d get the expected response:

> ipython -wthread
"Welcome to iPython"
>>> import myapp
# application loads
# you instantly have a prompt
>>>

Solution:

Tell iPython to launch your app after it starts:

> ipython -wthread -c "import myapp" -i
# app loads
"Welcome to iPython"
# you've got a shell
>>>

Anyway, for other threading backends in iPython, the shell doesn’t block while the application loads. Perhaps there is a way to get the wx backend to behave the same. In the mean time, I’m using this workaround without any serious issue. Perhaps I’ll find a better solution or submit a patch to iPython one of these days.

Converting financial CSV data to OFX or QIF import files

As a side project, I created a CSV to OFX converter that applies custom mappings to CSV data to export them to QIF or OFX files. This morning, I added the ability to override the built-in mappings with custom mappings that suite your needs.

Basically, you can take financial data from any institution and modify one of the existing mappings to provide the information needed for each export format. Once you’ve done that for your bank, you can import CSV data and export OFX or QIF for import into whichever financial software you prefer.

csv2ofx requires wxPython. You can retrieve the latest source with git:

> git clone git://github.com/mulicheng/csv2ofx.git 
> cd csv2ofx
> # use csv2ofx from the source directory
> ./csv2ofx
> # or install it site-wide
> python setup.py install
> # csv2ofx installed in path

You may also download a zip or tar archive from github if you prefer to grab a copy but don’t want to track the source repository. Go to the source repository for csv2ofx and click the download link.

There are directions in src/mappings.py and the README file for modifying mappings to suite your needs.

Contributions are welcome. Enjoy.

Update: 02/17/10
Added MS Money Rep and UBS support to csv2ofx

Custom Derived Classes for wxPython XRC resources

First of all, this isn’t a topic that is bran new or which requires new documentation. I have learned a few quirks about the process for creating custom controls, panels, frames, and other elements with XRC files in wxPython and I thought I’d write up a little post.

Let me point you to the two most important resources. The wxPython wiki has pretty much all you need to now to get started. First is the TwoStageCreation process that the XRC implementation uses to create classes. The second is the UsingXmlResources page.
There is also the XRCTutorial. But you should be past that if you’re trying to create custom controls.

Here is a summary of the basic process. Continue reading “Custom Derived Classes for wxPython XRC resources”