Archive for the 'Software' Category

Upgrading Software

I can never resist. When a new version is available I have to have it. Upgrade to WordPress 3.3 was seamless. Devils is always in the details though. I’ll probably find a post or a plugin that doesn’t work the same way.

Oh well, I’m on the latest and greatest right?!

Git Rebasing Tips

So there you are, using git to store, track and share your code with a few others. Maybe many others. You make a change, someone else makes a change, one or the other of you pulls the others changes and BAM.. you get that merge commit. It really isn’t harmful of course, but it cleans up your repository a lot to get rid of them. So…

  1. When you pull, use –rebase.
    git pull --rebase

    There. It did it for you. All your commits go after the commits on the remote origin.

  2. You forgot to do that. No prob…
    git pull # oh man.. I forgot.. merge commit
    git rebase origin/master # There.  Pretty easy isn't it.

So of course, you might have the same issue if you were working on a branch. You check out master, make some commits, go back to your branch, make some more, etc. It works the same way.

# on your branch
git rebase master

There. You can pull in the master branch changes but put all your branch changes after so you don’t have to have the merge commit again.

So you wrote that branch, changed master, and then you merge your branch back to master and get a merge commit again. In this case there isn’t a ref to rebase off of so intuitively like remote/origin or master. No prob, just use gitk or git log or whatever to find out what the sha1 of the old master was before you made your merge.

git rebase 

There. No merge commit again.

Now, there are a bunch of more advanced things rebase can do. You can use it to edit commits that weren’t quite right. You can also change the order of commits, edit the commit message, drop commits completely, and other cool things. You can do it all interactively too…

git rebase -i  # fun for all!

I leave you to study that part on your own.

YUI Image Uploader gets a code repository

I’ve created a github project for the YUI Image Uploader.

You can download the scripts and example page here:

This will also facilitate user contributed changes and backend scripts.

MS Money and UBS Mappings for csv2ofx

I added the MS Money Report mapping from the csv2ofx wiki and merged in Arthur Liu’s UBS mapping to csv2ofx.

A quick reminder for the way mappings work: The built in mappings are made to work as an example for your own CSV files. You might happen to have a CSV file from one of the banks or software used by the example mappings, but likely, you have your own bank and own CSV file format. Copy the src/csv2ofx/ file to your own location and name it Then edit/add mappings as you need to suite your own needs.

As always, contributed mappings are welcome back.


Converting financial CSV data to OFX or QIF import files

As a side project, I created a CSV to OFX converter that applies custom mappings to CSV data to export them to QIF or OFX files. This morning, I added the ability to override the built-in mappings with custom mappings that suite your needs.

Basically, you can take financial data from any institution and modify one of the existing mappings to provide the information needed for each export format. Once you’ve done that for your bank, you can import CSV data and export OFX or QIF for import into whichever financial software you prefer.

csv2ofx requires wxPython. You can retrieve the latest source with git:

> git clone git:// 
> cd csv2ofx
> # use csv2ofx from the source directory
> ./csv2ofx
> # or install it site-wide
> python install
> # csv2ofx installed in path

You may also download a zip or tar archive from github if you prefer to grab a copy but don’t want to track the source repository. Go to the source repository for csv2ofx and click the download link.

There are directions in src/ and the README file for modifying mappings to suite your needs.

Contributions are welcome. Enjoy.

Update: 02/17/10
Added MS Money Rep and UBS support to csv2ofx

A few reasons braid is better than 40 lines of Rake.

I’ve been doing a lot of looking at the git subtree merge strategy for tracking remote projects. Ideally, our remote projects would be tracked with submodules. The issue is, we are developing our own libraries for use in our own internal projects. We commonly do major development on the libraries and the submodules at the same time. While submodules work, we find the overhead of having to repeatedly commit in each submodule and then in the parent project somewhat prohibitive to being productive. A good source code management system should help, not get in your way of being productive. While git helps and is out of the way for the most part, we found this not to be the case when developing our own submodules. (They are much better suited when you’re tracking a 3rd party library.)

That being said, the subtree merge strategy lets you use your repository more naturally. When you commit, everything is committed and when you pull, you don’t have to worry about updating the submodules to the correct state, switching to the master branch, or spending time worrying about whether or not you just broke the state of your working directory. On the other hand, subtree merges have a little bit more overhead to deal with when you set them up and when you want to push back to the library you’ve made changes to. Referring to the link above, it takes a number of commands to pull in the library and you need to understand what is happening.

The braid project has been set up to make managing subtree merges a little easier. It has been criticized for being too complex and able to be replaced with subtrees by themselves (see the 40 lines of Rake post.)

Here are a few reasons I think braid is not too complex.

  • It does diff’s correctly.

    In order to find the diff of your merged in subtree against the library, you can use git diff-tree. This is quite daunting to do correctly in a shell. In order to get a patch that really represents what the diff is, you have to do a diff against just the subtree, not the project root or path. You get this by using git rev-parse to find the pulled in tree:

    # you need the base revision of the last time you pulled the library in (via fetch) 
    # and did the merge.  The revision is not what you see in gitk or by browsing the parent
    # of the merge, you find it with rev-parse.
    $ git rev-parse :
    # next, you need the sha1 of just the subtree directory of the commit that you want to 
    # get the diff for (HEAD or a previous commit will work too)
    # this can be retrieved with ls-tree
    $ git ls-tree HEAD path/to/lib
    # after parsing out those two sha1 ids, you can use diff-tree to get the correct diff
    $ git diff-tree   --src-prefix=path/to/lib --dst-prefix=path/to/lib

    Part of the problem with a shell script is finding the sha1 ids. When was the last time you imported the library? What if there are multiple parents? Braid takes care of these things by storing some meta information about the subtrees it manages in a .braids file. It will save you a lot of time later.

  • Braid stores the merge strategy for you.

    When you shell script your solution, there is no meta information about your merge strategy. You can manually add additional scripts to merge with a squash or full type merge, but you’ll have to remember to use the correct script when you update your library. Braid takes care of this with it’s meta data again.

  • Braid caches for you.

    There is a problem with fetching huge remote repositories in git. The creator of braid pointed this thread out to me that outlines the problem. Braid can maintain a cache of the remote repository for you which allows you to use the subtree merge strategy effectively without worrying about git’s fetch problem.