A Note on porting MATLAB to Python/NumPy

As I am getting into some of the more mathy parts of my thesis I’ve ended up doing a lot of work porting MATLAB code to python and I wanted to share my experience and wisdom for others who might try this as well.

First of all why am I doing this? Well I prefer working with open tools and as a friend of mine recently pointed out if you are using proprietary software to “prove” something it can’t really be called valid because you can’t confirm what the software is really doing. A good point indeed. Now I know there is Octave (which if you are a MATLAB user I highly recommend you check out it’s 99.9% compatible and of course totally free and open source) which I’ve used quite a bit, in fact that’s what I was using to run the MATLAB code while porting. But I prefer having my code in python because I find MATLAB code harder to read and the architecture of code in MATLAB just isn’t quite to my liking. Also it’s much handier to have everything in python for when I bring the application back to computer vision because I’ll have OpenCV to work with. And finally as a personal preference I also like working with my “samples as rows” in my data matrices  probably because of the computer vision I do.

So if you’re like me and you’re porting MATLAB to NumPy and you’ve found your way here then lets review the basic task at hand. We’re actually after 3 main goals here and it’s worth pausing and considering them separately:

1) Port from MATLAB to NumPy

2) Switch from “by cols” mentality to “by rows” mentality

3) Clean up the architecture and probably restructure as a class

Step 1 - Having done this a few times the first thing I recommend is this: write the code line for line the same only using NumPy, yeah I know you want to get in there and make it nicely formatted as a class or other but trust me do a one to one conversion first and leave it as “by cols”.  You can use def myFunc(*args) in python to mimic MATLAB’s nargin functionality. There is a great document on the NumPy website that highlights the main differences and what to use where, notably how NumPy handles by element operations and dot products is different so read up! I should also mention that for step 1 (we’re leaving it as by col for now) if you are doing any reshaping of the data using built-in functions NumPy defaults to row major while MATLAB uses column major. You can force NumPy to prioritize columns in reshape by using the “F” option in either flatten or reshape.

Afterwards you’ll want to test your output to make sure it gives the same result. If you end by having to work backwards and determine where the python code deviates numerically here are a few simple things that can help:

Python/NumPy MATLAB/Octave
savetxt('data.csv',X,delimiter=',') csvwrite('data.csv',X)
X=loadtxt('data.csv',delimiter=',') X=csvread('data.csv')
print X disp(X)
raw_input() pause()

Also work with the same input! That should be obvious. Then work your way through using print statements, saving data out and waiting for user input if required. NumPy and MATLAB print slightly differently but it is pretty easy to compare across. (I used to have a formatted print statement for one of them that matched the other but I lost it damn, shouldn’t be too tough to redo if you need it though, there also may be something online already I never checked).

Step 2 - switch from “by cols” to “by rows” now by this I mean that in a data matrix containing samples MATLAB users tend to think of each column as a sample while other people tend to thing of each row as a sample. It’s really just a different way of arranging the data but it can be tricky to revert. When you are switching col to row paradigm you’ll need to switch some of the equations around - get out a pen and paper and write out a simple case of the math for both row wise and col wise and check it through. It took me a very long time to find a simple ordering bug that resulted in a square matrix rather than a scalar. gah!

Step 3 - Now finally go nuts and format it nicely! make it a class encapsulate it whatever floats your boat! I’ve also found that a lot of MATLAB code was clearly written to test if something works and thus when you actually go to use it it’s quite awkward so I’ve found the usability goes up a lot with python.

And that’s it I hope this helps someone out there. I mainly just really wanted to write this up because I just successfully finished a tricky port and was really stoked about it. I’ve got one more to do so maybe I’ll refine this post a bit shortly!

tl;dr NumPy »> MATLAB

read more

Going Places is Awesome! but Travelling Blows..

I just got back from an awesome trip to India for 2 weeks but it made me realize that while going places is awesome travelling blows.

Story time, let me begin:

At the last-minute before leaving Canada our contact from IIT Kanpur requested we bring our projector to run the ARPool demo because they were having a tough time finding one that matched the specs we need (it’s important we get a good enough throw length or we can’t project on the whole pool table). Now bringing the projector was something I had been trying to avoid because it’s heavy and awkward to travel with and I had some concerns over getting it into India. Too late now I packed it up quickly before heading to Ottawa to meet Andrew for our flight. We decided that rather than have more bags Andrew would take one bigger suitcase and put the projector in his suitcase, a decision he’ll later come to regret… dun dun duhH! We also decided to go checked baggage at the last-minute because we were worried about having issues with all of our liquids and gels on carry-on, a decision we will also come to regret (yeah seriously never again, I was already on team carry-on before this but now I’ll never falter).

We fly Ottawa to Chicago where the flights are apparently notoriously delayed and ours was no exception causing us to eventually miss our flight from Heathrow to Delhi. Getting re-booked wasn’t too big a deal really  but we did have to wait in a massive queue (see what I did there). We finally get to Delhi at 4:00 AM local time, it takes us a super long time to get through customs thanks to another very long line and a border agent who can’t speak a word of English. See there was this one box we didn’t know how to fill out we weren’t sure if it was supposed to be where we were landing in India, where we were flying from or where we were actually from. The border guard just kept pointing at the blank and I kept trying to ask what to write then he looks at Andrew’s which is also blank, stamps it, lets him into India and continues to stare at me, what the heck!? Eventually I to get into India to and we head over to the baggage carousel where we wait and wait and wait. Our baggage didn’t make it - great.

We fill out all the forms and give IIT Kanpur as the address and phone number for delivering our baggage - all a little sketchy am I ever going to see this luggage? Next challenge, we were able to get an email off to the conference organizers letting them know we were delayed while we were in Heathrow but we can’t see their response without WiFi of which there is none (see WiFi rant). We pay for access to the airport lounge where we shower and finally get WiFi and are able to talk to someone at IIT Kanpur. After a lot of back and forth we finally settle on a new plan to get to Kanpur which is to meet their student volunteer in Delhi the next night at 1:00 AM and then take the train.

After another minor adventure of actually finding their student volunteer the following evening we started making our way to Kanpur. To his credit the student volunteer was great after we found him. The sleeper train itself was an experience, they really pack you in there wow, I prefer to call the bunks shelves as that’s more what it seemed like lol. The train is 6 hours and departed at 6:30 in the morning. After arriving in Kanpur we still had to drive for about an hour to get to the IIT. At this point we’d essentially been travelling for over 40 hours, had not slept when it was local night and I really had no idea what day it was. After getting to IIT Kanpur we make a few arrangements for the ARPool demo before finally getting some sleep.

Back to our luggage situation: so the day after we arrived in Kanpur we call the airport and our bags are in Delhi but there is a problem with Andrew’s bag - it didn’t clear customs because of the projector grrrrr. My bag is allegedly being shipped to Kanpur though. I did actually get it the next morning which wasn’t so bad and in the mean time we had been able to buy some new underwear and shirts on campus for not much $$. The rest of our time at IIT Kanpur was fairly enjoyable you can read about it here.

After the festival wrapped up we drove 2 hours to Lucknow for a flight back to Delhi to meet the student volunteer from BITS Pilani. We then travelled to Pilani another 6 hours in a vehicle (note that it’s only 200 Km… welcome to driving in India). I should mention now that Andrew still doesn’t have his luggage because customs is only open from 11:00 AM to 3:00 PM on weekdays. We meet the organizers in Pilani and check out the space for our exhibit (it’s 3:00 AM). The space is good but they also couldn’t get a projector with the right specs. We decide that Andrew will travel back to Delhi in the morning while I will remain in Pilani and see if we can figure something out.

The next day (seems weird saying that as I went to bed at 5:00 and Andrew didn’t even sleep but he does that a fair bit) Andrew heads back to Delhi and I start planning how to set up ARPool. We come up with a solid plan for mounting and then actually relax for the rest of the day. Andrew returns with the projector that evening after running around Delhi and the airport all day and having to pay a $300 import tax on the projector. The demo at the festival itself was a success!

We spend the next 5 days relaxing and touring around India seeing the sites which was incredible! We still had to do a fair amount of driving from place to place but it was nice to see the country side. I’ll make another blog post about the actual trip and post photos because this post is all about the travel shenanigans!

On the way back we get to the airport at around 10:00 PM for our flight which leaves at 3:40 in the morning, the rep from our tour company actually has to talk to the airport guard to let us in the airport because we’re too early for our flight. I sleep until about 1:15 at which point I go over to check in. The line is moving super slow and I kind of start the whole discussion with those around me of whether or not we’re going to make it if they don’t pick up the pace. At this point we realize that our fight out of Delhi has been delayed 4 hours and that’s why the line is so slow - they’re re-booking everyone. I wait in the line for about another hour and a half until it’s my turn. I get re-booked Heathrow to Ottawa direct - that’s actually better than what I had before and now I’m getting home sooner sweet! if only that was the end of it. I headed through security and enjoyed my now complimentary breakfast - gee thanks and 2 corona, I don’t care that it’s 5:00 AM!

I land in Heathrow and make my way to Terminal 3 (I know this airport good now) where I have to go through security again. Somehow I get selected to have both my bags completely searched and they throw my stuff everywhere. This is the second time today that my bag has been completely searched. This doesn’t really bug me for privacy reasons it bugs me because it was slow and stupid - what’s the point of looking at my bag under x-ray if you’re going to throw my stuff everywhere? I make it to the flight counter and present this paper that British Airlines gave me to Air Canada and they stare blankly apparently they don’t know what this paper is wtf! I go back to BA and it turns into BA telling Air Canada that I am booking on this flight and confirmed and Air Canada saying I’m not while in the meantime the flight takes off with me still in Heathrow - awesome guys! I blame BA, Air Canada and British Airport Security for delaying me so long at security for missing that flight. BA then books me on a flight to Toronto later that afternoon which worked out okay. I have to transfer back to Terminal 5 for this flight and of course go through security again, but hey do they check my bags this time when I have plenty of time - nope! After my flight and a 3 hour bus ride and I was finally back home - roughly 37 hours of travelling.

Seems like on the way there I got screwed by checking my bags and on the way back I got screwed by going carry-on, there’s just no winning!

tl;dr going places is awesome but travelling blows

read more