Making the seeeduino stalker (v1) work with avrdude and a Makefile

I got one of these Seeeduino Stalker v1 boards recently, for $22 as they were having a fire sale on them over on Seeeduino Studio to make way for version 2.0. The board carries an Atmel 328 and has an RTC chip plus a micro SD card already soldered on.

When I got the board, I got an UARTSBee programmer to go with it. This is basically just a USB to Serial adapter (FTDI chip). I got the version 4.0 board which unfortunately has 6 pins outgoing, which makes it incompatible with the 5 pin input of the Stalker. It is, however, easily possible to solder 5 pins onto the holes provided directly behind the 6 pin output. Then the programmer can be stuck straight on to the Stalker's input port.

To get started with programming the Stalker I downloaded the arduino software from the CrossAVR repository on the openSUSE build service. This is a java based GUI which simply provides an editor for text files and uses gcc to compile and avrdude to upload the software to the arduino (the Stalker). Using the arduino GUI was quite easy and uploading worked without a problem.

However, I wanted to use a standard Makefile and avrdude to flash the software to the Stalker. This turned out to be a bit more complicated than I thought because the arduino GUI knew that it was necessary to pull the DTR before flashing the software. This had the effect of resetting the Stalker, putting it into the bootloader. Only then will the Stalker accept a firmware flash. What confused me was that by running strace on the arduino GUI I saw that it was calling avrdude with -p stk200v1. When I tried this directly with avrdude it didn't work (because avrdude wasn't pulling on the DTR line to reset the Stalker so it would spring into the bootloader and accept a flash). It turns out that avrdude has already thought of this and provides a -p arduino option. Using this worked out of the box. Below is a video of the Stalker with the UARTSBee programmer stuck on vertically. Attached to it is a breadboard with some LEDS just to show the pins set to output and blinking.

bundesliga2go with mongodb, zeromq, gevent, web.py and websockets

bundesliga2go (switch to mongo branch) is a project which I started with Vlad so that we could look at the german Bundesliga results on a smartphone. It was originally our Hackweek 6 project. The results are provided by the webservice available at OpenLigaDB. They are provided in XML format.

The first thing we noticed was that it was going to be difficult and slow to just write a mobile web client which would query OpenLigaDB every time that a result or update was required. That is because when matches are in progress (typically Saturday at 15:30 german time) OpenLigaDB can be quite sluggish. Also, it seemed ridiculous to query already available and final results over a live internet connection with a mobile phone. On top, it can get expensive. We figured we could provide a better solution using modern smartphone browsers' offline mode.

So, the first thing to do was to cache the responses from OpenLigaDB. This was done by creating a suds (SOAP client for Python) synchronise script which simply asked OpenLigaDB for match data and stored this in a SQLite database. We then created an API (Python) for accessing the local database and for ensuring it was kept up to date with OpenLigaDB (we used APScheduler to schedule updates), especially when matches were in progress. The mobile client frontend was done with jQuery mobile. The actual web server itself was done with web.py.

Before long it became apparent that SQLite (or any relational database) might not be the most appropriate storage solution. The kind of data we were storing was more suited to a document type storage form so we moved to mongodb. This was helpful because we could now use techniques such as map/reduce to quickly compile the Bundesliga table and top scorers.

We implemented a first working draft by having the web.py server return JSON to the mobile browser. The browser stored the data using localStorage. This meant that it was only necessary to talk to the web.py server when updates were requested. Obviously, we needed to get around the security fence that browsers erect around remote JSON. We thought about using jsonp to do this but that seemed clumsy. What we came up with was CORS - the client first sends a HTTP OPTIONS request to the web.py server. The server authorises the client to use JSON returned over standard GET or POST and the client follows up immediately with the actual GET/POST request whereupon the web.py server returns the JSON data. The client then parses, stores and displays the data.

This worked fairly well until we hit the issue of live updating. On match day it is of course highly desirable to get live updates (something like 'push'). As the mobile application is HTML5 based, websockets are the obvious solution - except that they are not supported by every browser. We figured we'd provide websockets where available and try to degrade gracefully to long polling or the CORS method if necessary.

With respect to the actual live updates, what we essentially wanted to do is to push the updates to the client as soon as we stored them in the local database. This screams for an event based, non-blocking solution, so we chose zeromq for the messaging and gevent with gevent-websocket for the non-blocking websocket part. Some preliminary testing shows that actually does work as a broadcast server. More testing is necessary, though.

With websockets we are running into problems with some browsers. Testing on desktop machines, only Opera 11 supports version 7 of the websocket protocol. Firefox 7 and Chrome are already at version 8 of the protocol. The gevent-websocket library has not yet been updated to support version 8. The author is actively working on it though.

The code is available on github. Check out the mongo branch. This will be merged into master at some stage but I'm not really up to speed with git at all and I'd rather invest time in getting the application done than in having a super clean git repository.