GSoC status report #2

It has been more than two weeks since the last (and somewhat late) status report, so here is the update about the new features I implemented the last two weeks.

OS X like “Go back to version” for Notes

Rainy now supports a TimeMachine like archiving of notes that are synced. Every note that is changed or deleted in the syncing process is saved with a revision number in a special archive table. Via the REST API the client can query the history for any given note, and possibly restore any previous versions (much like Dropbox file history). Since this a feature specific to Rainy and not part of the Tomboy REST API 1.0 specification, there is no support in any client whatsoever, yet. However, I plan to support it in the HTML5 web interface that I’ll be working on in the next weeks. During implementing, I hope to find any shortcomings in the API and fix them, so that at the end of the GSoC I can extend the API specification so that others can catch up on that. With the feature present in the Web interface, at least a user can view and restore older versions in the browser, in case he lost some important changes or wants to recover an already deleted note.

The note archive also supports the recently introduced note encryption, so make sure you don’t forget your password AND lose all your authenticated device keys (=OAuth tokens), else the whole archive will be unusable!

Approaching Midterm

My GSoC  is split up into pretty much two parts: In the first half (until mid-term), do all important server-side stuff like the new database support, note encryption and go-back-to-any archiving. The second half is more client-centric, as it covers a creation of an HTML5 backend (=client) to access the notes.

Since I managed to finish the server-side stuff a little early, I am right now polishing the code a bit and add more unit tests, to make sure new features are working as intended, stabilize the code, and to actually have something to show at the mid-term evaluation to my mentor. There is nothing more satisfying to a developer, than a list of (mostly) green lights after pressing the “Run tests” button 🙂 So far Rainy has around 120 unit tests (although some are modified derivations of tomboy-library), but most tests actually test against the real REST API with HTTP requests to a local server,  not against any mockups or mocked servers with predefined responses).

GSoC 2013 – First progress report

As you might know, I was accepted to work on Rainy during Google Summer of Code 2013. It is a great opportunity to turn Rainy into something great, and I am happy to give my first status update about what I’ve been doing the last three weeks and what I am up to in the next time.

Rainy has grown to an extend where I thought it might be a good idea to have public website (not only a blog about it) online, and for some month the home of Rainy has been and still is located at http://dynalon.github.io/Rainy. Check for releases, news, infos and updates that are Rainy specific.

Week 1: PostgreSQL Support & Cleanup

In the first week, I planned to add PostgreSQL support. Since I plan to add more database backends in the future (MySQL, MSSQL) I first had to refactor some classes and interfaces that allow to plug in other backends at a later point. This was the first time I got in contact with the Inversion of Control pattern, as I ended up refactoring most of Rainy’s classes to use constructor-injection and auto wiring using ServiceStack’s Funq IoC container implementation. This was very fun to learn, and I like the concept, especially since ServiceStack uses the IoC  constructor injection pattern everywhere and you can easily hook into it.

After the refactoring, I added the PostgreSQL backend, which was very easy since I already used ServiceStack.OrmLite as an O/R mapper which brings the necessary abstractions.

So, Rainy can now be used using either the SQLite or PostgreSQL backend, allowing to scale for a much larger number of users. My first benchmarks show that SQLite is actually faster than Postgre, but I measured only using a small number of notes and users. For larger databases I think PostgreSQL will outperform SQLite.

Week 2 & 3: Server side encryption

This is one of my favorite tasks that I added to the GSoC proposal, and I think will be the most valueable one. Especially since recent events in the media should have raised awareness of data privacy in the cloud to a broader audience 🙂

Unfortunately, Tomboy and Tomdroid do not yet support full client side encryption of notes, that would enable us to store them safely in the cloud. To overcome this, we can at least implement server-side encryption in Rainy. Notes are encrypted on the fly using AES256 before they are stored in the database, and decrypted transparently upon requesting the notes. The required keys should of course not be stored in plaintext in Rainy’s database, so that it is non-fatal if someday a Rainy server-instance gets hacked and the database is made public.

The crypto implementation was one of the harder pieces in my proposal, and I am happy to tell that it is done and has landed in the git master. For anybody interested in the details how the crypto and the key management works, I wrote developer docs that can be viewed here. Since a basic understanding of encryption is required, I will discus only a easy-to-understand approach here, for details see the developer docs.

What the encryption does

• works completely in the background and on the fly, no setup on the clients is necessary
• encrypts every note with a per-note key, which is itself encrypted with the users master key
• the master key is not stored in the database, but part of the users oauth credentials. That way, if the database gets public the notes can’t be decrypted without the users master key
• if you lose a device (say your cell phone with Tomdroid) and you immediatley revoke the access token for the device before an attacker finds it, the attacker cannot sync again (but might access the notes that are stored on your phone). No need for changing the password or revoke all tokens
• Change your password at any time, without re-authenticating any device or client

What the encryption does not

• replace client-side encryption (which is sadly not available yet)
• protect your from malicous server administrators – they can still modify Rainy’s code and hijack your keys which should only reside in the RAM

What’s next

For the next weeks I’ve planned to add a Time-Machine like “Go back to version” function into Rainy, that will save all note versions into the database and allows to access any older revision. Very much like the TimeMachine mode os OS X or Dropbox allows to go back to any previous version of a file.

This requires the Tomboy REST API to be extended and might not land into the client software too soon. But it  will  be in the  HTML5 Webinterface client that I will also be doing as part of the 2nd half of the GSoC. More about that in a future blog post 😉