[OH-Dev] Server backup status

[Asheesh Laroia]

Excerpts from Asheesh Laroia's message of Tue Mar 27 18:09:46 -0400 2012:
> Hey all,
> 
> There's been an open ticket for creating a reasonable plan for restoring 
> the OpenHatch site in case of a disaster on our main server.
> 
> As background: every night, we encrypt a system backup and send it to 
> rsync.net, who is donating space for us for that purpose.
> 
> I spent a few hours today doing the following:
> 
> * Preparing a virtual machine (right now, on rackspace cloud) to download 
> a backup from rsync.net, and making this download process automatic.
> 
> * Chrooting into that backup, and writing commands into a Makefile that 
> restore that backup into working order.
> 
> The first of these, since it downloads the backup, requires secrets that I 
> shouldn't make public -- the encryption and SSH keys we use for rsync.net. 
> If you're on the Login Team, I can share this repo with you if you're 
> quite careful with it.
> 
> The second of these is available here: 
> https://github.com/openhatch/oh-restore
> 
> *Crucially*, the public oh-restore repository has a "make check" command 
> that is the restore process's final self-check. If you want to know, "Can 
> I believe that these backups restore what's needed to make the site work?" 
> then you should make sure you're satisfied with that stanza. Right now, it 
> only tests that the Django app can run; it needs to start testing the blog 
> and wiki and bug tracker, too.
> 
> I'll complete the automatic restore process shortly (as per 0.12.02 
> release goals). Once that's done and shipped in 0.12.02 and tested to my 
> satisfaction, I aim to add a weekly task to Jenkins where anyone can see 
> that it's tested periodically and works (probably won't be ready this 
> month).

Sorry to reply to my own mail. Further resources if you're interested:

* http://openhatch.org/bugs/issue527
* http://openhatch.org/bugs/issue528
* http://openhatch.org/bugs/issue530
* https://openhatch.org/wiki/Backups_of_the_live_site

As the automated restore comes to fruition, I'll update those, so feel free
to add yourself as nosy to those bugs.

-- Asheesh.