This site is an archive; learn more about 8 years of OpenHatch.

[OH-Dev] Current state of backups

Asheesh Laroia asheesh at
Tue May 1 17:28:17 UTC 2012

Hi all Devel folks,

Summary: The main OpenHatch server has daily backups, and we have a 
Jenkins job (with public logs) that, weekly, makes sure the server can be 
restored from those backups.


You can see the Jenkins job log here:

What we do is:

* Spin up a new virtual machine on Rackspace Cloud

* Log into it, and download the latest backups from

* chroot into the backup

* run a few commands to restore the server (e.g., restore MySQL from 
backups in *.sql.gz files, rather than relying on /var/lib/mysql ; start 
Apache; etc.)

* run a few commands to check that the backup actually restored things 

It's driven by an ad-hoc combination of Python, shell, and GNU make. You 
can find those scripts here:

When the job finishes, if it's successful, we rely on me to manually 
remove the virtual machine that it created.

There is some room for improvement, but I think we're on solid ground at 
this point.

-- Asheesh.

More information about the Devel mailing list