[OH-Dev] [issue531] Either adjust our use of zc.buildout so it actually works, or transition us to virtualenv

[Asheesh Laroia]

New submission from Asheesh Laroia <asheesh at asheesh.org>:

The current state of things
---------------------------

We use a tool called 'zc.buildout' (zc is short for 'Zope Corporation' that
first wrote the tool) to create a little Python environment for OpenHatch's
code and its dependencies.

The reason we want to create a little Python environment is that the OpenHatch
site, which is contained in a Python module called 'mysite', depends on other
Python modules. We use two files, buildout.cfg and setup.py, to specify what
version of those modules we want, and where to download them.

This has the following advantages:

* Users run two simple commands to get started:

 python bootstrap.py
 bin/buildout

Buildout then installs the dependencies

* The dependency modules don't have to be installed as root.

This is crucial -- the main point of the "little Python environment" is
that users can run the OpenHatch code without altering their main system's
set of Python modules.

What's bad about it
-------------------

There are approximately three problems with the setup we have now:

1. It is not transparent

When 'buildout' runs, it is not clear about why it is doing what
it is doing. It is also not clear about where it is downloading
dependencies from.

This leads to people feeling a little confused -- buildout appears
to be a magic black box that successfully builds a dev environment...
...unless it fails to.

2. In some environments, buildout seems to refuse to download anything

This is pretty killer (in a bad way). It means that new contributors
are faced with utter failure and no reasonable way to debug it.

This is the most show-stopper of the problems with buildout. Careful
reading of the buildout code might help understand why it does this,
if you can create an environment that reproduces it.

(One easy way to create such an environment, I think, is to take our
Vagrant configuration and run it against an Ubuntu Lucid virtual machine.)

One downside to buildout is that it seems to be a whole bunch of complex
code, and it downloads buildout whenver 'bootstrap' runs.

3. If the system Python environment includes a version of Django newer
   than what we use in OpenHatch, Buildout uses that newer version,
   often leading to incompatibility

If you do:

 sudo apt-get install python-django

and you get a 1.3.x version of Django, and you then run buildout, buildout
will let the system provide that 1.3.x version to the OpenHatch code.

Often there are backwards-incompatible changes between versions of Django.
The result is that a new contributor will say, "Gosh, I wonder why the OpenHatch
code is so broken" and, since they didn't get any error message or otherwise,
think we just ship broken code.

We use a buildout addon called djangorecipe, which I *think* is supposed to be
able to avoid this situation, but my new opinion on djangorecipe is that for the
small things we want to do, we should just ditch djangorecipe.

Reasonable possible solutions
-----------------------------

There are about two ways forward, as far as I can tell:

1. Diagnose the problems with buildout, and fix it

If we can root-cause and fix the download problem, that would be enough for
me!

2. Replace buildout with the a simpler setup based on virtualenv

I think that this is probably even better. To see an example of how another
project uses virtualenv and setup.py, check out the install instructions
for debexpo:

 $ git clone git://git.debian.org/debexpo/debexpo.git 

There, you can read the README file.

The debexpo documentation suggests using a virtualenv whose files live in
~/.virtualenvs/expo/. I think it would be better if, instead, the virtualenv
lives in "." -- the directory that holds the code.

----------
messages: 2287
nosy: paulproteus
priority: bug
status: unread
title: Either adjust our use of zc.buildout so it actually works, or transition us to virtualenv

__________________________________________
Roundup issue tracker <bugs at openhatch.org>
<https://openhatch.org/bugs/issue531>
__________________________________________