Box hugger no more

I've been using zc.buildout for years now to build the Pleiades infrastructure in a replicable way, and it's served the project well. The process of setting up a production environment has been this:

1. ssh in to production

2. git clone https://github.com/isawnyu/pleiades3-buildout $HOME

3. cd $HOME

4. virtualenv

5. bin/python bootstrap.py

6. bin/buildout install zope2

7. bin/buildout

8. Copy database to $HOME/var

9. Symlink $HOME to pleiades-production

10. supervisor restart all

The buildout script yields a database server, multiple app servers, an nginx load balancer, and Varnish caching proxy. Subsequent minor site releases are just a matter of pulling buildout configuration files and re-running step 6. Buildout is repeatable and, with care, idempotent. But the other steps are less repeatable and more manual. That I use screen to keep that shell alive shows exactly how much manual intervention is required. Although I've automated half the process, I've been the kind of devops person Subbu Allamaraju calls a "box hugger". Subbu says:

Two steps to cure box hugging – first, internalize the idea that the box you’ve just finished setting up meticulously is going to burst into flames the very next minute, second treat operations the same way as you would treat software development.

I'm well on the second step: all the Pleiades buildout configuration is version controlled, if not fully tested. But the first step, not so much.

I'm using a new project to reform my ways and be much less of a box hugger. To internalize the ephemeral nature of servers, I'm teaching myself to provision and configure Vagrant VMs with Ansible. My goal is to be able to deploy this project's sites to their production server using only Ansible, never logging in at all. Having no database in this project (it's all based on XML, on GitHub) makes this goal easier to hit. And it turns out that setting up Solr isn't going to be too tough, either. Thanks to this ansible-multi-solr project I've learned to write my own very basic Solr and Tomcat playbook. With just two commands

$ vagrant up
$ ansible-playbook setup.yml -k -i setup_hosts

and a short wait, I get a running Solr instance at http://192.168.35.10:8983/solr/.

I'm late to the party, I know, but Vagrant is killer. I'm also using it to test packaging and installation of Shapely and Fiona. I believe it was Whit Morris who directed me to Vagrant. Thanks, Whit!

I've used neither Chef or Puppet and chose Ansible because it's Python, uses familiar stuff like SSH and JSON, and because the playbook concept is a reasonable leap for me from Buildout. I'm enjoying it very much and hope to be able to contribute something to the project in time.

Thanks, Subbu, for providing the impetus I needed to make the leap from box hugging! I really would rather be developing than deploying and administering, and feel like I'm beginning to get a grip on the tools that will make that possible.