The bitExpert Vagrant setup

Forced Motivated by Robert Reiz and his blog post about how to set-up a Vagrant environment I thought that it`ll be a good idea to quickly outline how we typically configure Vagrant in our projects. The bare minmium VagrantFile looks like this:

VAGRANTFILE_API_VERSION = "2"

Vagrant.require_version ">= 1.6.0"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

    config.vm.define "wheezy" do |wheezy|
        wheezy.vm.box      = 'debian-wheezy-64-puppet361'
        wheezy.vm.box_url  = 'http://vagrantboxes.loc/wheezy64_puppet361.box'
        wheezy.vm.hostname = 'myhost'

        # Forward a port from the guest to the host
        wheezy.vm.network :forwarded_port, guest: 8080, host: 8080

        # Share an additional folder to the guest VM
        wheezy.vm.synced_folder ".", "/vagrant", owner: "www-data", group: "vagrant"

        # This shell provisioner installs librarian-puppet and runs it to install
        # puppet modules. After that it just runs puppet
        wheezy.vm.provision "shell" do |s|
            s.path = 'shell/bootstrap.sh'
            s.args = "-d /vagrant/puppet/ -m init.pp -c /vagrant/puppet/hiera.yaml -v"
        end
    end
end

I want to outline a few important things here:

• We have a very strict naming convention for the different base boxes used by Vagrant. As use can see there is a wrong relation between the name of the box and the box url. In addition to that the box name contains all information needed to understand what the box itself contains. In the examle above you can clearly see that the Box is a Debian Wheezy 64bit system and it includes Puppet Version 3.6.1. Puppet itself is apt-pinned in the box as well as on all involved servers to avoid any side-effects with different puppet versions on the different boxes. Very rarely do we upgrade Puppet.
• In most projects we use the simple port-forwarding set-up of puppet because it's simple and works quite well for us. When using port forwarding we make sure that at least the web server is running on the same port inside the virtual machine as it is reachable from the host. This simplifies the application configuration and makes sure we can interact with the application from the host as well as from within the virtual machine. This makes it possible for us to run Phing scripts on the host which interact with the application inside the virtual machine without knowing that the script actually runs on the host. To make this possible we add the hostname used by the application to the hosts /etc/hosts file as well to the /etc/hosts file in the virtual machine.
• In most all projects we currently use Virtualbox as virtual machine provider. This is can be quite slow when the application is configured wrong. Make sure that you put any logfiles or cache folders outside of the shared folder to improve performance. Other options are to use a different virtual machine provider, e.g. VMWare Workstation which seems to perform very, very well. Or try using the rsync folders feature of Vagrant.
• For the provisioning we do use Puppet but we do not use the Puppet Provisioner but the Shell Provisoner. We use this set-up mostly for one reason: librarian-puppet. If you are not familar with librarian-puppet think of it as Composer for Puppet modules, or npm if you are a Javascript guy. There used to be a Vagrant plugin for librarian-puppet but it had one problem: It would try to execute librarian-puppet on the host than rather inside the virtual machine and I could not get it to run on Windows.It did not find a Puppet executable although it was accessible via the PATH variable. After about 2 days of hardcode Puppet/Ruby debugging (on Windows!) I changed the plan and switched to the work-a-round with the shell provisioner. A very stripped down version of the shell provisioner looks like this:

# librarian-puppet might need git installed
git --version >/dev/null 2>&1 || emergency "Git client not found. Please install git before running this script."

# Check for ruby gem to install librarian-puppet if missing
gem help >/dev/null 2>&1 || emergency "Gem not found. Please install gem before running this script."

# install required puppet modules via librarian-puppet
if [ `gem query --local | grep librarian-puppet | wc -l` -eq 0 ]; then
  info "Installing librarian-puppet"
  gem install librarian-puppet -v 2.0.1
  info "Installing puppet module dependencies"
  cd "${arg_d}" && librarian-puppet install --clean
else
  info "Updating puppet module dependencies"
  cd "${arg_d}" && librarian-puppet update
fi

# run puppet
info "Excuting puppet apply"
puppet apply ${arg_v} --hiera_config ${arg_c} --modulepath=${arg_d}/modules/ ${arg_d}/manifests/${arg_m}

First of all we check that git and gem are installed and then install librarian-puppet as well as the required puppet dependencies. After that we trigger the puppet run via puppet apply. The script is used within the virtual machine to run puppet as well during the provisioning process on each of the servers (with slightly different parameters obviously). I will blog a bit more about the whole set up in the next few blog posts and share some more insights in how we do use Puppet at bitExpert right now.