Automating Your SOA Ruby Development Environment Using Foreman and Bash

The Monolithic App

When you have one monolithic ruby stack,  getting a developers machine to run the stack for smoke testing etc. is really easy.  If the application is just Rails based without any background jobs you simply start up your database,  rvm or rbenv  into the correct environment and then type

rails s 

If you have a delayed jobs you might also run something like the following in another terminal window

rake delayed_job:work

If that is your entire stack it might be a reasonable workflow. But then let’s say that you also add Sunspot so that you can add search to your app. It’s time to automate your stack startup. Using the Foreman gem the majority of the heavy lifting for this is really easy.
 
You add the gem to your Gemfile

gem foreman

Create a Procfile that looks something like this

web: bundle exec thin start -p $PORT
worker: bundle exec rake delayed_job:work
solr: bundle exec rake sunspot:solr:start

Install the gems and run forman

bundle install
foreman start

As time goes on your dev teams starts to grow, but each developer is using a different set of database credentials for their local database. Because database.yml has been checked in, each developer stomps on the other one’s database.yml whenever they commit code. You are not at the point of wanting to force common development machine configurations so you use environment variables and come up with a database.yml that looks something like this.

default: &default
  adapter: mysql2
  encoding: utf8
  pool: 5
  timeout: 5000
  username: <%=ENV['MYAPP_DB_USER_ID']%>
  <% if ENV['MYAPP_DB_USER_PASSWORD']%>
  password: <%= ENV['MYAPP_DB_USER_PASSWORD']%>
  <% end %>
  <% if ENV['MYAPP_DB_HOSTNAME']%>
  host: <%= ENV['MYAPP_DB_HOSTNAME']%>
  <% end %>

development:
  <<: *default
  database: <%=ENV['MYAPP_DB_NAME_PREFIX']%>_development

test:
  <<: *default
  database: <%=ENV['MYAPP_DB_NAME_PREFIX']%>_test

Now each developer can set their local environment variables to match their database setup, database.yml can be checked in and developers are not constantly over-writing other developers database configuration files.

Breaking Things Up To Scale

Some more time passes, your app usage and features grow until you finally need to break up the application to scale. You start off by breaking your app up into two services. Initially developers use separate terminal windows to start up each service when they need to run the environment locally. But then the usage numbers go up, the team starts to break out more functionality into services and before you know it you have 8 different services that need to be started.
 
Foreman works great when you are running everything from one root directory, but when you need to start to use multiple directories it doesn’t seem to provide a clean solution. We ran into this on a project and came up with the following solution.
 
First we created a common repo for setting up our dev environment. Then we created a file called common.env using the Foreman .env file syntax.

DB_HOSTNAME=localhost
DB_USER_ID=my_user
DB_USER_PASSWORD=mypass

APP1_DB_NAME_PREFIX=app1
APP1_PORT=3001

APP2_DB_NAME_PREFIX=app2
APP2_PORT=3002

APP3_DB_NAME_PREFIX=app3
APP3_PORT=3003

Each user then copies this file to .env in the same directory and makes and modifications they may want for their local environment. We then add the requirement that everyone on the team organize their code directory the same way so that our scripts know how to find things.

/app1
/app2
/app3
/developer-setup

If everyone is using RVM you can create a small shell script that can be run from the developer-setup directory under a common version of ruby and gemset that all of the services will share.

cd ../app1
bundle install
cd ../app2
bundle install
cd ../app3
bundle install
cd ../developer-setup

Then we create a script for every service so that it can be started up from our developer-tools directory

cd ../app1
bundle exec rails s -p ${APP1_PORT}

Finally we create a Procfile to start up our environment.

app1: ./start_app1.sh 
app2: ./start_app2.sh 
app3: ./start_app3.sh 

Now we can use “foreman start” from our developer-setup directory to start up our development environment. All of our services use the same version of Ruby. If yours are using different versions you will need to have separate developer-setup projects for each variant that will need to be kicked off individually.

Automate Everything

By automating your environment developers are not spending precious minutes starting up and shutting down multiple services. After a while those minutes add up to hours and the hours add up to days that your team can use to add new features or fix other parts of the system.