Friday, January 9, 2015

Setting Environment Variables for Docker with Fig

For the past few months I've been playing around with Docker, and so far I've had a ton of fun. The documentation is excellent, and in one simple command you can start experimenting. After going through the tutorial, one of my first goals was to figure out the best way to create a Docker image for a spring-boot service. My initial goals were to make it easy to set environment variables, since our projects follow the twelve-factor methodology. One of the several factors we follow is the third factor (III. Config), which recommends storing the config in the environment. While this has many benefits, one of the downsides is the tendency to create a lot of environment variables because it's quick, easy, and well defined. This makes it difficult to configure, test, and run the service. But as we will see, Fig will not only make it easy to set environment variables, it will also provide many other benefits.

Docker Image
Let's first start with a pretend service called the logging service. It's a Java service created with spring-boot. Here is a basic Dockerfile:

FROM dockerfile/java:oracle-java8
COPY logging-service-0.1.0.jar /data/
EXPOSE 8080
CMD ["java", "-jar", "logging-service-0.1.0.jar"]
FROM - the base image I start with. In this case it's the dockerfile/java base image with the Oracle JDK 8 tag.
COPY - here I copy over the jar so it's present in the image
EXPOSE - this tells Docker the container will be listening on port 8080 at runtime
CMD - here I've defined a default command to run which will start the service

Next we need to build the image:
docker build --tag="jlorenzen/logging-service:v1" .

Docker Run
Now we could run our new service by executing this command:
docker run -dP jlorenzen/logging-service

That's great but let's imagine the logging-service requires the following environment variables: ENV_1 and ENV_2. Here is how you would run the service while also setting the environment variables:
docker run -dP -e ENV_1=value1 -e ENV_2=value2 jlorenzen/logging-service

That's a basic example, but you can image how nasty it could get if your service required a dozen or more environment variables. The docker run command also has some other nice options for setting environment variables. For example, when using the -e option, if you provide just the name like -e ENV_1 without a value, than that variables current value will be used. Or you can use the --env-file option to specify a file that contains a list of environment variables. While this all works, it's really not enjoyable having to remember all those options and commands. That is where Fig can help. And it not only helps us easily set environment variables, but it also makes creating containers simpler and reproducable by anyone anywhere.

Fig
Fig is basically a simple utility that wraps Docker making it easier to create and manage Docker containers. In our case we will use it to run our logging-service image and set the environment variables. Here is a simple fig.yml file:
logging-service:
  image: jlorenzen/logging-service
  ports: 
   - "8080"
  environment:
   - ENV_1
   - ENV_2
As you can see I didn't specify any values for the environment variables. That's because I already have them defined in my host using direnv and Fig will just automatically use them. So in my case I have a local .envrc file that contains the following:
export ENV_1=value1
export ENV_2=value2
This allows me to set all my environment variables in one place. Here is the command I can use to start the container:
fig up

That's it! Much simpler than the corresponding docker run command.

Ideal World
What would be the best of both worlds is if Fig supported the docker run --env-file option and that it could read in a file containing export commands which is required by direnv. It seems support for the --env-file option in Fig is coming soon, so we are halfway there

Tuesday, November 18, 2014

Example Using Grails Promises

I was recently playing around with the Asynchronous Programming features in Grails using Promises, and wanted to share an example that went a little beyond a simple example. In case you are using an older version of Grails, the asynchronous features where added in Grails 2.3. While there are a lot of useful asynchronous features in Grails, for this article I'll only focus on using Promises. Promises are a common concept being introduced in many concurrency frameworks. They are similar to Java's java.util.concurrent.Future class, but like all things with Grails/Groovy, Grails has made them easier to use.

First, before showing you an example, go ahead and run grails console under an existing grails project. If you don't have one, install grails (see GVM) and run grails create-app. Using the grails console will allow you to quickly run these examples and experiment on your own.

Basic Example

import static grails.async.Promises.task
import static grails.async.Promises.waitAll

def task1 = task {
    println "task1 - starting"
    Thread.sleep(5000)
    println "task1 - ending"
}

def task2 = task {
    println "task2 - starting"
    Thread.sleep(1000)
    println "task2 - ending"
}

waitAll(task1, task2)
This would output:
task1 - starting
task2 - starting
task2 - ending
task1 - ending

More Complex Example
Let's say you wanted to list the states of 5 zip codes. Here is what that would look like if we did it synchronously:

["74172", "64840", "67202", "68508", "37201"].each { z ->
    println "getting state for zip code: $z"
    def response = new URL("http://zip.getziptastic.com/v2/US/$z").content.text
    def json = grails.converters.JSON.parse(response)
    println "zip code $z is in state $json.state"
}



And the output for that would look like:
getting state for zip code: 74172
zip code 74172 is in state Oklahoma
getting state for zip code: 64840
zip code 64840 is in state Missouri
getting state for zip code: 67202
zip code 67202 is in state Kansas
getting state for zip code: 68508
zip code 68508 is in state Nebraska
getting state for zip code: 37201
zip code 37201 is in state Tennessee

And here is what it would look like using Grails Promises to make it asynchronous:

import static grails.async.Promises.task
import static grails.async.Promises.waitAll

def tasks = ["74172", "64840", "67202", "68508", "37201"].collect { z ->
    task {
        println "getting state for zip code: $z"
        def response = new URL("http://zip.getziptastic.com/v2/US/$z").content.text
        def json = grails.converters.JSON.parse(response)
        println "zip code $z is in state $json.state"
    }
}

waitAll(tasks)

The asynchronous output would look like this:
getting state for zip code: 37201
getting state for zip code: 68508
getting state for zip code: 67202
getting state for zip code: 64840
getting state for zip code: 74172
zip code 74172 is in state Oklahoma
zip code 37201 is in state Tennessee
zip code 64840 is in state Missouri
zip code 68508 is in state Nebraska
zip code 67202 is in state Kansas

Each time you run the asynchronous version it will output a different order because the tasks are running asynchronously. The waitAll() method will block until all tasks complete.

Thanks to jeremydanderson for helping me figure out how best to use the collect method.

Tuesday, November 11, 2014

Groovy Spring Bean for Static Factory

I started playing around with Grails again recently and ran into a problem trying to create a bean in the Grails resources.groovy file for a static factory. After several frustrating hours trying to find the right combination, I eventually stumbled upon an answer.

The factory I was trying to create a bean from was the JAX-RS Client API class ClientBuilder.newClient() which returns a Client object.

Here is what the bean definition looks like in my Grails resources.groovy file:

import javax.ws.rs.client.ClientBuilder

beans = {
    httpClient(ClientBuilder) { bean ->
        bean.factoryMethod = 'newClient'
        bean.destroyMethod = 'close'
    }
}

Then in your Grails service or controller you can autowire or inject the bean by doing the following:
class FooService {
    def httpClient

    def get(url) {
        return httpClient.target(url).request().get()
    }
}

Hope it helps the next person.