Continuous Deployment with Freight

Early on at Sentry we set up Jenkins to automatically deploy after a passing build. Eventually we wanted better control over where we were deploying, and when (i.e. branch FOO to staging). To solve this we started with simple parameterized builds, and effectively had something working. Unfortunately when it came down to adding external controls we hit the age-old API issues within Jenkins itself.

Enter Freight

When we started going heads down on Sentry this year, it was a goal to make sure our tooling was the most functional and accessible for error monitoring. While we don’t want to invest a lot of time into those things, we want to ensure our day-to-day workflows are smooth. We started off by looking at what tools exist, and found things like GitHub’s Heaven. Originally going down that path, we quickly (though reluctantly) decided we were better off writing our own, as we’re not rubyists, and some of the design decisions simply didn’t make sense for our security model.

Enter Freight. Freight is heavily inspired by GitHub’s Heaven project, but it has a few key distinctions:

• It’s not coupled to GitHub
• It can run entirely behind a firewall

Beyond that there’s a lot of things in common, but at the core they’re both just task runners.

A Simple Core

Just like Heaven, Freight is entirely focused on executing deploys, primarily through simple commands.

A few key things are already done for you:

• Multiple apps and environments.
• Management of the source repository (unique per app).
• Basic workspace management (repository is copied over, and deploy is executed from new location).
• Abstractions for deploy provider, validation checks, and notifiers.

In our case, we simply plug in our existing Fabric deploy scripts (using the Shell provider):

{
    "command": "bin/fab -a -i {ssh_key} -R {environment} {task}:sha={sha}"
}

Freight automatically fills these in with the appropriate values at runtime.

We’ve also bundled in checks for GitHub Context (we use CircleCI), as well as notifications for Slack.

API Driven

Freight is an API-first project. It’s becoming what is the modern standard in reusable and accessible tooling. The choice allows us the ability to quickly whip up a React-based frontend, as well as integrations such as hubot-freight:

We’ve also built out a very basic command line utility, freight-cli:

$ freight deploy --help
Usage: freight deploy [OPTIONS] APP

Options:
  --env TEXT
  --ref TEXT
  -f, --force
  --help       Show this message and exit.

$ freight deploy getsentry --env staging
Created new Task with ID = 404

$ freight tail 404
(waiting for output..)
>> Check has passed: github
>> Running ['git', 'fetch', '--all', '-p']
Fetching origin
Warning: Permanently added 'github.com,192.30.252.131' (RSA) to the list of known hosts.
>> Running ['git', 'clone', '/tmp/freight-repo-1', '/tmp/freight-workspace-70653104e33011e496e95a5015beb482']
Cloning into '/tmp/freight-workspace-70653104e33011e496e95a5015beb482'...
done.

An Early Preview

We’re huge believers in open source, and while Freight is fairly early, we’ve kept all of it’s development public. The source, as well as the intended goals are all available on GitHub. While today the setup requires you to have a little bit of knowledge around how Python packages work, it’s fairly quick to run on Heroku.

It’s our hope to continue the open source tradition that started Sentry wherever possible, and developer services are the lifeblood of a lot of what we do. Give it a try and let us know what you think.