Stop Wasting Precious Time

  • Author Scott Lowe
  • Published 27 Oct 2023
  • Category tutorial

Google defines Continuous Integration (CI) as:

a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.

We're big fans of CI because it allows us to move faster with more confidence. Good CI practices bring with them a peace of mind for your dev ops people–have you ever fucked up a manual deploy before? I have, and it wasn't fun. Luckily, most of these stressful, yet critical processes can be delegated to robots without much work, which frees your developers to focus on what they do best: building things.

In this post, we'll walk through Castle's current CI setup in hopes that this will help others join the club. Be warned, non-nerds, it's gonna get technical.

our current setup

Castle runs Jenkins on a $10 / month Digital Ocean droplet. I like Ubuntu, so if you pick another flavor of linux then I can't guarantee that the commands in this post will work (also, be sure to install git and any other dependencies your project has). Once you've got Jenkins up and running, you'll need to create a new account on GitHub specifically for Jenkins to have access to your repos. After creating the GitHub account, ssh into your new droplet and run sudo su jenkins to switch to the jenkins user, then generate new ssh keys and add them to your new GitHub account. Give that user access to your repo (read & write). Finally, use the browser to login to Jenkins, click Manage Jenkins in the left sidebar, click Manage Plugins, then install both the Git plugin and the GitHub plugin.

Now you're ready to configure your Jenkins jobs. We maintain 3 jobs for each repo:

Let's go through them in reverse order, as deploying is easily the riskiest of these operations–so you really don't want it to be done manually by humans! If you're going to just set up one of these jobs to get your feet wet, automating your deploy is where I recommend you start.

Deploy the release anytime a PR is merged into master

In order for this to work, you'll need your project to be using the ssh remote (git remote set-url origin [your ssh remote, ex: git@github.com:castle-dev/castle-dev.github.io.git] if not) on your prod server, the jenkins user on your droplet must have ssh access to your prod server (copy the contents from jenkin's pulic key ~/.ssh/id_rsa.pub on the CI server into ~/.ssh/authorized_keys on the prod server if not), and the user account you're sshing into on prod needs read access to your repo (generate a new ssh key pair with ssh-keygen for that user and add it as a deploy key on the repo if not). Once all this is done, you're ready to configure the job in Jenkins.

To do this, create a new item from the Jenkins sidebar. Select Freestyle project and call it whatever you'd like (we use [repo name] deploy). Now, fill in the config:

Here's an example deploy process from our web app:

ssh [user]@[host] <<EOF
  cd [project folder] && git checkout master && git pull && gulp build:dist && cp -r build/dist/* /var/www/public/ && exit
EOF

Notice the <<EOF/EOF tags & how all the commands after sshing are on one line? That's important, otherwise Jenkins will report fails as passing when using ssh. Also, note that gulp build:dist and cp -r build/dist/* /var/www/public/ are commands that are specific to our web app project and your results will likely vary.

And that's it! Now, everytime someone adds new commits to the master branch (which if you're trying at all will only be when someone approves a PR from develop into master), Jenkins will ssh into your production server and update your app. There's also an easy-to-setup Slack plugin that you can install to have Jenkins notify a Slack channel about build results, but that exercise is left for the reader.

Bump the version anytime a PR is merged into develop

If you've made it to here then the hard work is done, as going from 1 to 2 is much easier than going from 0 to 1 when it comes to computers.

Before we discuss the details, let's reflect on the inspiration for this one. Software moves very quickly, so developers have conventions for describing their changes over time. Semantic Versioning or semver for short has become the standard for tagging software updates by dividing all changes into 3 distinct buckets: major, minor, and patch. Major updates are the ones that introduce breaking changes into the application, minor updates are the ones that add new features without breaking any existing functionality, and patch updates are the ones that fix bugs but don't add any new functionality.

At Castle, we bump the version everytime a feature PR is merged into develop, and the type of bump (major, minor, or patch) is determined by the changes included in the pull request. To communicate this information from the individual who built the feature to the individual who's reviewing it, we append major, minor, or patch on the end of the title of all PRs from feature branches to develop. We started using semver by just manually updating the version number and creating a corresponding git tag when merging PRs, but that quickly became tedius so we used some gulp magic to automate it like so. That gulpfile contains 3 tasks for updating the version number: gulp bump:major, gulp bump:minor, and gulp bump:patch. This was much better than changing the numbers by hand every time, but after a while checking out develop, pulling, running the appropriate gulp command, pushing the new commit, and finally pushing the new tag grew old as well.

To that end, we setup Jenkins to automatically run the appropriate bump command when a PR is merged into develop for us, and now we don't ever have to think about it again. Now let's dive in!

Create a new job (or copy the deploy job) and configure it like so:

Here's the example bump process from our web app:

git checkout develop
git pull origin develop
type=$(git show | grep -oE '[^ ]+$' | tail -1)
if [ $type = "major" ] || [ $type = "minor" ] || [ $type = "patch" ]
then
gulp bump:$type
git push origin develop
git push origin develop --tags
fi

The only complicated stuff here is the type assignment, $(git show | grep -oE '[^ ]+$' | tail -1) which says replace me with the last word of the last line of the last commit message. If that matches major, minor or patch then Jenkins will run the appropriate bump command and push the commit and tag up to GitHub. (Note that I'm not a whiz with refspec yet so this job will run more often than just when PRs are merged into develop but that's okay because the shell script only takes action when the previous commit message ends in major, minor or patch which won't be an issue if the only way new commits get to develop is through a merged feature PR.)

Lint & run unit tests anytime a new commit is pushed to GitHub

So you've done the heavy lifting, now let's take it to the next level. At Castle, we lint and unit test every commit that is pushed to GitHub. With all the hard work out of the way, this job is suprisingly easy to set up:

And finally, here's an example lint & test process from our web app:

bower install
npm install
gulp build
gulp jshint
gulp test:unit

Thanks for reading, hope it helps! :]