Setting up a Django CI with Jenkins and Git

After experimenting with Django for a while now, I decided that it is time for a Continous Integration for Django on my server in order to start some long-term projects. This is why in this post I will describe how to setup Jenkins for Django projects on Debian servers to detect changes to a Git repository, run tests in case of changes and deploy them on your Apache Server.

Note: This article is rather old and describes a very manual way of doing things. A more sophisticated way to bootstrap Jenkins in an as-code fashion can be found here.

I assume we have a Django project d_project at /srv/www/django/d_project running in a virtual environment at /srv/www/django/d_project/py-env.

Installing Jenkins

There are different ways to install Jenkins; here are 2 examples:

• You could register the Jenkins repository in your package manager and install it
• Since I already have a running Tomcat, I decided to download the latest jenkins war and deploy it in my Tomcat server instead

Consequently, since Jenkins is running within my Tomcat, the Jenkins user is - and in the following I will also assume that it is - tomcat.

Apache Configuration

Most likely you will run your Jenkins/Tomcat server behind an Apache Server or something similar. If Jenkins is running behind Apache, please ensure that your Proxy settings are correct. Here an example proxy configuration snippet for your Apache 2.x Virtual Host file:

Further, Django projects can be hosted via Apache with mod_wsgi.

Virtual Host File

Optional - Enable Coverage Reports

In case you want to use coverage reports, please install Violations and Cobertura Plugin in Jenkins with the help of its Plugin Manager. Also, you will need django-jenkins and coverage for your python environment:

(py-env)$ pip install django-jenkins
(py-env)$ pip install coverage

In order to use django-jenkins for your project, you have to add django_jenkins to your project’s INSTALLED_APPS. From then on you can execute tests using python manage.py jenkins or (with coverage reports) python manage.py jenkins --enable-coverage.

Also, you can define in your project’s settings.py which Apps should be tested by the jenkins runner by simply adding them to PROJECT_APPS.

Optional - Install Cleanup Plugin

I use Jenkins’ Workspace Cleanup Plugin to ensure that some files always get deleted before a new build starts. This is especially useful, if you only want a subset of files to be deleted/refreshed for sure. The Plugin can of course be installed via the Jenkins Plugin Manager. After the installation, inside each project configuration tab, you now have the option to activate a workspace sweep and define patterns.

Git

For Jenkins to handle Git repositories, you first have to install the GIT plugin. Of course you will also need Git available on your server and there are different ways to achieve that…

One efficient way to manage self-hosted Git repositories is gitolite. I like to use gitolite in combination with redmine and the redmine_git_hosting plugin. In case you use the same setup, you could do the following to enable Jenkins access to your repositories:

• Create a Jenkins CI user in redmine and give him read/write permissions to repositories.
• Create a public RSA key for the Jenkins user in $JENKINS_HOME/.ssh/ and save it in the redmine_git_hosting plugin. Be sure to NOT use a passphrase for the key, since otherwise Jenkins will be asked to enter the password each time he tries to access the repository.
• Make sure that your Git server is in the known_hosts list of your jenkins user. If not, you can manually use Git and you will be asked whether to add the server to the known_hosts list or not. From then on your jenkins user will be able to access the repository.
• Now, in redmine you can just add the Jenkins CI user to any project in which you need a CI.

Allowing Jenkins to restart Apache

Before we start here, please note that a publicly accessible dev version might lead to security issues!

In case you want Jenkins to also deploy your commit, it has to be allowed to restart Apache, so Apache can detect the changes to the code base. Add the following to your sudoers file:

tomcat ALL=NOPASSWD: /usr/sbin/apachectl

This enables our Jenkins user tomcat to have root access to apachectl.

Creating a new build job

In your Jenkins UI:

1. Create a new job

• Select New Item
• Specify build name and choose Freestyle project

2. Configure the job

• Under Advanced Project Options select Advanced and define check Use custom workspace
• Define the workspace to be /srv/www/django/d_project
• In Source Code Management select Git and define the repository URL (in case you use redmine with the git_hosting_plugin, this can be found in the Overview of your redmine project). Also, you can specify the branches to build - for a more sophisticated git project structure this is very important, but in this post we will simply build the master branch.
• In Build Triggers select Poll SCM and define a schedule such as H/30 * * * *. This will let Jenkins poll the repository every 30 minutes.

• In Build, specify a command to run the build/deploy script (e.g. /srv/www/django/deploy.sh)

Build/Deploy Script

The deploy.sh could look something like this:

Basically this script installs the project’s requirements, migrates model changes, lets Apache reload the source files and runs the tests.

That’s it! We now got a working basic Django CI! Happy integrating!