This is the beginning of a series about full Jenkins automation. I am calling that approach jenkins-as-code. The goal is to configure every aspect of Jenkins and its pipelines from a central git repository. We will leverage groovy scripting, jobDSL and shared libraries to not only codify the build/deploy pipelines (pipeline-as-code), but to also bootstrap and configure Jenkins from scratch (e.g., credentials, authorization, theme and job setup).
I recently started to move away from my SRE/DevOps position to dive deeper into the world of Information Security Engineering. This series is a good way to share some knowledge about Jenkins and at the same time have a manual for myself to be able to setup a Jenkins in a pure -as-code fashion.
It is usually a good idea to have as much as possible setup in an as-code approach, because it offers:
- clear definitions and automation, i.e., clear state definition and reproducibility
- rollouts from central servers (offers audit logs)
- a workflow like common software development, i.e., testing and reviewing is easier
Since Jenkins 2.x we can use pipeline-as-code approaches like Travis or CircleCI do. However, the overall (plugin) configuration and setup of job interfaces is not covered by pipeline-as-code. For a complete jenkins-as-code approach we also want our configuration and job interfaces treated as-code and be able to rollout any changes dynamically through a jenkins job interface without the need to restart Jenkins.
In the first part of this series we will discuss the basics:
- What are the layers in jenkins-as-code?
- What are the relations between those layers?
Next, we will create a Jenkins docker image packaged with a configuration/seeding job interface and pipeline definition, which uses a central shared library from GitHub to rollout configuration and seeding changes. Finally, we will have a closer look at contents of that central shared library.
Layers and Terminology
In order to avoid confusion it is very important to define the different layers and terms which will be used throughout this series.
Jenkins-as-code describes an approach to codify and automate every layer of Jenkins. In order to achieve its goal the approach leverages jobDSL, configuration and pipeline definition scripts.
A job interface describes the job in the Jenkins UI. We use a job interface to declare input parameters and to trigger the job. Preferably, a job interface is created using the Jenkins jobDSL plugin. When a job is triggered it loads the pipeline definition.
A pipeline definition is a groovy script which is loaded by a job interface when a job is triggered. Those groovy scripts are also known as pipeline-as-code - an approach also known from cloud CIs like Travis or CircleCI.
A configuration script is a groovy script, which interacts with the Jenkins eco-system to configure Jenkins and its plugins.
A jobDSL script is a groovy script which programmatically describes job interfaces. It is read by the Jenkins jobDSL plugin to create the job interfaces.
Seeding refers to creating all job interfaces from a single pipeline (the seeding job). That pipeline executes the jobDSL scripts in order to create all job interfaces.
Layers and their Interaction
Layers in jenkins-as-code:
Interaction between the layers:
Jenkins Docker Image
Now that we have covered some basics, let us begin with the hands-on work and create a docker image for our Jenkins.
We will use the following directory structure:
docker-image ├── Dockerfile ├── plugins.txt ├── init-dsl │ └── ConfigurationAndSeedingPipelineDSL.groovy └── init.groovy.d └── init.groovy
The following sub-sections show the files in detail.
Dockerfile and Plugins
We want to build an image which automatically bootstraps our Jenkins with an initial job interface for configuration and seeding.
Also, we must ensure that some plugins are installed.
Initial Pipeline Interface
The seed and configuration job interface is created through a jobDSL script.
init.groovy.d for initial setup
init.groovy.d is a directory in the jenkins home directory, which contains configuration scripts. Configuration scripts inside that directory are executed in alphabetical order at Jenkins boot time. This is ideal for setting up seeding and configuration job interfaces.
This script first adds a SSH private deploy key to Jenkins with access permissions to the shared library repository. Further, the script creates a configuration and seed job from a jobDSL script. That job will be used to pull the shared library from git and rollout its configuration and jobDSL definitions.
When you build this docker image and start the container,
init.groovy.d will add your (mounted) SSH deploy key and create the configuration and seeding job interface. When this interface is triggered, it will load the central shared library and run our pipeline defintions.
Shared Library Repository
Lets have a look now at how our shared jenkins library looks like.
We will use the following directory structure for our shared library:
shared-library ├── resources │ ├── config │ │ ├── auth.groovy │ │ ├── credentials.groovy │ │ ├── slack.groovy │ │ ├── theme.groovy │ │ ├── sshd.groovy │ │ ├── github.groovy │ │ ├── timezone.groovy │ │ ├── baseURL.groovy │ │ ├── globalEnvVars.groovy │ │ ├── globalSharedLibrary.groovy │ │ └── slaves.groovy │ ├── init │ │ └── ConfigurationAndSeedingPipeline.groovy │ └── jobDSL └── vars
vars directory will contain our job pipeline (build/deploy). Those pipelines will be called from a
Jenkinsfile inside one of our projects, but we will focus on that in a later part of this series and can leave it empty. Also the
resources/jobDSL directory which is intended for job definitions will be left out for now. We will come back to in a later part of this series. For now we focus on the
resources/init holds initial configuration and seeding pipelines which leverage scripts in
resources/jobDSL to configure and seed jobs.
In addition, we need a configuration and seeding pipeline script in the shared library repository.
The scripts loaded and executed in this job definition, e.g.,
credentials.groovy are discussed in the next part of this jenkins-as-code series.
We now have a pipeline definition for configuration and seeding. That pipeline defintion is triggered by an initial job interface for configuration and seeding. The initial job interface is bootstrapped by our docker image (
The next part of the jenkins-as-code series will focus on the configuration scripts, e.g., OAuth authentication, theming, slack and credentials.