Continuous Integration and Delivery pipeline for the Department of Transport’s On-Demand Transport platform

There are several tooling options that you can use when selecting a CI/CD pipeline and the chosen solution for this environment was Jenkins. It is a widely popular, open source, very well integrated into AWS and provides many plugins to make it a flexible solution.

The solution was required to be highly available, scalable, cost effective, and expandable to other projects and teams. It also needed to provide a mechanism for manual decision gates to manage the deployment of the software into the controlled environments. The defined pipelines should also follow the same infrastructure as code principle that is used to define the application’s infrastructure and be maintained in the source code repository of the application.

There are several ways that Jenkins can be deployed on AWS but to meet the requirements defined above, the following architecture was used:

• A single Jenkins controller/primary is deployed to an EC2 instance with an Auto Scaling Group to provide the required high availability.
• A spot fleet is used for running the Jenkins build agents responsible for building the software artefacts, providing a cost effective and scalable compute platform.
• The EC2 deployment agents are provisioned by Jenkins, allowing the deployment to the various environments. These agents run with an instance profile that allows the agent to assume a cross account role to deploy the infrastructure and application artefacts only to the environment in which it is entitled.

Pipeline

The pipeline is triggered by a pull request to the Master branch of the source code repository, where the build and unit tests are executed. A static code analysis is also performed and only if these steps pass and the appropriate quality gates are met, is the code deployed automatically to the development environment.

Each stage in the pipeline calls out to “hooks” using a standard naming convention to allow the stages to be customised for each project or component.

Build Pipeline

Automated Development Pipeline

A deployment to the test environment is then triggered following a successful deployment to the development environment where the pipeline awaits an approval before continuing. This process is repeated in the remaining environments until it eventually reaches the production environment.

Approved Deployment Pipeline