Imagine a product development paradigm where you build software so that it is always in a state where it could be put into production. A deployment pipeline is essentially; an automated implementation of your application’s build, deploy, test and release process.
From a business perspective, deployment automation allows you to get changes – planned or unplanned, into production reliably and quickly. This allows you to gather user feedback as early as possible and develop and maintain software that is lean, contains only features stakeholders need and avoid software bloat.
With Agile development, teams are able to create functionality ready for use more frequently. This can create a backlog for operations teams as they struggle to deploy the applications as quickly as new builds are ready.
In addition manual deployments come with a series of downsides
a) manual deployments that are not consistent across environments
b) slow; neither repeatable nor reliable
c) require documentation (often outdated)
d) manual deployments hinder collaboration as not all team members can execute the deployments
To keep pace with the increase in deployment demand you need to create a deployment pipeline. Without deployment automation, your company will struggle to unlock the full potential of Agile.
A deployment pipeline helps to deal with the challenge of feature release backlog -and bridges the gap between development and operations teams by breaking up your build into stages. Each stage providing increasing confidence until the product is end user ready.
Continuous Delivery (CD) is a software development strategy that enables organisations to deliver new features to users fast and efficiently. The foundation of CD is to create a repeatable, reliable and incrementally improving process for taking software from concept to customer.
Continuous Delivery enables a constant flow of changes into production via an automated software production line. Continuous delivery can be thought of as an extension of continuous integration, aiming at minimising lead time, the time elapsed between developers writing new code and this new functionality being used by live users, in production.
As the difference between each build decreases, the work becomes more tedious for operations teams since consecutive packages are likely to use the same set of resources and do not need major changes in middleware configuration or infrastructure. This increases demands on operations personnel and makes their job repetitious.
To achieve continuous deployment, the team relies on infrastructure that automates the various steps leading up to deployment. After each integration successfully meets these release criteria, the live application is updated with new code.
A good build pipeline will have a relatively high degree of parallelization. The pipeline tasks won’t all run in series – otherwise you are effectively daisy-chaining together a collection of build jobs, and the value of the system is significantly reduced.
For small projects parallelisation is less of an issue but with a large project that has slow unit and acceptance tests, and deployment steps take a long time to complete, then the whole pipeline may take hours to complete.
The working day could have ended before the final stage is executed. This is far from ideal because if there’s a problem with any part of my pipeline, you want to know about it fast – so that you can fix it. A good practice is to parallelise the tasks where possible. Acceptance tests, unit-test coverage and static analysis can often be performed in parallel.
You can imagine your deployment pipeline as a series of jobs or steps on your Continuous Integration server. Each job increases the confidence in the software and brings it closer to release. Initial phases are short and provide feedback as quickly as possible, later steps typically require more time and possibly require human interaction. If any build in the pipeline fails consecutive steps are not triggered. The key takeaway here is; automation. The whole process is fully automated and requires almost no human effort.
The 5 Stages of a deployment pipeline
1) Commit stage
The purpose here is to provide fast feedback for developers and prepare application binaries. It checks out new sources, compiles them, executes unit tests, executes simple smoke tests, prepares the application package ready for deploy, executes static code analysis tools, possibly verifies there are no architecture breaches. This phase executes fast – no more than 10 minutes. A successful finish states that the application works correctly at a technical level.
2) Automated acceptance test stage
The purpose here is to assert that the system works at both the functional and nonfunctional level. This stage ensures us that the application delivers value to the end user. To achieve this you would usually deploy the full application and go through the most important and common user paths. This phase is significantly longer than the commit stage; it could take hours.
3) Manual test stage
Commonly known as User Acceptance Testing (UAT). Testers can deploy the application with one click on a chosen environment. The application is possibly deployed to staging environments. This phase involves human verification of the current application. This stage should only involve tests that cannot be automated and integrated into to the automated acceptance test stage.
4) Capacity stage
Depending on how close the application is to final release is to verify application performance. The phase can be done in parallel to the manual test stage.
Typically load tests are run to verify that the system can serve a defined level of service under “live” load conditions. It is assessed whether the application is fit for purpose from a non-functional perspective in regards to response time and throughput. While it can be almost fully automated, the outcome usually depends on a human decision as to whether the current stage is acceptable.
5) Release stage
The goal here is to deliver the application to end users. Depending on the scenario, you could be upgrading a production environment, sending packaged software or deploying an update on client devices. Fully implemented Continuous Delivery might perform this phase automatically if all previous stages were successful. Typically it will be a human decision to release the product here.