CI/CD Pipelines & Use Cases: Everything DevOps Consulting Has Taught Me

Throughout my DevOps consulting career, I've realized that the true value of our work lies in optimizing processes to free up developer time. A key part of optimizing DevOps processes is understanding the difference between manual (on-demand) and automated (triggered) processes.

In this post, we’ll dive into both manual and automated triggers, exploring their roles and integration in CI/CD pipelines. We’ll also tackle the differences between Continuous Delivery and Continuous Deployment, and why one might be a more realistic goal than the other. By understanding these mechanisms and exploring automation opportunities, you can streamline workflows, reduce errors, and boost productivity in your development process.

Understanding the Use Case: Triggered vs. On-Demand

In CI/CD and GitOps, the distinction between manual (on-demand) and automated (triggered) processes is crucial. Each serves different purposes and needs within the development cycle.

Two primary mechanisms will start a pipeline:

1. Manual/On-Demand/Ad-Hoc: “I need something to happen”
2. Automated/Triggered: “Something happened”

Manual/On-Demand/Ad-Hoc Triggers

Manual triggers are typically used when direct human intervention is necessary. This might be for operations requiring a specific timing or condition that’s not easily automated. For instance:

• Scenario: A developer needs to test a new feature under specific conditions.
• Action: The developer clicks a button to spin up a new virtual machine (VM) where the feature can be deployed and evaluated in isolation.

These manual triggers are especially helpful for infrequent tasks or tasks requiring contextual decisions that software (currently) cannot effectively make.

Automated/Triggered Processes

Conversely, automated triggers are about responsiveness and efficiency, forming the backbone of continuous integration and delivery practices:

• Scenario: A developer commits new code to a repository.
• Action: This commit automatically triggers a series of actions — the code is built, tested, and deployed without requiring human intervention.

Automated processes are essential for maintaining the flow of CI/CD and for enabling practices like GitOps, where the state of live applications is continuously synchronized with changes made in version control.

Integration of approval and quality gates

Even within largely automated processes, there is room for controlled manual intervention. Approval gates or quality checks can be integrated to ensure that every change meets the necessary standards before it proceeds to the next stage. This hybrid approach ensures both speed and safety in software deployment.

The Drive Toward Automation

If a process within any of your existing workflows is manual, it’s worth exploring whether and how it can be automated. Automating repetitive and predictable tasks reduces the likelihood of errors and frees up valuable developer time for more complex problems that require human creativity and intuition.

What is the difference between Continuous Delivery and Continuous Deployment?

In CI/CD, the abbreviated “D” is an ambiguous distinction between Delivery and Deployment. So, what’s the difference? The Delivery vs Deployment debate is a point of frequent contention. (See our esteemed Director of DevOps Consulting’s post, Continuous Delivery.)

Let’s start by defining our terms as broadly as possible:

• Delivery: The process of putting a piece of software in an appropriate place, ready for deployment.

• Deployment: The process of putting software on the right resources and configuring it to be accessible for use by its consumers.

Therefore, Continuous Delivery is an automated, no-human-intervention process of putting every change to the product into a deployment-ready state, typically a versioned package.

Meanwhile, Continuous Deployment is the automated, no-human-intervention process of putting that package onto the proper resources to be accessed by its consumers.

Is Continuous Delivery better than Continuous Deployment?

In the real world, Continuous Delivery is a much more realistic goal than Continuous Deployment. Continuous Deployment requires a comprehensive automated test strategy to ensure that each deployment does not break existing functionalities or introduce new bugs. It demands a high level of confidence in the testing process, which can be challenging to achieve without significant investment in testing frameworks and technologies.

Continuous Delivery can be achieved with much less investment, as the deployment of the changes—while still ideally an automated process itself—can be manually triggered.

Other mechanisms and strategies for CI/CD

There are several mechanisms and patterns that bridge the gap between Delivery and Deployment. Two of the most useful are Feature Flags and Deployment Gates.

Feature Flags wrap new and old code in an if/then statement: if the flag is on, then the new code is executed. If not, the old code runs. This decouples Deployment from Release and, when used properly, can dramatically improve the flow of change while reducing risk.

Deployment Gates add a layer of protection in a Continuous Deployment pipeline: the pipeline will trigger and start the deployment, but it will wait before actually performing the deployment. A human (or human-involved) process is required to approve the pipeline to complete the deployment.

Conclusion

The distinction between manual and automated processes is a fundamental aspect of optimizing DevOps workflows. By understanding and leveraging both on-demand and triggered mechanisms, organizations can effectively streamline their CI/CD pipelines, reducing errors and boosting productivity.

Thank you for joining me in this installment of Everything DevOps Consulting Has Taught Me. Check out Bitovi’s Community Discord to pick my brain about CI/CD!