Continuous Integration Is Just a Script Runner
Receive an event from GitHub → clone the repository → run some tests → report the results. Those were the good old days of continuous integration (CI), a simple service that revolutionized the way development teams shipped software.
It made us faster at writing more reliable tests and helped us catch bugs earlier so that we shipped fewer to production. The term CI became common, and the practice spread like fire, helping teams ship software faster and more often.
But with the increasing complexity of cloud native systems, our dependence on CI has expanded far beyond its original scope. The first step of right-sizing the role of CI is acceptance: CI is just a script runner.
What CI Has Become
Application development has evolved immensely over the years. We used to compile our different applications and run them in multiple virtual machines, but nowadays we build them as different containers and deploy them on Kubernetes. They all depend on each other and need different storage that must be provisioned. Then Kubernetes, Terraform, serverless, end-to-end testing and cloud providers’ authentication entered the room.
You need to make sure all these different components work together as expected at every change to your codebase. You don’t want to manually load your binaries in your VM or to manually apply your Kubernetes manifests in production. You want continuous delivery. This was a genuine need for the industry, and the people who were better positioned to help you solve this problem, in an increasingly complex landscape, were CI providers like CircleCI, Travis, Jenkins, etc.
So they started to transform, from CI to full-blown software delivery platforms, offering complex pipelines, multiple runner types, composable configuration, integration to every possible platform. They placed themselves in the middle of the software delivery life cycle, successfully enabling it, automating as much as possible.
In doing so, they also became the biggest bottleneck for faster delivery.
How CI Is Holding You Back
In theory, having a tool automating everything you need to build, test and deploy to your system can be a great competitive advantage and can save a lot of time instead of running and managing these kinds of operations yourself.
In practice, though, those platforms started as task runners: They started as generic and technology agnostic. They never had any concept of what running a test means, much less the difference between an integration test and an end-to-end test, and how they affect a CI run, or any mechanism to define that.
That became a problem as soon as your application grew in complexity. CI service providers tried to solve this problem in many ways — offering custom flexible configuration options that only made it impossible to switch to other providers. They offered hosted runners for when you need to run heavy workloads, but you have to provide infrastructure yourself. They offer secrets and configuration but with poor tooling around it, trapping you in their ecosystems.
These tools don’t have any insights into how your stack is built, what your services depend on, what parts of your code have changed between pull requests and what needs to be rebuilt. Sure they allow you to declare dependencies between steps, but maintaining the cognitive load of the actual application-level dependencies are left as an exercise for the end user.
What was supposed to be a simple tool for scheduling and running tasks became a complex monolithic agent, difficult to debug, hardly descriptive of your system, slowing your team down.
The Future of CI
I believe teams are starting to realize how complex and bloated those systems have become. We need the automation now owned by CI systems, but we need it to de-silo it from CI service providers.
The future of CI requires us to stop for a second and start again from what we were promised at the beginning: simple version control software integration, scheduling and task runner. And we need to seriously take a look at what CI should be doing verses what we gave it control to do.
I would love to see mature platform and DevOps teams working on the following features.
Portable Pipelines
You should be able to run your pipeline regardless of your CI/CD provider of choice. Your pipeline should be able to express the complexity of your application and be dependency aware, easy to configure and edit as your application or stack grows. It should live in your repository and be completely independent and platform agnostic.
Our current provider bumped their prices? I want to grab my bags and set up shop on another platform, as easy and fast as possible. Or I can set up an internal platform and bring my pipeline there.
I want to change my pipeline because my application or stack changes, not because my provider has become too costly or because they are not supporting certain platforms anymore.
Debuggable Pipelines
You should be able to easily debug your pipelines. We should stop thinking of CI providers as inaccessible black boxes, and we should aim at being able to run pipelines from anywhere: Your deploy dependency is broken? I don’t want to access the logs from my machine; I want to run the pipeline from my laptop and check the execution logs in real time.
I want to try to debug a new test setup before even pushing it to my repository.
The times of commit, push and wait for the CI to see if the changes are valid are over.
Composable Blocks
Microservices architecture taught us that, in certain scenarios, splitting up your complex domain in smaller subdomains can be beneficial. Why aren’t we doing so with our delivery process? Why are we preferring “smart” all-encompassing solutions over composability? We should strive to describe our system using composable blocks and the dependencies between them.
Leave CI to do what it’s best designed to do: integrate with our version control system, scheduling and executing tasks, and reporting errors.
Conclusion
The CI platform became a delivery platform out of necessity and opportunity, turning into a velocity bottleneck for serious development teams. It’s our job as engineers to sit down and evaluate if the current approach is sustainable in the long run and how much control we want to take back from CI vendors.
Our team at Garden has been thinking about how to tackle these problems for years. The complexity of modern-day pipelines has driven more people to search for tools that allow them to codify vendor-agnostic pipelines, run them everywhere and speed up their delivery cycles.
Join us at KubeCon + CloudNativeCon North America this Nov. 6-9 in Chicago for more on Kubernetes and the cloud native ecosystem.
