How Cloud Foundry Has Felt the Kubernetes Effect
Cloud Foundry sponsored this post.
I’ve been reflecting recently on how much the Cloud Foundry project has changed in just five years.
One of my observations has been how different the open source ecosystem and enterprise computing market have become since 2015. As a result, the co-evolution of Cloud Foundry and the broader ecosystem around it resulted in pulling and pushing one another in different directions.
It has felt simultaneously like the fastest five years of my career — and the longest. This is because of the duality of this constant push/pull of rapid-fire change and the slow transformation underlying it all.
The initial idea for what is now called Cloud Foundry came about within VMware in 2009. It wasn’t until April of 2011 that the project was officially announced. I remember thinking clearly, as I watched the announcement video stream, that they were doing exactly what the market needed. The announcement was all about a platform that would focus on making application development and deployment easier. For the next several years, I observed the product as it evolved (and changed hands). In 2014, I got a little closer to the community when I wrote my first service broker integration into the platform. (For those who are unfamiliar, a service broker is how to integrate capabilities such as databases into the platform.)
By 2015, a young open source community had formed around the now open source Cloud Foundry project. The Cloud Foundry Foundation sprang into existence in late January of 2015. This is when I transitioned from observer to my role serving the community.
Changes to the Underlying Architecture
A ton has changed in the platform since 2015 — both in the internal architecture and the capabilities exposed to developers using it.
When the Cloud Foundry Foundation started, the community was in the midst of a major rewrite of the underlying architecture of the platform. The shift was from the initial (mostly) Ruby codebase (called the DEA architecture) to a new architecture, based on Go and called Diego. This was one of those major architectural shifts that is so difficult to get right. The challenges are never about the new architecture itself (not to diminish the work of the community in building it) but are in the transition: the competing demands for new features in the existing architecture and the desire to have less of a moving target for the new architecture. I’ve lived through many of these transitions in my career, and one of the things that has most impressed me about the Cloud Foundry community is how it handled this change.
Open collaboration can be messy, for sure, but the result was an orderly transition that gave the ecosystem of downstream distributions and end-users plenty of lead time to make the switch. The formal transition from the DEA architecture to Diego (and all of its subsystems) started in November of 2016 when Diego reached its 1.0 release. That release milestone indicated two key things to the community: it was at the targeted level of feature parity with the DEAs, and it had achieved the 250,000 container scale target hoped for by some of the downstream distributions.
Warden and Garden
Concurrent with the development of Diego itself, the component of Cloud Foundry responsible for the node-level container management underwent major changes as well. In the DEA architecture, this part of the architecture was a component called Warden. Warden pre-dated Docker by at least two years but was not a technology ever exposed to end-users. As Diego was being created, a parallel and complementary rewrite of Warden was happening. That effort was called Garden.
Garden’s design was thoughtful, in that it anticipated the need for easily changing the lowest level details of its code to support emerging OS-level container features. As early as 2015, the Cloud Foundry community was building implementations of Garden to support both Linux-based hosts and Microsoft Windows-based hosts. The ability to change the underlying OS-level container technology similarly allowed the Cloud Foundry platform to adopt the runC library donated by Docker to the Open Container Initiative. In fact, the Cloud Foundry project was the second project (second only to Docker itself) to adopt runC and have it running at massive scale in production clusters throughout the ecosystem.
Why Diego Mattered
The introduction of Diego opened up new feature possibilities. Container to container networking and volume services were two of the first major capabilities added after Diego was well underway. Both sets of features could be looked at as internal optimizations (ex: reduction of North/South traffic) but were more important as developer experience enhancements. With the C2C networking capabilities, more complex application to application logic could be implemented. It was also the beginning of Cloud Foundry’s embrace of the broader open source networking world (more on that in a bit). Volume services expanded the breadth of application types that could be hosted in the platform, allowing operators to offer application developers filesystem mounts to network-addressable storage devices.
Container Networking
Going back to the world of networking, it’s true that the container-to-container networking efforts added a host of valuable capabilities for developers. It also represented the point of inflection where the Cloud Foundry community started to take advantage of developing standards and projects in the service mesh space. The effort started with adopting the Container Networking Interface (CNI), a specification for container-based platforms to interact with an underlying networking layer for easier configuration. The next phase was the adoption of Envoy Proxy in 2017. More recently, the entire Cloud Foundry networking stack is targeting Istio + Envoy as the default components to support both cluster ingress/egress and cross-container networking capabilities.
Kubernetes + Cloud Foundry
More recently, the Cloud Foundry community has embraced another open source project: Kubernetes. This started with the donation of Kubo, a project jointly developed by Pivotal, Google and VMware. Kubo quickly became the Cloud Foundry Container Runtime (while the Elastic Runtime, better known as the traditional Cloud Foundry PaaS, was renamed the Application Runtime). This gave our community the confidence needed to take further steps towards adopting Kubernetes as the underlying container orchestration layer.
Project Eirini is the effort to allow Cloud Foundry clusters to use Kubernetes within their architecture, and to eventually (still in progress) replace the Diego code-base.
So many other changes and extension projects have been created by the community over the last five years that it’s hard to keep track. Some projects have evolved into key components of most Cloud Foundry installations. Others have served their purpose and are now retired to the Foundation’s attic.
What Hasn’t Changed?
Two things that have not changed in the last five years are worth highlighting: our community’s focus on providing a world-class enterprise developer experience, and the community’s commitment to continuous evolution of the platform. Neither of these invariants are “technology.” They are the core ethos of the community. We have been bringing the thousands of organizations using Cloud Foundry and the hundreds of thousands of developers deploying software into the platform on a journey. We keep learning how to better serve the enterprise developer and keep iterating on and expanding, that experience. We also keep evolving the architecture, harvesting the best of what the broader open source world has to offer. To be sure, we make these changes thoughtfully and carefully. We allow for the evolution to gradually introduce new capabilities while releasing frequently.
Quite a bit has changed, but what doesn’t change is the Cloud Foundry community’s commitment to bringing our ecosystem on a journey. I’m excited to see where we go from here.
Visit our birthday timeline and watch our birthday video.
Feature image via Pixabay.
