How Service Mesh Transforms Cloud Native App Development

Sep 23rd, 2019 1:00pm by Brendan Burns

Brendan Burns, Distinguished Engineer, Microsoft

Brendan Burns is a Kubernetes co-founder, and now a Distinguished Engineer at Microsoft, leading development efforts for Azure Container Service (AKS), Azure Container Instances, Azure Cloud Shell and Azure Resource Manager. He lives with his family in Seattle, WA.

Kubernetes is transformative for many reasons, but one of the most important is that its API was designed with the developer in mind. Instead of infrastructure objects like machines, disks and networks, in Kubernetes you create Deployments and Services. You are still using compute, networking and storage, but the resources you create are aligned to application development.

However, running processes inside containers or Pods in Kubernetes is not the whole story. An application is more than its running code, it is also defined by its network communication. These messages and characteristics like topology, routing, metrics and access control define the distributed system as much as its container image.

Thus, it is clear that an application-oriented API that defines application communication is a critical part of the cloud native developer experience, and these APIs have come to be known generically as Service Mesh APIs. Service Mesh APIs enable application developers to express the connectivity for their distributed application using declarative resources that are developer-friendly. The concrete details of how this application connectivity is achieved is left to the specific mesh implementation like, Consul, Linkerd, Istio or Envoy.

An application is more than its running code, it is also defined by its network communication. These messages and characteristics like topology, routing, metrics and access control define the distributed system as much as its container image.

The decoupling between Service Mesh user and Service Mesh implementation is an example of a critical Kubernetes design principle. Namely modular composability and decoupled layering of infrastructure using declarative APIs. This design principle has led Kubernetes to be made up of numerous generic pluggable APIs. Examples of this include cloud-provider bindings that allow Kubernetes to work across Azure, AWS, Google and more; as well as the CNI (Container Networking Interface), CSI (Container Storage Interface) and CRI (Container Runtime Interface) which generically define how a container interacts with Compute, Networking and Storage.

The Service Mesh Interface (SMI) provides the same generic API for Service Mesh concepts. SMI is a generic interface that presents core Service Mesh concepts in an implementation neutral way. It captures the core ideas in Service Mesh but provides an abstraction above any specific Service Mesh implementation like Istio, Linkerd or Consul.

Separating API specification from implementation has created value throughout the Kubernetes ecosystem, and that same value can be realized for Service Mesh technologies. A generic API enables people to build portable applications that can be easily migrated from on-premise to cloud-based clusters, or to migrate between various public clouds. Further, standardizing our discussions on generic interfaces enables a worldwide community to discuss and, more importantly, teach the same set of concepts and resources regardless of how they are implemented. A common interface means developers can build useful tooling that can be used by the entire ecosystem.

Perhaps most importantly, a generic API future proofs newer technology like Service Mesh. When an API and implementation are tightly bound together, taking on a new API (or set of APIs) is a fraught choice. If the implementation turns out, after a time, to be unsuitable, then changing out the implementation requires major surgery and relearning of concepts and tooling since the API and implementation are joined at the hip. A generic API changes this equation, since the user is abstracted away from the implementation of the API, the implementation can be swapped with little change. This means that the success of adopting a mesh isn’t bound up in the success of any particular implementation, and that in turn means that adopting a newer technology like Service Mesh has fewer risks.

The Service Mesh Interface is designed to be a simple API that is easy to understand and implement, thus it is defined by only four concepts:

A Route defines a path into or out of an application via either HTTP or TCP
A TrafficTarget describes whether an application may call (or receive calls) on a particular set of Routes.

A TrafficSplit describes how traffic should be split between two (or more) Kubernetes services for the purposes of experimentation.

TrafficMetrics describes a generic metrics endpoint for obtaining application traffic statistics.

With this API the core application-oriented structure of an application can be defined in a developer-oriented manner.

Of course, the primary concern of such an interface is that it is a “lowest-common denominator” and thus useful to none. This is always a concern with interface APIs. The way to address it is by embracing both vendor-specific annotations, and evolution of the spec over time. Our inspiration in this evolution is OpenGL which has been successful despite similar tensions between generic API and specific graphics card capabilities. Within the SMI, APIs vendor-specific annotations will always exist and won’t be portable. However, the most popular vendor-specific features will be embraced and added to the specification over time. In this way, the evolution aspires to ensure that most users can be successful in managing their Service Meshes via the SMI API.

Kubernetes marks the beginning of the transformation to an application and developer-oriented cloud API. SMI is an important next step down this road.