How has the recent turmoil within the OpenAI offices changed your plans to use GPT in a business process or product in 2024?
Increased uncertainty means we are more likely to evaluate alternative AI chatbots and LLMs.
No change in plans, though we will keep an eye on the situation.
With Sam Altman back in charge, we are more likely to go all-in with GPT and LLMs.
What recent turmoil?
Networking / Service Mesh

ZeroLB, a New Decentralized Pattern for Load Balancing

The load balancer market hasn’t evolved with the massive architecture changes that are taking place in the application landscape today. Service mesh and the ZeroLB operating model provide users a path delivering services with greater agility and functionality, wrapped in a self-service consumption model.
Aug 10th, 2021 12:08am by
Featued image for: ZeroLB, a New Decentralized Pattern for Load Balancing
Feature image via Pixabay.

Marco Palladino
Marco Palladino is an inventor, software developer and internet entrepreneur based in San Francisco. As the CTO and co-founder of Kong, he is Kong’s co-author, responsible for the design and delivery of the company’s products, while also providing technical thought leadership around APIs and microservices within both Kong and the external software community. Prior to Kong, Marco co-founded Mashape in 2010, which became the largest API marketplace and was acquired by RapidAPI in 2017.

With advancements in technology-driven by the microservices era — which started in 2013 and 2014 with the release of Docker and Kubernetes — new architectural patterns have emerged to provide decentralized load balancing, yet portable across various platforms and clouds. The old monolithic and centralized load balancer, a technology largely stuck in the early 2000s, becomes deprecated in this new distributed world.

The most common breed of load balancers being deployed across every application — centralized load balancers — are a legacy technology. They don’t work well in our new distributed and decentralized world. Remnants of a monolithic legacy way of doing things that did not adapt to modern best practices, centralized load balancers prevent users and organizations from effectively transitioning to the cloud native and container-based era.

Because centralized load balancers introduce single points of failure, they cause unnecessary network latency. They also have a flat view of upstream services, so they are unable to provide much intelligence, identity and observability to services. In addition, they are expensive and frequently not portable across multiple clouds. Finally, centralized load balancers have limited features, including lack of set-up support for modern zero-trust security, tracing and metrics, and self-healing capabilities across applications.

In this article, I will deconstruct these statements, and introduce a new breed of load balancing that is natively decentralized: the ZeroLB architecture.

The Woes of Centralized Load Balancers

Traditional load balancers introduce a series of problems in our applications that get exponentially worse the more decoupled and distributed our applications become. In a typical decentralized application, centralized load balancers become a single point of failure for our services:

By introducing a centralized load balancer, we also increase network performance by adding one extra hop in the network that we don’t typically need:

In addition, when we decouple centralized load balancers for high availability, we typically need one load balancer per service which increases complexity in our architectures, further reducing the performance, and increasing our operational costs:

When deploying on a public cloud, we may be tempted to utilize their native elastic load balancers, which will in turn increase costs quickly, since they are very expensive, Also, they are not portable across clouds and therefore cause a fragmented experience for developers or operators who are replicating these deployments across environments:

All of these limitations while not extending the baseline connectivity functionality that modern service builders require such as self-healing routing, canary releases, observability, zero-trust security and more. In the cases where they do find ways to implement this functionality, the implementation is most commonly inconsistent across implementations and typically not portable.

ZeroLB: Decentralized Load Balancing

Introducing ZeroLB, a new pattern for implementing load balancing capabilities that is more performant, less costly to implement, portable across every cloud and built for the new microservices era. Many organizations today are adopting what essentially is ZeroLB with great success via client-side load balancing implementations, yet many more are still looking at centralized load balancers for their decentralized architectures.

With the introduction of the “ZeroLB” terminology, the goal is to evangelize this new load balancing pattern in a clear and concise way, in order to increase its adoption in the world and better more reliable applications. ZeroLB can be built with custom implementations, or via the adoption of a general-purpose service mesh that is portable and can run across our applications on both VMs and containers — and on every cloud — such as the Cloud Native Computing Foundation‘s Kuma, which natively supports all of the above, leverages Envoy proxy and has open governance.

By leveraging decentralized data plane proxies deployed next to our services, we can load balance traffic without any single point of failure, and in a faster way too since we don’t have to hop to a centralized load balancer anymore:

One of the core functionalities we target in service mesh is making application connectivity “portable” across both VMs and containers — and across every cloud vendor. The ZeroLB approach standardizes how the load balancing capabilities are being implemented across different applications and environments. Since this model functions in a decentralized model (running on the same underlying VM or Pod), we save money by not having to spin up any cloud load balancer for our upstream services:

All of this baseline connectivity and load balancing capabilities are provided while also enabling advanced (and frankly, modern) service connectivity features like self-healing, service routing, canary releases, health-checking and circuit breakers and zero-trust security:

Wrapping Up

The load balancer market hasn’t evolved with the massive architecture changes that are taking place in the application landscape today. The way users expect “connectivity” to be consumed has changed. Developers expect the platforms they deploy against to function natively in a multicloud world. They require lower operating costs, with a higher bar of functionality.

Service mesh and the ZeroLB operating model provide users a path delivering services with greater agility and functionality, wrapped in a self-service consumption model. This model is born in a multicloud world where application portability is a baseline requirement and decentralized applications are the new normal.

Developers and operators are having to manage much more complex environments than they had to previously, and many times the pain points of that complexity come from the challenges of legacy technology choices impacting modern designs. Embracing the ZeroLB operating model sets teams up to achieve optimum connectivity in the “cloud-connected” era.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: Docker.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.