API Gateway, Ingress Controller or Service Mesh: When to Use What and Why

In just about every conversation on ingress controllers and service meshes, we hear some variation of the questions, “How is this tool different from an API gateway?” or “Do I need both an API gateway and an ingress controller (or service mesh) in Kubernetes?”

This confusion is understandable for two reasons:

• Ingress controllers and service meshes can fulfill many API gateway use cases.
• Some vendors position their API gateway tool as an alternative to using an ingress controller or service mesh — or they roll multiple capabilities into one tool.

Here, we will tackle how these tools differ and which to use for Kubernetes-specific API gateway use cases. For a deeper dive, including demos, watch the webinar “API Gateway Use Cases for Kubernetes.”

Definitions

At their cores, API gateways, ingress controllers and service meshes are each a type of proxy, designed to get traffic into and around your environments.

What Is an API Gateway?

An API gateway routes API requests from a client to the appropriate services. But a big misunderstanding about this simple definition is the idea that an API gateway is a unique piece of technology. It’s not. Rather, “API gateway” describes a set of use cases that can be implemented via different types of proxies, most commonly an ADC or load balancer and reverse proxy, and increasingly an ingress controller or service mesh. In fact, we often see users, from startup to enterprise, deploying out-of-the-box NGINX as an API gateway with reverse proxies, web servers or load balancers, and customizing configurations to meet their use case needs.

NGINX, now a part of F5, is the company behind the popular open source project, NGINX. NGINX offers a suite of technologies to develop and deliver modern applications including NGINX Plus for load balancing, App Protect for security, and NGINX Ingress Controller to get control of Kubernetes.

Learn More

The latest from NGINX

There isn’t a lot of agreement in the industry about what capabilities are “must haves” for a tool to serve as an API gateway. We typically see customers requiring the following abilities (grouped by use case):

Resilience Use Cases

• A/B testing, canary deployments and blue-green deployments
• Protocol transformation (between JSON and XML, for example)
• Rate limiting
• Service discovery

Traffic Management Use Cases

• Method-based routing and matching
• Request/response header and body manipulation
• Request routing at Layer 7
• Retries and keepalives

Security Use Cases

• API schema enforcement
• Client authentication and authorization
• Custom responses
• Fine-grained access control
• TLS termination

Almost all these use cases are commonly used in Kubernetes. Protocol transformation and request/response header and body manipulation are less common since they’re generally tied to legacy APIs that aren’t well-suited for Kubernetes and microservices environments. They also tend to be indicative of monolithic applications that are less likely to run in Kubernetes.

What Is an Ingress Controller?

An ingress controller is a specialized Layer 4 and Layer 7 proxy that gets traffic into Kubernetes, to the services, and back out again (referred to as ingress-egress or north-south traffic). In addition to traffic management, ingress controllers can also be used for visibility and troubleshooting, security and identity, and all but the most advanced API gateway use cases.

What Is a Service Mesh?

A service mesh handles traffic flowing between Kubernetes services (referred to as service-to-service or east-west traffic). It is commonly used to achieve end-to-end encryption (E2EE) and for applying TLS to all traffic. A service mesh can be used as a distributed (lightweight) API gateway very close to the apps, made possible on the data plane level by service mesh sidecars.

Note: Choosing a service mesh is its own journey that is worth some consideration.

Use Kubernetes Native Tools for Kubernetes Environments

So how do you decide which tool is right for you? We’ll make it simple: If you need API gateway functionality inside Kubernetes, it’s usually best to choose a tool that can be configured using native Kubernetes config tooling such as YAML. Typically, that’s an ingress controller or service mesh. But we hear you saying, “My API gateway tool has so many more features than my ingress controller (or service mesh). Aren’t I missing out?” No! More features do not equal better tools, especially within Kubernetes where tool complexity can be a killer.

Note: “Kubernetes native” (not the same as Knative) refers to tools that were designed and built for Kubernetes. Typically, they work with the Kubernetes CLI, can be installed using Helm and integrate with Kubernetes features.

Most Kubernetes users prefer tools they can configure in a Kubernetes native way because that avoids changes to the development or GitOps experience. A YAML-friendly tool provides three major benefits:

• YAML is a familiar language to Kubernetes teams, so the learning curve is low or even nonexistent if you’re using an existing Kubernetes tool for API gateway functionality. This helps your teams work within their existing skill set without the need to learn how to configure a new tool that they might only use occasionally.
• You can automate a YAML-friendly tool in the same fashion as your other Kubernetes tools. Anything that cleanly fits into your workflows will be popular with your team, increasing the probability that they use it.
• You can shrink your Kubernetes traffic-management tool stack by using Kubernetes native tools already in the stack. Every extra hop matters, and there’s no reason to add unnecessary latency or single points of failure. And of course, reducing the number of technologies deployed within Kubernetes is also good for your budget and overall security.

North-South API Gateway Use Cases: Use an Ingress Controller

Ingress controllers have the potential to enable many API gateway use cases. In addition to the ones outlined in Definitions, we find organizations most value an ingress controller that can implement:

• Offload of authentication and authorization
• Authorization-based routing
• Layer 7 level routing and matching (HTTP, HTTP/S, headers, cookies, methods)
• Protocol compatibility (HTTP, HTTP/2, WebSocket, gRPC)
• Rate limiting

Sample Scenario: Method-Level Routing

You want to implement method-level matching and routing using the ingress controller to reject the POST method in API requests.

Some attackers look for vulnerabilities in APIs by sending request types that don’t comply with an API definition — for example, sending POST requests to an API that is defined to accept only GET requests. Web application firewalls (WAF) can’t detect these kinds of attacks. They examine only request strings and bodies for attacks, so it’s best practice to use an API gateway at the ingress layer to block bad requests.

As an example, suppose the new API /coffee/{coffee-store}/brand was just added to your cluster. The first step is to expose the API using an ingress controller simply by adding the API to the upstreams field.

To enable method-level matching, you add a /coffee/{coffee-store}/brand path to the routes field and add two conditions that use the $request_method variable to distinguish between GET and POST requests. Any traffic using the HTTP GET method is passed automatically to the coffee service. Traffic using the POST method is directed to an error page with the message "You are rejected!" And just like that, you’ve protected the new API from unwanted POST traffic.

For more details on how you can use method-level routing and matching with error pages, check out these ingress controller docs. Additionally, you can dive into a security-related example of using an ingress controller for API gateway functionality.

East-West API Gateway Use Cases: Use a Service Mesh

A service mesh is not required, or even initially helpful, for most API gateway use cases because most of what you might want to accomplish can, and ought to, happen at the ingress layer. But as your architecture increases in complexity, you’re more likely to get value from using a service mesh. The use cases we find most beneficial are related to E2EE and traffic splitting, such as A/B testing, canary deployments and blue-green deployments.

Sample Scenario: Canary Deployment

You want to set up a canary deployment between services with conditional routing based on HTTP/S criteria.

The advantage is that you can gradually roll out API changes — such as new functions or versions — without affecting most of your production traffic.

Currently, your ingress controller routes traffic between two services managed by NGINX service mesh: Coffee.frontdoor.svc and Tea.frontdoor.svc. These services receive traffic from ingress controller and route it to the appropriate app functions, including Tea.cream1.svc. You decide to refactor Tea.cream1.svc, calling the new version Tea.cream2.svc. You want your beta testers to provide feedback on the new functionality, so you configure a canary traffic split based on the beta testers’ unique session cookie, ensuring your regular users only experience Tea.cream1.svc.

Using a service mesh, you begin by creating a traffic split between all services fronted by Tea.frontdoor.svc, including Tea.cream1.svc and Tea.cream2.svc. To enable the conditional routing, you create an HTTPRouteGroup resource (named tea-hrg) and associate it with the traffic split, the result being that only requests from your beta users (requests with the session cookie set to version=beta) are routed from Tea.frontdoor.svc to Tea.cream2.svc. Your regular users continue to experience only version 1 services behind Tea.frontdoor.svc.

This example starts your canary deployment with a 0-100 split, meaning all your beta testers experience Tea.cream2.svc, but of course you could start with whatever ratio that aligns to your beta-testing strategy. Once your beta testing is complete, you can use a simple canary deployment (without the cookie routing) to test the resilience of Tea.cream2.svc.

Check out these docs for more details on traffic splits with a service mesh. The above traffic split configuration is self-referential as the root service is also listed as a backend service. This configuration is not currently supported by the Service Mesh Interface specification (smi-spec) However, the spec is currently in alpha and subject to change.

When (and How) to Use an API Gateway Tool for Kubernetes Apps

Though most API gateway use cases for Kubernetes can (and should) be addressed by an ingress controller or a service mesh, there are some specialized situations where an API gateway tool is suitable.

Business Requirements

Using both an ingress controller and an API gateway inside Kubernetes can provide flexibility for organizations to achieve business requirements. Some scenarios include:

• Your API gateway team isn’t familiar with Kubernetes and doesn’t use YAML. For example, if they’re comfortable with NGINX config, then it eases friction and lessens the learning curve if they deploy NGINX as an API gateway in Kubernetes.
• Your Platform Ops team prefers to dedicate the ingress controller solution to app traffic management only.
• You have an API gateway use case that only applies to one of the services in your cluster. Rather than using an ingress controller to apply a policy to all your north-south traffic, you can deploy an API gateway to apply the policy only where it’s needed.

Migrating APIs into Kubernetes Environments

When migrating existing APIs into Kubernetes environments, you can publish those APIs to an API gateway tool that’s deployed outside of Kubernetes. In this scenario, API traffic is typically routed through an external load balancer (for load balancing between clusters), then to a load balancer configured to serve as an API gateway, and finally to the ingress controller or Gateway API module within your Kubernetes cluster.

The Future of Gateway API for API Gateway Use Cases

This conversation would be incomplete without a brief discussion of the Kubernetes Gateway API (which is not the same as an API gateway). Generally seen as the future successor of the Ingress API, the Gateway API can be implemented for both north-south and east-west traffic. This means an implementation could perform ingress controller capabilities, service mesh capabilities or both. Ultimately, there’s potential for Gateway API implementations to act as an API gateway for all your Kubernetes traffic.

Gateway API is in beta, and there are numerous vendors, including NGINX, experimenting with implementations. It’s worth keeping a close eye on the innovation in this space and maybe even start experimenting with the beta version yourself.

Watch this short video to learn more about how an API gateway differs from the Gateway API:

https://youtu.be/GQOf4t4KGbw

Conclusion: Right Tools for the Right Use Case

For Kubernetes newcomers and even for folks with a decent amount of experience, APIs can be painfully confusing. We hope these rules of the road can provide guidance on how to build out your Kubernetes architecture effectively and efficiently.

Of course, your mileage may vary, and your use case or situation may be unique. But if you stick to Kubernetes native tools to simplify your tool stack and only consider using a separate API gateway (particularly outside of Kubernetes) for very specific situations like those outlined above, your journey should be much smoother.