Netflix Discovers Severe Kubernetes HTTP/2 Vulnerabilities
Taking a look at how the internet’s HTTP/2 protocol works, Netflix engineers discovered a set of vulnerabilities in Kubernetes. The main issue was found in the net/https library of the Go language. Because it was found in this particular library, every version and every component of Kubernetes is affected. These two issues have been assigned base scores of 7.5 (which earns a high severity) by the Kubernetes Product Security Committee. Those two vulnerabilities are:
- CVE-2019-9512 Ping Flood. This enables an attacker to send continual ping requests to an HTTP/2 peer, causing the peer to create an internal queue of responses. When this happens a server’s CPU and memory can be consumed, which can lead to a denial of service.
- CVE-2019-9514 Reset Flood. This enables an attacker to open a number of streams, which are used to send invalid requests to solicit RS_STREAM frames from the peer. If the peer queues the RST_STREAM frames improperly, a server’s CPU and memory can be consumed, which can lead to a denial of service.
All eight of the vulnerabilities affect Kubernetes. And although these are still awaiting analysis from NIST, Kubernetes has already issued patches that are found in the following builds:
- Kubernetes v1.15.3 – go1.12.9
- Kubernetes v1.14.6 – go1.12.9
- Kubernetes v1.13.10 – go1.11.13
It is crucial that users upgrade each and every instance of Kubernetes (often abbreviated to “K8s”) employed to one of the above iterations, otherwise, they will be vulnerable to attacks. Of course, how you upgrade your Kubernetes instances will depend on how they were deployed (on a data center server, via Google Cloud, AWS, etc.). Make sure to consult the official Kubernetes documentation on the upgrade process for your deployment.
How These Vulnerabilities Work
Each of the Kubernetes vulnerabilities work via the following steps:
- A malicious client requests a vulnerable server to do X which generates a response.
- The client refuses to read the response.
- The refusal manipulates the server’s queue management code.
- If the server poorly handles its queues, the client can force it to consume both excessive CPU cycles and memory while the requests are processed.
The above steps create a denial of service, which can bring down the server.
It is important to note that the vulnerabilities (as described by Netflix) do not allow an attacker to modify or leak sensitive information. Instead, the discovered flaws allow a Denial of Service (DoS) to be perpetrated on vulnerable servers.
The K8s Complexity
Stephan Fabel, director of product for Linux distributor Canonical, had this to say of the Kubernetes vulnerabilities. “For the recent denial of service attack with K8s it is interesting to note this is not a flaw unique to K8s. But it does highlight that K8s is a complex application built on a foundation of technologies that take security seriously. These packages are patched or are getting patched across the ecosystem,” he said.
Fabel continues by reminding admins that Kubernetes is not an island unto itself: “Kubernetes is not standalone, nor is its security. By securing the entire deployment from the operating system to application and to the network, Kubernetes can remain secure. Tools like system updates (unattended-upgrades), role-based access control, local firewalls (ufw) and network segmentation aid in this. This is not specific to K8s, a layered security approach is required for all systems,” he said.