Cloud Foundry HTTP 2 Project Thwarted by GoLang Indifference
A project to bring HTTP/2 to the CloudFoundry application development platform ran into a roadblock when the keepers of the Go Language did not respond to requests, with sufficient swiftness anyway, for supporting the HTTP/2 over TCP “upgrade flow” process.
As a result, the Cloud Foundry Go Router reverse proxy removes headers that would let a CF application know it can send and receive HTTP/2 traffic. Such capability could be coded in, bypassing the Go language library entirely, but the project team doesn’t want to take on the responsibility for supporting such a potentially widely-used function.
Carson Long, a software engineer for VMware Tanzu, the company’s Kubernetes distribution, spoke about this challenge at this year’s virtual Cloud Foundry Summit held earlier this month.
Faster in Bundles
Launched in 2015 to expedite web traffic, the HTTP/2 has become one of the most requested developer features (alongside gRPC). It speeds web traffic by breaking traffic into binary-encoded frames which can then be multiplexed within a single TCP connection. About 45% of the top websites support HTTP/2, as do all the major browsers.
Long is part of a team to bring HTTP/2 to Cloud Foundry. This involves ensuring HTTP/2 is supported at each step, including load balancers (surprisingly difficult given the variety of load balancers), and the Cloud Foundry Go Router itself (with an addition of two lines of code).
But the most difficult compatibility change, however, turned out to be in checking if the end application, built on Cloud Foundry, can recognize HTTP/2 traffic. HTTP/2 relies on mTLS for encryption, which is terminated in Cloud Foundry not by the app itself but rather by the Envoy proxy. This means that Envoy also talks directly with Transport Layer Security (TLS) Application-Layer Protocol Negotiation (ALPN) to determine whether HTTP/2 can be used or not.
When it terminates the mTLS, Envoy then sends the packet payload, now in plain text, to the application using another protocol, HTTP/2 over TCP (H2C).
“The problem we quickly ran into was how do we determine if the server or app actually accepts HTTP/2. There’s no way of knowing in this situation, so clearly this we can’t just forward HTTP/2 every time,” Long noted. An app that only accepts HTTP/1 will just drop HTTP/2 packets and the client would never find out.
The good news is is that H2C comes with an upgrade flow, in which the app is asked, through a few extra headers, if it accepts HTTP/2. If an affirmative response is given then HTTP/2 packets can be sent.
The team ran into a problem, however, in that the Go Standard library does not support the H2C upgrade flow. They filed a request in GitHub, Long said. “We pretty much got no response from the GoLang team,” Long said. “That was not encouraging and pretty much put a stop to this upgrade flow.”
The issue, submitted in May, is currently marked as “under investigation” by the GoLang maintainers.
Until the issue is resolved, the development team is looking at a number of different options.
They have written a “health check,” a separate H2C process that would run once to determine if the app supports HTTP/2 and if so, would continue to send HTTP/2 packets. Another option would be for the app itself to alert the Go Router that it can accept HTTP/2 traffic through a manifest entry. A third option, the one currently being pursued, is to set up “routing destinations” to communicate between the Go Router and the CF app.
In the meantime, the team has written its own HTTP/2 detection code (internally dubbed “H2 Awesome”) as a prototype, which Long demonstrates in a demo following the presentation.
