How WebAssembly Could Streamline Cloud Native Computing
As a small, fast, secure, cross-platform bytecode, WebAssembly should be ideal for cloud native patterns. Docker founder Solomon Hykes famously tweeted that WASM and WASI (the WebAssembly System Interface for safe access to system APIs), would have removed the need to create the container technology.
But since WASM gives you a binary rather than entire environment, unless you move the entire environment into the WebAssembly module, it’s not a replacement for containers. It’s a way of running arbitrary code where you want it, with the least possible overhead and the fewest (known) security issues.
That’s ideal for embedding third-party code where Kubernetes workloads need it. For example, the Envoy proxy (often used for network routing and load balancing in Kubernetes) already uses WebAssembly as a way to write filters, Open Policy Agent policy rules can be written in WASM and Kubewarden uses WebAssembly modules for admissions policies.
In other words, stop saying “WASM is the end of containers;” start saying “WASM plus containers.”
Kubernetes Helps WebAssembly
If you think of cloud native processes as something you create from an API when you need them at scale, scale up as you need more capacity and remove them when you don’t, WASM is an excellent way to run those processes. But you still need an environment to run them in. Cosmonic CEO Liam Randall compares WebAssembly to a virtual, and potentially universal, CPU because it acts as a universal compilation target but notes “there’s no virtual operating system that sits in there.”
And unless they’re embedded in another workload like Envoy filters or used as serverless functions in an environment like Cloudflare Workers, you need an orchestrator to coordinate them. “Scheduling and orchestration is still an open area for sever-side WASM,” noted Renee Shah, a principal at the venture capital firm Amplify Partners.
There aren’t yet WASM-specific scheduling and orchestration systems, and it may not make sense to develop those from scratch rather than integrating with existing options. Kubernetes (K8s) certainly isn’t the only choice for orchestrating cloud processes, but it’s so well known and widely adopted that it’s logical to think about using it with WebAssembly.
With Krustlet, which is now a Sandbox project of the Cloud Native Computing Foundation, Kubernetes can natively orchestrate WebAssembly runtimes. Krustlets are Kubernetes Kubelets written in Rust. When the Kubernetes API schedules pods onto a Krustlet, Krustlet runs them in a WASM runtime; WebAssembly and container workloads running in the same cluster can communicate with each other and interact with the rest of the cluster using features like secrets, volume mounts and init containers.
If WASM+WASI existed in 2008, we wouldn’t have needed to create Docker. That’s how important it is. Webassembly on the server is the future of computing. A standardized system interface was the missing link. Let’s hope WASI is up to the task! https://t.co/wnXQg4kwa4
— Solomon Hykes (@solomonstre) March 27, 2019
Matt Butcher, head of the Microsoft team building container and WebAssembly technologies like Helm, Draft and Krustlet, told The New Stack that Kubernetes is a “hidden gem” for WebAssembly because it brings not just scheduling but other key services.
The startup time for a WebAssembly runtime is in the 10 milliseconds range, compared to several seconds for a container (or several minutes for a virtual machine), which makes it a good match for serverless — but also for scaling.
Kubernetes “also brings with it the storage layer, which turns out to be a very big deal; you want to be able to move your workload around, you want to make sure that it can attach to the right storage wherever it lands, and that would have been a monster project to write on our own,” Butcher said. “Kubernetes is mature in that area. Secrets management, the networking layer; it’s like getting a bunch of freebies. Here’s a problem I don’t have to worry about, because Kubernetes will do it. That was one of the things that make Krustlet such a great experience starting up, because we could go from very basic to pretty interesting configurations because we only had to hook into Kubernetes instead of writing our own implementations.”
WebAssembly can take advantage of other container constructs too; with WAGI, the WebAssembly Gateway Interface as a workaround for building HTTP handlers that compile to WASM, you can reference and pull modules from OCI registries. (Kubewarden and Krustlet also take advantage of OCI artifacts to distribute WASM modules through OCI registries).
“The nice part about WebAssembly and containers is that they both play on the same trend of breaking up monoliths,” Shah pointed out. “If a company already uses microservices, it’s relatively straightforward to add new ones powered by WebAssembly.” And with Krustlet, they can run side by side in Kubernetes.
Better, Faster, Smaller: WASM Helps Kubernetes
There are plenty of areas in which WASM will be useful that Kubernetes won’t be relevant, although another orchestrator may be needed for some of them. But there are also Kubernetes scenarios that WASM will improve significantly, so WebAssembly may become a common Kubernetes workload — again, without replacing containers.
WebAssembly isn’t about compiling an entire app into a module instead of a container; you’re creating a binary, and that fits in very well with the cloud native principles of having code that does one thing and does it well. WASM modules running in Kubernetes can take on some tasks where containers have been the only option but aren’t well suited for, but in most cases these WASM “pods” are going to be doing new work that wasn’t possible before, and maybe in new places.
What WASM in Kubernetes will be particularly good for is scenarios that need high density, fast availability and scaling, or to run with limited resources, Butcher told The New Stack. It’s also a lot of work to correctly secure Kubernetes and the default runtime configuration of containers. Containers aren’t a security boundary; WebAssembly does a lot more to prevent code escapes. The security of the WASM sandbox and the “denied by default” way WASM modules require explicit permissions for any capabilities have obvious advantages here.
The startup time for a WebAssembly runtime is in the 10 milliseconds range, compared to several seconds for a container (or several minutes for a virtual machine), which makes it a good match for serverless — but also for scaling.
“When you’re doing the kind of thing where you want to take a workload and say ‘OK, bring me up to five instances, well that’s not enough, bring me up 15 instances,’ you get instant scale and very high throughput,” Butcher said. That’s also useful for scale to zero (where, despite the name, you typically never scale below one instance because of the latency for a cold start), he notes. “It costs money to have a server that you have sitting there waiting for somebody to use it. Can we make it appear to be always available, but only because it’s so fast to start up that the end user is blissfully unaware that the process wasn’t running until they requested it?”
There’s a lot of emphasis on making container sizes smaller, but they’re often too large for edge environments with constrained memory and storage or minimal processing power. Running a 2MB WebAssembly module instead of a 25MB Docker container means workloads can be managed by Kubernetes in more environments (perhaps using minimized versions of K8s, like K3s or Kind).
That can significantly improve workload density, which might reduce cloud bills, Butcher noted: “There’s an old joke that cloud services are really just paying somebody else’s electrical bill, and density improvements mean lower cost.”
But the far more heterogeneous compute environments of edge are perhaps the biggest opportunity for the combination of WASM and Kubernetes, which could mean that “the cloud” moves beyond hyperscale data centers.
“We’ve been thinking about the cluster as a thing that lives in the data center, and we’ve deployed them as things that live in a data center,” Butcher said. That might be AKS on Azure or Kubernetes on bare metal servers, but it’s still a data center. “What Krustlet solves for us that we weren’t able to solve with Kubernetes and containers was that we needed to be able to extend Kubernetes clusters outside of just the data center and to devices that were getting a little more exotic,” he said.
Supporting Arm silicon was a big part of that, but so are the microcontrollers used in IoT devices, the many hardware AI acceleration options and the heterogeneous CPUs that even Intel is now producing. “We’ve got exotic architectures, we’ve got new emerging architectures, and we want a cross-platform, cross-architecture story and we want a cross-operating system story,” Butcher said.
Randall, of Cosmonic, echoed this idea. “The diversity of CPUs has made more obvious what limitations that containers bring to the ecosystem,” he said. “In a CI/CD world. if you have to target dozens or hundreds of different CPUs, you need dozens or hundreds of specific CPU images.” Plus, he noted, “a lot of small devices will never run Linux.”
“What WebAssembly brings is we let go of our assumption that we’re building for a single CPU, because WebAssembly is simply a compilation target that we can bring our existing code and applications to.”
Edge — but Experimental
Because modules can run on many different devices without needing to be recompiled, Butcher suggested WebAssembly offers the “write once, run anywhere” promise familiar from Java but “in a form that’s really amenable to cloud native development.”
That could be part of a very diverse Kubernetes environment thanks to projects like Akri, which attaches IoT and other devices (from an IP camera to a field-programmable gate array) to a Kubernetes cluster as resources. “Wouldn’t it be awesome if we could make the cluster expand outward from just the data center into different devices elsewhere?” Butcher asked. “Wouldn’t it be cool if I had a cluster that lived on my phone and my laptop and whatever devices were closest to me, and as I got into proximity, they’d join and leave the cluster?”
Recently, the Akri and Krustlet teams have been experimenting with compiling Akri to WASM so it can be deployed into a Kubernetes cluster using Krustlet, which makes it even smaller and faster for the edge.
However, that depends on WASI to deliver the virtual operating system primitives that WASM lacks, and WASI is still emerging and can take a lot of work to make useful in key areas like networking. “The WASI specification still doesn’t have guest side network access figured out; it’s always a matter of piping everything through the host runtime. It doesn’t have some of basic library stuff figured out,” Butcher noted.
Fintan Ryan, a senior analyst at the analyst firm Gartner, told The New Stack, “WASI needs to move beyond an experimental stage for any significant adoption.” And while they are being developed, those robust APIs are still six or 12 months away.
When they do arrive, more tools will need to be built and more integration work done between the WebAssembly and Kubernetes ecosystem, Butcher said. “What will it look like when our WebAssembly modules can open up a thousand server sockets and listen on those? How will Kubernetes wire it up so it’s got a thousand routes going into the WebAssembly module?”
In the meantime, he suggested, “WAGI and Krustlet and things like that give us a good way to get people in the door and looking at the technology today, hopefully getting excited about it and having their own ideas, and starting up their own GitHub repositories and startups and working groups.”
One of the ways he hopes to get people excited may provoke some strong reactions: a WAGI provider for Krustlet that, he says, “will give you a quick, easy developer experience to build applications in — I kid you not, 1996 style CGI — because the technology is rock solid and everybody in the universe has support for this!
“It’s a step we can take today to get this technology in front of people so they can go build small applications of microservices on it.”
The Possibilities of WASM plus Kubernetes
WASM could make the developer experience with Kubernetes as easy as Docker and containers, Butcher hopes: “I want a nice quick workflow where I can write an app and, somewhere in the background, it gets deployed out to somewhere like a Kubernetes cluster running Krustlet.”
Building containers may be easy, but deploying, testing and sharing the test environment with colleagues is more fraught, he said. “That is still a little bit slow and it has some rough edges, and we’re hoping that WebAssembly might be able to smooth over some of those as well.”
Cosmonic’s wasmCloud, now a CNCF Sandbox project, also aims to improve developer and experience, and also relies on Krustlet for some scenarios, although with its own runtime rather than the Bytecode Alliance’s Wasmtime runtime.
“The combination of Kubernetes and WebAssembly enables if you want to take wasmCloud and put it into a set of Kubernetes servers on multiple clouds and have them talk to each other,” Randall said. One customer, a large European bank, is experimenting with using it to fail over from one cloud Kubernetes service to another, for availability when one of its cloud providers has an outage.
There are also some fundamental patterns in microservices that future features in WebAssembly might be able to improve on.
“Your typical microservice involves every developer standing up their own HTTP server and writing their JSON to object serialization,” Butcher pointed out. That adds up to around a thousand lines of boilerplate code in every microservice that has to be written, tested, secured and updated. The WebAssembly nanoprocess model puts the HTTP layer and serialization/deserialization in the WASM runtime.
“When two modules happen to be sitting next to each other on the same runtime, they just communicate directly,” he said. “But when they get spread across the network, then the host runtime takes care of the traffic between the two and the developer doesn’t have to maintain it. And the security mode is great because you don’t have to update every piece of code every time OpenSSL has a bug.”
That’s still some time off. “We’re probably about a year out from being able to really even start solving that problem,” Butcher noted. “If we solve it, we’ll be able to build an entirely new class of applications.”
That’s still an “if” with a lot of moving pieces. “Nanoprocess is seen as even more experimental than WASI,” Ryan cautioned.
“The promise is huge, the amount of work to get there is also huge.”
Not all of the promise of WebAssembly and Kubernetes is that distant though. Another heterogeneous, resource-constrained area where Kubernetes is increasingly important, especially with tools like Kubeflow, and where WebAssembly could deliver significant gains: machine learning.
WASI doesn’t just deliver low-level OS interfaces like storage and networking; it can also offer high-level interfaces for something like a neural network. Trained machine learning models have to run on a wide range of devices with platform-specific runtimes and use different specialist hardware accelerators. Because it’s platform-independent, WASM would simplify deployment and the WASI-nn interface will let WebAssembly talk to multiple machine learning runtimes like ONNX, OpenVINO, PyTorch and TensorFlow.
“Machine learning is another area where WebAssembly might be safe enough, but low enough on the stack that it’ll be a little bit easier to integrate with the hardware than containers have been,” Butcher suggested.
“We’re seeing WebAssembly as a glue layer for some of the niggly parts that were hard to do with containers, that we might be able to do more easily with WebAssembly, and then join up the container ecosystem with some of the harder bits of, say, hardware management or deploying machine learning algorithms.
“I think that’s where you start to really see the story of how containers and WebAssembly are complementary to each other.”