How Google Cloud Run Combines Serverless and On-Demand Kubernetes

Google’s new Cloud Run service is a serverless platform built on Knative, the runtime environment that extends Kubernetes for serverless workloads, and the Functions Framework, which Google is also open sourcing for Node.js 10 today. But unlike other serverless functions where you typically bring code written specifically to run as a function and be triggered by events, Cloud Run lets you package up existing code in a Docker container.

That container can run in the fully managed serverless Cloud Run environment, but because it’s using Knative it can also run on Google Kubernetes Engine, so you can add on-demand, pay-as-you-go code to your existing Kubernetes clusters.

You decide where to run the container when you deploy it rather than when you package it, because many customers are asking for the same serverless functionality but half of them are looking for simplicity and half of them are looking for more flexibility, Oren Teich, director of product management director Google Cloud, explained to us. “I’m already using Kubernetes, I already have my cluster, my ops team manages it, maybe there are regulatory concerns. We want to avoid the problem of ‘because I wanted X, I can’t do Y’.”

“Cloud Run on GKE allows the exact same developer experience on a managed operational infrastructure on GKE. The developer can take the code they’ve written and have the exact experience in the portal or at the Gcloud command line and target what destination they want. You can use the same tools; you can have one experience, you can learn the APIs and how the user interface works and take the workload and move it onto any Knative compliant cluster.”

The idea is to improve and simplify the developer experience, and he expects that some organizations will use a mix of serverless and GKE deployments for Cloud Run for different developers.

“We’ve heard from customers that they love the idea of serverless and they love the managed infrastructure and they love handing off the operational concerns. But they want to be able to run arbitrary workloads. They want to be able to take proprietary software or old software like a 15-year old Java binary and still get all the benefits [of serverless].”

Cloud Run offers a combination of Docker and serverless. “[Docker] is the standardized format to package stuff up, with an ecosystem on both side, and we can make it easier to create developer experiences to get to the next generation and deliver the promise we had ten years ago but with the flexibility that people need to run real stuff.”

Where to Run Cloud Run

The code you can deploy to Cloud Run needs to be a stateless HTTP container that listens for requests on a port that will be configured as (but should not be hardcode as) 8080, starts an HTTP server within four minutes of receiving a request, responds within the request timeout, and is compiled for 64-bit Linux.

“Your container should not rely on local state,” Steren Gianni of Google’s serverless team explained in a session at Next. “You can use consistent storage like databases and cloud storage but inside a container you can only rely on temporary state. The thing you create will not necessarily be there next time you invoke the container.” The limitations are specified in the container runtime contract.

If you don’t already have your code in a container, you build it in the usual way and put it in the container registry.

Deploy the container to Cloud Run by pasting in the URL of the container registry and what you get back is an HTTPS URL. Google demoed Cloud Run with a Python script calling LibreOffice (a Java binary) with a Python script, installing fonts and using Flask to convert Word documents to PDF; that added up to a 2Gb container that deployed in seven seconds over conference Wi-Fi and took ten seconds to start but then processed a thousand PFDs in a minute.

This is a stable HTTPS URL that you can expose to the internet, if you want to use it for a web site or an API endpoint or a mobile backend or a Webhook. Or you can use it internally to refactor a monolith into microservices that scale independently.

The fully managed Cloud Run service is the simplest option. “You just give us a container and everything else is taken care of. We deploy it, we set up HTTPS, we set up the endpoint, we set up the load balancing, we set up the routing, we set up the scaling. There are no parameters to tweak. This isn’t a complicated orchestration system, this is a very easy to use developer-friendly product.”

Cloud Run is pay as you use, charged per 100 milliseconds of usage. “It scales from zero so you only pay when a request comes in. This isn’t a web task that you have to keep running 24/7 and pay for it even if there’s no traffic; if there’s no traffic, we shut things off and when traffic comes in we run it and we scale it up.”

In Cloud Run, the container runs on top of the gVisor sandbox that Google open sourced last year as an optional replacement for the default Kubernetes runc runtime. That does limit what Cloud Run can run because it doesn’t support all the Linux system calls, but Teich says that in practice this is rarely a problem. “There are a few odds and ends that you might hit but the reality of what we see for customers is that as long as they are stateless HTTP containers [they run] … Every time we’ve found a rough edge, it’s usually very, very quick for us to fix it.” If your code does have issues in gVisor, Cloud Run on GKE doesn’t use a container sandbox.

There are other limitations that might make Cloud Run on GKE a better choice for some workloads. Cloud Run allocates a single vCPU to a container and you can’t change that; if you want more control over resources, Cloud Run on GKE lets you choose more machine sizes and more RAM and CPU or use custom machine types, including GPUs and TPUs for acceleration. Workloads can run side by side with the existing cluster workloads and they also have access to the VPC and the Compute Engine networks.

Cloud Run on GKE has the same autoscaling, including scaling down to zero, but it isn’t pay for use in quite the same way. The price (which hasn’t yet been published) comes out of the GKE resources you purchase (and you can use the GKE usage metering to charge that on to internal users).

Because Cloud Run is built on Knative, using the Kubernetes Custom Resource Definitions that provide the necessary abstractions, you could start by containerizing an existing application and running that on its own, then move it to GKE to run as part of a Kubernetes deployment without needing to repackage it; you just choose the GKE cluster as the location in the Cloud Run console when you fill in the URL of the container you want to run instead of picking a Google region.

If you’re using Knative on another Kubernetes cluster outside the Google cloud, you can add the Functions Framework to it to and be able to migrate containers or serverless code to Cloud Run. In fact, Teich suggests, “through the Knative samples and the reference API this is all based on, any cloud provider that implements Knative is able to take advantage of the same benefits. Knative is the reference implantation behind Cloud Run.”

Because it’s Kubernetes underneath, Cloud Run can take advantage of the Kubernetes ecosystem. Cloud Run integrates with Stackdriver for monitoring and logging, but you can also monitor with Datadog (using the Datadog agent for Cloud Run on GKE and the standard Google cloud monitoring APIs on the fully managed version), or use NodeSource’s N|Solid runtime for Cloud Run on GKE to get real-time performance metrics and custom notifications for Node.js code.

If you’re experimenting with Cloud Run on GKE, it’s worth noting that you can’t disable Cloud Run on GKE in the beta service; to stop it running you have to delete the cluster, so pick a non-essential cluster to try it out on. And if you don’t have an existing application that you want to try on Cloud Run, StackBlitz is working on integration so you can use its one-click system to create a new app, GitHub repo and Cloud Run endpoint, ready to start coding and testing.

Initially, Cloud Run is only available in US regions, but Google says it will be coming to European regions soon.

Google introduced Cloud Run during its annual Google Next conference, held this week in San Francisco.