OpenCost: Open Source Collaboration on Kubernetes Cost Standards
Kubernetes cost management company Kubecost, working with cloud, vendor and user partners, has submitted an open source project for managing Kubernetes costs to the Cloud Native Computing Foundation. Called OpenCost, it combines a specification as well as a Golang implementation of these detailed requirements.
“Starting to talk with more and more partners, there was no agreement on how to measure cost in Kubernetes,” said Webb Brown, co-founder and CEO of Stackwatch, which created Kubecost, said in an interview.
“One of the big problems when you move to Kubernetes, is that costs are essentially a black box. No longer do you have kind of an individual VM or machine for a single application or a single team. In Kubernetes land, everybody’s running in the same cluster typically or a small set of clusters. This really lets you untangle those costs very accurately, show chargeback or just provide cost visibility across your organization with a really high degree of accuracy.”
Its partners in the project include Amazon Web Services (AWS), Armory, D2iQ, Google, Adobe, SUSE, Mindcurv and New Relic. They’ve been working to create a standard for Kubernetes cost monitoring in any environment.
In announcing the submission to CNCF, he said, “From the start, our goal has been to enable teams using Kubernetes to operate with a single model for measuring and managing cost across all of their environments. We consistently hear from users and our partners that standardization across cloud platforms, Kubernetes distributions, and teams is strongly needed. To most effectively drive a standard, we feel it is important that OpenCost is not driven by any single entity — and that the project continues to evolve and thrive within the broader Kubernetes community. We believe the CNCF is the best organization to help with this goal.”
CNCF has yet to announce whether it will accept the project.
The project is based on the Kubecost engine, which is open sourced under an Apache 2.0 license. It includes:
- Real-time cost allocation by Kubernetes service, deployment, namespace, label, statefulset, daemonset, pod and container
- Dynamic asset pricing enabled by integrations with AWS, Azure, and GCP billing APIs
- Supports on-premises k8s clusters with custom pricing sheets
- Allocation for in-cluster resources like CPU, GPU, memory and persistent volumes.
- Allocation for AWS and GCP out-of-cluster resources like RDS instances and S3 buckets with key (optional)
- The ability to export pricing data to Prometheus
OpenCost aims to standardize cost tracking, allocation, methodologies and measurements to help teams using Kubernetes more easily understand their infrastructure costs. OpenCost will provide guidance across various in-cluster resource types — including compute, storage, network, and load balancers — and will address this problem by providing standard cost models, best practices and guidance. Using OpenCost, development teams can align on how to allocate workloads costs, and ensure they are doing so in a standard, uniform manner. This standard enables teams to operate with a single cost-allocation model across their organizations and all their infrastructure environments. The goal is also to create standardization across cloud platforms, Kubernetes distributions, and teams using Kubernetes, the company reported.
“One thing that we’ve consistently heard from both our end users and customers is that they’re excited by the benefits of Kubernetes. They want to adopt Kubernetes. But this lack of cost visibility presents itself as friction to do that. So … we’re removing some of the barriers for teams to adopt Kubernetes at scale. So I think that ultimately, by providing this better cost allocation engine, it hopefully will help teams adopt more and more Kubernetes as they mature,” said Alex Thilen, head of business development at Stackwatch.
In November, Stackwatch announced the expansion of Kubecost to give users a single view into both Kubernetes and “out of cluster” spending on cloud resources. But it’s not the only company tackling the financial aspects of Kubernetes. Others include CAST AI, finout and Harness.
A report from the CNCF and the FinOps Foundation attributed lack of cost monitoring to overspend. It reported that only 38% of respondents can predict their monthly cloud bill within a 10% margin of error. However, cost overruns rank far down on the list of challenges Kubernetes users reported in a Canonical survey. (Lack of in-house skills/limited manpower ranked as the top challenge.)
“We very much want to have other folks join us,” Brown said. “We want more contributions. We want more launch partners, both from end-users and vendors. So part of this announcement is getting the OpenCost initiative out into the world and asking for that feedback and asking for others to join us.”
