How Arktos Takes Kubernetes to the Next Level
Arktos is an open source project designed for large-scale cloud infrastructure. It’s a cluster management system, as well as a container and VM orchestration system. The project is licensed under Apache 2.0 and is fully open and public.
Arktos was evolved from the Kubernetes codebase and features a lot of similar API objects — like pods and replica sets. But Arktos has core design changes, including fundamental improvements in areas like VM orchestration, scalability and multitenancy.
Why We Need Arktos
There are already projects like Kubernetes and OpenStack in open source communities. Both of them can be used to build cloud infrastructure, to some extent. There are also several proprietary Infrastructure as a Service (IaaS) platforms, by public cloud providers like Amazon or Microsoft. So why do we need Arktos?
When we started Project Arktos, we had the following ideas in our mind:
- Today’s cloud IaaS layer focuses on VM and resource management. Containers are managed by a separate system. Can we have a simplified architecture that manages both containers and VMs as first-class citizens?
- Application orchestration is the norm in the container world — such as replica set, stateful set, etc. But they are not used in the traditional VM world. Can these orchestration paradigms also be applied to VM applications?
- The modern cloud has tens of thousands, or even millions, of hosts. But existing open source projects usually support only a few thousand hosts. Can we have an open source platform that supports the public cloud level of scalability?
- Edge computing and 5G are increasingly being deployed. Can a cloud platform intelligently coordinate resource allocation between cloud data centers and edge sites? Can we have built-in support for service offloading and enhanced security for that?
When we looked around the industry, we couldn’t find an open source project that meets the above requirements. While Kubernetes is the de-facto standard of container orchestration, it doesn’t have capabilities of VM orchestration, multitenancy, or cloud scalability. OpenStack Nova has been around for many years, but it doesn’t have good support for containers — nor does it have very good scalability. And neither one has special considerations for edge computing.
That’s why we started Arktos. We want to provide an open source solution for a large-scale new cloud infrastructure.
The Key Features of Arktos
To support the above vision, we built the following key features into Arktos:
Unified Container/VM Orchestration
With more and more containers running on bare metal, today’s cloud data centers have separate orchestration stacks for containers and VMs. This introduces a lot of problems — such as separated resource pools, duplicated components, increased maintenance and operation cost.
Arktos introduces native support of VM, in addition to the mature container support inherited from Kubernetes. Now cloud providers can have a unified resource pool and only need to maintain and operate one system. Customers can run container workloads and legacy VM workloads in one system.
Cloud-Scale Scalability
Arktos provides a public cloud level of scalability. Our design aims to support 300,000 hosts per region and 100,000 hosts per cluster.
From backend storage to the frontend API server, all control plane components can scale-out and are highly available. Workloads are partitioned and handled by different instances.
Cloud-Edge Scheduling and Offloading
We are working on a global scheduler that helps coordinate resource allocation among multicloud data centers and edge sites. It will intelligently place workloads on the best cloud or edge site, based on application traffic patterns and global resource availability. We are also exploring built-in support for offloading applications and cloud services to edge sites.
Built-in Multitenancy
As I mentioned earlier, Arktos has evolved from the Kubernetes codebase. Kubernetes doesn’t support multitenancy, but multitenancy is a must for a cloud platform — especially for public cloud.
Arktos has built-in multitenancy support, based on the idea of a virtual cluster. It provides strong tenancy isolation. All tenants are sharing a single physical cluster, but they all feel they are using the cluster exclusively. They are not aware of the existence of other tenants at all.
Conclusion
These features are still under active development. As of this writing, Arktos has three releases and its GitHub repository has more than 700 commits. Thank you to all of the contributors! We are also collaborating with some academic researchers to explore more challenging ideas.
In upcoming posts, we will talk about these features in detail. If you have any questions, please contact us in Arktos slack channel (invitation to the channel) or file an issue of question type on our github repo.