Why Platform9 Migrated Its Managed Kubernetes Service to Packet from AWS
There’s a core economic theory called “economies of scale.” The larger an organization is, the more cost-effectively it can produce widgets, meaning it can undercut smaller competitors on price.
In the context of building applications in the cloud, however, managed Kubernetes provider Platform9 found that this basic economic explanation does not hold water. In 2017, the managed Kubernetes provider started moving away from Amazon Web Services, switching to buying bare metal from the much smaller Packet. The biggest motivator was the cost.
The Importance of Cost
Whether it’s the cost of storage, the cost of running Kubernetes or the cost of serverless, cloud costs are a major concern for almost all companies running in the public cloud, but especially for managed service providers like Platform9. AWS cost structures are also incredibly complex.
“We were happy with AWS as a product and as a service. The main problem we had with AWS was costs,” explained Platform9 CEO Sirish Raghuram. “There’s a zillion levers you can pull to optimize costs, we were spending so much of our time trying to manage and optimize costs.” Platform9 reached a point where investing additional money was inevitable: either in a way to control AWS costs or in moving to another solution that would lead to lower costs.
Given that Platform9 had two other compelling reasons to move away from AWS, it seemed like investing in a solution that would lead to long-term lower costs was the best bet.
It’s no secret that many companies, especially retailers, view AWS as a mortal enemy. Platform9 had prospective customers say they could not use any service that was hosted on AWS. Moving to a non-Amazon solution meant that Platform9 could work with these customers.
The third reason to move away from AWS IaaS offerings was to let Platform9 use its own platform. This wouldn’t be as apparent to customers, but gave the Platform9 team the satisfaction of using their own product and feeling like they were genuinely “walking the walk.” In order for this to be possible, Platform9 needed to move to bare metal.
Why Packet?
Infrastructure service provider Packet is a relatively small bare metal provider that works primarily with startups with between 100 and 500 employees. According to Chris Wright, director of revenue marketing at Packet, one of Packet’s primary value propositions is lowering infrastructure costs: “We tend to hear from these hypergrowth startups that their company is growing 2x or 3x year over year, but their cloud infrastructure costs are growing 5x or 10x.”
In fact, AWS does have a bare-metal service that would have allowed Platform9 to dogfood its own platform — but it costs over twice as much as similar hardware from Packet, according to Cody Hill, director of technology at Platform9.
Packet wasn’t the only option Platform9 considered. What set Packet apart in late 2017, as the project was getting started, was the API-driven nature of everything. “We could leverage their API to access the bare metal infrastructure and use our product to provide a cloud service on top,” Raghuram explains.
“There’s a zillion levers you can pull to optimize costs, we were spending so much of our time trying to manage and optimize costs.”– Sirish Raghuram.
“You don’t have to put in a service ticket, you don’t have to wait,” Hill explains. Everything, from configuring networks to procuring IP address, is done with automation. With Packet as the programmable infrastructure layer and Platform9 as the cloud service, the team had a complete alternative to AWS.
Well, almost.
Migrating between environments — especially when there are production customers and SLAs to mind — is always a complex undertaking. There was only one major unpleasant surprise: storage. AWS had a complete solution that includes robust (if sometimes expensive) storage options. With the combination of Platform9 and Packet, storage was the only thing missing.
The first third-party storage solution Platform9 used with Packet was, in Raghuram’s words, “a train wreck.” The challenges were not just with the storage solution, but also with piecing everything together — now there were three moving parts instead of one, and the integration between the three wasn’t always perfect.
At the beginning of the shift, Platform9 used a blue-green deployment pattern to move from Amazon to Packet. At first, only the VM-based workloads were moved while RDS databases stayed on Amazon, meaning Platform9 had to connect the Amazon and Packet workloads to give the VMs access to the databases.
Platform9 has since moved to a managed database and no longer uses RDS. In fact, Platform9 now uses AWS only for disaster recovery, in case Packet were to go totally offline — the primary sites do not use Amazon’s services at all.
Takeaway
Since moving to Packet, Platform9 also signed one major client that considers Amazon a competitor. Raghuram is confident that the contract would not have happened if Platform9 were still on AWS.
On an infrastructure to infrastructure basis, Platform9 is saving about 63% compared to the costs on AWS. That number doesn’t account for some of the additional investment needed to manage the integration, but even with accounting for increased engineering headcount, the savings are still around 45%.
“I would say the database service and the storage service were the two areas we struggled and ended up having to invest a fair amount of time to get a replacement in place,” Raghuram says. Those are the areas he’s hoping to see other companies step in a provide the final piece of the puzzle, making it that much easier to replace Amazon’s soup-to-nuts solution in the future.
