How AWS Uses Chef in a Post-Container World
You’re likely running some form of server on Amazon Web Services right now, somewhere in the world. If so, you’re also entrusting your systems to the reliability and stability of the folks at AWS: that they’ve thought through their DevOps practices and have your back when it comes to scaling, outages, and backup. And you’d be correct to place your trust there: Amazon’s AWS track record only gets better every year as billions of applications successfully complete their business goals within its confines.
But just how does Amazon maintain such a spectacularly science-fiction sized hardware and software hosting system? The answer is not only DevOps, it’s also Chef. At Chefconf this past week, we sat down with Jonathan Weiss, senior software development manager and Mark Rambow, software development manager, both from Amazon’s AWS team. We traced the lines of control and practice inside their company to find just what it is that allows them to not only operate at scale, but to allow everyone else in the world to operate at scale as well.
Their answers were fascinating. “We believe in what we call, ‘working backwards from the customer.’ Which means, we always try to start with the customer problem, how can we resolve that, how can we remove that friction for the customer and work backwards to the technical problem,” said Weiss. “We always like to put this to a dedicated team to solve. We call this a ‘two pizza’ team. A team you can feed with two pizzas. And then that small group of people owns solving this problem and bringing the DevOps aspect to it, and that means defining the problem, designing the solution, implementing the solution, then deploying, operating it and maintaining it to a degree.”
“Our aim is to make infrastructure effortless for customers, right? So when you go to the cloud, it’s just convenient to get new server instances running in Docker containers deployments is a problem that is solved for our customers,” said Rambow. “We like to evangelize what we do internally to our customers because they can benefit from the agility that we live currently, so that they can adopt the same best practices.”
The pair also said that Chef continues to be useful to their teams, even in a container-based world. Configuration management still covers a lot of needed use cases for enterprises, from compliance to deployments. Weiss pointed out that a major area where enterprises continue to need help is around, “The famous legacy application, of which there exists a lot more than of greenfield applications. These are usually way harder to get into something like Docker or Kubernetes but it’s easier to put them in an image and replicate them with something like Chef so that you’re able to at least lift and shift them into the cloud where you can run flexible workloads, you can benefit from things like auto-scaling, even though you’re not able to automate them as much as you would like to.”
In this Edition:
- 1:43: How does DevOps work inside Amazon?
- 5:36: What are you seeing from customers in their technology decisions that are requiring you to add features?
- 7:11: How have those internal processes you’ve built end up externalized?
- 14:06: For AWS, how is configuration management evolving as a practice?
- 21:42: What is OpsWorks?
- 26:32: How do you see the architecture and infrastructure you’ve developed evolving as customers get more sophisticated about what they’re trying to do?