KubeCon+CloudNativeCon sponsored this podcast as part of a series of interviews with Kubernetes end users. Listen to the previous stories about Spotify’s golden path to Kubernetes adoption had many twist and turns and from one server to Kubernetes, a startup’s story.
A lot of the time, it’s harder to convince your friends and family than a stranger. The first group is usually more decisive and direct with you. The same goes for your work family. When you’re building an internal infrastructure for autonomous teams, it becomes your job to not only provide that technical backbone, but to act as sales and customer support.
Nobody said internal developer advocacy would be easier.
The sixth episode of The New Stack Analysts End User Series, we talk with Simone Sciarrati, the engineering team lead at Meltwater media intelligence platform about the autonomous engineering culture, molding developer experience, and those tough technological decisions. Our Publisher Alex Williams co-hosted this conversation, along with Cheryl Hung from the Cloud Native Computing Foundation and Ken Owens of Mastercard.
About four years ago, Meltwater moved from having two departments — engineering and operations — to breaking down into fully autonomous product teams, each given a high level of trust. With this in mind, Sciarrati’s team became the one whose project was to actually serve internal customers — namely their coworkers — by building, maintaining and supporting the Meltwater distributed infrastructure.
With great autonomy comes, well, great decision-making. The company moved to Amazon Web Services, which saw numerous teams adopting Elastic Beanstalk and Amazon Elastic Container Service (ECS). Then it became the infrastructure team’s job to build something more compelling than that.
“Since we cannot mandate that other teams come and use our platform, it’s been challenging because we had to go out and do customer discovery, understand the needs of the other teams and then design a product that would fit with their way of working,” Sciarrati said.
This alternative to ECS is a home-grown Kubernetes platform with a logging platform, as well as continuous delivery and continuous integration or CI/CD metrics, alerting and support for HashiCorp’s Terraform.
As the autonomous teams were already applying GitOps principles to development, that led the two Foundation Mission teams to follow the same way of treating code — everything going through Github — for the infrastructure side. They also chose Terraform, which Sciarrati pegs as the most mature way to manage infrastructure, and a hosted version of Drone for CI/CD, which he says provides the fastest and best user experience.
It’s been a notable success as he estimates that every four to five months they’ve doubled the number of deployments, leading to 70% to 80% of Meltwater teams using at least some of the services running on their platform. But that didn’t come easily despite being in the same building, working for the same company.
“Of course, it took a lot of time because we need to go out and advertise, evangelize and convince not only the engineers, but also the view of engineering managers that it’s a good choice. You should invest the time to come and use our platform because it’s better than something else,” Sciarrati said.
Part of this internal developer advocacy is abstracting unnecessary Kubernetes complexity, while still allowing each autonomous developer to check the logs or to write code, if they wish.
“I’ve noticed that it’s a lot of just complexity that the development teams don’t really get any value from to try to understand that level of detail when really they just want to, like with Drone, they just want to push and have their code deployed. And then they want to move onto to having their automated testing kickoff. They really don’t want to have to try to figure out if their cluster is behaving the way it should, right? That’s really not their job,” Owens said.
When you are selling to your fellow devs, Sciarrati has found it’s important also to not do what he calls a Big Bang Approach where you try to test and automate everything at once. Instead, accepting that the environment is constantly in flux, Meltwater’s infrastructure teams try to make incremental updates on top of Kubernetes at each release.
Now, once the majority of his Meltwater teammates are sold, Sciarrati rounds out this podcast with the constant decisions to be made on behalf of these internal customers, the trade-offs taken, and the ways to continue to improve that developer experience. After all, autonomy doesn’t have to mean loyalty if teammates find something better.
Amazon Web Services is a sponsor of The New Stack.