Why Cloud Native Open Source Is Critical for Twitter and Spotify
At last count, social media giant Twitter enjoys around 353 million active users, and streaming music service Spotify has 356 million active listeners. In both cases, open source tools and platforms for cloud native environments have served as the cornerstones for their tremendous growth.
In this latest episode of The New Stack Makers podcast, Spotify Senior Staff Engineer Dave Zolotusky, and Twitter Developer Experience Lead and Manager for Engineering Effectiveness Jasmine James discussed the role of open source software in their respective organizations. Katie Gamanji, ecosystem manager of the Cloud Native Computing Foundation and Alex Williams, founder and publisher of TNS, co-hosted this interview.
Why Cloud Native Open Source is Critical for Twitter and Spotify
One commonality between the two services: Kubernetes.
“There’s a lot of innovation that happens with open source internally, and one of the [examples] I can name — that is actually a CNCF project — that’s on my radar right now is Kubernetes,” James said. “Everybody’s probably familiar with that one — we’re going through a large migration that I’m helping accelerate, so I would say that that’s probably the biggest example that I can talk about right now.”
Spotify began migrating to Kubernetes a few years ago and today a significant amount of its backend operations runs on Kubernetes, Zolotusky explained. “We spent many years building various types of infrastructure ourselves, including container orchestration. And obviously, none of those things have taken off the way like Kubernetes has. So, in that sense, we’re doing our best to stop building our own stuff and use much more open source or at least industry standards,” Zolotusky said.
Open source tools and platforms play a critical role in how both companies scale their services. In Twitter’s case, for example, “scale is always kind of at the forefront of the conversation,” James said. “I think that one of the things we always like to think about is how we can really make it easy to scale based on metrics or monitoring or things of that nature, so that we don’t have to manually do it. So, obviously, having applications containerized and having very focused microservices that can easily grow or be spun down if they’re not needed at a certain time are key,” James said. “So, we have a very good process around architecting these services and making sure that that’s considered, and we have a very robust technical design process that all applications go through internally to make sure that this is something that we can keep considering.”
