Robinhood’s Kubernetes Journey: A Path More Treacherous than It Appears
Welcome to The New Stack Makers: Scaling New Heights, a series of interviews with engineering managers who talk about the problems they have faced and the resolutions they sought, conducted by guest host Scalyr CEO Christine Heckart.
The challenges engineering leaders face define how technology decisions are made to support the demands of rapid-growth businesses.
Heckart’s first interview is with Adam Wolff, Robinhood’s former vice president of engineering who recalls two years ago when Kubernetes looked so right — and then the difficulties that followed when Wolff mandated the platform’s adoption. Robinhood is an online investment service, offering a free app for trading on financial exchanges.
Robinhood’s Kubernetes Journey: A Path More Treacherous Than it Appears
“So I read all the marketing materials for Kubernetes,” Wolff said. “And I, like, played with a demo. I was like, ‘this looks great. This seems like exactly what we need.'”
Now, two years later, the Robinhood team is still working on its migration to Kubernetes, said Wolff, who left Robinhood after this interview was recorded.
“We’ve made a lot of progress, but it’s been much more difficult than we expected,” Wolff said. “And there’s a lot of reasons for this. And I think our team is not alone, in finding that this path is more treacherous than it appears.”
In retrospect, it would have been better to focus more on how to achieve faster releases with immutable infrastructure, he said, especially for a company with a few hundred engineers.
“For a company like Robinhood to deploy technology like Kubernetes, it’s not just one team, you have an infrastructure platform team that’s looking to deploy this, you have a security team that needs to sort of agree on how we’re going to do this and what the potential gotchas are,” Wolff said. “And then your product teams need to decide that it’s worth it to adopt this technology. Each one of these teams will bring their own goals, desires, fantasies to this project.”
Adopting Kubernetes has not been the only challenge for Robinhood. Last week, Bloomberg reported that a number of customers had reported compromised accounts but they had no one to call to get their matter resolved. In a statement to Bloomberg, Robinhood has said a limited number of accounts had been targeted by cybercriminals. The problem, they say, is that customers’ personal emails used for Robinhood accounts were accessed outside of the Robinhood platform. A Robinhood spokesperson said they are working with the affected customers to better secure their accounts.
Today, the lessons learned are about the fundamentals, the basics that an engineer gets taught. Sure, the engineering is cool, the new stuff is exciting. But the engineering does not lead the business. It’s the business that leads the engineering.
