Tools / Sponsored

Ramin Keene of Fuzzbox on Experimenting in Production

24 Apr 2019 5:00pm, by

This podcast is sponsored by InfluxData and KubeCon+CloudNativeCon. InfluxData delivers a complete open-source platform built specifically for metrics, events, and other time- based data — a modern time-series platform. KubeCon + CloudNativeCon, May 20-23 in Barcelona, will feature key maintainers behind popular projects like Kubernetes, Prometheus, gRPC, Envoy, OpenTracing and other domain experts, adopters, and end users to further the education and advancement of cloud native computing.

Ramin Keene of Fuzzbox on Experimenting in Production

Also available on Apple Podcasts, Google Podcasts, Overcast, PlayerFM, Pocket Casts, Spotify, Stitcher, TuneIn

On this episode of The New Stack Makers podcast, TC Currie is joined by Ramin Keene, CEO and founder of Fuzzbox, a brand new company that makes it safe to experiment in production.  She caught up with him at the LaunchDarkly Trajectory convention in Oakland, California where he ended his talk with the question “If it were completely safe to experiment in production, why wouldn’t you?

Keene said he’s had to make a shift that is common across the industry where he used to be able to know his company’s entire stack and knew exactly what to do to fix it.  But with the growth and addition of complexity, he’s now about three levels removed from what the engineers are doing.

Quoting founder Charity Majors he said testing in production means testing using “real users, real traffic, real scale, and real unpredictabilities.”  

It also means extending the software dev cycle beyond the release. Software used to be shipped on a disk, which is a clear end to the release cycle. But with continuous delivery, such clear distinctions are a thing of the past. 

At the highest level, experience leads to new knowledge, testing doesn’t.  Testing is about verification, confirmation, and integration, while experimentation is about discovery, learning and the unknown.

“You don’t know what you don’t know,” said Keene. And that’s where experimentation comes in. 

One of the challenges of this new way of thinking is creating a good experiment. Coming from an SRE engineering background, he thought he could just have an idea and throw out a test but his data scientists chastised him by pointing out how many unintended changes would be affected by the ‘one’ change he thought he was testing for.  It’s critical, he said, to take the time to learn how to create an experiment.

Listen in to hear Keene talk about the three concepts behind experimentation in production, what Fuzzbox actually provides its customers and the danger of sick canaries.

In This Edition:

2:09: Coming from the waterfall mentality, and using microservices as an organizational problem
4:50: The difference between testing in production and experimenting in production
11:30: How Fuzzbox helps its clients make it safe to experiment off production to help them in production
13:37: So what’s the stack at FuzzBox?
16:51: When did you start the company?
21:07: Who are the most likely engineers that will most likely want to be playing with experimentation in production?

LaunchDarkly is a sponsor of The New Stack.

Feature image by Michal Jarmoluk from Pixabay.

The New Stack is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: LaunchDarkly.