Isaac Mosquera, CTO at Armory, Chats about Canaries and Testing Microservices in Production
Isaac Mosquera, chief technology officer and co-founder at Armory.io sat down to talk about testing microservices in this latest episode of The New Stack Makers podcast, hosted by San Francisco correspondent TC Currie.
Within the Armory platform is a product called Canary which is one tool to help test microservices in production. The name comes from the traditional miners’ practice of taking a canary with them into coal mines. When mined, coal can release an odorless toxic gas that would kill miners where they stood. The canary became an early warning system. If the canary died, the miners left the mine, quickly.
Similarly, Canary launches a small bit of code into production to see how it effects the system as a whole, and discover is this element introducing a lot more risk into the production environment. “That’s a lot cheaper way to do it than set up manual QA testing to assess the risk,” Mosquera said.
Canary and Feature Flags
Canary is one tool in the expanded toolbox needed to test the more complicated platforms using distributed systems. Deploying new microservices still requires old-school methods of integration and regression, but requires additional testing tools and automation. It’s impossible for a human to do regression testing across all services, said Mosquera. So what other tools can we use to minimize the risk? Feature flags are a part of that toolbox.
“Canary-ing is more of an engineering exercise, and feature flags are more of a product and marketing exercise,” he explained. Feature flags provide the ability to expose a particular feature at a particular date to a particular set or group of people. The flags limit who gets affected by the launch of a new feature, from allowing your manager to see the changes, to a group of beta customers so you can assess the impact before deploying to all production users.
Automation Is Key
Armory automates everything because they are typically the source of errors in software deployment, he said. The product comes loaded with all the metrics that are associated with a typical application, e.g. CPU, memory, disc, bytes i/o, queue sizes, and so on.
In addition to these system level, engineering-focused metrics, the software rolls in data from Jira or ServiceNow into Canary to determine how the launch affected the business and automatically roll it back if the effect is negative to either the system or to the business. “You want as many sources of data so you can get a clear signal if it is a bad deployment,” said Mosquera.
“We get the engineers back to doing what they do best which is innovating and writing code, and what they’re actually good at.”
Listen in to find out what causes outages, why machine learning will never replace Canary, and why you can’t organize agile deployments across distributed services with waterfall management methods.
In This Edition:
3:56: What is the Canary platform?
7:41: Canary vs. Feature Flags and why you need both.
13:48: Automate the QA Process: Humans introduce risk.
19:43: Pulling Canary out of Spinnaker.
21:25: Why managing microservices with waterfall business processes is failing.
27:03: How big is big enough to start using Armory?
Feature image via Pixabay.