Monitoring Distributed Architectures

“The mainframe lasted for a while, but as IT got decentralized, the physical machine became the fundamental unit that people used in order to think about their infrastructure and their architecture — having the host as the center of the universe,” says Alexis Lê-Quôc, CTO and Co-Founder of Datadog, during this conversation with Alex Williams and co-host Donnie Berkholz of RedMonk about monitoring distributed architecture. It’s a fitting conversation, especially with the news today that SignalFx raised $20 million in a Series B round for its modern app monitoring service.

For more episodes of “The New Stack Analysts” check out the podcast section.

#34: Monitoring Distributed Architectures

Alexis continues, “You wouldn’t think in terms of, ‘they’re just apps that run somewhere, and someone’s taking care of the ether in which they run.'”

Instead, he observes, people percieve, an app as running in the machine. “It’s there. I can see it,’ or, ‘it’s in the data center next door.’ I clearly map an application to a machine to the point where, when I say ‘the database,’ I mean ‘the application’ but I also mean ‘the machine that runs the database.'”

“So, we have this universe where machines matter, and then we start to distribute applications, just because the machines can scale-up only to a point, and yet the demand for processing — the demand for apps — outstrips the scale at which we scale-up the machines. So we need to distribute — instead of having one database, I have a cluster of databases, and then I have multiple clusters. There’s still a relatively static mapping between part of the architecture and machines.”

“A lot of demand right now is around Infrastructure as a Service. People want machines. Twenty years from now we may come back and say ‘this was dumb, because that set us back decades, because we carried over a lot of the baggage we had from the physical environment.'”

Alexis posits that this attachment to the notion of “the host” is the central fallacy of monitoring distributed architecture. “The host works as the center of the monitoring view because it’s stable. It exists for a period of time that we as people can relate to,” Alexis concludes, “but that’s also, conversely, why it doesn’t work for heavily distributed architectures.”

“The move to distributed architectures where things change all the time, forces monitoring to go from very imperative to go to something much more declarative.”

Digital image courtesy of The Getty’s Open Content Program.

Datadog and SignalFx are sponsors of The New Stack.