Netsil Visualizes the Performance of Microservices
Microservices architectures require rethinking of traditional application performance management approaches to provide greater visibility into applications and their myriad dependencies.
This week, Netsil came out of stealth with its solution to that problem, unveiling its Application Operations Center (AOC), billed as the “Google Maps of microservices applications.”
Netsil takes a network-centric approach, offering a real-time map of all the component containers along with the communications structure of their many dependencies.
“Unlike APM approaches that require an agent within your Java or .NET application, … Netsil operates completely outside your containers and application code, so your application container is completely unmodified. It uses a network packet as a source of truth,” said Harjot Gill, CEO and co-founder.
The San Francisco-based startup, founded in 2016, grew out of research at the University of Pennsylvania and the founders’ experience at Google and Twitter.
Chief operating officer and co-founder Shariq Rizvi. former director of engineering at Twitter said as that service grew, it became painfully obvious that traditional application performance monitoring (APM) approaches that are programming-language specific leave too many blind spots. In addition, a wide range of critical services, such as databases, load balancers, service discovery and DNS, that are impractical to instrument using APM.
“It became clear that one of the big operational challenges was service dependencies,” he said. “Now that all these services are talking to each other, the issue was doing things like stateful deploys, manage service-level objectives… There was a need for this real-time map of who’s talking to whom. Instead of that map, we had engineers spending a lot of time looking at logs.
“It was clear this was a new world, this world of microservices and APIs, that there ought to be a map with no code change at all, and inside these maps, some metrics for service-level indicators. At Google, we used to call them the golden signals of monitoring the service…things like throughput, latency for every service.”
The AOC generates maps that automatically discover every Docker container, Kubernetes pod, host and service endpoint, along with all the interactions among them. The maps also capture key service health metrics for API calls, database queries, DNS lookups and other service interactions.
It involves installing one agent per node. With Kubernetes, most customers just install a container, then install a daemon on all the worker nodes. It starts looking at all the network traffic and collects all the metadata on which services are communicating and the protocols they use.
Its real-time analysis of packets includes calls to external services such as AWS RDS, AWS DynamoDB, API calls to Google Maps, Salesforce, Stripe, Twilio and more.
The company says the AOC is especially useful with Kubernetes, as it can create maps of hosts, namespaces, services and pods, and drill down to diagnose problems in such areas as service configuration, service availability and service creation.
The service includes more than 70 integrations with services such as JMX, NGINX, and MySQL and allows users to be alerted to issues such as latency, errors and throughput before they reach a critical stage. Users can set their own critical alert rules by user-defined groups, on tags and attributes, or globally and reuse them across multiple similar services. Alerts can be made with Pagerduty, email or webhook integrations.
He said he doesn’t foresee the AOC replacing logging products, which are more traditionally used for things like auditing or deeper root-cause analysis and debugging, it does see it taking on monitoring and APM competitors.
“If you’re talking about real-time visibility for incident response or CI/CD use cases, understanding performance issues, that’s where logging is not a great fit,” Gill said.
For now, the service merely draws attention to problem areas, but the company plans to add more.
“We’re planning to bring insights, so we can call attention to interesting data and interesting trends that are happening in the cloud. That’s all in the roadmap. We want to [provide the ability for] operators to ask questions and get answers,” Gill said.
Limetray, a marketing and technology company serving the restaurant industry, is among the early users vouching for Netsil. It stack includes Kubernetes, Docker and AWS.
“As we add more features and services for our customers, one of the biggest challenges has been understanding service dependencies. Netsil maps and metrics have been instrumental for us to understand transaction flows and quickly identify root causes of latency and errors before they impact our customers,” said Sooraj Elamana, vice president of engineering at Limetray.
And Ursheet Parikh, partner at Mayfield Fund, one of the lead investors in Netsil, points to the complexity that comes with the popularity of Kubernetes and Docker.
“This shift [to microservices architecture] is exposing new challenges and creating opportunities to rethink entire category of products such as APM. We are very excited about Netsil’s radically innovative approach that delivers pervasive observability for DevOps with immense simplicity and ease of use.”
