Cloud Services / Data / Tools

Confluent’s Kafka Platform Becomes an Integrated Service for Azure

2 Feb 2021 3:00am, by

Confluent’s commercial Kafka platform for managing streaming data now can work seamlessly with Microsoft Azure such that Azure customers can access Confluent Cloud without “ever leaving the Azure portal,” Dan Rosanova, head of product management at Confluent, told The New Stack.

As “one of the first partners to build this type of integration with Microsoft,” Rosanova said, Confluent and Microsoft engineering teams have been collaborating to code a custom interface in Azure specifically for Confluent, Rosanova said.

Previously, Confluent users had to switch between Confluent Cloud and Azure interfaces, while now, “everything they need is in one place,” Rosanova said. Users can use their same cloud identity to interact with Confluent Cloud and can also view and track their Confluent Cloud resources from within the Azure portal. “That was not possible without this new native integration. We’ve had payment integration with the three major cloud providers for some time, but this goes beyond that to user experience and identity integration.”

According to Confluent, 80% of the Fortune 100 companies have adopted Kafka for data streaming, often across multiple cloud environments located in different geographical locations around the world. But Kafka, like Kubernetes, can pose difficulties managing it without support platforms and interfaces. Confluent’s raison d’être is to thus provide platform support that an in-house team of Kafka experts would otherwise have to provide.

One of the idiosyncrasies associated with Azure that required configuration work by Confluent’s engineers involved how Azure “works off a resource provider for the control plane,” in order to create and manage resources, Rosanova said. “This is a little unique to Azure, but has many advantages including integrated authentication and the ability to create deployment manifests for many different types of resources that all get deployed together,” Rosanova said. “Confluent basically has a resource provider in Azure now.”

Confluent Cloud users on Azure will quickly notice how authentication is integrated with their  Azure accounts. “This means your Azure accounts, which are backed by Azure Active Directory, are used for authentication with Confluent Cloud,” Rosanova said. “The second thing is that you now have a resource visible in Azure that is your Confluent Cloud instance. So this actually goes far beyond a marketplace offering.”

While noting that Confluent supports single sign-on integration with both Amazon Web Services (AWS) and Google Cloud, Rosanova said Confluent is “in talks with other cloud providers about similar integrations, but each provider works differently so the integrations may be different as well.”

“We certainly value all of our cloud partners and understand that customers want to use the cloud infrastructure provider of their choice, so we will do everything we can to be the best streaming service provider in an authentic way that feels natural to the user of that cloud provider,” Rosanova said.

With Confluent Cloud, Confluent said Azure customers can now access managed connectors to Azure and Microsoft services, including:

  • Azure Functions.
  • Azure Blob Storage.
  • Azure Event Hubs.
  • Azure Data Lake Storage Gen2.
  • Microsoft SQL Server.

Feature image par David Mark de Pixabay

A newsletter digest of the week’s most important stories & analyses.