Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
Kubernetes / Operations

Kubernetes for Developers with a Distributed App Runtime

Diagrid shares a demo of how Dapr makes Kubernetes accessible for developers by creating a separation of concerns.
Aug 15th, 2023 1:37pm by
Featued image for: Kubernetes for Developers with a Distributed App Runtime

Dapr is a distributed application runtime that does the busy work by codifying platform requirements through APIs accessed over HTTP or gRPC. It makes Kubernetes accessible for developers by, in many respects, creating a separation of concerns. Dapr is an open source project under the Cloud Native Computing Foundation. The Dapr project was originally created at Microsoft. Diagrid was founded by members of that founding team to create new offerings based on Dapr as well as continue to lead the development of Dapr with other contributing organizations and the community.

“We thought, ‘Hey how can we do this better?'” said Mark Fussell, founder, and CEO of Diagrid and Dapr’s co-creator, during a demo with The New Stack. Diagrid provides a service, Diagrid Conductor, that manages Dapr on Kubernetes.

“It helps you operate and manage that on top of Kubernetes,” Fussell said about Conductor. So think of it as a managed version of Dapr on Kubernetes.”

Communication, state management, managing secrets — even things like workflows get codified with APIs in Dapr. Dapr users can use any language of their choice.

“I mean, Kubernetes has got nothing to do with developers,” Fussell said. “And yet they’re told to build on top of it all. So we’ve seen by using these APIs, developers are anywhere between 20 and 40% more productive building their applications.”

Dapr runs on a sidecar model. A developer writes their code, and Dapr appears as a sidecar when the application runs. It does the heavy lifting for the developer to reach their goals.

In the demo, Fussell provided examples of the API calls using the example of a shopping cart application to do a localhost call to the Dapr sidecar, running on port 3500.  An invoke method gets called for an order method, and Dapr then does the discovery, secure call with retries, and other requirements.

Using a sidecar method comes down to a separation of concerns. A user may update the libraries separately that often get bundled apart. Also, with the separation, a new version of Dapr eliminates the task of updating the application code.

“Often, when you pull all these runtimes together into one application, you don’t know whether it’s the runtime or your application that’s having issues,” Fussell said. “So the separation of concerns, particularly in a distributed application environment, means that you can update things independently, resolve issues independently.”

And the latency is almost minuscule, he said. It takes milliseconds for service invocation.

“So you get great performance as well as separation of concerns. So yeah, so that’s what Dapr is,” Fussell said.

Fussell said that the concept of components reflects a core aspect of the APIs. It’s how the infrastructure gets integrated into the application. State gets put into state stores. A state store from a shopping cart application that users may swap between different cloud services or a local development machine to the cloud, thereby providing code portability.

“It gives us dynamic behavior about how you tie infrastructure to your application without having to pull in a specific SDK or throw away your code and having to learn these things,” Fussell said.

The Dapr technology reflects what we heard several years ago when technologists said that Kubernetes would eventually disappear. Dapr makes Kubernetes accessible to developers. They still know Kubernetes sits underneath, but they do not need to understand the inner working as much as a team did in 2016. Kubernetes had no use for developers back then. Dapr allows developers to work on applications across cloud services, a dynamic seen now more often across the cloud native landscape.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: Pragma, The New Stack.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.