Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.

CNCF, Eclipse Explore Kubernetes-Driven Internet of Things/Edge Computing

Sep 28th, 2018 8:31am by
Featued image for: CNCF, Eclipse Explore Kubernetes-Driven Internet of Things/Edge Computing

The Eclipse Foundation and the Cloud Native Computing Foundation have partnered to create a Kubernetes Internet-of-Things (IoT)/Edge Computing Working Group.

As IoT technologies become increasingly considered for industrial-cloud use, “the idea of collaborating with Kubernetes became more and more interesting to both groups,” said Mike Milinkovich, executive director of the Eclipse Foundation. “There’s interest in seeing how this stack plays out over time.”

The CNCF has 29 projects in various stages of maturity, and members such as VMware and Siemens are interested in investigating the K8S synergies with IoT and edge computing.

The Eclipse IoT Working Group has been around for six years, and has amassed nearly 3 million lines of code across 36 projects, supported by 42 companies, such as Bosch and Red Hat. The groups aim to be “the open source center of gravity for the Internet of Things,” Milinkovich said.

Eclipse-member Bosch, for instance, has over 300 factories it wants to manage in a cloud-friendly way, and also wants to extend this work to the company’s own customers.

One project that Bosch is leading is Eclipse Hono, which does large-scale MQTT data ingress from as potentially many as millions of end devices, converting it into AMQP and then routing it to the application that cares about the data from that particular device. Hono runs atop of Kubernetes.

“Historically, Kubernetes has been very focused on HTTP and TLS for the request-response transport layer, but what will we do when we have potentially millions of devices” that don’t run on these protocols? Milinkovich asked.

In such a scenario of horizontal scaling, Kubernetes could provide a standardized control plane and manage the applications for edge and hybrid-cloud operations. It could be used to manage an edge computing deployment, or multiple edge computing deployments. K8s could also manage multiple IoT gateways.

“When you start talking about the full breadth of IoT, there are so many use cases and user requirements, that there is no one way to do this,” Milinkovich said.

The first bit of work the group will take on is to settle on some common terminology, for terms such as “IoT gateway,”  as well as a common understanding of some user requirements. Documenting use cases and establishing some reference architectures are also on the to-do list.

“To enable this distributed computing infrastructure that we need to have, we also need to have a common set of tools, and a common set of infrastructure that will help us enable that,” Milinkovich said. The working group will initially look at the core CNCF technologies such as Kubernetes but expand into other potentially useful technologies as well, such as the Linkerd service mesh or the Prometheus monitor.

The group plans to hold regular online meetings, in-conference meetings at events such as Kubecon,  as well as foster ongoing discussions on mailing lists.

The Cloud Native Computing Foundation and Red Hat are sponsors of The New Stack.

Feature image: Imgflip meme generator.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: The New Stack.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.