Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.

VMware, AWS Partnership Provides a Container On-ramp to the Cloud

Oct 14th, 2016 7:47am by
Featued image for: VMware, AWS Partnership Provides a Container On-ramp to the Cloud

In the latest act of its long-term — and thus far, successful — strategy to forestall any chance of hypervisors being rendered obsolete by containerization, VMware and Amazon Web Services announced Thursday afternoon that they are making Amazon Web Services the “primary public cloud” deployment platform for vSphere-driven virtual machines.

In so doing — and as the VMware is expected to confirm next week at its conference in Barcelona — it cuts the ribbon on an end-to-end hybrid deployment scheme that enables developers to build application containers using Docker tools, yet has those containers be rendered and deployed as vSphere virtual machines, on the AWS cloud.

As a VMware spokesperson confirmed to The New Stack late Thursday, the partnership does enable vSphere Integrated Containers (VIC) to be deployed seamlessly on AWS public cloud infrastructure, using VMware’s existing tools.

No More Binary Choice

“This becomes the primary public cloud offering of VMware,” said CEO Pat Gelsinger during a press conference Thursday [pictured above at right], “sold, supported, and delivered by VMware. And the result is, we give our customers this ability to seamlessly operate across this hybrid environment that we are announcing today.”

Developers’ and administrators’ first taste of VMware Cloud on AWS will come by way of a limited participation beta program, for which applications are being taken now.

“If you think about it, the vast majority of enterprises in the world are virtualized with VMware,” said AWS CEO Andy Jassy [pictured at left above], during the same press conference. “And they have gotten used to running the software and tools that VMware offers to run their on-premises infrastructure. Because AWS and VMware didn’t provide a seamless experience, and a seamless opportunity to run that same software on-prem and in AWS, they, in fact, did have to make a binary decision, which was, ‘Either I use the VMware software, and it’s hard to use AWS in the public cloud; or I use AWS in the public cloud, and I have to leave behind the VMware software.’”

That binary choice to which Jassy referred used to be reinforced by VMware’s decision to compete with AWS in the public cloud space — a decision which VMware now admits didn’t pay sufficient dividends.

The Big Boss Moves In

During the VMworld 2016 conference in Las Vegas just seven weeks ago, VMware announced a partnership that enabled VMware services to be supported on IBM Cloud. The company had already tried to compete against the Big Three cloud providers with vCloud Air, without much success. Customers had seen deployment options for VMware VMs on Microsoft Azure, though word was held back a potential AWS integration, even though it was obvious that VMware’s NSX network virtualization platform was being adapted for AWS. VMware executives’ facial expressions at the time were signal enough to analysts that something was brewing.


“This new offering is a native, fully managed VMware environment on the AWS Cloud that can be accessed on an hourly, on-demand basis or in subscription form,” writes AWS Chief Evangelist Jeff Barr, in a company blog post Thursday. “It includes the same core VMware technologies that customers run today in their data centers today including vSphere Hypervisor (ESXi), Virtual SAN (vSAN), and the NSX network virtualization platform, and is designed to provide a clean, seamless experience.”


VMware VP for Cloud Products Mark Lohmeyer and AWS General Manager for Cloud Strategy Matt Wood took a few minutes to demonstrate Barr’s point, making a seamless deployment of a VMware virtual machine to AWS’ servers in Ireland, using only the latest versions of VMware’s native tools. Although that demo did not involve Docker-style containers, it did serve as clear proof that NSX had been successfully re-engineered to provide customer infrastructure support service for AWS.

VIC is VMware’s version of containers designed to be hosted on NSX. By extending the NSX platform onto the world’s largest public cloud, VMware makes it not only feasible but to a measurable extent, desirable for an enterprise to have a hybrid cloud deployment option that is fully integrated into its existing environment. This way, containerized environments would not have to be compartmentalized to separate servers — virtual ones, or even physical. Microsegmentation is how VMware handles the problem of subdividing any workloads that require exclusive policies, effectively treating containerized workloads as though they were separate, without bending the network to make them separate.

Control Plane

NSX is where VMware’s microsegmentation capability comes from.

[SCM]actwin,88,83,88,83;http:// AWS Summits Live Stream - Google Chrome chrome 10/13/2016 , 4:47:07 PM

“By linking [AWS] to a customer’s existing, on-prem environment,” said Lohmeyer, “we can provide a common control plane across those two worlds that enables a whole set of new, powerful hybrid use cases.”  He then went on to demonstrate what he described as “the entire VMware SDDC software stack running on AWS — ESXi, VSAN for storage, NSX for networking, and of course, all the enterprise capabilities of those products, whether it’s NSX microsegmentation or VSAN flash storage.”

In a VMware company blog post Wednesday, senior staff architect Frank Denneman noted, “I believe one of the strengths of VMware Cloud on AWS service is that it allows administrators, operation teams, and architects to use their existing skill set and tools to consume AWS infrastructure.  You can move workloads to the cloud without having to re-platform them in any way, no conversion of virtual machines, no repackaging and very important no extensive testing, you just migrate the VM.”

As we saw in Las Vegas, VMware is adapting its existing tools so that administrators will be able to treat both VM-based and container-based (using VIC) workloads as equal partners. So the virtue that Denneman pointed out would be extended here as well. Such an extension would enable vSphere’s existing Distributed Resource Scheduler (DRS) to serve as a container scheduling agent in these systems, forestalling the need to use Kubernetes, Apache Mesos, or Docker Swarm.

Odd Man Out

In January 2015, Docker SVP Scott Johnston suggested that the widespread use of containers would eventually lead to a single, unified toolset for all operational workloads. With the AWS, VMware integration, that prediction may yet come true, but not quite the way Johnston might have preferred.


As Denneman notes, the key value proposition for VMware Cloud on AWS is that it leaves the administrative tool set alone. That’s also the value proposition for VIC — while it adopts Docker in the development phase, it stops Docker cold at the gateway to its administrative suite. It also provides a legitimate — maybe even preferable — alternative to Amazon’s own EC2 Container Service that elevates Amazon’s role rather than subjugating it, and that also doesn’t exclude Docker from the development suite where it’s already pervasive.

VMware’s counter to the threat of containerization to is to use its stronghold in virtual infrastructure: accepting and absorbing containers, while laying the foundations for a network infrastructure that takes over control of containers. And with the help of a new corporate parent named Michael Dell, VMware’s in the perfect position now to pull this off.

Docker is a sponsor of The New Stack.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: The New Stack, Docker.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.