Docker, Density, Data Gravity and the New Complexity of SDN

Apologetic for not having his “usual set-up,” co-host Michael Coté valiantly participates in this installment of The New Stack Analysts, an edition for which host and The New Stack Founder Alex Williams has chosen to discuss a topic that’s been covered here extensively – networking.

For more episodes, check out the podcast section of The New Stack.

#29: Docker, Density, Data Gravity and the New Complexity of SDN

At the time of recording this episode, Alex was still researching his January 29, 2015 in-depth post, “SDN, Docker and the Real Changes Ahead,” and he engages Michael in quite a bit of thinking out on the topic of SDN.

Referring to a John Willis presentation that he captured at DockerCon Europe, Alex summarizes the transition from traditional, centralized software-defined networking to the current trend toward decentralized, distributed SDN, and says that according to Willis there are two major factors affecting the way we think about networking.

One issue is density: “the amount of virtual machines on any physical server, and how heavy they are because they carry that operating system.” Docker containers being more light-weight, “you may have the possibilty of thousands of containers on a server,” says Alex.

“Secondly,” he continues, “data gravity comes into play because we have lots of services and apps that might be inside of these containers,” requiring more thought toward management and orchestration.

“You start to think about the developments over the past several years, and one of the things John cites is this whole idea of how compute resources are allocated. It used to be that the life-span of compute might be hours, or even months, and then it went down to minutes.”

“It could go to milliseconds,” says Alex, noting AWS Lambda’s intention to provide instant compute capability. “That sets up an interesting conundrum in the networking space.”

With all of these containers with all of this data gravity, how does one obtain the compute to manage it sufficiently and best optimize it? One proposed solution is to “swarm” compute resources from place to place, but this raises further issues with design patterns.

“All of these concerns are concerns of distributed applications,” says Michael. “A distributed application basically means you’re running on more than one server. It’s a lot more complicated than that, but you’re splitting up your application over all different kinds of compute, in different ways, and you’re also trying to take advantage of the fungibility, or people would say ‘elasticity’ or ‘versitability,’ of that. Now we have cloud apps so we’ve got some more triangulation on what’s going on out there.”

Alex notes the trend toward automation in Willis’ discussion of the capability to do segmentation and provide services in the data plane itself.

“If we didn’t have automation we wouldn’t have cloud stuff,” Michael says. “Much of how we experience computers nowadays would not be cool if we didn’t have automation. There’s this notion of ‘batch job’ where every twenty-four hours you can do something. Can you imagine how relaxing it would be if Facebook was just a daily email you received? That would be nice. But we’ve chosen not to live that way. Instead, because of automation, we can go there every second and check on things,” says Michael.

“It’s great.”

Feature image [CC BY-SA 3.0] via Wikipedia.