The Real Docker Ecosystem Launches with Plugins
The “batteries included but removable” philosophy around which Docker Inc. says it based its development strategy, might not mean very much if the batteries that are included are the “preferred” variety. You could remove them, but why?
When Docker Inc. purchased SocketPlane last February, it said its intent was to instill the company with the talent necessary for it to develop an API around networking. That worried many in the community that Weaveworks — among the most visible contributors to containers’ networking capability — would be relegated to “also-ran” status, or as a provider of “substitute batteries,” if you will.
Monday at DockerCon 2015, Weaveworks CEO Alexis Richardson took the stage with Flocker maker ClusterHQ’s CTO Luke Marsden and Glider Labs principal Jeff Lindsay, in the first public demonstration of the most recent iteration of Docker containers with support for a new and more “composable” form of plugins. This new iteration is the culmination of a project begun at ClusterHQ called PowerStrip, with the goal of enabling extensibility specifically so that containers could retain their states once moved.
“Composable”
Left to Right: Jeff Lindsay, Luke Marsden and Alexis Richardson.
“We’ve been working closely with Docker on delivering the Docker Open Extensible Platform,” said Marsden to a packed exhibit hall during Monday’s sessions. “We’ve been doing that because we believe that customers want choice, combined with consistent Docker user experience.”
In practice, explained Glider Labs’ Lindsay, every application presents the world with a different profile. Although storage and networking are common domains for extensibility, there are tremendous arrays of options for both, each of which may have its own complications. Because extensibility requirements can be so particular, it’s possible that no single vocabulary of API functions may be able to satisfy all foreseeable vendors’ solutions.
Left to Right: Luke Marsden, Jeff Lindsay and Alexis Richardson.
“Plugins provide users with choice, freedom, and the implementation of underlying storage drivers and other things in Docker,” said Lindsay. “So far, vendors have been able to wrap the API, to sort of integrate with Docker. But that actually is not a very ‘composable’ system. You couldn’t use Flocker and Weave together, and there just wasn’t a good way to really extend Docker.”
Put another way, Docker wasn’t something that could be “built out,” to borrow a phrase from the hardware world that means “customize-ability.”
Flocker Woven with Weave
In a demo Monday, the developers showed how Flocker and Weave could be utilized together, truly for the first time, in a scenario that container managers may soon face in the order of everyday management. A container providing a web site is moved from one location to another. Along the way, it would be ideal if it were to maintain its existing state, as well as maintain links to the database which had already been bound to its existing session.
Flocker provides what ClusterHQ calls ‘container data management,’ while Weave provides a container-oriented form of SDN. Both would have to work seamlessly in order for the transfer to work. Although technically it may be inaccurate to say so, what appeared to happen from the web site user’s perspective was a form of live migration — a method that VMware virtual machines undergo regularly, and which is a hallmark of truly fluid cloud platforms. In any event, during the demo, the web site container moved from one IP address to another (Weave) while retaining its links to its state and its underlying database (Flocker).
“Here’s another piece of the puzzle to get enterprise apps into production,” said Weaveworks’ Richardson: “You’ve got an app, you’ve containerized it, and now you need it to be available, reliable, maintainable, you need to update it. You will need to manage state. What we’re showing you today is one solution to that.” He conceded that these new plugins are presently experimental, but nevertheless functional.
“You can take your pick of the type of technology you’re going to use to solve this problem,” said Richardson. “You can pick and choose based on specific properties, that will lead to the right solution for your enterprise app. And this ability to have a management of state, but remaining within the simple Docker [context], is an incredible step forward in terms of achieving goals that customers have been asking for, for some time.”
The Dichotomy Dies Down
It was a public acknowledgement among Docker’s most well-known supporters that a purely stateless architecture may not only be infeasible, but could actually be a bad idea. Luke Marsden only drove that point home, during an interview later Monday with The New Stack.
“I think that the stateful versus stateless dichotomy was never really an argument for me, never really a contentious issue for me,” Marsden told us. “There was a debate over whether the application components should be stateful or stateless.”
One example he offered was WordPress, which mixes state into its app and has been criticized for having done so. “No disrespect to the WordPress folks, but that style of application design has largely gone away with the 12-factor manifesto from Heroku. But every application has data at its core. There will always be a database, key/value stores, caching layers, that need to be deployed as part of applications. So the transitions from so-called ‘stateful architectures’ to ‘stateless architectures’ are just ones where the application tier becomes stateless, but not where apps don’t have databases in them any more.
“Pretty much every meaningful app has a database or NoSQL or a key/value store in it,” continued Marsdan, “especially with microservices, if you listen to Adrian Cockcroft, because you can’t upgrade them individually because they share database schemas, and all sorts of nastiness. There’s actually a proliferation of different data services in modern microservices architectures. It’s just that the application tier is stateless, and scales more easily as a result. But that doesn’t mean there isn’t a place for stateful containers, because the promise of Docker is being able to consistently deploy the same application, including its databases, on your laptop with exactly the same versions of your PostgreSQL or MySQL or MongoDB, and then consistently deploy that application and stage it in production.”
The move from testing to production was a consistent theme throughout many of Monday’s presentations, and also the topic of discussion outside those sessions. There appeared to be measurable relief among many participants Monday that purely stateless applications was no longer being presented as an ideal state for 5, 10, or 20 years into the future. Maybe statelessness has its place in microservices architecture, but perhaps that’s not everyplace.
“When Heroku originally came up with the idea that applications should be stateless,” remarked Marsden during his interview with us, “they did themselves a great favor by avoiding having to solve the hard problems of managing stateful containers, by convincing developers that they should not put state in their app, and by creating an ecosystem of externalized data services that makes sense in the Heroku world. In a completely public PaaS, you can have these specialist companies that get really good at managing Mongo on AWS, and because it’s AWS it’s in the same region, so the latencies are low enough that it works for the app to just connect to this public thing. But the great thing that Docker has done is make applications portable across different environments.
“If you really want to be able to do that, you need to be able to capture the entire app,” he continued, “because you can’t seriously conceive of running an application in a private data center — which many enterprises will continue to want to do for a long time — and have it access a database-as-a-service on Amazon. That probably doesn’t make sense, a) because of latency, and b) because why do they want to run their stuff on-premises? Probably so they can keep their data local to them within some region. I think if Docker is going to fulfill that promise of portability, and a consistent developer experience across different environments, then tackling state is essential.”
More of our interview with Luke Marsden, along with his boss, ClusterHQ CEO Mark Davis, later in The New Stack.
Docker and Weaveworks are sponsors of The New Stack.
