Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
API Management / Microservices

Mashape Opens Kong, a Microservices API Gateway Built on Nginx

Apr 28th, 2015 4:00pm by
Featued image for: Mashape Opens Kong, a Microservices API Gateway Built on Nginx
Feature image via Flickr Creative Commons.

Tuesday morning, API platform provider Mashape made official its open sourcing of, and free access to, a microservices management layer called Kong. Mashape originally produced Kong internally as part of its development and continuous management platform for its API hub.

Now, as Mashape CEO Augusto Marietti told The New Stack, having been inspired by the example of Netflix, the company will freely distribute the keys to its kingdom.

The Eighth Wonder of the World

Kong is intended to help companies with monolithic environments to move faster to a microservices architecture, Marietti said. Kong automates the custom services and logic for microservices, Marietti said.

“For those who have microservices architecture, [Kong] doesn’t change the way it’s orchestrated, but it makes it lighter because it can remove a lot of common functionality from, say, seven or ten or twenty microservices and make all of them centralized on Kong … making it lighter and more maintainable.”

Kong is built atop the well-known Nginx reverse proxy server,  Marietti said. Its role in this new ecosystem is to provide services (note, not “micro”) that microservices engineers will commonly use, or should be using, in the course of everyday development. One of these services is a RESTful API interface for the enrollment of APIs themselves in a kind of registry, which Marietti cautioned me should not be called a “repository” for reasons I’ll explain in a bit.

The other service is a platform for plugins, providing a base of common functions from which microservices would frequently be borrowing. One of these plugins should sound familiar: authentication. For example, an OAuth plugin would make it possible for an API to deliver authentication both to and for microservices on the backend. This way, services being delivered over disparate platforms can be assured they’re “speaking” to the services they claim to be.

“It takes out all of the headaches with dealing with authentication protocols and systems,” said the CEO. Logging is another good example of a common service that may be exposed to microservices by plugins, through Kong.

If you think of Kong as a common storage and access mechanism for the services that microservices would use, then perhaps “repository” is a fitting term in a general sense, he noted. But Kong is not a collection of the microservices themselves, the way Docker Hub collects images of containers together into “repos.” The APIs you use are still your own, hosted on your own platforms.

Centralizing Distribution

If the ideal of microservices architecture is to succeed in the long-term, then someone will need to orchestrate it. The first thought people have about how to go about this task is to build and deploy something they can call a “management platform” for microservices.

That’s hard to accomplish when one of the guiding principles of the architecture, as articulated by continuous integration advocate Martin Fowler, is decentralized control. It’s hard to build a hub around decentralized control; just ask anyone who’s tried communism.

Still, that doesn’t stop people from trying because, as any veteran communist will tell you, you have to succumb to capitalism sometime. As another example,’s Vamp tries to be a cross-platform platform where services can be deployed on any number of cloud services, and yet co-exist as the same application.

“I think for a while, we’ve been living in a hybrid era where a lot of companies are not microservices-centric, but are not any more monolithic,” explained Mashape’s Augusto Marietti, relating tales of the days before the revolution.

In the picture Marietti draws for us of this hybrid era, major corporations have been running their businesses on ancient code bases. The way they’ve been dealing with modernization has been to build wrappers around their existing code bases, then use those wrappers to expose those bases as services to mobile services. He cited a case of a major bank whose name everyone here would recognize, whose code base is so deeply embedded in its infrastructure that, out of fear of what they might find or which ancient gods they might disturb, no one dives down deep enough into the infrastructure to uncover it.

Thus the bank’s mobile solution becomes something like a deep-water drilling rig, pulling up code like core samples.

For the next five years at least, Marietti projects, major corporations will have to continue working this way: effectively tapping down deep into their existing code base, then wrapping microservices around whatever they pull up. Certainly, no one’s building new monolithic architectures that archaeologists will have to dig up twenty years from now — or at least, if they are, they’ve declared themselves top secret and we’d require a government warrant to talk about them.

“Kong is perfect for this transition from the old world to the new world,” said Marietti. “Hopefully in the future, all the companies will look like Netflix, which is 100 percent running on a decoupled, API-centric microservices architecture.”

Version 0.2.0-2 of the Kong microservices management layer is now downloadable from GitHub, and more information about its architecture and utilization is available from

Group Created with Sketch.
TNS owner Insight Partners is an investor in: The New Stack, Docker.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.