Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
DevOps / Microservices / Tech Life

When Will the Microservices Hangover Hit?

In 2021, businesses may be waking up to a microservices hangover. Complicated and redundant implementations, increased overheads, and more.
May 25th, 2021 11:00am by
Featued image for: When Will the Microservices Hangover Hit?
Feature image via Pixabay.

Nočnica Fee
Nica helps teams adopt serverless and optimize their costs on AWS. She is a Serverless Developer Advocate for New Relic.

The microservices movement was a boon for every company that needed to “digitally transform” overnight, after everything that happened in 2020. Microservices can allow IT to move faster and be more flexible. Code is for the most part easier to understand because it’s restricted to one business function. Also, at a time when Kubernetes experts are harder and harder to come by, microservices are an easy choice at a time when companies need to move fast and not break too much.

However, in 2021, many businesses may be waking up to a microservices hangover. Complicated, redundant implementations; overall increased overheads; lack of governance. These issues are causing technology teams to take a hard look at the promises of microservices.

A Double-Edged Microsword

The beauty (and ugliness) of microservice architecture is that you can have components written in twelve programming languages. To put it another way, developers like to brag about how they saved time and resources by rewriting a service in Go or simplified some new features with a cool Python ORM [object–relational mapping]. But then you wake up a few quarters later and your stack is written in 12 languages, and it’s not clear that you even have developers who know those languages anymore.

Onboarding with microservices is much faster. That’s good, right? Yes, but it also means the person committing changes in their first month has zero chance of understanding the system as a whole. Microservice adherents would say that’s a feature, not a bug. Now, go into any successful microservices shop and ask anyone — even a senior dev — to draw the architecture. You’ll either get a bewildered look or impressionistic spaghetti, and that can be a real problem for any company with plans to scale.

Microservices also talk a lot about “speed” and immediacy, but those terms get conflated. Sure it’s fast when it comes to getting into production, but that doesn’t necessarily mean speed of performance. The microservices architectures that works for a company selling pet toys may not support a business that depends on streaming 4K video. And as consumer expectations grow, even technologically mundane brands are going to have to start thinking more about how quickly and reliably they can serve their customers.

Is the Answer a Monolithic Architecture?

Well, not necessarily. A good example of how monolithic cultures fail is found in the concept of “holacracy,” a management style attempted (and mostly discarded) by a few major Silicon Valley companies. It’s the organizational version of a monolith: equal and standardized units that all theoretically work together as a whole, where anyone can make requests of anyone without any filters or middlemen. In practice, of course, problems abound.

That’s why monoliths, which are praised for their interoperability, can actually become productivity nightmares. In software, this can feel constraining — we’re asking devs to screw in screws, but only giving them a hammer. At the same time, it’s hard to strike a balance with governance: if we need three pre-approvals for every new idea, we stifle innovation.

Culture Is the Real Glue

The pandemic has given me lots of time to reflect on the culture of our dev team. Ultimately, successful software approaches — whether microservices, monolithic, or somewhere in between — are characterized by a consistent approach that reflects the development culture.

That’s right, MBAs: dev teams have culture too. Execs need to know, love and foster these cultures, and make them work with the organization’s overall development approach. Starting with the dev teams, see what technical culture they like and what’s been successful. Is it cowboy and cavalier, where devs fly solo and make changes? Is there a loose framework, within which people can innovate? Or do we work best with strong technical cohesion and inviolable processes?

The right approach is not about monolith vs. microservices — rather, it’s predicated on thoughtful and intentional analysis of ITOps and your business. Microservices can move fast and break things. A monolithic architecture comes with stability, predictability and better visibility. How to win? Recognize and guide your dev culture, and try to meet in the middle as you draw up the bigger picture.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: Pragma, Real.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.