Hype-Driven Development, from Frameworks to Microservices
Last week Marek Kirejczyk‘s essay on Medium about the dangers of hype driven development, which he described as “how teams bring doom on themselves,” triggered a massive online discussion about what’s driving today’s technology decisions.
In a nutshell, hype driven development (HDD for short) has been a long-standing buzzword — or anti-buzzword? — that’s been humming around for years, but Kirejczyk — the VP of Engineering at a small dev shop — tried to capture it with a definitive essay.
HDD happens when a development team picks the “newest, hottest technology” to use on a project, based on how the technology is trending on Twitter, or on a number of enthusiastic blog posts or conference talks. A key symptom of HDD is that contrary to the hype, progress doesn’t speed up when the new tech is developed but just the opposite: work slows to a crawl.
Kirejczyk warned that HDD comes in many flavors: Stack Overflow-driven development, conference-driven development, and of course, loudest guy-driven development. “When one guy is talking all the time about this new framework/lib/tech’s that he has no experience with, but talks about it all the time and finally the team decides to use it,” he wrote.
HDD can happen in any project, from new JavaScript frameworks to new trendy combinations like Elixir and Phoenix (“or put your favorite language/framework pair here”).
He even devotes a whole case study to microservices. This particular scenario, according to Kirejczyk, begins with IT managers suspiciously eyeing their whole application, suddenly convinced that it’s plagued with a “monolithic” architecture that’s riddled with spaghetti code. This scenario moves from a “Let’s rewrite everything as microservices” to an outcome of “It’s now way slower to develop the app, difficult to deploy, and we spend a lot of time tracking bugs across multiple systems.”
Ultimately Kirejczyk concedes that microservices “with the right investment might pay off as a way to scale the system and team,” but also that “Microservices are extracted not written. You must be this tall to use microservices.”
Kirejczyk’s bubble-bursting message drew lots of reactions from developers around the web, both positive and negative.
At least one product developer took his criticism even further, complaining in a comment that in the end even extracting microservices “usually leads to a distributed monolith, not a service architecture.” and calling it evidence to support the theory that microservices “are merely hype, and not a well-considered and legitimate pursuit.
“The same can be said of frameworks.”
After going on to disparage the entire Rails community, this product developer ultimately decided that there’s really no hope for anyone. “Having a deep enough grasp of the fundamentals helps us steer clear of mere hype, but even the pursuit of ‘no hype’ can be riddled with and under the influence of unrealized hype.”
With dozens of professional developers weighing in, the discussion turned into a very productive one. An older developer pointed out that the ultimate question is how do we know we’re choosing wisely when it comes to emerging technologies? “Currently I feel that the only way is to run a small project with the new technology to the very end — i.e. deliver it to production. And be ready to admit that the decision was wrong.”
Another commenter suggested the answer might be the Architectural Tradeoff Analysis Method from Carnegie Mellon’s Software Engineering Institute (which comes with its own conceptual flow diagram).
Developer Bensan George even linked to his own essay listing the key questions to ask when adding new tech to your infrastructure. (“If at the end of the day, you can still find the benefits outweighing the cost, then pat yourself on the back for putting your idea through some form of rigor.”)
Stéphane Colson, the editor of Linagora Engineering, also warned about cognitive biases based on a technology’s novelty or a perceived authoritativeness, but also about our countervailing bias towards the status quo (and suggested his own essay, “Managing biases as a tester,” for further reading)
Meanwhile, in a discussion on Reddit, someone pointed to another essay from Jason Kester, the head of a small development company, titled simply “Happiness is a boring stack.”
In the comments on Medium, the Associate Project Manager at Infobeans Systems India also shared a theory that hype-driven development wasn’t necessarily originating with developers. “Sometimes clients and their managers who sometimes comes up with buzz words…just to prove that they know technology. This brings the dev teams in a very tight spot and they have to just jump on the ‘Hype-Wagon’ so they appear cool in front of everyone.”
Another commenter saw a world where brilliant developers quickly master any new technology faster than mere mortals — creating its own special chaos. “[T]he brilliant developers bring in new technologies and then they go on to the next big project leaving the solid but not brilliant developers holding the bag.”
Kirejczyk’s original essay had recommended “Test and research before you decide,” making decisions based on actual experiences and calculating the potential time-savings. But to this end, it had also argued for the hiring of experienced people, which led one commenter to suggest one more important rule: “No software religion. People who want to use one tool for everything are a plague on software teams.”
Another developer also urged a more responsible approach, adding “when someone actually focuses more on the tools as a programmer than on the solutions to real problems, that’s a sort of a professional narcissism…”
Developer Anthony Comito complained, “I also like how people refer to ‘new technologies’ like it’s some Minority Report-esque technological leap, but really it’s just a new JavaScript library.”
Medium also lets an essay’s author recommend comments left by readers, which offered a nice chance to hear some alternative opinions. Marek gave his approval to a comic strip titled “The Problem Is Not The Tool Itself” from a series titled CommitStrip, which shares the cautionary tale of a developer who suggested the entire team migrate to NoSQL because he’d forgotten to optimize his MySQL queries.
But the essay on Medium also drew some interesting reactions on Twitter (where at least one small software company CEO responded “guilty as charged.”) And an IBM design director was one of the readers who suggested there’s another force at work, tweeting “One motive for HypeDD is resume-padding: using current job as a coding boot camp for next job.”
When Kirejczyk’s post turned up on Reddit, another poster even argued that “purging all ‘hype’ from your development process mostly just serves your employer at the expense of your resume,” and recommended, “selectively climbing on a bandwagon now and then.”
Another Redditor seemed to agree that just because you keep up-to-date on new technologies doesn’t make you a trend-chasing hipster. “There are a lot of slow shifts happening and not following them will make you obsolete. For example microservices, containers and container orchestrators aren’t going away.”
Another shared their own personal strategy of seeing if the hype lasts more than 18 months: “I understand not jumping on bandwagons, but sometimes the bandwagon is great because it has music and money and booze.”
One of the livelier posts on Reddit even seemed to suggest that HDD was the spice of life. “The bottom line is hype is subjective, you have to use your brain to know when it will help the resume and when it’s just goofing off because you can. You’ll absolutely never cleanse the world of hype or snake-oil or pseudo-science and I for one would rather you didn’t make the world quite ‘perfect’ because without a little variety and fun, what would we have?”
Back on Medium, Kirejczyk’s essay drew over 1,400 “likes” — but there were also commenters who disagreed with the essay’s entire premise. One senior Java developer pushed back, writing that “I’ve been in projects with microservices and NoSQL being great choices.”
Zachary Friedman, a lead software engineer, also commented that while he liked the article overall, “I really don’t think you’ve done a good enough job convincing me that this is categorically a negative thing… what tends to scare me more is badly coupled code, poorly defined interfaces, and wasted time reinventing the wheel.”