Celebrating 20+ Years of Domain Driven Design (DDD) and EIP
Twenty years is a long time in anyone’s life, but for an information technology book it is pretty much an entire lifetime.
Unless, of course, you are talking about Domain Driven Design (DDD) by Eric Evans, and Enterprise Integration Patterns (EIP) by Gregor Hohpe and Bobby Woolf, two seminal books that recently celebrated their 20th birthdays.
According to Microsoft: Domain-Driven Design (DDD) is a collection of principles and patterns that help developers craft elegant object systems. Properly applied it can lead to software abstractions called domain models. These models encapsulate complex business logic, closing the gap between business reality and code.”
Meanwhile, Enterprise Integration Patterns provides a consistent vocabulary and visual notation framework to describe large-scale integration solutions across many technologies, an Amazon blurb about the book said. It provides a catalog of 65 patterns, with real-world solutions to help engineers design effective messaging solutions for their enterprise.
Yet, while much has changed, the design patterns, principles, and practices in these classic books are still very relevant 20 years later — in some ways more than ever.
A review for DDD on Amazon, Dec. 31, 2022, said: “Great book for creating APIs. After reading this book, I felt head and shoulders above my peers on how to create APIs.”
Similarly, a recent review for EIP on Amazon said: “It’s 2023 and the book is STILL relevant…because this book isn’t about any particular technology, it’s about explaining patterns that you can reach for again and again.”
Interestingly, both books understand that component design and integration go hand-in-hand. For example, DDD’s Aggregate pattern warns of the perils of shared database locking schemes for distributed transactions.
“DDD and EIP are counterparts that fit well together. EIP focuses on helping multiple applications work together as one whereas DDD shows how to model domain functionality within an application,” Woolf told The New Stack. “It’s like the lines and the boxes — your architecture needs both” added Hohpe.
One clear reason DDD and EIP remain so relevant is that they are not primarily books about technology. They are foundational books, focusing on concepts and trade-offs, while using technology as examples rather than as the main subject.
The list of patterns formalized in EIP is the foundation for several successful commercial integration products, such as Mule, Camel, and WSO2 Micro Integrator. Each of these products offers domain-specific language templates for the patterns, turning the design patterns into an executable domain language.
Both DDD and EIP have been proven over the past two decades to simplify the design and development process for complex systems and have recently been shown to be very applicable to microservices patterns.
A lot of pixels have been spilled in support of the monolith versus microservices debates. One thing it seems everyone can agree on however is the complexity of microservices. This complexity breaks down into two dimensions: the inherent complexity of the business domain and the technical complexity of building and operating a distributed system.
As Martin Fowler says in his introduction to DDD, in reference to the common complaint about the complexity of software systems, “…the heart of this complexity is the essential intricacy of the problem domain itself.”
In other words, automating business functions is complex because the business functions themselves are complex. This remains true regardless of the technology used to implement the functions.
The challenges of designing and implementing complex systems have not gone away in the past 20 years. If anything, they may have become more pronounced because of the increased popularity of distributed computing and networked integration, and the continued disruption of traditional business models using web and mobile apps in the cloud.
The DDD and EIP Authors Talk about What’s Changed
“The state of the art of DDD is definitely better than it was twenty years ago,” said Eric Evans.
“I’d say the biggest changes have been a result of the massive change in systems architecture over that time. Now we can often make smaller, more specialized models bounded in microservices,” he added.
Moreover, Hohpe noted that “Many of the fundamental challenges of building distributed systems, such as asynchrony, idempotency, eventual consistency, remain.”
Yet, what changed is that these days almost any system that is built, especially on top of cloud platforms, is distributed, Hohpe said.
“EIP’s introduction starts with ‘Interesting applications rarely live in isolation.’ You could probably scratch ‘interesting’ now – these days hardly any application lives in isolation,” he added.
One thing that’s clearly happened over the past 20 years is that DDD has been adopted as the default approach for reducing the complexity of microservices design, and EIP for reducing the complexity of implementing those designs (especially for events or “messages” as Hohpe insists on calling them).
“The ubiquity of messaging in the architecture makes it much more natural to model with events,” observed Evans. “The need to define what the messages mean increases the visibility of language in the system. In these and other ways, the current technology platform supports DDD better than the one that dominated 20 years ago. It is still very hard! I think it always will be. But it seems like there has been a little reduction of the accidental complexity,” he added.
Meanwhile, Woolf explained that what has become easier in 20 years is the ubiquity of integrating applications with HTTP and REST.
“We originally wrote the book for queued messaging systems, but integration with unqueued transports has become more sophisticated,” he added.
Some Examples of Using DDD and EIP
Let’s say, for example, that you are redesigning your monolithic application into cloud native microservices for handling seasonal workloads, just as we did for Citi’s payment processing systems in 2013.
First, you would identify your business domains, identify the microservices for those domains, and determine their granularity and where to draw the interface boundaries, using the language of DDD.
Then you would design the connections among them using EIP, taking into account coupling and operational aspects such as latency or behavior under high load, and deciding when to use request/response and when to use asynchronous messaging.
At last year’s HPTS conference, Colin Breck a software engineer at Tesla, described their implementation of the event-driven microservices pattern. (He didn’t post his slides but gave the link to a related video.)
In the Tesla application, some of the events are triggered by automotive telemetry and passed using messages to microservices that store and retrieve them for analysis, such as finding the nearest open charging station for a car generating a low battery event.
In another example of the event-driven pattern, Walmart replaced an RPC-based system for inventory inquiry with a system that propagated events to a queryable common data store whenever there was a change in available stock for a particular item — as described by Randy Shoup at the same conference.
What’s Next for DDD and EIP?
Neither book has so far required an update, although the authors have added reference material and updated implementation examples on their websites (for DDD see Domain Language and for EIP see Gregor’s Ramblings and The Architect Elevator).
However, “There have been some important changes in how we model domains. Most obviously, domain events as first-class members of the domain model. Functional programming becoming mainstream has also helped, by loosening up the way people think about models,” said Evans.
Indeed, there are preconceptions about what a model should look like or be about. And Evans’ book was meant to provide illustrations or examples of the principles, rather than the thing itself, he said.
Such preconceptions may short-circuit thinking.
Hohpe agrees that there has been a boost in interest in DDD (and to some extent EIP), but said he is disappointed when folks regurgitate the terms without any additional insight.
“You end up with dozens of blog posts on ‘when building an [Event-Driven Architecture] EDA/microservices [solution], define a Ubiquitous Language,’” he said.
“That’s not wrong, but it’s a bit like saying ‘To win the race, you should drive fast.’ Running the risk of becoming the ‘old grumpy person,’ I’d like folks to build onto these concepts and evolve them. Luckily, we also have nice examples like Vlad Khononov’s DDD book or EIP icons being integrated into open source products and tooling.” Hohpe added.
In addition to the lessons about using the concepts and principles with appropriate care, it can be important to trace the evolution of modern systems in the context of the DDD and EIP patterns.
“I think a service mesh like Netflix OSS and Istio owes a lot of its capabilities to the enterprise integration philosophy that what a requestor expects and what a provider performs can be very different. Still, they can work together successfully with enough mediation,” said Woolf.
Moreover, APIs have become a first-class programming artifact, whereas the service-oriented architecture was still a new thing when Woolf co-wrote EIP, and its focus on service interfaces hadn’t yet taken hold, he said.
Yet, “Now, developers focus less on individual patterns like Request-Reply, Command, and Document, and more on services with APIs taking for granted that a requestor will use Request-Reply to invoke each service task,” he added.
Advice for Readers of DDD and EIP
To really make domain modeling pay, you have to be broad-minded, Evans said. When on a project, Evans noted that he thinks about where the context boundaries are, but not about what entities or value objects they should have. “I think: What is the trickiest, most confusing part of what we are doing? That is the part we should be trying to model,” he said.
“The crux is, to find that important and excruciating part of the domain. Get concrete examples that show why it’s hard. Experiment with the language until you can clearly and concisely talk about those examples,” Evans said. “That is the start of our language and a first sketch of our model. These models can be radically different from the examples (and patterns) in my book. These are the fun ones for me.”
Hohpe said he believes that if you would ask any developer if an event is a message (EIP clearly says so) and you’ll hear long tirades mixing asynchrony, message brokers, and coupling but without giving any clear answer.
“Semantic Diffusion has become the industry norm. I’d like us to be more precise with our language about distributed systems,” he said.
Meanwhile, diagram notation has caught on and become commonly used to document integration solutions. But the diagram is not the solution; software architecture diagrams are a representation.
“In jumping directly from an EIP-style diagram to a solution, developers might forget about the richness of the patterns that the diagram represents,” Woolf said.
Times have changed. Monolithic scale-up designs assumed that networks are unreliable and that you could only rely on what was on the same server. Now networks are more reliable and faster than ever. Assumptions of cloud native computing practices include redundancy via replication and duplication that assume networks are involved, unlike some traditional monolithic systems.
Microservices are the dominant architectural approach for cloud native computing, and this approach assumes the network will (almost always) be there.
Along with that, RESTful and other APIs (including asynchronous APIs) have become the dominant implementation style. I would go so far as to say that the web has infiltrated the enterprise — it’s no longer something only for constructing bridges between enterprises and their customers, but within the enterprise as well.
But through all of this, the complexity of the business domain hasn’t changed. If anything, more of the business domain is in scope for IT than ever, due to increasing digitization.
The principles and concepts in the seminal DDD and EIP books are as helpful as ever — if not more so — in tackling that complexity, from design to implementation.
Eric Newcomer is CTO and Principal Analyst at Intellyx, LLC. He is co-author of “Principles of Transaction Processing,” the second edition of which was published in 2009.