Debunking Three Myths About Event-Driven Architecture
Event-driven architecture enables building highly available and reliable applications that can scale to handle traffic spikes. However, there are some misconceptions around its use. This article discusses some of these misconceptions and debunks them with some practical examples.
Myth #1: Difficult to Manage
Event-driven applications are designed around event brokers, which connect event producers and consumers. However, having to include an event broker has created an impression that it increases the complexity of the deployment and raises management and monitoring overhead.
When cloud computing rose in popularity, IT teams started to abandon on-premises event brokers in favor of cloud-hosted, managed messaging services such as AWS SQS/SNS, Azure Event Grid, Event Hubs and Google PubSub. These services eliminated the burden of managing and scaling event brokers.
With the rise of serverless computing, this reached another level. Organizations moved their entire event-driven applications into cloud infrastructure-as-a-service (IaaS) platforms, to save money and lessen management complexity. Then functions-as-a-service (FaaS) platforms, such as AWS Lambda and Google Cloud Run, seamlessly integrated with existing messaging services to provide an improved developer experience. This further motivated developers to move their event production and consumption logic into cloud platforms.
Today, there are many organizations using cloud infrastructure to build and deploy their event-driven applications. Lower management complexity, reduced total cost of ownership (TCO) and improved developer experiences are key factors that have contributed to organizations making the switch.
So, we can no longer say that event-driven applications are complex to manage.
Figure 1: Cloud native programming languages like Ballerina use serverless technologies to build event-driven applications. (Source: Imesh Gunaratne)
Myth #2: Hard to Understand
Event-driven applications are often criticized for being hard to understand when it comes to execution flow. Their asynchronous and loosely coupled nature made it difficult to trace the control flow of an application. For example, an event producer does not know where the events it is producing will end up. Similarly, the event consumer has no idea who produced the event. Without the right documentation, it is hard to understand the architecture as a whole.
Standards like AsyncAPI and CloudEvents help document event-driven applications in terms of listing exposed asynchronous operations with the structure of messages they produce or consume and the event brokers they are associated with. The AsyncAPI specification produces machine-readable documentation for event-driven APIs, just as the Open API Specification does for REST-based APIs. It documents event producers and consumers of an application, along with the events they exchange. This provides a single source of truth for the application in terms of control flow. Apart from that, the specification can be used to generate the implementation code and the validation logic.
Figure 2: A sample AsyncAPI specification document
CloudEvents is a specification that standardizes how event publishers describe their events. It introduces a common set of attributes to wrap events that are exchanged between disparate event-driven systems. This allows developers to reuse the logic and tooling required to read and write events across multiple cloud platforms.
Standards like the above make it easier to build portable and interoperable event-driven applications in the future so they will no longer be difficult to understand.
Myth #3: Difficult to Test and Debug
Event-driven applications are often made of distributed, loosely coupled components that communicate over the network using asynchronous events. Owing to the unreliable nature of the network, these events might get lost before reaching the destination. In such situations, users often tend to blame the lack of flexibility in tracing the event path to figure out what went wrong. Also, the asynchronous nature of these systems makes it hard to debug the control flow of applications.
However, with the help of rich observability tools, this can be eliminated. Distributed tracing tools like Zipkin and Jaeger enable developers to trace the path of an event from its origin to its destination. They also provide insights on which components had intercepted the event during its journey and what mutation they caused to the event’s body. Not only that, if an event is lost during transit or delivered more than once, these scenarios can be easily tracked with these tools.
Figure 3: Tracing the path of an HTTP request with Jaeger (Source: Jaeger)
Moreover, strategies like event sourcing capture state-changing operations in a system as events into an append-only event log. Distributed commit logs, such as Apache Kafka, serve this purpose. Later, those events can be replayed from the beginning to derive the current status of the system. This allows developers to travel back in time to any point in the system and replay events to quickly reproduce erroneous states in the system. That can be done without even touching the production system, which enables rapid debugging capabilities.
Closing Thoughts
Due to their distributed and loosely coupled nature, event-driven systems are often misunderstood as complex, hard-to-understand and hard-to-test architectures. However, as the technology progresses, those misconceptions have begun to disappear.
Cloud computing and serverless platforms have absorbed the management complexity of event-driven systems. Standards like AsyncAPI and CloudEvents make systems interoperable and well documented. Distributed tracing tools and centralized logging make it simple to test and debug event-driven systems in productions. All of this helps organizations build event-driven systems better than in the past.