Cell-Based Architecture: A New Decentralized Approach for Cloud Native Patterns
This is the first article of a three-part series outlining the new cell-based architecture, which was published as an open specification on GitHub last summer. Our approach creates a pragmatic and technology-neutral reference architecture that addresses the requirement for agility. It can be instantiated to create an effective and agile method for digital enterprises, deployed in private, public or hybrid cloud environments.
When I present the new architecture at technology events, one common question is the reason WSO2 is defining a new reference architecture into an already crowded market. This article lists the motivating factors.
As a technology provider, WSO2 has been involved in the integration space for more than a decade, helping over 500 customers implement various digital-driven solutions. Paul Fremantle, CTO and co-founder of WSO2 (and the co-author of cell-based architecture), and I have been deeply involved in most of the implementations. Paul and I also have more than 20 years of experience in the distributed computing space. The above factors led us to discover the mismatch between existing architecture patterns and what enterprises truly require.
Layered and Centralized Architecture
Over the past two decades, we have used layered and centralized architecture patterns, which have resulted in creating many centers of excellence (CoE) groups and technology stacks. These layers have created gates, which, in turn, have slowed the rapid movement and flexibility that agile teams require for iterative development and continuous delivery. Owing to this, even organizations that practice agile methodologies are only partly agile (agile or fast waterfall). Our approach aims to minimize centralized dependencies and move integration into the agile development pipeline, making organizations more responsive to business needs. Based on the results from the most recent “State of Agile Report,” research only four percent of companies benefiting from the actual result of agility actually spend time and money on practicing agile methodologies.
Brownfield and Greenfield
With the introduction of microservices architectures (MSA) and cloud native infrastructures, architects started designing architectures entirely as greenfield (I would call these PowerPoint architectures). However, the reality is that many enterprises need brownfield systems and data, which are required to run the business. As existing reference architectures address either greenfield or brownfield, we realized that enterprises require a hybrid architecture approach that can reuse existing systems and data as well as build new systems in greenfield.
Reference Implementations
We conducted precise research before defining the new architecture and, during that exercise, we identified that most existing reference architectures were reference implementations. The rationale behind claiming those as reference implementations were owing to defining implementations based on a vendor-specific technology. We wanted to make the new reference architecture ultimately based on technology and be vendor-neutral. That way, an architect could refer to the defined reference architecture and build solutions based on the desired technologies that already exist in the enterprise or plan to introduce them in the future.
Architecture and Technology Agility
Existing reference architectures are also limited in architectural agility in that they block adopting new or comparable technologies for projects. As the CoEs have standardized technologies across the organization, each project team has to stick to those specific technologies even if they are not the ideal fit for the project. This underutilization of technology is a side effect of the centralized layered architecture and it led us to think about the need for a decentralized and modular architecture.
The Gap Between Architecture, Implementation and Deployment
To a certain extent, agile methodologies helped to narrow the gap between architecture, development and deployment. However, there is a disconnect between those three steps in the application lifecycle; this results in broken systems for the business. Establishing seamless connectivity in each lifecycle stage with a common architecture construct is crucial to improve agility (by having an iterative architecture and continuous delivery). Our approach combines the application and system architecture using the same architecture construct and this naturally connects the architect, developer and DevOps engineer.
Governance and Dependency Management
While microservices and containers increased the flexibility of development and deployment for applications, both patterns increased complexity by introducing many moving parts. With this in mind, we wanted to implement a light governance framework with observability, which would enable dependency management, impact analysis and a monitoring mold into the architecture.
We started the project by conducting thorough research on the market. Scholarly papers published by various research institutes, books and online content from other vendors were some of the sources for the content. This research validated that our motivation factors were real.
We narrowed down our research into four areas — quantum computing, Kubernetes, microbiology and systems biology — to find an entry point for the project. Quantum computing and Kubernetes led us to think more about decentralized architectures and the market’s movement towards the same. Microbiology and systems biology gave us insight into how independent and self-operated units (cells) connect and build numerous extensive systems. Also, there were other exciting characteristics of biological cells mapped to the architecture construct we were looking at, such as how the membrane of a cell provides a clear cell boundary and protects the cell’s internals, as well as creating a single communication channel.
After the preliminary research work, we looked at the reality for the enterprise: mainly the disconnectivity of business expectations and the technical definitions. We concluded that regardless of the service development style (SOA or MSA), organizations seek solutions for business problems from services, which essentially require to connect multiple technical services and expose a business savvy API. As a result, (microservices are required to connect using a composite service or an API gateway.
Conclusion
In summary, our motivation factors behind introducing this architecture pattern were based on teething issues faced by architects while moving to the microservices and cloud native approach, which requires connecting with a large number of monolithic applications that already reside in the enterprise. In the next article of this series, I will give an overview of the cell-based architecture.
