Building Microservices, Toiling With the Monolith and What it Takes to Get it Right
“Not all of a large system will be well designed.”
The words of Eric Evans echo throughout this piece, as we learn about how one company made a strategic design decision to move portions of their monolithic code to microservices and bounded context.
Microservices are the hot new thing … at least in theory. There still aren’t a lot of stories about them in practice, and when there are, they are more about microservices advice, and quite a few about the organizational pitfalls, which are both important topics to talk about. But what about if it went right? This piece takes us through the process of one mid-sized commerce site making the transition to microservices.
When Your Team Needs to Make a Change
Carlos Buenosvinos joined the Atrapalo team about a year and a half ago now. He says that Atrapalo, like many companies in Barcelona, were not only monolithic but had bad team practices: not performing testing and not built for scaling. The popular restaurants-activities-and-travel-deals website was experiencing high turnover, with people leaving every week or two “because of a lack of innovation” in the company.
Buenosvinos — having been through a similar process at his previous company, eMagister classified websites group — recognized the need to break away from the monolith in order to spark innovation and to be more competitive with faster delivery, which in turn would create better team unity and retention.
Following the same pattern of his work at eMagister, the Atrapalo leadership decided to use RESTful and domain-driven design in the bounded context of microservices.
“Microservices is a good tool for a tech lead, CTO, whatever, to increase productivity of larger teams and to invest in the careers of the developers because they are going to be learning about new architecture, new technologies,” he explained.
But Buenosvinos doesn’t think microservices are for everyone, and he doesn’t recommend them for smaller teams.
“You can apply microservices in small teams but in general the main goal is maximum productivity in splitting teams so they can focus on tasks.”
He continued, “If you have a small team of ten guys developing, you can make the trip to microservices, but I wouldn’t recommend it.”
To the opposite, Atrapalo’s monolithic architecture was causing more trouble than it was worth.
“For 70 developers, branches, your team has to be really well-behaved to go monolith with different teams and projects and the products all connecting in synchronous and asynchronous ways.” He said that as they broke off into microservices, “all these conflicts disappeared.”
If teams are more productive in microservices than on the monolith, Buenosvinos recommends you go for it; however, “If you can deal with some conflicts when you are merging, developing and deploying, it’s OK. It’s better to stay in the monolith than go to a distributing system.”
And this doesn’t mean that you have to go all microservices or bust. “In one scenario, some of the parts of the monolith are not worth it to move away, or on the other hand it can be productive” to make the move.
Plus, once you start to break away, that doesn’t mean you’re married to the idea. “If anything goes not as expected, you just stay in a system with a monolith, but with some microservices, and you work with separate projects.”
How to Start the Move Toward Microservices
You can’t just close your eyes, cross your arms, make a wish and Whammy! your monolith has magically broken off into microservices. The transition can be a long and drawn-out process that can be inherently painful as teams and responsibilities shift.
As with all transformations that affect your team dynamic, you need to dedicate a lot of time and patience to communicating what is the plan of action and to explain to your team that they’re not breaking up, it’s just more “I do this, you do that.” And still, not everyone’s going to like it.
At Atrapalo, it was the same. Buenosvinos said, “Most of the people were really happy about learning new things, and there were some guys who weren’t very happy about that, but they left the company.”
After that initial exodus, Atrapalo had almost no turnover in the first year of implementing microservices. He believes that’s because:
“You have to explain to the developers that the main goal of whatever we are doing here is releasing values to the users as soon as you can, pushing things to production faster with a higher quality.”
Deciding Where to Split Your Team and Your Code
Buenosvinos says you must always concentrate to “split in terms of team, not code. Just one repository and everything was deployed at the same time,” with fixes on airplane travel, hotels, etc.
He applied the Mikado Method of experimentation and learning how to reorganize:
- Set a goal: How do you envision your future?
- Experiments: To find the prerequisites.
- Visualization: Mapping out steps one and two.
- Undo: After you’ve figured out which goal or pre-req has broken your system, roll it back.
He said, to start, take a look at the team you have and who is the best to maintain it. Then ask, “What’s the easiest part to split and to train the team about breaking the monolith?”
He talked about how Amazon can have one hundred developers dedicated to reviews and checkout, but with a team of about 70, Atrapalo had to be more creative.
“You can’t have developers dedicated to a specific product,” saying that he would assign some products to some teams, while three projects can be maintained by just one. “Some dependencies in that sprint need more resources in service engineering and frontend or whatever, so we move [team members].” Then they hire contractors if they need a new skill met, or they train team members.
At Atrapalo, “We started looking at the whole Atrapalo stuff and then looked for small pieces that you can break away and give to small teams.”
For this travel site, the first thing they isolated and broke off was the users’ identity access. Next it was the checkout and catalog; followed by hotels catalog, plane and train travel; and then reviews. Atrapalo now has one API (application programming interface) for each product — three or four — as well as for each of the six microservices, each synchronously based on RESTful APIs.
Buenosvinos said that “The monolith is still there working quite well. Each of the products that are still running on the monolith, we have to take away. The end goal is when you have a website with bounded context.”
How to Decide on Your Microservices Language(s)
A strong principle of microservices that seems to have arisen is the freedom for any micro-team to use whatever language they need to get the job done. However, most of the time, companies aren’t reinventing the wheel, but rather using as much of the monolith’s language as possible.
Buenosvinos contends that when microservices are developed in the same language, it’s much easier. “It’s normal to say, ‘Hey guys, our main technology is,’ but [then use] probably two or three languages maximum.” Atrapalo works mostly with PHP, and then some Node and Java for things like streaming videos.
Other microservices pioneers like Netflix only use Java in general, with some Python and Node. Buenosvinos said, “They’ve decided this is the tech they are going to be using in 80 to 90 percent of the cases.”
Buenosvinos recommends that when you are looking for a language with which to start moving away from the monolith, or simply when you are developing a new service, you need to think about the community you are serving, the support you are offering, and is it easy to recruit. Atrapalo went with PHP because it’s easier to hire in lower salaried Barcelona than the more-complex-to-learn and thus more-expensive-to-hire Java.
“Imagine that you have people that are working on the monolith. You can take them and put them on different services,” he said, “and you can continue with the same language as the monolith.”
Is Scrum Essential for Microservices?
“People are probably the most difficult part to handle. There are always people that are not happy with changes. As my CEO told me, ‘You aren’t paid to be friends with developers’.”
However, managing the team is, of course, the most important part of the microservices transition. Buenosvinos said that is why already being a Scrum, or at least Agile company is easier before making that move.
“It’s not required but it really helps, but you could run microservices with waterfall, applying any Agile methodology releasing and testing,” He says Scrum is “the easiest approach because we are running microservices on top of our Agile structure.”
“So many people are talking about the Scrum but not applying it in hard environments … in big teams,” Buenosvinos warned. He said that both microservices and Scrum are about the details. “How the Scrum master can notice when something is becoming a problem.” He said that, particularly as you transition to microservices, you need a strong Scrum leader that can look at the team and ask the right questions:
- How many people can work on this one?
- What are our user stories?
- How will we manage releases and teamwork?
Also, the Agile methodology in place often affects how you organize the teams.“As Scrum says, small teams up to six to eight people work more effectively than a team of 12,” Buenosvinos said.
“We are trying to split the company into smaller teams to go faster, to deploy independently, to use different tech, to hire different people.”
He says that, within six months of starting to implement microservices, “The rest happens naturally — starting to deploy every week then twice per week.”
How Bounded Context Brings About Microservices Transformation
Buenosvinos argues that microservices may be a buzzword and hot hashtag, but it’s not a new concept, saying that it really goes back to around 2004, where it started out with the name bounded context, and how it interacts with domain-driven design.
He says microservices is basically the same.
“Domain-driven design is about explaining what your company does in isolated parts, in performing these specific parts, and you can have a dedicated team to do this stuff, and you can have different tools for different teams.”
Strategic design principles of domain-driven design look to maintain the distillation of models across multiple models:
- Bounded context: When you have multiple code bases, you end up with unreliability and bugginess. This gets even worse with situations like microservices where people are using different technologies and programming to do it all. The bounded context principle makes sure you put down specific boundaries for team organization, who deals with what part of the app.
- Continuous integration: Microservices help limit Conway’s Law and the fact that more people in the same bounded context will splinter off, but on the other side you want to prevent more splintering. To prevent this, frequently merge all code and automate testing to catch bugs.
- Context map: “When connections must be made between different contexts, they tend to bleed into each other.” Map the existing terrain, highlighting any sharing and clarifying any bounded context.
Buenosvinos continued that “Microservices is a cool word but bounded context is more mature in terms of discussion.” He describes the journey Atrapalo is going through as more bounded context and domain-driven design. “Domain-driven design promotes architecture that is not coupled to the framework so people can move from one framework to another.”
How do you Measure the Success of Microservices?
“It’s not cheap to go to microservices at all. It’s not easy to understand. It’s not for everyone and it’s not for all developers. You have to be on the mid-to-high skill-sets,” Buenosvinos said.
Essentially, the moral of this story is that microservices is not for everybody, but when it works, you come out the other side with faster deployment and a innovative, faster-moving team.
Atrapalo’s team of about 70 developers has been going through the transition to microservices for more than a year now. “In that time, they’ve gone from deploying once every two weeks to about three times per day.”
Of course, for Buenosvinos, success is measured differently:
“To take a big team and to get them working with best practices, it’s the thing that you can be most proud of.”
Have you tried microservices? Comment below about how your process worked (or didn’t.)
Feature image: “barcelona” by Osamu Kaneko is licensed under CC BY 2.0.
