There’s more than one way to skim a monolith. There seem to be two popular ways system designers do it — diving in head first or taking forever to chip away, by being too careless or too cautious.
In the latest Agile Roundabout meetup in London, Kobalt Music technical manager Jeremy Ryan discussed the approaches his company is trying in an effort to move to a more agile architecture, one involving the route of gradual change.
To understand the challenges facing Ryan and his team, let’s start with a brief primer on the tangled web of the finances in the music industry. Kobalt offers services around promotion and collection of music royalties. This March alone, the company processed half a billion royalty payments.
Imagine all this happening atop a full-on monolith. “The only way to cope with that amount of data is a giant database,” Ryan said.
In Kobalt’s case, this involves a heavy architecture with “a honking great Oracle Database,” a Java Swing user interface (UI) for internal usage, a web UI built with PHP for external users like music publishers, and a mobile app for the artists to track where it’s all coming from.
He said “It is a monolith but that’s not too bad, and it scales reasonably well so far. But Oracle is just that — you kind of throw money at it and it’ll scale. But that kind of rises exponentially the more you want to scale.”
However, the company has reached the point that it knows the system will scale for awhile longer, but Ryan sees a future where the company won’t be able to afford to scale it anymore, so eventually, it will need to chop that database up.
Ryan says the company isn’t looking to be a pioneer of microservices architecture, saying it would settle for just smaller services for now, with ideally, full adoption of microservices later.
Right now, the company does a weekly release of its internal software stack, with hundreds of issues addressed. If even one of the integrations fails, the release doesn’t happen. Ryan admits it’s kind of “doable” now with 80 developers, but the company is looking to scale and at least double that, which he predicts will also bring double the bugs. Microservices could definitely offer some benefits in such as scenario, like being able to release isolated pieces.
The company’s designers need to think forward, a couple years ahead, to when they will eventually scale their team past the point the Oracle Database will not work for them anymore.
When and Where to Begin
Simply put, step by step. Many companies have failed to go to microservices because they go too small too fast. Kobalt isn’t looking to repeat that mistakes. Their end goal isn’t 100 percent microservices either, but rather quite a few microservices, some services, and things they don’t think it will be able to break up, that it won’t be able to make into “neat and easily deployable services,” Ryan said.
So, when do you start? Where do you start?
For the first question, the answer is easy — it needs to start now before the Oracle database stops scaling. Ryan thinks the company has a year or two left, after having invested heavily in some more Oracle hardware. But there will come a time when the company has reached its scaling limit.
Now for the question of how to start breaking up the application into smaller services. For Kobalt, it starts with the easy stuff.
“You can look at your application and find some nice seams you can cut,” Ryan said. You may “have some things that don’t need to be live, though everything is live right now.”
Currently, everything looks to the Oracle Database in real time, but he says not everything needs to be.
For example, when you stream something on Spotify, it could be up to two years for the artist to make any money off of that listen. Typically the artist will just get a check but won’t understand where it came from.
“We’re trying to make that a bit shorter and pretty much the only way we can do that is if we can cut down the number of steps in that chain of people, using our collection society which can talk to customers all over the world,” instead of collection societies in any give country, Ryan said.
He says the web portal doesn’t need to show live data.
“If they’ve been waiting two years to see where all this info is coming from, they can wait two hours.”
And, of course, as new projects come along, they want them built untethered from the database and service-oriented from the start. And that those services are as small and task-specific as possible.
However, Ryan believes there’s bound to be things they can’t chop up into smaller services or microservices. For now, the company will start by chopping up “when things really start to bite us.”
Ryan’s strongest advice is not to try to do everything at once.
Using the bar illustration above, Ryan pointed out that if the design team tried to go from Point A all the way to Point D directly, “We would almost certainly fail or it would take an inordinately long time, probably too long.”
Instead, they are looking to chop up into slightly smaller services that they will then chop up again into even smaller services until they get as close as they can to a microservices architecture.
And since this massive database has never been confronted before, they have to be prepared to adapt. And to remember that you don’t necessarily have to have microservices, just that they are small enough.
However, an audience member pointed out that this could be an inefficient way to do it, re-architecting more than once and focusing only on short-term, quick wins.
Ryan responded that he was laying out the plan now, not that he was predicting the future.
“We know roughly where we want to go and we know vaguely how we’re going to get to it, but we are able to change that,” Ryan said.
“This is what we are open to change because we’re prepared to throw all of this [plan] away if necessary.”
But Always Work on Testing along the Way
And of course, test, test, test. Ryan contends that Kobalt has pretty good test coverage but that it takes too long — half an hour to 90 minutes — something that becomes more of a problem when you try to give more autonomy to more developers to release on their own, and when they need to know quickly if something is broken. They want to get it down to two or three minutes per test.
Microservices require continuous deployment and continuous integration, which implies continuous testing.
Sell It up the Foodchain
How can you sell this long-term investment, short-term disruption? For Kobalt, this is an easier sell.
“The easiest way to sell to upper-level management is showing the bill,” Ryan said. “Oracle is quite good but if you can say projected cost as we continue to grow: Where is the stop point and how can we avoid that?”
If you can not only save time to market but save money directly, microservices can be a compelling move for most companies. It’s not the only answer and not the perfect one. It’s about finding the right fit and transition for your team and your architecture.
Feature image via Pixabay.