Do You Need a Managed Database?
We have become used to moving applications and data center infrastructure into the cloud. Some even find the experience is as seamless and pain-free as cloud providers promise.
But when it comes to the databases at the heart of many organizations, particularly online transaction processing systems that demand sub-millisecond response and rock-solid resilience, the stakes are considerably higher. Getting it wrong is at best a crisis, at worst, an existential threat to your entire business.
So, if you’re running a real-time, NoSQL installation, are conscious that speed or scalability is becoming an increasing challenge, and have begun looking wistfully at the virtually infinite range of database services the cloud can offer, where do you start? What is it that you need to be considering to successfully make the leap?
The Cost of Managing a Cloud Database
Perhaps you first need to take a step back and contemplate whether your current setup can be moved at all. For some applications, on-premises is the only option.
This might be true of financial companies that are bound by industry regulations, said Tzach Livyatan, vice president of product at ScyllaDB, which produces an open source, distributed NoSQL platform.
Likewise, for organizations operating in particularly sensitive or high-security domains, such as national defense or cybersecurity, a move to the cloud might simply not be permissible.
But let’s assume you do have the freedom to move your database to the cloud. Don’t forget that someone is still going to have to manage it.
What does database management in the cloud encompass? Pretty much everything that managing your database on-prem involves.
“You usually don’t need to install on the cloud,” Livyatan told The New Stack. “But you do need to upgrade the database routinely, you need to operate the OS and third-party packages. Every week or every month, there is some patch release of either Linux or the database or something that you need to take care of.”
And of course, you need to keep on top of scaling, as well as the security groups, identity management and firewalls needed to protect your data.
You may well have a team of skilled people capable of carrying out all these functions, but you need to consider whether managing your database day-to-day is really the best use of those skills — whether on-prem or in the cloud.
“Often you will find that the most expensive part of running a database is the people who run it, because it’s skilled engineers,” Livyatan said. “In many cases, the team itself costs more than the database operation for a year.”
And that impacts the value you can generate from that team. In a recent TNS podcast, Keith Sader, director of engineering at adtech vendor GumGum, told how his company decided to switch to ScyllaDB’s managed service. GumGum made the move from a self-installed, self-managed Apache Cassandra setup on Amazon Web Services (AWS) because, he said, “managing Cassandra databases is not really what our business makes money at.”
Even hosting its Cassandra installation on AWS left Gum Gum with a “too many cooks in the kitchen problem.”
Mitigating the Risks of Vendor Lock-in
For all these reasons, it becomes less a question of moving to the cloud, as moving to a managed cloud database service.
But what factors do they need to weigh up when deciding between competing services? Customers generally want to run their data wherever their application is already deployed, Livyatan said, to ensure the lowest possible latency and to avoid potential data-transit costs between clouds.
This might make multicloud a less attractive option for some customers. Working with a single vendor also makes both technical and admin tasks simpler. After all, each cloud has its own toolsets and nuances, while dealing with different billing systems and rate cards just makes life unnecessarily complicated.
Regardless of whether you’re using one cloud provider or multiple clouds, you should be concerned about the potential for vendor lock-in. On a simple level, abandoning your on-prem infrastructure locks you into the cloud in a generic sense — it’s difficult to move back to an on-prem infrastructure that no longer exists.
But there’s an expertise issue as well. Once your database experts become, say, “AWS experts,” it can be harder to contemplate moving to other platforms if necessary.
In the case of ScyllaDB’s managed database service (ScyllaDB Cloud), Livyatan said, “we mitigate this risk to some extent because we’re using a standard API. So worst case, because we are using an Apache Cassandra API and a DynamoDB API, you can always just fall back to running your own ScyllaDB [enterprise or open source], or even move to another database without much work, because it’s a standard API.”
It’s always a good idea to have an exit strategy for major parts of your stack, including your database, but you need to weigh several factors carefully before using it.
Take a long view on the size and likely cost of the infrastructure involved, Livyatan advised, and figure out best- and worst-case scenarios with your potential providers. Some platforms, he noted, can get expensive if they start handling millions of requests per seconds.
A Matter of Trust
“You want to decide on a path that a year from now will give you a reasonable cost and performance,” Livyatan said. “A year from now, you can be either very successful or maybe not as successful.” So it makes sense to price both scenarios.
And even if everything goes as planned, peaks and troughs may be inevitable, in which case a managed service can help smooth the loads.
Sader, of the adtech vendor GumGum, told how this works for his company on the New Stack Makers podcast: “At this point, we’re coming into our busy point of the year. Ads really pick up in Q4. So really what we do is we reach out so we go, ‘Hey, we need more nodes in these regions, can you make that happen for us?’ They go, ‘Yep, give us the things.’ We pay the money. And it happens.‘”
Which illustrates how important it is, if you choose a managed database product, to be able to “trust” your partner. This trust can be informed by the maturity of the vendor, Livyatan said, and its ability to provide the level of service required, to the level of simply ensuring the data you write will be there when you want to retrieve it.
But it’s also trust in the vendor’s level of expertise. Some providers will offer multiple open source products as a managed service, he said. But users need to ask themselves whether they’re confident the provider will have the deep expertise to fix a problem when it arises — or whether you’ll need to rely on that open source project’s broader “community” for help.
In such cases, you might feel more comfortable knowing the vendor is dealing with the people who wrote and own the core product you are relying on, Livyatan suggested.
Who Manages the Migration?
Whatever option you choose, there is still the issue of managing the migration itself.
“The first question is, are you migrating from a compatible database or a similar database, like Apache Cassandra or DynamoDB to ScyllaDB, or vice versa?” asked Livyatan. “Then your application can stay roughly the same, your data model can stay roughly the same, and we’re “only” talking about moving data from place to place.”
However, moving to the cloud might be seen as a chance to make a more dramatic switch from your existing database. “Then you have to change your data modeling,” Livyatan said. “You have to change your application. And then we’re talking about a different scale. Probably you need to rewrite another application on the side, or just change it. So of course, it’s more complex.” And the more data you’re looking to move, the longer this task potentially takes.
Sader’s advice is to adjust your own setup if possible, and “cleave as close to the vanilla execution as you can before you transition over to a managed solution.” The transition will be smoother, he added, “if you can make it a drop-in replacement.”
Either way, you will have to face the question of how much, if any, downtime your organization is able to tolerate, or whether you want to operate with both platforms running for a period before phasing into the new database.
At which point, your choice of new vendor, and their ability to manage the sort of service you need, will really come into play.
“I wouldn’t call it trivial,” says Livyatan. “But since we’ve done it many times, we’ve got some expertise in it.”
Check out this recent episode of The New Stack Makers podcast for more about GumGum’s decision to use ScyllaDB’s managed database platform.
