How Paylocity Scaled Up Community in the Cloud with MongoDB
Relying on vague reasons and requirements to rationalize a cloud migration project — such as “we need to modernize” or “our competitors are doing it”— won’t generate optimal results. It might even produce more problems than the one it was intended to solve.
Success in the cloud, especially when migrating a product or service built and run on-premises, is rooted in clear business goals and the technical requirements for supporting them. These create the specific framework for understanding (and measuring) what success looks like, and then bringing those conditions to life.
That’s been precisely the case for Paylocity, a leading provider of payroll and human capital management (HCM) software solutions to handle everything from payroll to recruiting and onboarding. While these are longstanding human resources needs, modern platforms that serve them, like Paylocity, are also increasingly looking to apply software to more abstract (but no less important) requirements like culture — when you may only see your coworker on-screen instead of in a traditional office.
In that vein, Paylocity’s Community product had become one of its fastest-growing tools. Community enables employers and employees to connect and post messages online, share feedback, post videos and even celebrate birthdays. Like much of Paylocity’s platform, Community was built and run on-premises.
From a database standpoint, Paylocity opted for a relational database management system (RDBMS). But the company’s developers were beginning to bump into the outer limits of what was possible with their own infrastructure. In order to maintain momentum and continue providing the caliber of service clients had come to expect, Paylocity needed a cost-effective, scalable, and high-performing document database with minimal maintenance requirements.
Encouraged to find the best tool for the job, Paylocity ran several options through its own evaluation checklist and testing parts of its real application on them. But none were able to check all the boxes for a seamless migration until it landed on MongoDB.
“We needed a platform that was kept up-to-date with new query updates, new performance optimizations and removing that operational burden that you can have with on-prem or other solutions,” John Wiegert, Paylocity’s principal engineer, told The New Stack. The need was only getting more pressing, with Paylocity expecting 60% year-over-year growth on its platform.
Using MongoDB Atlas has essentially reduced the company’s database maintenance effort to zero, while giving it five times the performance speeds, at one-fifth of the cost of its previous RDBMS solution.
Here’s how the solution came about, and what the company learned.
Scale, Two Ways
“Scale” essentially had a double meaning for Paylocity. Not only was Community experiencing 40% growth in its monthly active users, but it was also simultaneously rolling out new features — like video capabilities — at a 90% usage clip. And all the while, handling more than 100 million API requests.
“This is a very actively developed application,” Wiegert said.
The company realized its traditional RDBMS infrastructure wouldn’t be able to accommodate this type of growth, according to Wiegert and Adam McElhinney, Paylocity’s chief data science officer and vice president of data insights.
“We saw the growth numbers look like this, and we knew that we had hard caps and our traditional technology that at some point we’re going to hit a ceiling on what we can scale to,” McElhinney said.
The potential to upgrade what was possible in terms of product development and growth — and the potential downside of sticking with the status quo, including maintenance problems, cost overruns, lack of flexibility and poor developer experience — made a compelling business and technical case for migrating the Community application to MongoDB’s cloud-based developer data platform.
“It enabled us to continue that growth trajectory without users seeing any disruption,” McElhinney said.
Taking infrastructure maintenance and other operational toil from Weigert and his team’s plates was one of the big selling points of MongoDB Atlas, as was returning flexibility and choice to developers as they continue to add new features.
“We have so much less operational burden. We can just focus more and more on building applications and features, which is what we do best as developers,” Wiegert said. “We don’t have to worry about how big [customers’] usage gets.”
It’s been enough of a success story that McElhinney said Paylocity is now actively looking at increasing the company’s cloud footprint. With MongoDB Atlas and the developer data platform, his teams can utilize a tightly integrated collection of data and application infrastructure building blocks to rapidly innovate and address any application need at scale.
3 Tips for Your Own Cloud Migration
Teams looking to adopt or grow a similar hybrid model — and especially to make the case for moving an existing app to the cloud — can learn from Paylocity’s experience of moving its Community product to MongoDB Atlas. Here are three big takeaways from Paylocity’s story:
1. Start with a Business Outcome.
Remember, success in the cloud doesn’t come from fuzzy goals.
In Paylocity’s case, it started with setting specific objectives based on a business goal. Only then could they identify the technical solution needed to achieve it. Don’t try it the other way around.
“Something we really think is important is to make sure that all of your technology initiatives start with a non-technology or business outcome,” McElhinney said.
Paylocity needed to simultaneously support rapid product development and rapid customer growth; moving off an RDBMS and onto MongoDB’s flexible document model was the technical goal that came out of that — not vice versa.
2. Make Sure Your Team Can Tackle the Learning Curve.
“One thing I’d really reiterate for people that are considering doing something similar is: Make sure you really think through what the training plan for your people looks like,” McElhinney said.
A sink-or-swim approach to training and learning new tools or technologies is likely going to slow you down or cause the kinds of disruptions that become visible to end users, something that Paylocity wanted to eliminate. Your vendors should be resources: when you evaluate a new tool provider, always ask what support they offer for training.
“We’ve found MongoDB a great partner [in terms of] providing us workshops, great documentation and just other training materials to help our developers,” McElhinney said.
3. Rethink Your Design Patterns.
There are multiple models for moving workloads from an on-premises environment to the cloud, but they don’t all offer the same results. The lift-and-shift approach merely replicates the complexities and drawbacks of the traditional data center.
McElhinney said that in order to take full advantage of cloud capabilities, teams need to ask: “How can I re-architect this to take advantage of this different paradigm?”
As Paylocity’s data science team has expanded its cloud footprint, for example, it has had to rethink how it operationalizes machine learning models. It is embracing serverless architectures to optimize and take full advantage of cloud native capabilities.
“We need to run on the order of tens of millions of machine learning model predictions in a given month,” McElhinney said. “Leveraging serverless architectures gives us a way to provide those predictions quickly so that clients get data rapidly, regardless of what the backend workloads look like.”
