How HashiCorp Nomad Helps Relax Cannabis Compliance and Kubernetes Complexity
For legal cannabis producers and suppliers, compliance is the difference between blossoming your business and losing your license. Governing bodies closely regulate every stage of the cannabis industry, from the greenhouse to the end retailer. Ample Organics provides the tech infrastructure to help cannabis companies meet compliance. In essence, we give complete visibility into the entire supply chain of any cannabis product they produce.
We started as a medical marijuana software platform in Toronto in 2014. When recreational marijuana became legal in October 2018, our business boomed. Everyone jumped on the cannabis train, and many of them became our clients. Today, about 70% of the Canadian market, including some of Canada’s largest licensed producers, run on our seed-to-sale software platform to track and report every stage of production and sales.
But managing such a wide-ranging and dynamic service isn’t easy. Developing and deploying new features and functionality can be time-consuming and resource-intensive. And we face unique challenges around data residency, as, by law, Canadian cannabis companies have to house all client data within the national borders — which limits our options for data centers and service partners.
To comply with regulations and continue to meet the business needs of our customers and our own company, we needed to rethink the way we develop and deploy features and tools, as well as the underlying IT infrastructure to support it.
Kubernetes Caused Mental Health Problems
In the past — before we joined the team — the company had adopted Kubernetes for orchestrating its containerized development environment. Everyone thought it was the proverbial hammer and every IT problem was seen as a nail.
While the initial rollout worked well when we were a tiny startup, it was evident early on that it wasn’t the right tool long-term for a rapidly growing and maturing business. Since Kubernetes doesn’t offer vendor support and because our unique data residency requirements prohibited us from contracting with a U.S. managed service provider, we were left on our own to manage, configure and bend Kubernetes to our needs.
Over time, that created a number of bottlenecks and headaches to deal with. In particular, each client had their own cluster that had to be manually edited, because we didn’t have federation in place. As we began tiering our services, the volume and velocity of changes meant that only certain customers got what they asked for.
Worse, it created real concerns about uptime and reliability. We’ve primarily been an Amazon Web Services shop for years. The majority of cloud providers have a data center located in the Montreal, QC region, meaning that geographical redundancy with a reasonable disaster recovery strategy were basically impossible.
Those uptime issues spilled into other areas. We breached SLAs with our clients. We couldn’t meet deadlines for other projects that clients pay for, such as enhanced features and functionalities. And none of it was a path to sustainable growth and success in an industry sure to keep expanding for years to come.
We realized that we needed an orchestrator that’s scalable and efficient with automated workflows. It had to be cloud-agnostic, for portability from one data center to another, and needed to provide options for securely and efficiently moving data around — to help us serve customers worldwide while adhering to data residency requirements.
Fatal Crash Ends Kubernetes’ Reign
We’d known about HashiCorp from our experience using their other tools, such as the open source version of Vault, and Terraform to deploy our resources. So it only made sense to look into its orchestration tool, Nomad.
Coincidentally, our Kubernetes cluster had crashed in September 2019. It was an opportune time to put Nomad to the test, so we assigned one team to rebuild our Kubernetes cluster and another to try to build a Nomad cluster on the side. We had our clients set up and running in two weeks on the Nomad cluster, but we weren’t able to get that Kubernetes cluster back in operation, which made our decision to adopt Nomad a no-brainer.
In the past, we needed five people to manage Kubernetes, but Nomad and our other HashiCorp tools require just two.
Nomad is easy to cluster up. We converted our Kubernetes deployment manifest to Nomad job files, then tested it. And since it’s a single binary, it’s simple to configure to our specific needs, which eliminates much of the complexity we faced with Kubernetes. More importantly, Nomad’s agnostic infrastructure resource pool and automated workflows let us deploy and manage our containers and apps across on-prem and any private or public cloud environment, which dramatically expands our data center options while still meeting our data residency obligations.
Nomad has a nice web UI, so it even works for some of our Tier 1 support teams. If they’re not command-line savvy, they can still get to the UI and find stuff by clicking around. They wouldn’t touch Kubernetes, fearing that a keystroke might collapse everything. Now they’re able to take some of their support issues off our hands.
As part of the HashiCorp ecosystem, Nomad is the missing link for creating complete, hosting-agnostic solutions. We use Terraform to deploy resources for old and new clients, while Nomad makes sure our containers are up and running with the proper resources. It has better resource allocation and integration than Kubernetes, so we don’t have to deploy as many resources now. We use Vault for secrets management and Consul for service discovery, to reduce the amount of load balancers we need. The Consul UI gives a clear view of what’s running, which version each client is on — and other things like that. We’ve offset our costs about 5-10% with Consul.
Nomad Solved Our Orchestration Issues
From a mental health perspective, all this has been wonderful. We’ve stabilized our performance, improved our security, and we’re able to fit more clients and containers into our resources for greater productivity and efficiency.
In the past, we needed five people to manage Kubernetes, but Nomad and our other HashiCorp tools require just two. With the additional people available now, we can repurpose team members who were previously occupied with administrative work. That has helped us increase our client project deployments, from just eight per week to more than 125 customer deployments within two hours.
Not only can we do more customer work faster, we also do it better. Before, at least one or two of our deployments experienced failures on a weekly basis. With Nomad, it’s now one in 125, and only on occasion. And when we do experience an issue that’s just beyond our capabilities, we rely on HashiCorp’s support team and resources to get the answers and help we need.
A lot of people think Kubernetes is the proverbial hammer that will magically solve any and all of your IT needs. We see quite a few threads in various Reddit subreddits where people post wishing they’d tried Nomad earlier, and they can’t because they already invested too much into their particular Kubernetes setup. But we’d recommend taking a look anyway, regardless of the size of your company. HashiCorp tools have 90% of what you need from an enterprise perspective in the free, open source version, with good documentation and tutorials.