API Management / DevOps / Software Development / Contributed

What Scaling Shopify’s Checkout Taught Me about Building Great E-Commerce Apps

22 Jun 2022 8:00am, by
Harry Brundage
Harry Brundage is the co-founder and CEO of Gadget, a serverless stack for e-commerce app developers. Gadget offers a full-stack development platform that helps developers start, build, and scale amazing apps. Before founding Gadget, Harry spent seven years at Shopify, where he led many of the company's engineering teams. At Shopify, Harry worked on the biggest technical threats to the platform’s success, like the core product's frontend, the platform's flash-sale readiness and key commerce primitives.

Last year, Shopify, the biggest e-commerce platform in the US, saw its partner ecosystem generate $32 billion in revenue which is up 45% from the year prior. Online brands are spending more time and money customizing their storefronts, and the functionality that powers them.

As a result, developers have shifted their focus to working on software that helps power unique shopping experiences. Their apps can range from simple workflow automations to customized cart notifications to AI-powered product recommendations.

Building e-commerce apps present a unique set of challenges that need to be navigated carefully. In addition to the myriad of technologies and APIs needed to build a good e-commerce app, developers also need to take e-commerce-specific challenges into account, such as additional security requirements that come into effect when they are dealing with people’s money.

I experienced this firsthand at Shopify where I led its scaling efforts while running the biggest flash sales on the internet, including Kylie Jenner’s viral makeup frenzy.

It was a time-intensive, tedious process that had my team constantly trying (and sometimes failing) to build for massive surges in traffic. Just about everything in our app that could break, did break — the database, connection pools, web server processes, load balancers, caches and even the frontend. We revisited every minutia over and again until we finally got it right.

There I learned the best way to build great e-commerce apps:

1. Build like it’s 1995 — with a backend

Frontend-focused apps offer users a great experience and enable rich and responsive interactions. But every e-commerce app eventually needs a server-side component.

It’s nearly impossible to get away with building 100% on your app’s front end without any server-to-server communication due to webhook processing, large data volumes and security concerns.

If your app’s backend is set up effectively, it can sync and store data on demand. This allows you to keep a fresh copy of the data you need to power your app, while escaping the underlying platform’s restrictions (such as API rate-limits). Bite this bullet as early as possible — backend needs are an inevitable challenge as your app begins to scale.

2. Start with serverless. Start with scale

The moment your app goes to market on Shopify or any other e-commerce platform, every merchant on the planet can install it. That means you need to be ready to handle any traffic volume increase right away, whether you’re dealing with the largest retailer or a small mom-and-pop shop.

You may not always know when big merchants will install your app, so the ability to automatically scale your e-commerce offerings in response to incoming traffic works best to minimize cost. Serverless platforms solve this problem by matching server-side power directly to your needs, handling any ebbs and flows of traffic with ease. Serverless is a great match for bursty e-commerce workloads where events like Black Friday, flash sales and exclusive merch drops are commonplace.

You should evaluate serverless offerings ranging from functions (Google Cloud Functions and AWS Lambda) to databases (DynamoDB, FaunaDB) to full-stack serverless offerings (Gadget).

3. Security is a necessity — so is compliance

Storing customer orders and personal information is a different ballgame than storing cat photos. Carelessness puts you at risk of security breaches like those that have hit Apple, ​​Facebook and other big tech companies.

Be vigilant about compliance with data collection rules such as Europe’s General Data Protection Regulation or California’s Consumer Privacy Act. On Shopify, for example, every app has to adhere to “the right to be forgotten,” meaning all personal data is erased once Shopify notifies you to do so. From the beginning, you should try to avoid storing sensitive customer data within your app — and if you need to, you should have a strategy for erasing it as requested.

Adding to the complexity, e-commerce apps are often written to be multitenant so different merchants are able to install the same app without fuss. This requires careful permissions management in your database, tagging each record with the tenant it belongs to in order to ensure each merchant can only access their own data. These types of data issues become harder to fix the longer you leave them unchecked.

As developers, dealing with these security and scalability concerns isn’t what we set out to do when we tackle new problems. Fortunately, there are more tools at our disposal than ever to remove points of friction and help us focus our time on what matters most: creating something new and exciting for our customers.

By building smarter, you can set your app and merchants up for success despite the looming threat of that next gigantic flash sale!

Feature image via Pixabay.