Creating ‘Offline-First’ Applications for Wi-Fi Dead Zones
Working “in the field” or simply being out on the shop floor used to mean you were also on your own. The decisions you made and the advice you could give were based on whatever information you carried with you, whether in your head, on paper or, maybe, on some sort of digital device.
But that was before we got online. Now, an array of mobile and embedded devices, together with ubiquitous connectivity, mean we should be able to take advantage of the cloud, and all the data stored in it, wherever we are, whether our task is delivering Pepsi to remote mom and pop stores, drilling for oil at sea or even carrying out the most delicate orthopedic surgery.
Certainly, a generation of developers has been happily creating apps with REST APIs and the cloud or remote data centers to give workers and devices out in the field — or simply walking the shop floor — the ability to share and update data, and carry out business.
But that backend connectivity doesn’t come for free. There’s a cost to pumping large amounts of data to the cloud, processing it and sending it back out again — environmentally, financially and ultimately in terms of performance and availability.
And even in the United States, mobile coverage maps still show large gaps. Outside of those blank spaces, the forces of nature can still take down cell towers or cut off data centers. And let’s not forget the mayhem that an overworked admin or malformed update can unleash.
Taking an edge-first approach only partly solves the problem because what we’re talking about is something that’s happening beyond even the edge. And this presents developers with some real challenges.
Modern developers have “the bliss of unlimited resources that are always available on demand instantly,” according to Dave McCarthy, vice president of cloud and edge infrastructure services at the market intelligence firm IDC.
But they face a jolt when it comes to developing applications that will run in the Internet of Things (IoT) or “field environments” where they can’t guarantee those resources are always available, he told The New Stack: “The first word you learn as a developer with IoT is the word ‘constrained.’”
Then, McCarthy said, they must face the question of, “How do you maintain the integrity of your application when those things happen? Those are things you typically don’t run into in the data center or the cloud.”
So perhaps, what we need to be thinking about is “offline-first” applications. But what does offline first actually look like?
The starting point is to assume that connectivity cannot always be guaranteed, but “aim to ensure that wherever you go, things work,” Mark Gamble, product and solutions marketing director at Couchbase, told The New Stack. “If you walk into the subway, you’re still able to play that game or make that appointment or sell that thing.”
Why Offline Caching Isn’t Enough
This approach to offline-first app building assumes that data – some or all – is held and processed on the mobile or remote device, rather than in the cloud or back at a data center.
One option for achieving this is to simply cache data on the handset or device to keep apps running during internet disruptions, then syncing it when connectivity is restored. However, this leaves developers facing further questions, Gamble said. What is the upper limit of data stored in the local cache? After all, the longer the device is disconnected, the larger the amount of data stored. An intercontinental flight means a long time offline.
And where is the cache being held, exactly? Is it simply in memory,and likely to die when the battery does? Or is it persistent? Is it encrypted while on device and while it’s being synced back to the cloud when connectivity is restored?
“Offline caching is good for an occasional blip,” argued Gamble. “The problems start to arise when you’re caching too much, and you start to run against cache limits. And that’s where you start to risk data loss. Data corruption is bad, but ‘where did it go?’ is even worse.”
Alternatively, you could consider embedding a full database to the application runningon the device itself, meaning information can be queried and stored without the need for connectivity at all. One option is to use an open source database platform like SQLite, to store and process data on the device.
But “a typical application is not one person and one set of data working all by themselves forever,” Gamble said. “Eventually, that data needs to be shared across an ecosystem for other users.”
Which means developers still need to consider how to sync that data back to the cloud, as well as to other devices running the app, and SQLite doesn’t include sync as standard. This can lead developers to cobble together their own stacks, either with custom code or by integrating third-party technologies. But this can be time-consuming and creates issues around standardization, maintenance, updates and repeatability.
Couchbase’s solution combines its own lightweight NoSQL database, Couchbase Lite, which can be embedded to apps running on the device, but also syncs with Couchbase Capella in the cloud when there’s connectivity. When there isn’t connectivity, applications will still function by leveraging locally stored data.
Additionally, Couchbase’s sync service, called Capella App Services, allows apps to sync only the data that’s changed while offline, versus syncing everything, cutting down on traffic and speeding up the sync process overall, said Gamble. This is obviously going to be more efficient. Its syncing is bidirectional, he added.
“There’s also this notion of directing the flow of synchronization through filters — in a large retail chain you don’t want to sync all the data to every store; maximize efficiency by syncing only the data needed for each individual store.”
Syncing in the Field
The platform also offers developers a variety of topologies depending on the environment they’re working in. With peer-to-peer syncing, devices cut off from the cloud can sync with each other without a central control point. So, users or devices can continue to share data locally, then, when connectivity is available, sync back to the cloud.
Combined with the broader Couchbase product line, developers can then build multi-tier architectures.
One Couchbase customer is medical device maker Arthrex, which, among other things, embeds the database in some of its arthroscopic devices used in surgery to capture data, which can then be synced with the hospital’s on-premises systems and ultimately to the cloud, where surgeons access it for post-operative review and recommendations.
Likewise, military applications can benefit from the ability to quickly sync data between field personnel and potentially to a vehicle, to a command center and upwards. But such applications will also need to be able to continue operating if any given layer is knocked out and must give operators confidence that they control precisely what data is being transmitted and that it is secure.
The choice of topology is intertwined with user experience — at most when the application is being used by a human. If store or hospitality staff are using tablets to check inventory or process orders, they need to get the information quickly, and a hop to the cloud and back might simply take too long compared to accessing locally stored data. But the app must still be fast and intuitive enough that it doesn’t break the flow of the customer interaction unnecessarily, wherever the data is being pulled from.
Ultimately, McCarthy said, an offline-first approach addresses many of the issues that dogged the first wave of IoT-type applications. These had the right intention in terms of harvesting large amounts of data, which could be used for tuning algorithms and delivering insight. But centralizing data and processing made for a disjointed workflow and patchy user experience when it came to relaying those insights back to users in the field.
And the “field” is not just “hostile” or remote environments. As McCarthy noted, retailers and hospitality businesses rely on multiple systems to make the in-store experience comparable to e-commerce — from inventory management to recommendations to queue control.
“I think you certainly need to design for the capability of being offline,” McCarthy said. “Because, if, for whatever reason, the store is unable to communicate to a central source, you don’t want to tell all your customers to go home.”
