Enterprise Adoption of APIs is Driven by Internal Integration Needs
The need to integrate the ballooning range of software-as-a-service (SaaS) tools being used within enterprises is driving the adoption of application programming interfaces (APIs.) While many think of APIs in the enterprise as being introduced as a way to leverage a microservices architecture or as part of a broader cloud migration effort, integration is actually the early driver for using APIs in many businesses across industry sectors.
“The number one use case for APIs occurring at the enterprise is integration,” said Mark Geene, CEO and co-founder of Cloud Elements. “The pervasiveness of the duplication of data is driving demand. An average enterprise is using 1,000 SaaS apps, and every one of those is its own island of data,” said Geene. “Not connecting tools leads to a lack of getting the full value out of your customer data or out of an aspect of your business.” Geene gives the example of one automative manufacturer that Cloud Elements has spoken with that is managing 600 applications “just in human capital.”
SaaS tools Driving Integration Needs
The increase in consumption of APIs as a major industry driver has been acknowledged by Gartner analyst, Mark O’Neill. Speaking at API Strategy and Practice in Boston last November, at APIdays in Paris in December, and in a recent post on LinkedIn, O’Neill argues that enterprises rely on external consumption of APIs across business unit sectors including marketing, sales operations, IT, finance, logistics and HR.
In marketing departments alone, this has led to the use of best-in-breed API stacks and integration platforms for building “macroservice architectures: big chunks of semi-closed applications that have relatively small pipes of data being exchanged between them,” wrote marketing technologist analyst Scott Brinker.
What has caught Zapier by surprise is the uptake in use of its new developer program tooling amongst enterprises creating their own integrations for internal teams.
A Cloud Elements study to be released shortly demonstrates the need for integration platforms to act as a mediation layer that can connect various SaaS tools with systems of record, in order to align data sets across various systems.
Cloud Elements’ research found that the greatest demand for integrations in enterprises comes from the need to connect cloud storage, CRM and marketing applications. In the second half of 2016, the integration platform also saw growing demand for syncing data from finance and helpdesk apps.
Geene points to two common industry use cases, one customer facing and one internal:
- 360 view of the customer: “One of the areas we hear is the disconnect between who is a customer and who am I supporting,” said Geene. “When a customer comes in and engages with a business, does the service management system know who that customer is, what they bought, or how they pay their bills? If you are going to engage your customer, you need to engage in the point of sale system, billing, any messaging systems you are using: customers expect to be known and not to have to repeat themselves each time they interact with your business.”
- Managing internal assets: “If you look at the proliferation of cloud storage along with on-prem, workers often can’t find core information they need to use to run their business, because it is all siloed in those systems, and they can’t stop that growing, with workers now all bringing in their own devices. They need to start building workflows that connect content from those silos, and that takes integration.”
Data from Clearbit demonstrates the growing problem enterprise faces in connecting the proliferation of SaaS products in use across their industry. Clearbit analyzes use of 298 common business SaaS tools amongst enterprise. Taking an industry example — the airline industry — amongst those enterprises with more than 1,000 employees, at least 25 percent are using ten or more of the SaaS tools that Clearbit tracks. And that only represents the tools that can be read via a business’ website. Amongst financial services with more than 1,000 employees, at least 21 percent use more than 10 SaaS tools.
One of the key challenges when managing and integrating so many software systems is that each system may have its own nomenclature for similar data fields. Integration platform Bedrock Data recently published an infographic showing how various software may have different names for the same database fields:
Detail from Bedrock Data’s infographic on how different SaaS tools describe the same data field
Todd Sundsted, Technology Executive and Leader at data metrics integration platform SumAll, said different names for the same data field is just one of the integration challenges for an enterprise if they try and solve each integration on their own. He lists other common pain points that any business must solve when trying to integrate various tools with data in their systems of record:
- Different ways time is calculated and displayed.
- Rate limits differences in allowable number and frequency of API calls that can be made.
- Dev, stage and production tokens.
- Lack of use of standards in currency, order and purchase transactions, and many other areas.
The Other Integration Need: Accessing Systems of Record
While connecting up the proliferation of SaaS tools used within an enterprise is one key driver of integration, Zapier has found that equally as important is the microservice-like functionality of opening up access to a data field within the systems of record.
The launch of its recent developer program was aimed at SaaS providers who might want to speed up the integration roadmap with Zapier and make sure they had their product’s functionalities available in Zapier’s ever-growing app catalog. Zapier’s in-house team builds new integrations based on customer demand, but for SaaS businesses, the new developer program provides a command line interface (CLI) tool so that businesses can offer their own integrations on the Zapier platform.
With that new functionality, what has caught Zapier by surprise is the uptake in use of the new developer program tooling amongst enterprises that are creating their own integrations for internal teams. “Teams within enterprises might have their one developer who usually hacks up a private integration that they share with the rest of their team. They might be hooking up their SQL database with Salesforce, for example, and creating an endpoint into their proprietary data,” said Bryan Helmig, co-founder and CTO at Zapier.
Helmig said that while enterprise developers could use Zapier’s in-browser tooling, many needed access to a CLI so that they could also add the private integration they create into testing and CI/CD workflows.
“The CLI is a way to get closer to the toolset,” said Helmig. “You can build a lot more knowledge around your schema.” Helmig said the unexpected market for their developer program has been amongst enterprises who will never need to grow their integration into a branded offering, but that need to solve internal integration pains. “Enterprises have lots of little janky scripts that never get updated and they don’t get the attention they need. So Zapier solves a lot of that integration stuff, and that is where we are seeing a lot of enterprises bake this in. They just need private zaps (what Zapier calls their integrations). We didn’t predict that type of internal stitching.”
Integration platform architectural considerations
Geene from Cloud Elements points to a number of architectural considerations that enterprises need to address when introducing APIs to integrate their SaaS tools and datasets.
For starters, in a throwback to the service oriented architecture days, Geene is seeing a greater use of canonical data models that document the complexity of an enterprise’s various databases and datasets stored in various tools, and identifies what data needs to be connected to get a more accurate, real-time picture of both customers and operations.
Along with that, Geene said there is a change in how IT perceives its own role. “What we are finding is that enterprise IT are becoming more of the mediation layer,” said Geene. He said that enterprises have to either build their own integration platforms to manage the complexity, leverage an enterprise service bus (ESB) or make use of an integration platform-as-a-service. Geene said that enterprises often have their own personalized data fields in particular systems of record datasets or within SaaS tools, and the complexity of making all of them available in a consolidated way is beyond the IT’s capacity to deliver.
Instead, IT’s focus should be on creating a canonical data model that “consolidates these SaaS tool APIs together and makes them available in a consistent way.” Geene points out, for example, that the marketing department may need a different view of the dataset than the finance department does, but IT can ensure that a consistent overall data architecture is maintained to deliver on both organizational needs.
“By the time they come to us, our customers already have done some of that data modeling work, and then they map everything into Cloud Elements,” said Geene.
While Cloud Elements is usually deployed in the cloud, some financial services clients, Geene said, do require deployment of the integration platform on premises. “But what we are finding more and more is that our customers want to deploy on a virtual private cloud, that’s the trend we are seeing many enterprises being most comfortable with in their hybrid cloud model.”
The API maturity path within enterprise
The growing use of integration platforms again points to a hybrid cloud infrastructure model that is allowing enterprises to keep competing against more tech savvy market entrants without losing pace or competitive advantage. Now they can add integrations and leverage microservices and even function-level benefits without slowing down and having to reorient their legacy architecture.
For many enterprises, it is becoming clearer that a path is emerging that moves from first integrating internal systems by introducing APIs; then creating microservices and functions for internal use (exposed via API); before moving towards opening those APIs and functions to partners, resellers and suppliers. In some cases, this is then followed by a stage of growing those APIs into fully-fledged products with external developer access, either monetized directly or to reach new customers and enter new markets.
Even last year, this used to be a strategy of the leaders of the pack, now it is consolidating as the mainstream way enterprises must operate.
Feature image: By Clem Onojeghuo on Unsplash.
