APIs Are Where Fun Happens
We are living the API economy, and it is fueling the evolution of the digital world. Every day, new companies whose products are APIs are being created. New forms of APIs — GraphQL, gRPC — and new vendors around API, especially API security, are being created every month. At a recent API world conference I attended, I served on a panel about API trends where I said: “For a long time, APIs were gateways to fun. Now it is time for APIs to be the fun.”
That is clearly where the world is going. APIs are where governance, orchestration, federation, distribution, data serving and “backends for frontends” are moving.
Let the fun begin.
APIs as Gateways to Fun in Data Sources
APIs are the way to access and manipulate data, and so they are clearly the interfaces to the data layer.
Figure 1: API interfaces to the data
APIs do not replace some other interfaces; they supplant them to address a different audience or to hide business logic.
A couple of independent observations here:
- Of course databases have SQL interfaces. But they now increasingly have APIs too. The reason is simple: APIs bring more developers, more apps, more uses to the data. Last month, Ed Anuff, chief product officer at DataStax, wrote an excellent article on APIs: “Why Databases Need APIs.” I would edit the title somewhat to be “Why Databases *Have* APIs.” Because it is all around us.
- Traditional analytical tools have been slower to catch the API wave, but that is changing. Headless business intelligence (BI) is becoming a thing. Power the BI app straight from the analytical system, bypassing custom “app/dashboarding” code. Cube.dev is a great example of this, with REST and GraphQL APIs being central to its offering.
But APIs don’t just provide interfaces to independently built data layers. They help build out the data layers. The most important aspect they help with is data federation.
APIs as Fun
Almost all enterprises have efforts to build out their data warehouse/lake. However, data lakes are difficult to maintain and primarily serve analytical needs. The output of all the analytics needs to be consumed, and there is data that is either not in the warehouse today, or will never be in the warehouse (quantity at hand, anyone?). Furthermore, there are several other secular trends that lead to decentralization of data:
- API economy: Say you are accessing a third-party API like Stripe. In that case, you (typically) do not have access to the raw data. Rather, you have access to the data through the APIs that the system provides. This is also true of the microservices that your organization might be building.
- Clouds tend to lock away the data (they make the egress fees really high), so the data stays where it is.
- Data residency requirements further help decentralize the data.
Net-net, a data federation approach is sometimes the only way to access decentralized data. And while you could build a SQL interface, it would typically only cover SQL databases, such is the approach used by Trino, for example. The right approach is an API tier, more particularly, a GraphQL API tier.
Figure 2: GraphQL as the federated data tier
In this tier, you put some metadata (which backend can do what), a scatter-and-gather execution optimization engine, and lightweight business logic for routing and assembly.
APIs also help round out traditional data architectures. Take ETL (extract, transform, load), for example. There are source systems (some accessed through bulk APIs) that are transformed and then loaded into warehouses. How does governance move around as data is moved around?
APIs as a Way of Organizational Control on the Data Layer
The biggest problem for enterprises is almost never the technology; it is the organization. APIs help unlock data sources from control of organizations and make enterprisewide capabilities easier. However, someone once said, “Show me the APIs that an organization exposes, and I will tell you its org structure.”
The fun is in taking an organization-specific API structure and converting it to a consumption-specific API structure. Again, GraphQL comes to the rescue here. Federation is its superpower, and it allows for slicing and dicing of APIs into parts and recombinations. GraphQL APIs go from being gateways to backends to being gateways for applications to do magic. Imagine the ability to shape something wonderful from the clay that the backends provide.
A federated GraphQL API architecture does exactly that.
Figure 3: Federation is a superpower of GraphQL
Summary
There is a generational shift from APIs being just gateways to backend data sources to APIs being a place where new data architectures are born and new capabilities are produced. APIs are going from gateways to fun to being the fun. Hang on tight!
