Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
API Management / Software Development

API Management Is a Commodity: What’s Next?

While API management has become a commodity, the industry must address five new areas: regulation, verticalization, security, foundational business infrastructure and no-code development.
Jun 6th, 2023 9:59am by
Featued image for: API Management Is a Commodity: What’s Next?

NEW YORK — The recent Global API Days conference delved into current API management trends, indicating that the market continues to grow with a clear trend toward breaking out components of current API management to create best-of-breed solutions for specific capabilities such as API security, federated marketplaces, and analytics, and testing.

The event featured a virtual Who’s Who in the API industry.

For instance, in a keynote, API Days co-founder and Chairman Mehdi Medjaoui, said that API management has become a commodity. Medjaoui listed five new areas for the $175B API software industry (growing 30% annually) to address: regulation, verticalization, security, foundational business infrastructure, and no code development.

Meanwhile, Gartner’s Chief of Research for Software Engineering, Mark O’Neill, said that 94% of organizations are adopting APIs and two-thirds are using some form of API management software. Gartner sees a trend toward lighter-weight gateways, and unbundling functionality for security, third-party API governance, and AI bot monetization.

O’Neill also noted that many organizations have multiple gateways. For example, an institutional gateway for internal APIs, a cloud gateway for cloud APIs, and an embedded gateway as part of a software package. He doesn’t see this changing any time soon, meaning the API industry will need to meet this challenge of multiple gateway instrumentation and governance.

LLMs Will Drive API Adoption in New Ways

Overall, APIs appear to be reaching new levels of adoption, as organizations rely on them for more and more daily business activities and new sources of revenue.

O’Neill said that Gartner is receiving a tremendous amount of client inquiries about ChatGPT, surpassing other topics in recent months. “Generative AI, such as ChatGPT, has the potential to act as an intelligent client to consume APIs, and is particularly well suited to APIs which implement the HATEOAS (Hypertext as the Engine of Application State) constraint of REST, in which servers respond dynamically with hyperlinks.”

Vedran Cindrić, founder and CEO of Treblle, an API observability startup, predicted that the influence of Large Language Model (LLM) adoption alone will drive as much as a 30% spike in API adoption as organizations adopt AI tools to modernize customer FAQs and document search capabilities. They will need tooling that keeps pace with the new types of API applications.

Asanka Abeysinghe, CTO at WSO2, agrees that AI chatbots will accelerate the adoption of APIs by enabling access to LLMs and facilitating data exchange. “As a result,” he said, “External API management and quality of services, including monetization and governance, will resurface as critical requirements. API management is evolving into a standardized practice, urging organizations to shift their focus towards deriving value from APIs and leveraging them to create tangible benefits.”

APIs Increasingly Adopted in Finance

Finance is experiencing API transformations.

Jason Kobus of JPMorgan Chase and Shira Jaffe of Plaid noted two major trends for APIs in financial services: increasing global regulation such as Open Banking APIs for sharing data, and strategic business-related APIs for embedded banking services, which make it easier to pay for an item or apply for a loan when engaged in another activity such as an F1 racing hobby or increasing the options for paying for an Uber trip.

Jaffe highlighted the trend toward using APIs for more personalized banking interfaces, while Kobus stressed the importance of how people use APIs to pay — they must be immediate, secure, and trustworthy.

Event-Driven APIs

Meanwhile, Gartner’s O’Neill highlighted the growing interest in event-driven APIs of all types — not just Async API, but also webhooks, WebSockets, and subscriptions within GraphQL. Customers have expressed the requirement to manage both request/response and async APIs with the same set of tools.

Shawn McAllister, CTO and chief product officer at Solace agrees with Gartner’s assessment. “Event-driven APIs are gaining popularity and are being leveraged by our clients alongside RESTful (request/response) APIs. The challenge to date has been the ability to design, catalog, discover and govern event-based APIs in the same way that organizations have long done with API Management tools. This is why Solace introduced an Event Portal, a toolset that provides those capabilities for event-driven systems.”

Alex Drag, director of product marketing at Gravitee said that it’s not a simple task to govern and secure event-driven APIs, especially when they are used in the same way as traditional REST APIs. The challenge is to manage the lifecycle for all APIs by supporting both the OpenAPI spec and the AsyncAPI spec, applying policy consistently at the message level, and exposing both types of APIs in a developer portal.

Portals, Analytics, Third-Party APIs

As API management software unbundles the gateway and adapts to the multi-gateway world, new and emerging software vendors are looking to fill the resulting requirement gaps for API design and development, security, analytics, portals, and marketplaces.

Alex Walling, field CTO for Rapid, sees that developers need a layer of abstraction on top of their existing API gateways, such as those from WSO2, Kong, and Apigee so that they can find APIs easily and check whether someone has already developed an API for what they need.

Moreover, Derric Gilling, CEO of Moesif, said he believes that API Gateways will become just one of the specialized pieces of the API stack developers and organizations will need to assemble to meet the growing adoption of APIs. He sees business models for APIs evolving beyond simply charging for API invocation counts, and the need for a specialized analytics solution to keep pace.

Along with the continued explosion of interest in APIs, especially as organizations use more third-party APIs, the development and testing process becomes more complex and time-consuming. “With increased API consumption, especially third party and internal APIs, developers need a better way to reduce dependency on flaky third-party, sandbox and incomplete APIs, so they can create an isolated environment and enable faster time to market,” said Uri Maoz, CEO and co-founder of WireMock.

What’s Next

APIs are front and center in a number of IT trends, including LLMs, event-driven architectures, third-party API productizations, SaaS-based software integrations, and financial services regulations, to name a few. The overall market is growing in terms of the number of APIs being developed and deployed, as well as in the variety of API protocols and usage patterns (SaaS integration, internal integration, API products, etc.).

APIs are truly becoming the next lingua franca of the internet, following in the footsteps of the incredible success of the web by providing machine-machine communication in addition to human-machine communication.

The API software industry is responding with innovative best-of-breed solutions for observability and analytics, lightweight gateways, federated portals, testing, security, and development productivity tools. This will create more work for customers to evaluate and assemble comprehensive API solutions, but the result will be a software stack that not only can better handle the explosion of APIs and API types but also provide better solutions for business process automation.

Eric Newcomer is CTO at Intellyx. He has served as CTO for leading integration vendors WSO2 and IONA Technologies and as Chief Architect for major enterprises such as Citibank and Credit Suisse. He has created some of the best-known industry standards and university textbooks in use today.

Gravitee is an Intellyx customer, and Solace and WSO2 are former Intellyx customers. None of the other organizations mentioned in this article is an Intellyx customer.

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.