I’ve talked recently about a second coming of API management, which in my opinion is a wave that has several fronts. One of those fronts I see rolling out is in the area of event-driven API infrastructure. API management for request and response API infrastructure is pretty well defined, but the same concepts applied to event-driven infrastructure represents a pretty significant opportunity within this second evolution. Something that will span HTTP 1.1, HTTP/2, and TCP API implementations, providing the essential API management building blocks we might take for granted with HTTP 1.1 implementations, but as API providers expand their API toolbox to possess a variety of implementation patterns, they are beginning to look for consistent management capabilities across all their APIs.
Over the last year I have had several questions from readers regarding who they should be talking to when it comes to API management for their Kafka, NATS, and other event-driven API infrastructure, including something as simple as better management for Webhooks. To help me better tune into the existing API management solutions out there that might help out in this area, as well as brainstorm what the next generation event-driven API management solutions might be, I wanted to explore this topic a little more here on the blog. Beginning with restating what some of the essential building blocks are for API management:
- Portal - A single URL to find out everything about an API, and get up and running working the resources that are available.
- On-Boarding - Think just about how you get a new developer to from landing on the home page of the portal to making their first API call, and then an application in production.
- Accounts - Allowing API consumers to sign up for an account, either for individual, or business access to API resources.
- Applications - Enable each account holder to register one or many applications which will be putting API resources to use.
- Authentication - Providing one, or multiple ways for API consumers to authenticate and get access to API resources.
- Services - Defining which services are available across one or many API paths providing HTTP, HTTP/2, or TCP access to a variety of business services.
- Logging - Every call to the API is logged via the API management layer, as well as the DNS, web server, file system, and database levels.
- Analysis - Understanding how APIs are being consumed, and how applications are putting API resources to use, identifying patterns across all API consumption.
- Usage - Quantifying usage across all accounts, and their applications, then reporting, billing, and reconciling usage with all API consumers.
- APIs - API access to accounts, authentication, services, logging, analysis, and usage of API resources.
Some of these building blocks when applied to event-driven APIs would focus on the business of APIs, helping companies potentially onboard and monetize new and existing customers with their event-driven API infrastructure. However, some of it moves into the critical awareness building elements of API management and would go a long way towards helping event-driven API providers develop a better understanding of how their resources are being putting to use. Providing much needed visibility into potentially high volume API infrastructure operating internally and serving a variety of partners. Helping not just manage event-driven API infrastructure, but helping us be smarter when it comes to what we do (or do not do) with our next generation API implementations.
Beyond the essentials of API management, I’d like to see how it can evolve and become more specialized to address the evolving needs of event-driven infrastructure. Here are few concepts I’ll be expanding on in future stories:
- Event Opportunities - Helping HTTP API providers understand the event-driven opportunities that exist with their existing request / response APIs. Helping them understand how evolving and maturing their APIs to be more event-driven can help reduce the load on their infrastructure, increase performance, and reduce friction for their consumers.
- Event Reporting - Provide specialized activity reports across APIs, and for consumers when it comes to the types of events that occurring, and provide all the other data points and dimensions when it comes to how, when, and why events are being triggered. Providing more visibility into the meaningful events that driving business, and keeping consumers engaged.
- Rate Limiting - I’m not sure you will want to be limiting requests in an event-driven world, but I am interested in better understanding what rate limiting might mean in an event-driven landscape. It might end up being something that is configured and defined by consumers, allowing them to set thresholds for how many, and how often they want to be receiving events, and possibly even allowing for sampling, and other types of limitation.
- Scaling - Exploring how publishers and consumers might be able to scale API availability and consumption using the API management infrastructure. Empowering consumers to dial up the scale of garden or firehose they wish to receive, and possibly scale underlying infrastructure to better meet demands.
- Monetization - This is one area I’ve had some very interesting conversation around in the last year, thinking more deeply about how event-driven infrastructure can be monetized similar to how API providers have been generating revenue from request and response APIs. There isn’t a lot of precedent out there when it comes to monetization and the development of plans around event-driven infrastructure, but it is something I’ll be brainstorming more.
These are just a few of the areas I’ll be brainstorming more around when it comes to API management for event-driven API infrastructure. As part of my research in this area I will be taking a fresh look at what existing API management provider offer when it comes to management for event-driven APIs. I am sure here are existing offerings out there, but it isn’t something that has show up on my radar. Beyond what already exists, I will be taking each individual building block of API management and writing posts about how it might be applied in an event-driven reality and to specific implementations like Kafka, NATS, or maybe even MQTT in an Internet of Things world—fleshing out what some of the killer features might be when it comes to the management of our event-driven API infrastructure.
I think there are a whole buffet of opportunities for the next generation of API when it comes to the ways in which the API toolbox is being expanded in recent years. Think about how API management can be applied to not just event-driven APIs, but GraphQL, gRPC, Webocksets, and other patterns. Each area will have its core needs to help onboard, authentication, log, and report on usage, but they will also have more nuanced needs when it comes to each area. Over the next couple of months I will be exploring these opportunities, and encouraging existing API providers to step up in these areas, and hopefully incentivize new commercial and open source providers to jump into the game and evolve how we view API management. Along the way I hope we also revive and continue investment in what is needed for API providers to be successful in managing their existing request and response structure, and not viewing API management as a relic of the past, and something that every API provider will need to develop the awareness they need to be successful.
