You don't manage the API, the API manages you

We live lives that are waveforms constantly changing with time, now positive, now negative. Only at moments of great serenity is it possible to find the pure, the informationless state of signal zero.

Gravity's Rainbow

Kick-Off for a 5-part series on API's

Inspired by a good, simple, API best practices post from Yahoo, I figured it might help to collect all the stuff I've learned in different companies and projects over the last years on API's - anything non-trivial beyond 12 Factor App, Web API Design and Serverless Architecture. For me, API's are the uniform access point to one or multiple systems. For the sake of this article I exclude API's within the same binary here, so you could argue I speak about stand-alone API's if you want (I don't like the term Web API or Front-End API too much).

Stand-Alone API's, and with it Microservices, are being promoted as "SOA, the good parts". Here, an API as front end to multiple systems is lean approach to SOA with right-sized governance, less canonical data models, and a distributed, scalable, containerised, architecture. Quite simply though, they are just a natural evolution of open ecosystems with ubiquitous, uniform HTTP interfaces, propelled by contemporary service-based business models leveraging duck testing for continuous delivery. A good example of such an API is Firebase for instance, a portal to a PaaS.

Over the course of this week I will share a few ideas I stumbled across in the last years. Some of them might have worked, others not, on some I might have worked directly, on others not. This is not a tutorial, this is a story.

My favourite dish is brussels sprouts haloumi or broccoli and tuna pasta or kale and lentil salad - Onion layers of service quality

The brassica plant has multiple layers but adjust them to the environment, forming brussels sprouts, broccoli, kale or cauliflower. It's the perfect example of evolution, because none of its incarnations is "better", they all fit an environmental condition perfectly. The plant, as a genus, ensures its survival, in the sense of the selfish gene. There is no "best fit", no single winner and no ideal state, but there is a local maximum (see James May's version of Chaos vs. Evolution) - some brassica used the onion layer principle and shielded it's core against bacteria by layering boundaries, while others protected the core with heavy armour or simply chose to distribute faster and outpace pest.

In evolution, your local context, your environment drives mutation - no wonder Google considers code archeology one of it's key success factors for having a singular, large, evolving code base. As such, knowledge about API's is really applied knowledge of the world, the web, HTTP, Hypermedia/REST, ontologies, simply relationships. Applied to a system of systems with various models of state and truth, a hostile environment and concurrent technological and non-technological processes fighting for resources, with information being the main resource. API's are the core of information exchange, and as such are the genes of our systems.

Part 1: You don't manage the API, the API manages you

API's are interfaces and as such naturally often leaky. Because the systems behind them are already leaky representations of the real world. But as API's also need to conform to certain technology requirements, they often become not only leaky, but also muddy. You cannot manage this mud, that's why I don't like the term "API Management" - it oversimplifies the interface, which is always dangerous. Because it's often the API, the contract, that stays, not the systems around it.

Just like Software, API's are always political. Who controls the interface controls the systems. That's why the term "API Management" is so tempting. Because it gives the impression the Management could get the control back - more about this is a follow-up post.

You have to understand the API not as a governance, or management, tool but as a boundary, a business card, a portal into the world of your systems with shared ownership. Currently there is 3 major models of building such an API boundary:

1. The gateway model, like Apigee or Axway. Sometimes called "Experience API" or "Web API"
2. The orchestration model, like FUSE, Layer 7 or WSO2, basically like an ESB for REST Services, often called a "Backend for Frontend" (BFF)
3. The distributed model, like Zuul or Træfɪk, basically dynamic reverse proxies with very little transformation logic but real semantic knowledge of the service topology (as made famous by Netflix)

API's are the Map to your landscape

An API is a flattened version of your service territory. You can stack API's to reduce complexity further, but this simplification always comes at the cost of different semantics that need to be explained. API's are a rabbit hole, they always stay a little bit mysterious just like genes. What I like about Nr. 3, the distributed model from above is that is is a real part of the architectural domain context, hence of its semantics. Strangely, though, I have not seen any of those distributed API gateways fully exposing those semantics - more about this is a follow-up post.

Some of the Nr. 1 do have such a UID mapping. They are rather obviously what we often call API Gateway or Edge Server, typically employed for non-functional reasons, e.g. security, throttling, backwards compatibility or caching. All valid use cases but often misused to stay with a big muddy ball of legacy systems behind it. And Nr. 2 is just as obvious for everyone who comes from an Enterprise Integration Patterns background, it is much heavier than models 1 or 3, might sometimes cause landgrab, but has the advantage of a clear separation of API logic which can be more complex then other business logic. Personally speaking, I tend to favour a mix of the model 2 and 3. I believe OSGi is a good model for a Microservice API,'s because it achieves both separation of concerns and dynamic relationships even with non-REST components (which is great e.g. for CQRS).

But I also like that, on the Pivotal Cloud, both model 1 and 3 can be used, with Apigee and Zuul with Eureka and Spring Hateoas working together as a Domain-Driven Architecture. The only problem with such an approach is that too much non-functional logic becomes part of the service, which becomes slow and non-resilient in itself, especially if not all relationships are RESTful which causes religious discussions only solvable by God himself.

API's, not services, are your teams

For instance, the (otherwise great) Self-Contained-Systems model (that is similar to 1+3) makes the mistake that it prefers a team per service, with the assumption that the domain of a service is a feature - in my experience that leads, thanks to Parkinson's Law and the IKEA Effect either to bloated systems or to frustration in the team. Often the domain of a service is not 1:1 a stand-alone API feature, especially across channels or in the absence of channels. I don't have an issue that, in some Microservices models, Web Components are a Microservice in itself as UI and better than a BFF. Maybe not particularly RESTful, because the semantics internalise representation instead of externalising them, but elegant.

Without too much references to Conway's law it must be clear that the choice of API defines your organization. The mix of model 2 and 3 allows smaller services and logic that is clearer per feature, with a shared ownership of the whole API - but comes at the cost that teams must understand, and synchronize dependencies, between multiple services. Cross-cutting teams aligned to real features, or to API's, tend to understand the platform better, which is great for distributed teams and high numbers of parallel features being developed. But be aware a platform can itself lead to a a distributed monolith that hinders refactoring.

If you build an API, understand you don't manage the API. At best, if you are lucky, you are managing some of the systems. And you get some consensus on how they can be accessed. This consensus is your API, your information DNA, and once it's distilled it's hard to change. The way your teams handle those interfaces is much more important, and over time the API will become a standard that will shape the systems rather than the other way round. Hopefully you have someone who cares for the API by then, call that person architect if you like.

You don't manage the API, the API manages you.