A calm API uses situational awareness

In “Now Urbanism” Jan 13, 2011 on Informal Urbanism Celine D’Cruz explains:

“It’s important to look at all these aspects … In 20 years what will the structure look like … what do we have to do to make it simple … it’s not so complex to repair”

This post is part 2 of the series "You don't manage the API, the API manages you", which is a story of my experiences with API's over the last years and by no means a guideline for others.

In my first post, I had argued that the the distributed model of stand-alone API gateways is interesting, because it has some semantics from within the system, but no tools on the market fully support it. While there are some standards, like RFC 5988 and JSON-LD, and some implementations like OData, if you like XML/Atom, JSON API (Katharsis) or HAL/ALPS (Spring Data Cloud), they are very limited and with it limit the use of REST itself.

REST Semantics vs. the distributed Model

You might have noted I tried to avoid the term REST in this series on API's. That's a pity, but I don't want to get into a passive-aggressive argument over REST terminologies. I am personally just not interested into the platonic ideal of the semantic web, because I believe there is no one ideal language and hence such semantics are always just an additional, hard to explain, abstraction. That does not mean I don't like standards like ROCA or NARWHL. I still believe they are valid in many cases, and if you have a document/distributed state kind of application model, e.g. a web shop user interface or the Guardian, you should use them, and probably together with some form of progressive enhancement e.g. isomorphic apps with React (React/Relay/GraphQl is a document patching framework hence perfect for this use-case). I have even made good experience with a workflow and insurance calculation engine based on hypermedia but all other cases, the good ol' blog and order examples, seem rather random to me in terms of semantics.

Many systems are naturally either more event based or service based than document* based, or simply or too fuzzy, reactive, multispeed-evolving or eventually consistent to fix all relationships at the moment of access like a timefreeze. Thus Messaging protocols such as Stomp or RPC protocols such as Thrift or gRPC might just be the better fit, and SPA's or Regressive Web Apps might represent the ephemeral state of a system better. I've chosen HTTP-aligned examples here, because the ubiquity of HTTP makes it in my opinion almost unavoidable if you want your services and API to be independent of your infrastructure. I don't see a need to adhere to REST if you want to use HTTP; using a technology in unintended ways is at the core of innovation and hacking. Otherwise we would only have one telephone per town and no internet.

API's only have human users

One of the issues with REST is that it places itself somewhere between a human to machine and a machine to machine interface, with hyperlinks suggesting application logic can be universally understood by both. That puts it into an uncanny valley, and so many competing standards to fill this void out there. The problem is, while an API might be deep-learned, even a machine to machine interface eventually has a human user: The developer. And it is better for the system and its users if the developer understands what the UI does. My apologies, but a nice understanding for an architect is only no. 2 on the priority list of system properties, after the service experience (try Apiary for that). That's why, personally, I prefer a well-documented RESTish API (say RMM Level 2) that maps to a nice domain model, with "domain events as state transitions" and prefer to rather break REST principles than DDD principles.

Web API's are at the top of the onion layers of abstraction of semantics, and as such the hardest to reconcile with REST principles. A good user experience does not modify whole documents, it's as simple as that. You can build a lot of magic, for instance JSON Forms, but that only makes your architecture harder to understand for the real users. In the phone example above, if you click the link, James May speaks about Poka-Yoke, the Japanese version of KISS. The ambiguity and dogma of rest makes it harder to use that it should actually be. REST is, thanks to HTTP, especially well suited for the constantly failing systems we build today, it should embrace these failures more.

Calm is the new cool

The standard failure of API design is to mix up API simplicity with domain simplicity. Most domains are not simple. The easiest example is versioning. An API and its dependencies should be as stable as possible (I used contract tests, behaviour tests, integration tests, monkey tests, circuit breakers, mocks, patching, polymorphic API's and so on to push this stability as far as possible) for consumers, but at the same time in every modern company the domain model is constantly evolving. Semantic Versioning can be used in API's, but there is simply still no technical standard. Content-Types are too rigid, other Headers too unsafe and unstandardised, but URL's surprisingly easy. I usually go with full backwards compatibility - and rather name an API after a very specific domain context, in the hope this would go away with the business process (e.g. EndOfDayPersonalAccountTransaction2016).

Pagination is frequently used as example for hypermedia, especially because the link relations are indeed standardised. However, it's not standardised how they should be used. What I miss in all the examples is a complete model of pagination. What is a page even? Are those examples clear whether they mean a sliding window or a snippet? Which point in time is the reference? How do you check for updated records? How do you handle filter, or language or channel changes? And how to you enable a UI refresh that feels natural? Nothing of that fits standard semantics.

Qualitative Domain Models

In Java world, Objects are still used as the standard data structure, because they have many powerful attributes (pun intended). At the same time, concepts of stateless and immutable systems have flattened their disadvantages. Libraries like Jackson have become a de-facto standard of object mapping. Regardless whether you call it REST or not, I always believe in the principle of URI and UID where possible as lowest level of "maturity" of any API. But what is unique in our onion layers of abstraction? Every representation of a domain object can differ slightly depending user context, adaptive risk systems for instance can change the quality or amount of data, a license might allow one user slightly more real-time data than another (e.g. market prices), a channel might require certain masking or filtering for regulatory reasons, and the order of fields might change for streaming. We can have UID's for all of those, but that's hard. Really hard.

This means there has to be something like a session as omni-channel UI state, mapped for every user, with all contextual settings applied to resource representations. All of that linked to real UID's and versions, and the database, linked to their MVCC. Doesn't that mean we just go back to Lotus Notes style of DB-backed applications, like Firebase, CouchDB, Cockroach or Ethereum promoting it? Alternatively, some API Management systems come with something like that, a distributed Session store. Both are an awkward hack to achieve a semi-time-freeze. Our systems deal with the real world now, there is no time freeze, and no single channel. We cannot just define a fixed set of context, between which we map when we like. We cannot apply a central planning approach anymore, where we dictate what's right and what's wrong for the user.

In architecture, Constant’s “New Babylon” is based on the concepts of sublime moments, Situations, or what we would nowadays call Emergence. Constant describes it as:

“I see new Babylon as a web […] a network covering the whole world”

Which sounds a lot like Ersilia or the Internet. If we really want the internet to span the world, to make it a calm technology, we need to build APIs that are as simple and flexible and beautiful as the real world. I like the word situations better than context, we have to build situational awareness into our API's. We need to build situative API's - and they might not be RESTful.

*) I know REST purists would take the word "document" almost as an insult, and I apologise for this. But the fact that representations are always a fixed artifact and state is passed like a token makes this the best metaphor for me