Over the last 5 or more years I've had kind of an abstinence from conferences and software architecture books. Industry focus was on Cloud, Serverless and ML, leaving system design stalling, with the occasional, rare exception (KNative, Learned Structures, ISA and Simon's ongoing quest for explainability come to mind).
Conference speakers still explain Agile, DevOps, ADR's, EDA and Resilience, while people still pile up tech debt and big balls of mud, just now using Serverless or Kubernetes. The 20 year anniversary edition of The Pragmatic Programmer, which I hold in my hands, says it is “just as relevant today as it was back then”. Given that I saw the limits of Agile, I became more interested in the product operations, SRE side of systems and how observability, explainability, human collaboration and supportability spins system evolution to converge towards simplicity (or not) and builds a community-centric narrative that hopefully enables, in the long term, better socio-technological structures.
Over the last year or so, though, I found myself surprised by an influx of interesting material, in particular in regards to bias, culture, empathy, failure, discovery, resilience (SRE) and risk awareness in orchestration complexity of socio-technical systems.
A welcome change from a long overdue self correction of VC-fueled get-rich-quick startups culture. Concrete examples are my favourite book Accelerate, which inspired a great Böckeler / Fowler talk at Craft Conf and a very entertaining self reflection by Tilkov, and other nice perspectives like George Fairbanks Continuous Design Talk and Nygard thinking about state, Videla's, Wichary's, Steenson's and Ullman's thoughts about weird languages, Design It! (and the 2nd edition of Release It!), but also some really great distributed systems research by Howard and Kleppmann challenging our perspective on concurrency, and new, humble ways to create frameworks.
Empathy for Entropy
It's not simply that Microservices have made microblogging-driven startups suddenly realize the value of BDUF. It rather is the agile move away from Enforcement towards Observation with better tools, empathetic user-centric techniques and ways of thinking about consistency and concurrency. It reminds me of my W-Jax talk 7 years ago, when I first read about Spanner and its formalization of time. When Kleppmann argues for OLEP and Local First, that the queue is the database, it reminds me how Spanner argues that the database is the queue. Both arrive at the same insight: That information and time are entangled, and that entropy or consistency are derived from that. As Howard beautifully analyzed, for our systems it’s easy to replace the concept of time with state transitions (Lamport clocks). This allows us to step away from seeing the system as uncontrollable, and us reactive, to focus on the real, human cause for entropy.Flat Ontology
This has interesting implications for consistency in the domain-driven design sense, because it means that changes in the model (from a business- or user-perspective, designed and desired or not) have just the same, equal impact on the system architecture as-is as production releases, technology choices, bugs, company culture, team composition and societal trends. All of those need to be observed and reacted to, ideally anticipated, in order to keep external consistency to the user, just like gardening over the seasons or wine blending over the years.There is a group of thought experiments called Theseus’ Ship which this art sometimes refers to. The basic idea is that a concept can stay the same even though everything material of if changes - even if every wood plank is replaced, it’s still Theseus’ ship.I was reminded of this when reading the example of the "Dutch East India Trading Company" (follow-up reading on Anarchy) staying the same despite everything around and inside it changing, in a book on Object-Oriented Ontology (OOO), a hip philosophical framework especially in art circles. And just today, reading the 20 year anniversary edition of The Pragmatic Programmer, they look at their own techniques through the lens of Theseus’ Ship.
While I don't agree with the OOO ideology on an everyday basis, particularly because it is highly subjective, while claiming to not be, and often unnecessarily binary, tribal and even hostile* ... so, while I don't agree with that, I do think it is a useful tool to think about Architecture in the System Design sense, because it equalizes objects, as an isolated concept, with events or changes, without a human perspective of time. It reminds me of why I love Erlang, because it is explicit about the observer outside and the actor inside the system, it embraces speculation about potential realities and risks. It opens a conversation everyone, not only programmers, can participate. In that sense, the 20th anniversary of PragProg is in itself an iteration, it is an actual learning on Architecture, not just a new experiment. I tried to explain that feeling clumsily in my book, 6 years ago, using ANT and Conway's Law, but OOO’s idea of a "flat ontology" is just a lot more elegant, and fits into more recent publications like adaptive Team Topologies.
Architecture Astronauts
OOO reminds me of a fierce discussion I had 5 years ago or so with a Domain Architect for an event-sourced system. They had designed an event bus with a canonical data model verified in a metadata master system. Their USP was a mandatory, human multi-stage process to get any metadata change approved, new events required at least one presentation before the architecture board - a monthly committee in a different time zone. Needless to say my approach was the complete opposite, back then Spotify-oriented, defining a set of accepted formats (eg JSON) and a few standard header fields, observing what goes on the bus and reaching out to teams with, for instance, long dead letter queues. One approach was ideological, prone to bias, focused on preserving hierarchy. The other observing, open, user-focused. In other words one was tall, steep (an elevator), the other flat. The ivory tower was what gave architects a bad name, the extreme end of the ivory tower was called Astronauts, who are more excited about defining abstract patterns than making lives of the end user easier.But actual Astronauts see changes in the earth earlier and clearer. They seem above, but actually their world is flat, a whole. They fundamentally care about impact. They leave to conduct experiments, and come back to the ground with new insights. They risk everything using the engineering they’ve helped to build - an engineering that inspires others. Similar to Theseus’ Ship, the stations are constantly changing, constantly accommodating new experiments. Astronauts and missions change, there is no ultimate goal but constantly rebuilding our understanding of the system from a slightly different vantage point, yet still within. 20 years later, I’m on the side of Astronauts, and I think we’d be better engineers if all of us sometimes think about the big impact, the world, the planet.
Seeing things as a whole is not hierarchical, it’s empathy. Looking at emergent patterns of microservice or event communication, network latency, error types, user behaviour, machine learning models or support cases and feedback has nothing to do with building a house or a bridge, but looking at the landscape, the environment, and society. Which is a way of looking at causality similar to what academic history often does, as hierarchies of culturally observable, overlapping effects.
In that sense, as Buckminster-Fuller said, "We are all astronauts". Pragmatic ones.
*) As can be seen in the numerous attacks against others in the book while praising Sci-Arch even though they only use OOO to claim a new star architecture (“Killing simplicity”) that ignores community context
No comments:
Post a Comment