Sunday, 31 May 2020

Observability, Debt or the Bret-Victor-Ization of distributed systems

I've been thinking how the different way of conceptualizing cost (in the broader sense of investment) in Cloud changes the tech debt metaphor. It never was a good metaphor to start with and allowed too many excuses, but I like the idea of expressing a suboptimal, incomplete or leaky level of abstraction somehow, as a dialectical, critical tool. I like declarative systems because they allow comparison of state over time, but they do limit expressiveness of our mental models, and omissions, when writing down code. Debt, with its pseudo-quantifiable touch is such a mental model limit. No one wants to keep ADR's for each of these*. How to solve this?


It's exciting to see how the next step in polyglot programming is taking stacks apart and designating a layer to developer experience for humans based on “progressive disclosure of complexity”, and how we argument for this is feedback time or, in other words, Software Delivery Performance. What a16z recently called The Decade of Design (but combined with craft and lean), the best example probably being Stripe which shows that beautiful API's and documentation and a beautiful website (and beautiful books) might after all correlate, and maybe because they take empathy to their heart (no surprises here for anyone who has seen or used PayPal). 

When I first saw this at Google working with Dremel / F1 and all the Data Mesh tools around it, an ontological, rhizomatic approach to data - without oversight, yet with structure, as side effect of, essentially DDD (or set / category theory, referring to ontology below). And the same when seeing Borg and the Service Mesh around it. Both were built as products with a builder focus, with "a builder" meaning universally anyone who wants to build something, meaning to contribute to a shared idea. When we say everything-as-code we need to go beyond engineering components, we, and that means everyone and the maximum of diverse perspectives, need to look at the product and all of its users. Similar to medical doctors who moved away from seeing "man as machine", or architects seeing the city or the house as a machine, we are slowly moving away from seeing a software system as a machine, perfectly controllable. Technology is not neutral, and a constant process and struggle that goes far beyond engineering.