Thursday 29 November 2018

An annotated Philosophy of Software Design

We build our computer (systems) the way we build our cities:
over time, without a plan, on top of ruins
Ellen Ullman

After much hype I've read John Ousterhout's A Philosophy of Software Design which he uses to teach his course at Stanford and presents as a personal experience "opinion piece". Its basic goal seems to be to develop an awareness and intuition about how and when to manage complexity emerging out of proper problem decomposition in Software Design.

Undergraduate students thus seem to be the main audience, which explains why, despite the title, architecture or non-abstract large system design, distributed systems, developer workflow and design thinking are not covered much, and why a prosaic, almost aphorismic writing style were chosen. My assumption is this is  also why barely any references or annotations are given. It's supposed to be the beginning of an iterative journey, similar to language learning. I figured it might help to share mine, though, to go the next step in that journey (Github would feel too official, so blog):

Preface, Iterative process to this book. Exercises in Programming Style 2014Beautiful Code, 2008Software Craftsmanship 2001 (Craftsperson) and before that The Pragmatic Programmer 1999 and even Programming Pearls 1986 developed iterative, small-wisdom, later developed into Kata's, based learning of software design.


Saturday 28 April 2018

Meshup

After a recent conference I had some good questions and discussions about the current state of services meshes (Istio in this case), thus decided to note down what I find interesting about them.

I'm a fan of infrastructure-level (polyglot) service meshes. The premise excites me as much as Android ten years ago. Working over those years with SOA, EDA, REST Hypermedia API's, API Management, Microservices and language-bound Services Meshes (Netflix / Pivotal, Lightbend's Reactive), I tend to be careful about the resolution of their promises though.

This post is about 3 lesser known effects of service meshes. My last post already covered complexity, emergence and observability in a more general way so I'll limit those topics.

Reasoning


Horizontal scalability in the multi-core age is one of the main arguments behind basically all modern stateless software architectures. Consistency in distributed systems, immutability and state-handling are often mentioned common properties, typically used to justify functional programming paradigms. To me it seems though, their most important common property is a complex, emergent network graph structure. Layers and tiers cannot represent contemporary systems anymore. Those systems have a complex adaptive graph structure in space (infrastructure, users, component interactions) and time (versioning / one codebaserainbow releases, experiments, DevOps, event order, routing).

We could observe a lot of frameworks in the last years quietly move towards declarative graph alterations. The first I remember were Android Intents and Puppet. But more recently it was ReactTensorflowBeamKubernetesEve (RIP) not to forget the re-emergence of SQL combined with flexible consistency models and stream processing.

All of those come with a robust, well-defined domain vocabulary and set of patterns that allows to precisely define desired behaviour. A graph encourages modularization and reuse, it allows for division of labor: Better specialization while making the overall concept better understandable. This, in turn, allows a wider, more diverse group of people to reason and converse about the behaviour of the system. The shared language and culture may hopefully enable them to learn alongside the system, what Nora Bateson calls "symmathesy". It requires all actors in the system to define goals and dependencies, versioned together in one codebase across layers and components, documentation, test (spec), customer support feedback and architectural decisions. That's why all good (micro) service-architecture principles contain continuous delivery and lean.


The biggest difference between those graph-based declarative approaches and Model-Driven concepts (MDD/MDA) is that they are bottom-up, and designed to support evolution*. Instead of requiring a canonical model, tribal ("bounded") domain language or strict interface contracts externally, it is very easy to implement domain-event messaging on the infrastructure level, because the infrastructure itself has meaning, allowing for independently composed distributed systems - in other words choreographed rather than orchestrated.

The declarative, domain-event-driven approach shares some advantage of MDA though: The vocabulary, patterns, and visible graph of dependencies. It makes it a lot easier to follow, though, and a lot harder to ignore. Once the implications of changes to the graphs are commonly understood, it's a lot easier to reason about the graph (see, for instance, the original Flume paper). On the low level that service meshes target, the infrastructure level, this quality makes it a lot easier to reason and iteratively learn the entire system (including the mesh itself and the DevOps process around), and to version, document, track and test the system.