Offline First

In temple architecture, the main room stands at a considerable distance from the garden; so dilute is the light there that no matter what the season, on fair days or cloudy, morning, midday, or evening, the pale, white glow scarcely varies. And the shadows at the interstices of the ribs seem strangely immobile, as if dust collected in the corners had become a part of the paper itself […] where dark and light are indistinguishable

Jun'ichirō Tanizaki, In Praise of Shadows, 1977

Hoodie.io, an awesome web app framework, started the “Offline First” initiative.

It struck me for two reasons: First, it’s the right way to go and I have been looking into Meteor or Rendr/Backbone for some time. In a world, where services converge across technologies, platforms and ecosystems, technology needs to go a similar way (see my W-Jax talk). Seconds, I was quite surprised the let’s-code-all-startups-in-javascript-scene is realizing that! I’ve had my troubles following all the JavaScript developments of recent years (with the exception of Node) because their use cases simply didn’t make too much sense for me, it all seemed to be exclusively fitted for kinda-social-cloud-powered-minimalistic-apps. Seeing this attitude changing makes me feel optimistic and encouraged.

Until recently, the web app world was like “The West” in Tanizaki’s book: Always perfect, brightly lit, accepting no natural error, resilience was only required in certain places like hardware. Tanizaki praises the shadow, and more the diffuse actually, as a juxtaposition to the always-bright, the efficient and the productive. In building resilient, adaptive, “Offline first” systems we now embrace the shadows, the connection problems, the version conflicts, the deadlocks. And more, we embrace the diffuse, the not-so-clear situations to be identified, auto-corrected or escalated to the user. This means we value adaptability over usability in some areas, giving the user control (if she wants or not) over the process. This means we have work closer with users, design better services, fail gracefully.

As an enterprise guy, where mobile applications have been in place for some time, my projects have always been “offline first”, i.e. “complex” rather than “complicated”. It’s the reason I looked into CouchDB for mobile sync. Even when I built applications mixed with WebViews and native views, this very distinction was made taking into consideration the connectivity context. Connectivity is a part of responsive, or adaptive, or reactive, design. And therefore a part of flow-oriented, resilient service systems that can adapt dynamically*. 

Or, as Ethan puts it, with a nice reference to How Buildings Learn and the aging process: “Shearing Layers”, which means acknowledging the gaps, designing for change where the system is exposed to natural forces and “invite our users into some of these problems” i.e. move conflict resolution to the user, and make visible the complexity - with a suggestion system at work.

Or, in Tanizaki’s words:

The quality that we call beauty […] must always grow from the realities of life, and our ancestors, forced to live in dark rooms, presently came to discover beauty in shadows

*) If you read my blog regularly you know I am not a fan of cybernetics the way Sussna is proposing it as an automated solution to cope with complexity. I like to quote Steve Jobs: “people don't know what they want until you show it to them”. However, I was happy Sussna did the JAX keynote, agreeing with him almost entirely, I fully agree to his Manifesto.

The application server is dead

Jonathan Harclerode and me gave the W-Jax 2012 Keynote, and have been giving it a few times since. Now, one year later, I'll write down the transcript in it's most recent version (for OpenSlava).

Optimism vs. Pessimism

Recently I've read a nice maxim by Bill Clinton "Every person has an optimist and a pessimist side. The trick is to be optimist on bad days and pessimist on good days".

In the technology world, we still live up to the optimism of the 50ies, it's considered anti-innovative, even anti-social, when concerns are brought up - unless you directly describe how you solved them. This begins with interviewers looking for  10x, Ninja and Rockstar programmers, assuming agile processes will work more productive amongst superstars, born with success and hyper-productive. Or people argue resilience is a bunch of fault-tolerant servers and session state in a distributed database that will make sure transactional safety for us (we don't care how), or delegating it to a messaging system that will encapsulate flow from us (we don't care how), or a hardware load balancer that will normalize access (we don't care how). But it's not relying on superpowers that creates successul networked systems. Evolution has told us: it's communication, the best communicating group of the fittest will win.

We're only at the very beginning of understanding complexity and peer-to-peer-networks. The military has a long history in understanding complex networked systems involving humans and therefore error. Looking at the current discussion around drones the problem does not seem to be ultimately solved. For our systems to achieve a kind of "Byzantine fault tolerance" against infrastructure errors, we need a new approach to concurrency and consistency. Once we include business-critical processes in distributed systems, we need to make sure information is valid - in the eye of the beholder. Usually, transaction safety is considered ubiquitous, but we need systems more like MVCC, that gives one user the impression of consistency, yet allowing concurrent modifications in limited scope. True Resilience is about having the user chose between optimistic and pessimistic locking, and providing a relevant consistency context for the user, hyper-consistency if you want. In military, logistics is key. In the same way, having access to trusted, coherent, resources, is key in our distributed systems.

Seek out feedback and learn from your mistakes as you go along

At the end of the tunnel, there is always light; It just might be a train.  

The Streets, Going through hell

The headline is a quote from the Palchinsky Principles, via Gojko Adzic's worthwhile talk "Make impacts, not software" (via @jbandi).

In May I visited the Lötschberg-Basistunnel, a recently completed 35km tunnel through the Swiss Alps. I learned two amazing things from the guide:

1. Most complex tunnel drills stay in the tunnel. It would be too expensive to get out of the tunnel (the costly heads are unmounted sometimes) – because they start plastering the inner shells right after it.
2. In the beginning it’s not really clear where the tunnel drill will take its way. As the head moves forward, rock formations can require changes in the trajectory.

Gojko Adzic merges Design Thinking, SMART objectives, architecture assumptions documentation and business case calculation into “Impact Mapping”, singing the old hymn about analysis-paralysis and Water-Scrum-Fall. He claims that pre-planned software development “it's not a road, it's a tunnel”; rather than sticking to a linear backlog, project should have “something like a GPS”, a vision. Funnily that's how, in fact, tunnels are built. Roads on the other hand often are grids. It's not the question of a tunnel or a road - the distance makes the difference. A straight planned product with tons of features in a neat grid will not enhance usability. Creating a vision – an impact that everyone understands – will.  Being able to define a long-term goal allows for adjustment. Rather than just to “divide and conquer”, this is the art of the product owner.

As someone who moved from startups and design/advertising into traditional consulting I am amused by the now-mature generation of Web 2.0 firms which start to look into the very same traditional processes: product planning, sticking to delivery, definition of requirements which are backed by business cases, business process management and simple interfaces. It's good to see that the mechanic and the organic approaches to software development finally merge, creating a mature, objective and tolerant way of building technology that helps the people using it. 

Something between Micro Services and ROA

A post from July 2013 on Microservices I never finished but now, in 2014, like to reference. It's unchanged, so take it with care and read Fowler's and Richardson's stuff first.

I've read two interesting articles recently one on Divide and Conquer and another one on Simplicity*. Both advocate a new pattern of maximum decoupled services called Micro Service Architecture (by Fred George).

...

This reminds me of NoFlo and Mathematical Visualizations and Abstractions. The strength of Micro Services is not the Continuous Deployment or Decoupling but enforcing to think about organization and the connections of functionalities, because one cannot simply tangle them.

...

However, Micro Services are still services, which means verbs. If they would be nouns, there would be no need to manage tangling because the complexity would already be in the hypermedia.

Maybe something in between is the future of Software Engineering, where Data is managed in REST but logic as Micro Services can be moved between the tiers, as horizontal view of vertical data.

....

*) The solution to this within Yammer is DropWizard, an approach going in the direction of Cell Architecture, with Autodeploy features added and a noun-based approach. A little bit like the Java answer to the 12 Factor App and Docker but, in my opinion, too much of a data expose layer with some PaaS candy.