Sunday 29 December 2013

On Flow-Based Programming

A late answer to Daniel's Post "The Future of Computing"

Since some time I am intrigued by Flow-Based Programming (FBP) programming. It feels natural to me because I've worked with vvvv back in university, doing some interactive work, early on with Node and Erlang, and share a general interest on the life cycle of systems, flow in architecture and how to bring process and flow into model versioning.

Daniel points out the social aspect to FBP when he links it to Bret Victor's "Future of Programming" arguing we should (I paraphrase) bring computing technologies to people rather than people to technologies. The way Alex puts it is a bit more extreme: "People shouldn't have to learn to code to apply software to these problems". The question is not technology, or how we present technology. There will always be professional engineers that like to use vim, and there will always be people who like to learn the quirks despite their original profession. And technology has already become a lot more accessible, easier to learn and distribute. On the other hand, however, people don't want to think abstract, externalizing their creativity into a formal language.

In Code, Georg Vrachliotis tells the story of how Architecture scholars reacted to the rise of the computer. From the Architecture Machine and the 1970s statement "We want to arrange matters so that the computer can be used as naturally and easily as a pencil" all the way up to the contemporary view of code and architecture cross-pollinating each other, at the section of programming a drawing, with technologies like CAAD - much like visual programming or FBP.

We can't bring technologies to people, neither can we bring people to technology. We can create, however, a new space that provides the tools for people to pick up and decide how much they want to engage with technology. We have to build systems and technology that is both easy and versatile, comfortable and challenging, stable and fluid. By doing so we avoid the System design trap - the more we engineer, the less natural a system feels. It reminds me of one of my favorite quotes from John Gall:
Trying to design a System in the hope that the System will somehow solve the Problem, rather than simply solving the Problem in the first place, is to present oneself with two problems in place of one

UPDATE:  On the discussion of common sense and using Nudge as a way of "Utopian Realism" I like to add the beautiful end of the Manifest for a New Realism:
Enlightenment still calls for a choice of position  and faith in mankind, in knowledge and in progress [...] What are needed are knowledge, truth and reality. Failing to accept them [...] means pursuing the ever-open alternative proposed by Dostoevsky’s Grand Inquisitor: the path of miracles, mystery and authority. 

Friday 27 December 2013

E-Books and Tablets are so 2013.

In Autumn I decided to switch over to use a tablet  (Nexus 7, maybe part of the problem) exclusively for private work. At the same time I decided to use E-Books, mainly because I had read so many bad books I threw away that keeping paper didn't make much sense anymore.

From Jan 1, 2014 on, I will switch back to a laptop and paper books for the bulk of my work. Here's why.

Why I dropped the Tablet (pun intended):

Main reason: I multitask. I know it's not efficient. Probably it's same kind of boredom that drives me to it, as I am certainly not suffering from ADHS. Unfortunately I am not a stream person, either. Tablets are made for streams, executed one by one, like packet switching or serial monogamy. They are built for either technology agnostics (think iPad for seniors) or very organized cloud-only stream-switchers (think Ben). Sure, some solutions like Cover try to address this with contexts, but it's still single task.

Having more than one E-Mail open? More than one browser instance? Switching between a Spreadsheet and a PDF staying at exact the same spot? Want to write an article with references (leaving aside there is not a single good quote-taking app out there)? Undo after you accidentally deleted all text? All that is currently impossible with the current state of most Android and iOS Apps. It gets worse with offline or limited connectivity (Hybrid Apps just crashing, Google Maps stays blank), where it's literally impossible to do research over more than a few browser tabs. Add technical problems (no ESC key when Apps freeze, Android updating in background just when you need to show your plane ticket) and questionable product decisions (Apple disabling USB Sync) it destroys all the productivity added in the first place.

I was positively surprised with the text input (thanks to an IVSO keyboard and Markup editors), on-screen note-taking, reader capabilities and TV connectivity (ChromeCast). Really, I loved the Nexus. The problem is though, it only works 90% and drives you crazy the other 10%. It's those 10% that tests usually not mention but made me unpack my 5-year old PC and get workin' again. Efficiently.

As for E-Books:

Main reason: No experience. I do read for both pleasure and knowledge extension, but the latter I primarily do via Readers like Pocket. A book is usually a long text I want to work with, i.e. a text I need to understand, cross-reference, highlight, read many times. All this is extremely cumbersome, at least with Kindle (I did only try Google Books as an alternative but found almost no books I am interested in. Same by the way for movies (less than on a regular plane flight) which one cannot even rent in English).

The Kindle app does not have a central place to look through your notes or highlights (apparently in the US store there is "my Highlights and Notes"). It does not allow multiple page markers or general sidenotes like a quick pencil brush. It's, like the tablet, for grandma reading, not for working.

But even with longer novels it's not pleasure. There is only a very rough feeling of process as the bar and remaining time are based on tags (for chapters) apparently randomly distributed through the books I've read. The process bar shows a general process even if there is lots of advertising and errata in the book - which also destroys the nice experience of finishing a book. I had to scroll though some multiple times until the index showed 100%, a few will probably never make it over 99%. Reading multiple books in parallel (yes, I multitask) is annoying because they all show up, randomly ordered and no grouping.

In addition, you cannot rent or nicely give an E-Book present. A voucher for Christmas, really? Why can't I transfer it in virtual gift wrap? Why is there no way to transfer a read book with a nice personal ex-libris note? That probably put the final nail in the coffin.

Sorry slick technologies, at the moment you just annoy me.

Inspired by XCKD 1309



Saturday 9 November 2013

Offline First

In temple architecture, the main room stands at a considerable distance from the garden; so dilute is the light there that no matter what the season, on fair days or cloudy, morning, midday, or evening, the pale, white glow scarcely varies. And the shadows at the interstices of the ribs seem strangely immobile, as if dust collected in the corners had become a part of the paper itself […] where dark and light are indistinguishable
Jun'ichirō Tanizaki, In Praise of Shadows, 1977

Hoodie.io, an awesome web app framework, started the “Offline First” initiative.

It struck me for two reasons: First, it’s the right way to go and I have been looking into Meteor or Rendr/Backbone for some time. In a world, where services converge across technologies, platforms and ecosystems, technology needs to go a similar way (see my W-Jax talk). Seconds, I was quite surprised the let’s-code-all-startups-in-javascript-scene is realizing that! I’ve had my troubles following all the JavaScript developments of recent years (with the exception of Node) because their use cases simply didn’t make too much sense for me, it all seemed to be exclusively fitted for kinda-social-cloud-powered-minimalistic-apps. Seeing this attitude changing makes me feel optimistic and encouraged.

Until recently, the web app world was like “The West” in Tanizaki’s book: Always perfect, brightly lit, accepting no natural error, resilience was only required in certain places like hardware. Tanizaki praises the shadow, and more the diffuse actually, as a juxtaposition to the always-bright, the efficient and the productive. In building resilient, adaptive, “Offline first” systems we now embrace the shadows, the connection problems, the version conflicts, the deadlocks. And more, we embrace the diffuse, the not-so-clear situations to be identified, auto-corrected or escalated to the user. This means we value adaptability over usability in some areas, giving the user control (if she wants or not) over the process. This means we have work closer with users, design better services, fail gracefully.

As an enterprise guy, where mobile applications have been in place for some time, my projects have always been “offline first”, i.e. “complex” rather than “complicated”. It’s the reason I looked into CouchDB for mobile sync. Even when I built applications mixed with WebViews and native views, this very distinction was made taking into consideration the connectivity context. Connectivity is a part of responsive, or adaptive, or reactive, design. And therefore a part of flow-oriented, resilient service systems that can adapt dynamically*. 

Or, as Ethan puts it, with a nice reference to How Buildings Learn and the aging process: “Shearing Layers”, which means acknowledging the gaps, designing for change where the system is exposed to natural forces and “invite our users into some of these problems” i.e. move conflict resolution to the user, and make visible the complexity - with a suggestion system at work.

Or, in Tanizaki’s words:
The quality that we call beauty […] must always grow from the realities of life, and our ancestors, forced to live in dark rooms, presently came to discover beauty in shadows

*) If you read my blog regularly you know I am not a fan of cybernetics the way Sussna is proposing it as an automated solution to cope with complexity. I like to quote Steve Jobs: “people don't know what they want until you show it to them”. However, I was happy Sussna did the JAX keynote, agreeing with him almost entirely, I fully agree to his Manifesto.


Monday 21 October 2013

The application server is dead

Jonathan Harclerode and me gave the W-Jax 2012 Keynote, and have been giving it a few times since. Now, one year later, I'll write down the transcript in it's most recent version (for OpenSlava).

Sunday 18 August 2013

Optimism vs. Pessimism

Recently I've read a nice maxim by Bill Clinton "Every person has an optimist and a pessimist side. The trick is to be optimist on bad days and pessimist on good days".

In the technology world, we still live up to the optimism of the 50ies, it's considered anti-innovative, even anti-social, when concerns are brought up - unless you directly describe how you solved them. This begins with interviewers looking for  10x, Ninja and Rockstar programmers, assuming agile processes will work more productive amongst superstars, born with success and hyper-productive. Or people argue resilience is a bunch of fault-tolerant servers and session state in a distributed database that will make sure transactional safety for us (we don't care how), or delegating it to a messaging system that will encapsulate flow from us (we don't care how), or a hardware load balancer that will normalize access (we don't care how). But it's not relying on superpowers that creates successul networked systems. Evolution has told us: it's communication, the best communicating group of the fittest will win.

We're only at the very beginning of understanding complexity and peer-to-peer-networks. The military has a long history in understanding complex networked systems involving humans and therefore error. Looking at the current discussion around drones the problem does not seem to be ultimately solved. For our systems to achieve a kind of "Byzantine fault tolerance" against infrastructure errors, we need a new approach to concurrency and consistency. Once we include business-critical processes in distributed systems, we need to make sure information is valid - in the eye of the beholder. Usually, transaction safety is considered ubiquitous, but we need systems more like MVCC, that gives one user the impression of consistency, yet allowing concurrent modifications in limited scope. True Resilience is about having the user chose between optimistic and pessimistic locking, and providing a relevant consistency context for the user, hyper-consistency if you want. In military, logistics is key. In the same way, having access to trusted, coherent, resources, is key in our distributed systems.

Sunday 21 July 2013

Seek out feedback and learn from your mistakes as you go along

At the end of the tunnel, there is always light; It just might be a train.  
The Streets, Going through hell

The headline is a quote from the Palchinsky Principles, via Gojko Adzic's worthwhile talk "Make impacts, not software" (via @jbandi).

In May I visited the Lötschberg-Basistunnel, a recently completed 35km tunnel through the Swiss Alps. I learned two amazing things from the guide:
  1. Most complex tunnel drills stay in the tunnel. It would be too expensive to get out of the tunnel (the costly heads are unmounted sometimes) – because they start plastering the inner shells right after it.
  2. In the beginning it’s not really clear where the tunnel drill will take its way. As the head moves forward, rock formations can require changes in the trajectory.

Gojko Adzic merges Design Thinking, SMART objectives, architecture assumptions documentation and business case calculation into “Impact Mapping”, singing the old hymn about analysis-paralysis and Water-Scrum-Fall. He claims that pre-planned software development “it's not a road, it's a tunnel”; rather than sticking to a linear backlog, project should have “something like a GPS”, a vision. Funnily that's how, in fact, tunnels are built. Roads on the other hand often are grids. It's not the question of a tunnel or a road - the distance makes the difference. A straight planned product with tons of features in a neat grid will not enhance usability. Creating a vision – an impact that everyone understands – will.  Being able to define a long-term goal allows for adjustment. Rather than just to “divide and conquer”, this is the art of the product owner.

As someone who moved from startups and design/advertising into traditional consulting I am amused by the now-mature generation of Web 2.0 firms which start to look into the very same traditional processes: product planning, sticking to delivery, definition of requirements which are backed by business cases, business process management and simple interfaces. It's good to see that the mechanic and the organic approaches to software development finally merge, creating a mature, objective and tolerant way of building technology that helps the people using it

Something between Micro Services and ROA

A post from July 2013 on Microservices I never finished but now, in 2014, like to reference. It's unchanged, so take it with care and read Fowler's and Richardson's stuff first.

I've read two interesting articles recently one on Divide and Conquer and another one on Simplicity*. Both advocate a new pattern of maximum decoupled services called Micro Service Architecture (by Fred George).

...

This reminds me of NoFlo and Mathematical Visualizations and Abstractions. The strength of Micro Services is not the Continuous Deployment or Decoupling but enforcing to think about organization and the connections of functionalities, because one cannot simply tangle them.

...

However, Micro Services are still services, which means verbs. If they would be nouns, there would be no need to manage tangling because the complexity would already be in the hypermedia.

Maybe something in between is the future of Software Engineering, where Data is managed in REST but logic as Micro Services can be moved between the tiers, as horizontal view of vertical data.

....

*) The solution to this within Yammer is DropWizard, an approach going in the direction of Cell Architecture, with Autodeploy features added and a noun-based approach. A little bit like the Java answer to the 12 Factor App and Docker but, in my opinion, too much of a data expose layer with some PaaS candy.

Sunday 26 May 2013

Design Thinking for Ecosystems

Bret Victor’s talk about “Drawing Dynamic Visualizations” (via Stilkov) inspired me to go a little further on design thinking. Not in particular because of his amazing visualizations but rather his quote of Alan Kay referencing Jaques Hadamard, who did a survey amongst great mathematicians what kind of tools they use.  In this study Hadamard found that only a few of them “claimed to use mathematical symbology at all […] they did it mostly in imagery or figurative terms”.

Alan Kay in Bret Victor's Talk

I am sometimes surprised to see hardcode-science-educated colleagues struggle when I apply techniques and methods learnt in design and architecture school. Sometimes even simple brainstorming seems to disqualify me as proper nerd peer. This also applies to systems or enterprise architecture when I draw systems in the size according to the importance of their interfaces on the whiteboard, print out code to show its asymmetries or refuse to include technical layering in early system drafts (in the end it’s all indirection).

Design Thinking is everything but new – d.school has shaping it for a while. But via the organizational implications mobile is putting upon enterprises, it’s entering the world of technology architecture rapidly. Agencies like R/GA, Frog or recently-Accenture-acquired Fjord take giant steps towards the convergence of innovative service and product design with technology. What they call “functional integration” is often coined an ecosystem in the technology world. It gets more interesting though if we ask why this integration has not reached all the platforms we know of. Apple is usually considered one of the best ecosystems – yet, its technical platforms are surprisingly heterogeneous and task-based, you could argue skeuomorphic in the way they act. Android and Windows 8 are much more “flow-based”, trying to support an externally planned long-running activity through multiple channels. In a way, they are more technical – but then again, in the long-term, they might very well bring the benefits of good design: a product that is unobtrusive, long-lasting, essential and pure.

Our architectures must become like good design, like an unobtrusive ecosystem that shaped the components living in it. As Triarchy writes:
“Deleuze and Guattari talking about the structural and genetic aspects of organisations and the need to consider the shape of the formed organisation on the structural plane as well as the evolution of the formed organisation on the genetic plane. This echoes the debate about whether to prioritise structures or processes in management. For D and G, it's necessary to do both.”

Or as Bjarne Stroustrip, again quoted by Bret Victor, said: “A lot of thinking about software development is focused on the group, the team, the company”. Evolution, i.e. predators, are important in emergent, especially in agile, iterative systems. However, systems don’t end in themselves; they need to fulfill a purpose. In a world where all kinds of systems are possible, design thinking can find this purpose.

UPDATE: A good write up on why design thinking is a discipline, not a process: http://www.fastcodesign.com/1663558/design-thinking-is-a-failed-experiment-so-whats-next

Thursday 25 April 2013

Ätzende Architekturmetaphern

On JAX 2013 I gave a talk "Ätzende Architekturmetaphern". The main thesis was that software architecture has arrived at a point very similar to one in the history of urbanism, where two main streams merged. The one coming from the tradition of society, the "organic" one. The other coming from the machine age and functionalism, the "mechanical one". Via a short detour over emergence, a new kind of theory materialized - what Vidler called "the third typology". It lead to a new approach of "bottom-up" research, cumulating in the analysis of informal architectures, i.e. slums. This new way of critical theory could very well apply to the current state of our profession, where we are looking for a way to include resilience, rhizomatic structures, information flows, evolutionary design and agility into our architecture.



As Prezi does not feature longer transcripts, this is the reading list (received on April 25th 2013):





Sunday 31 March 2013

Functional architecture


An addition to „Alphabet and Algorithm“: Mario Carpo goes on, telling how Giedion looked at repetitive tasks (and eventually the Japanese tea ceremony) as a precursor for our technology. In “Rethinking technology” I have once read the marvelous story of Oliver Evans and his magic machines. It reminded me a lot of the autocracy in “Walden” or “Just enough architecture”. Giedion was one of the ideologists behind CIAM, postulating separated districts and as means to achieve at a new architecture of “human scale”, a new humanism through the “functional city”. This made me think on Japanese cities and its structures that emerge out of simple rules, almost like A New Kind Of Science. If we could establish as simple rules and boundaries we could govern systems as efficient and elegant. Creating a functional architecture that actually matches organizational systems. If this would be possible we might eventually forget about technology.

Looking forward to read “Architectural Principles in the Age of Cybernetics” which I found researching for some keywords. It connects Giedion’s work to the Cyborg Manifesto – something I have become curious about again after watching “Real Humans”.

Saturday 23 March 2013

Architecture after art


In his “six memos for the next millennium”, Calvino quotes an introduction to the Chomsky/Piaget debate which lists two metaphors for the emergent development of life forms: The crystal, representing invariance, and the flame, representing stability in motion. The two, he claims, converge in the metaphor of the city: “The invisible cities … a singular symbol concentrating all my reflections, experiences and hypotheses”.

Ersilia, the most cited invisible city, standing for internet relations. (C)   Delavega, Ephemera + Lascarr


I am a huge fan of using urbanism as a model for software systems. The examples of using houses to illustrate relationships between components are just too static, representing at best the crystal. Often, leaky abstractions are drawn (like decorators to ornaments), adding no real value to the discussion.

One could argue that even the metaphor of a city is incorrect and we should rather rely on pure emergence within benevolent dictatorship. In my opinion this would just be the flame* (Not Nero’s hopefully). Viktor uses the Bonsai as a model for software architecture. Small decisions lead to a final picture – what Smolander calls the decision process in his four disciplines of architecture (the others being the blueprint, the literature and the language). I’d like to be free in the decisions, like the flame, but limited in its dimensions, like the crystal. What I like about the Bonsai model though, its reference to Ikebana, in Garr Reynolds words: “Empty space is as important as the positive elements […] Space allows other elements to […] connect”. Japanese cities are built differently from European ones. More central planning is applied, districts are clearly separated, but on the other hand the city is built in harmony with nature. Tatami mats are the traditionally used not only to cover Japanese floors but to actually define the floor plan. Japanese cities are complex patterns, defined by simple rules for spaces and floor plans. Applied to large software systems this would mean we take into consideration environmental qualities and turn them into simple rules, like Haiku poetry. Or, as Calvino says “Poetry is the great enemy of chance”.

Recently at the Prado it was fascinating to see how the style, e.g. of Tizian, of paintings was preceded by renaissance architecture. The other day it struck me in an exhibition on Paul Klee’s influence on Japanese architecture. Buildings like the Sendai Mediatheque reference his paintings in their dynamic structure – both the flame and the crystal. There is no reason why architecture cannot come out of art, it does not always have to be the other way around.

*) Interestingly, the Netherlands are very well known for planning cities top-down. In this context of the blog post it would make perfect sense to discard the metaphor.

Wednesday 27 February 2013

An architecture of predators and monuments

Mario Carpo’s, in his essay “Alphabet and Algorithm”, tells the story how Alberti invented the modern architect. He managed to invent plans which allowed the architect to stay away from the actual work, concentrate on the plan, yet keep all the fame. All of this in the best intention to foster collaboration and nurture the crafts. Many software (and particularly enterprise) architects see themselves in this tradition - unfortunately only the planning part. But Carpo goes on and explains how modern algorithms sorta turned the Renaissance around, bringing back participatory authorship into real-world architecture. While identity and idiosyncrasy become less important, the remix, versatility and sustainability in terms of longevity come back to architecture.

The rupture of individualism in architecture can be linked to the megastructure movements of the late 60ies. As Annette Urban points out in “Mythos Monument”, the very concepts which wanted to turn cities into moveable, flexible, flowing structures became the megastructures par excellance. By neglecting centralization, the city planners effectively turned themselves and their structures into the center. When Haussmann tore apart the arrondissements of Paris to allow industrialization and military vantage, rather than introducing the “service bus”, he froze the versatility. In realization that any structure is totalitarian or at least aristocracy, and thus can never provide actual needs for citizens, Architects like Superstudio began ironically playing with monuments and city limits. From them we learn that planning always ends in itself.


Courtesy: Archivio Superstudio, Firenze via Megastructure Reloaded

However, this reminds me of a story from Emergence. When W. Daniel Hillis's self-emerging AI algorithms showed signs of perfect structures, but less-than-optimal local maxima, he introduced predators. By doing so, he encouraged the algorithms to develop in the right direction, becoming stronger, and eventually near-optimal. The lesson for architects is: Become a predator, one that collaborates just like Alberti did.

Sunday 20 January 2013

Resilience, responsive design and vertical integration

Jason Grigsby had an excellent post recently on responsive design for mobile apps. In short he says that one always need to design in a responsive way, because the lines between screen sizes, input models and form factors are so blurry that, in fact, it's impossible to know how the user expects your app to work.

I like to put this in context with mixed model mobile apps (Frameworks like Calatrava) and the new trend infrastructure trend of "vertical integration", i.e. expand cell architectures into the hardware domain.

BlackBerry 10 is a new type of platform which is streamlined (managing user experience) and open at the same time. It allows polyglot programming, hence not only mixed model apps but a cross-tier, cross-device integration of components. You could have a database model in Java, covering almost all platforms. A UI which is responsivly designed for each platform, and business logic which is in a DSL. The latter is most imporant because, with vertical integration, it could be aware of the consistency models offered by the backend. Independent of data storage, the business layer could offer several consistency models. If one or the other model is favoured, only this kind of hardware could be scaled. Just like with Facebook's photo storage, all parameters can be negotiated by both server and client. Which will eventually lead to emergent and resilient systems which will be able to run for longer than most platforms.

These cross-device, cross-tier software systems work like swars. The benevolent dictator nudges the swarm, bringing in new systems, monitoring both user experience and infrastructure efficiency. Now the question would be - how to control? Agile software development aims toward one product, this is clearly not  enough. So how would be build, distribute and manage these kind of systems? How would we manage the API's to connect them, the directories to find the services and the resilience to learn from our mistakes?