Rhizomic Timeline

Tumble Tree is the most needed browser plugin ever. Since ages I am looking of a way a browser should me a cross-tab/cross-window long-term history. Imagine this would work for your systems architecture, with all the interfaces and processes. How they emerge, how they fade, how they converge, the whole rhizome - ideally with a time shift feature so you can also plan ahead.

An architecture can emerge.

Recently I had the joy of stumbling over an interesting discussion regarding architectural emergence. It started with a blog post by Igor. A nice and intriguing read that's basically saying that, although every system has an architecture and therefore emerging architectures are possible, they will be of limited complexity due to lack of evolutionary fitness. I like the idea of evolution and that Igor admits emergent architectures are possible. The original post lists a main key constraint to the quality of emergent architectures, though: „Developers do not anticipate upcoming requirements“. Furthermore, it is stated that “natural selection doesn’t think at all”.

Coming from this “reset to zero” (with local changes) idea the author argues that emergent architectures tend to be in higher technical debt than designed ones. Technical debt is not inherently a problem but can influence the system adaptability negatively in a way that it limits possible complexity.

I tend to disagree.

Firstly, I believe there is no lack of vision within development teams. Usually developers know what kind of product (from a high level perspective) is expected as a “goal”.

Secondly, I do not think that evolution is purely random, this argument would leave out evolutionary processes such as co-operative behavior and parent–offspring communication.  In fact, the fittest species are the ones who communicate most efficiently (Hi internet!). Hence I think that, though technical debt might be temporarily higher, communication will usually lead to quick changes. Nowadays we have the possibility to change technology layers or tiers by the means of refactoring and abstraction. In the EJB example of the original post, I don’t see why an upfront design decision to use EJB 2.1 with a change request to EJB 3 would be different from the team realizing that EJB 3 is the way to go and incrementally changing implementation (e.g. by assigning a team to refactoring). The more changes, the more YAGNI.

To some extent, these issues have been addressed by Alex. He is adding Dual Inheritance (Alex knows the terminology much better so forgive me any formal mistakes, please) theory to the formula, adding a bit of pessimism towards the outcome of the mentioned communication. In real life, debt would most likely not be detected and eliminated; it would be covered and not even sanctioned. This is a pretty good point, yet for some reason Alex limits it to myopic agile processes, which weakens it a bit.

Nevertheless, I still disagree.

Agile itself does not mean iterations are industrial or in any way myopic. The mentioned bias towards success is actually a bias towards “not failing”. Innovation can only take place if graceful failure is embraced and considered success, whereas procrastination is considered true failure. In Agile development, not failing can mean multiple things. In most of the cases, a developer (being an engineer) will realize that a technology might lead to not reaching the Sprint or Product goal and suggest adjustments accordingly. In my experience it’s not the developers being risk-averse but the management who would not accept not meeting milestones even if this would mean a better end product would be shipped.

Agile requires feedback: Between the users, product management, operations (read DevOps), project leads and the development team. This is also discussed by Alex in his following post. I very much agree with Alex that lack of feedback is one of the key reasons projects fail. I do, however, disagree that direct feedback stays myopic and does not change the ecosystem or culture around it (which somehow implies that no feedback is no problem). To stay with the metaphor of Evolution, this would ignore changes during the development of an organism, during childhood or via environmental changes.

My arguably optimistic assumption is that every developer wants to have success and deliver a good product. Given that he or she receives sufficient feedback, the developer will strive towards a “good architecture”. Because they want good feedback, interesting projects, reference letters, svn praise and all that in a flat world. In the democratic process which is happening between the developers, and between the teams (it’s good practice to have multiple teams, rather than having “one development team/stream for each system”), questions will arise and possible problems addressed. If you have, however, one architect or project manager designing and making decisions, he or she might indeed act as Alex pointed out: not acceptingchange eventually leading the project to grief, frustration and death. In the worst case, by turning this into a hindsight bias “problem of the developers” and trying an even tougher policy next time. 

In building architecture, an architect is someone who has an overall idea, mainly communicating, motivating and facilitating process change with the end goal in mind. No architect in the real world will not consider experts and listen to them every single day unless the keys are handed over. The beauty of evolution is that it has found to many ways to overcome its own randomness. As adaptability is actual fitness, evolution has found ways to incorporate feedback probably better than we humans do in everday life. A good architect is a feedback-broker who fosters evolutionary processes. Because he or she knows one day the actual world (as in society change, earthquakes or subprime crisis) will hit him hard.

I believe that software architects are still necessary. Instead of having "master architect" attitude, they should have a very similar role to building architects. Sticking to a major plan in one person’s head with the argument that emergence will lead to worse results is the dark side of choice management; it’s giving up adaptability in favor of predictability of failure. Accepting emergence, though, is the flipside. Channeling ideas coming from emergence into architectural decisions, embracing change coming from democracy within the development team and keeping all options open will eventually lead to even higher adaptability, more complex systems and – just by the way – a better life as evolved human beings.

On Mixed Model Mobile Apps

Charlie Kindel recently published "Apps must be cross-platform", an article I mostly agree with. Would only add Cross-Device and a grain of pervasiveness. My 2009 thesis supported the same argument, that cross-platform should neither be agnostic (like HTML5 or Java ME) nor purely generative, it should abstract business logic but embrace user experience. Therefore, I decided to finally publish some of the content (1,2) in concise form (Iteratively, as I am not allowed to publish the thesis itself), looking back.

In Architecture without Architects, Bernard Rudofsky claimed that architects should learn from premodern architectural forms. At IT architects we should learn from our users, embrace user experience. Apply "ilities" not general, but where they belong. Performance and Usability at the presentation tier. Reuse, scalability, maintainability and requirements traceability in the business tier.

Art made by walking. Source: Richard Long

Event-driven web applications

I gave a talk on JAX 2011 on event-driven web applications.

My argument was that AJAX, Push and Mobile Access have changed the way web applications need to work, that latency and efficiency is key now. Put a bit simpler: Where we had documents before, we now have constant interactions. Or, as David Gelernter put it, where we had places (like files) to put something, we know have time-based streams. It is the death of the application state and the rise of the event.

Because HTTP was meant to be structured information, it is stateless. All the software which has been developed for the web, including Java, is therefore built around the transition from stateless to stateful and back. In the future it might very well be that the 3-tier is being replaced by the tiers stateless data (HATEOAS), events and stateful user interface. This goes along the lines of SOFEA, MVVM, CQRS or the LMAX architecture.

Next generation non-blocking I/O frameworks like Node.js (or nowadays (this post is written in 2012) Vert.x) will allow purely agent and event based asynchronous behavior. This does not mean it must be purely on the client (like Yahoo Manhatten showed us) or even HTTP, but that tiers are going to be even more loosely coupled than we can imagine, and that API design will become key.

I would like to hear your ideas on that, whether we will go down that road or rather the other directions with technologies such as functional programming, actors (Erlang) or even Software Transactional Memory such as Akka (or Play).

A pattern language for emergence in systems architecture

(Disclaimer: Published 2012, Written 2009, from Thesis handed in 2010)

To create architecture is to put in order.

Put what in order? Functions and objects.

Le Corbusier’s quote from his Precisions No. 68 could be taken out of any software engineering book. Apparently there is a certain common sense between natural and digital architecture. Two of my thesis’ most important components are Design Patterns and Web Services. What do their prophets[1], the “Gang of Four”[2] and Roy Fielding, have in common?

Both reference the architecture theorist Christopher Alexander in their most important works. In doing so, they reference building architecture as important source of inspiration.

Referencing Alexander is a practice used by a majority of software engineering evangelists. As the name defined by the NATO suggests, software creation is interpreted rather as engineering process as opposed to a creative process (Pree 1995, p. 3). Control over software, like a construction engineer has control over a building, is the divine aim (Barbacci 1998, Kruchten et al. 2006). This implies a taxonomic approach[3] where all knowledge is inherent – a view of the world that goes back to the enlightenment and Cartesian ideals, when meta-spiritual societies like the Freemasons where founded. Their rationalism goes along with an architectural symbolism. And their rationalism eventually turns in to the field of cybernetics, which in turn, created information science[4]. This trunk of ideas, originating from a historicized view of architecture, turned out to be Pandora’s box of software engineering.

Baber (1982) uses a similar analogy referencing the major figure of cybernetics, Norbert Wiener, when he sets his famous fable of the city of “Moc”. The collapse of its infrastructure is used to explain a vicious circle of inadequate knowledge transfer. In his opinion, software designers need as much education as real architects: “The term software engineering was used [...] in a provocative than in a descriptive manner.” (p. 14). Since Donald Knuth’s, Frederick Brooks (1986) and Baber’s manifestos, computer science sought help in becoming an organized, designed taxonomy. This may be the reason “patterns have taken root in software” (Coplien 1995, p. xi). With patterns, Alexander’s ideas from architecture (Coad 1992) have also been adopted.

Information Science has since adopted a huge vocabulary from architecture. The mounting of an application’s building blocks, including compiling and linking[5], is called building. Admittedly, a compiler assembles – a reference to industrialized construction techniques. This thesis, of course, deals with frameworks. Furthermore, many metaphors regarding three-dimensional composed objects can be found in the pattern language of software engineering, e.g. stacks, platforms and pages (or even libraries). Finally, we constantly use space of a runtime environment. Consequently, software engineering could be seen as design of digital space – or architecture. To use the semantic scope of architecture was not compulsory. It merely was considered common knowledge – in contrast to mathematics. It opened, like Fielding (2000 p.18) mentions also a view, an interpretation to things. Like everything perceived in space. A common illustration of this case is the use of “table” instead of the term “relation” for database systems (Codd 1970). The use of language in these pictorial terms, as a process of semantics, is one of Wittgenstein’s main ideas. We will come back to this later – but keep in mind that process is an antonym to timeless.

It is important to point out that Alexander’s The timeless way of building has been released in 1979. According to Portugali (2000, p. 31-33) at this time architecture realized, that neither Le Corbusier’s utopian nor Friedmann’s epistemological nor Bertalanffy’s system theoretical approach to planning was going to work out in real life. In the quest for the ratification of order, the philosophy by Michael Foucault was considered useful. Forty (2000, p. 248) accredits Foucault’s idea of “space […] in the form of pattern and ordering” to this new perception of order as abstract scheme. The link to information science is obvious with Carole Brooke (2002, p. 54) pointing out that Foucault’s critical analysis questions the source of any ontology in regards to the power that lead to its order. Power here is a precondition for adoption, also in information systems (Conway 1999, p. 71). Lefebvre uses this aspect of power to criticize the modus operandi of “privilege[ing] a single concept, and make everything else to fit hat concept” (2000, p. 248). This critique resulted in a new architecture considering a designed space only one out of many possible manifestations of order. Soon the pattern ideal was considered a failure in building architecture (Alexander 2008), but flourished in its digital twin.

The era of post-structuralism was Alexander’s context. As he describes the idea of the pattern (1979, p. 147): “each pattern is a generic solution to some system of forces in the world. But the forces are never quite the same”. He still believed in a higher-order system, though a living one, with patterns as memes. He created a pattern language[6], a terminology to index force-configurations and solutions to resolve those forces, “morphological laws” (p. 90). In my opinion, the temptation to use Alexander’s approach in information science is founded in the holistic nature of patterns and a traditionally strong positivism in computer science. Patterns are generic solutions that exist similarly to a platonic idea; something that is rather discovered than invented (Coad 1992). In combination with calculation, it becomes un-questionable (Barry 2004, p. 87). Perception, however, always limits discovery. Ladyman (2002, p. 254) summarizes this as “Physics cannot explain the universe” (see Interlude 1). Alexander created a common language –in Wittgenstein’s sense of a language as expression of experience. A language translating a perception of being in time rather than a universal grammar[7]. Thus, the question if semantics are still the same for a sentence cannot always be answered. Debates in Philosophy of Science, like the discussion Habermas contra Foucault (Brooke 2002), underline this.

How did architecture move on after realizing a constant adaption is necessary?

Alexander considers Le Corbusier’s pragmatically totalitarian order inappropriate when he states “the radiant city [...] makes us feel bad” (p. 284). He believes that technically supported iterations can never be “actual” (Alexander 2008), hence frightening in an “uncanny” (Mori 1970) way. He implicitly favours Frank Llyod Wrights approach (p. 203) of an individualised romanticised[8] city, a common ground, more of a “bottom up” approach as Scheurer (2009, p. 49) put it. In the way he contrasts them, however, he approves the general idea of the city a structure inherently harmonious (Fishman 1982, p. 13) – although not centrally planned but evolved out of a “law” system of patterns. Later, according to Portugali (2000, pp. 9) the insight of the network structure of cities emerged. Instead of a system of complex patterns Wittgenstein’s Ideas were used to create interconnected concepts and categories, finally leading to the idea of self-organisation and the synergetic (2000, p. 261)[9]. Deleuze and Guattari went further and “in place of totalizing unity, [...] advocate locality, situatedness, rhizomic strategies [...] and assemblages” (Picon 2002, p. 1).

In software engineering, Alexanders’ idea of patterns is somewhat largely applied to object-oriented aspects (Fielding 2000, p. 16), whereas it should be applied to a more abstract level, virtuously viewing all “engineering principles and architectural style” (Perry & Wolf 1992, p. 42). In mobility context, IT architecture takes local aspects into consideration (Nyir 2005 or Interlude 3) – but how does locality, as a time-space defined context, a domain, affect patterns? What could we take out of this analysis? Patterns are inevitably connected[10] in a network. More importantly, patterns constantly evolve in the same way as technology and culture (Law 1991, p. 17). Only by using and critically questioning them they become useful. Patterns are a language that does not provide a generic solution but rather explains a certain context. Wittgenstein uses the “builders language” to draw an example of a simple, yet complete language. Complete, because it is in use and thus can evolve[11]. Blair (2006, p. 146) quotes Wittgenstein’s famous city analogy which brings us back to architecture theory:

...for these are, so to speak, suburbs of our language. (And how many houses or streets does it take before a town begins to be a town?) Our language can be seen as an ancient city: a maze of little streets and squares, of old and new houses...

Coming from fragmented artefacts, building architecture turns back to its roots by incorporating evolution. To the extent of Friedman’s “structure that would incorporate the unforeseeable”[12], Philip Johnson’s diffusion between glass walls and nature or architectural warfare as Weizman’s calls it “walking through walls”[13]. The layers become a grid which allows alignment rather than limiting movement. In the same way as in building architecture back in the day, digital architecture still wants to create order; at last local, domain-specific ones. Yet only in rearranging these orders, a system is kept alive.

The misconception in my opinion is that patterns represent divine ideas that can, if stacked properly, still form a perfect timeless system[14]. On the pro side, evangelists like Goodliffe (2009, p.25) state “The quality of your software city is directly related to how much town planning went into it”. He is supported by e.g. Keller (2005), who argues that perfect order can never be achieved, but recursively applied patterns can be used in enterprise architecture to lead this way. On the contrary, Monson-Haefel (2009, p. 186) provokes “It’s simply not possible to future-proof an architecture”, Carr (2005) sees IT as commodity and Hohmann (2003, p. 3) believes that “architectural "goodness," [...] can only be explored through actual use of the system”. The latter follows the path of building architecture, where Cartesian ideals have been replaced by a holistic view. An approach accepting change of semantics (or call it uncertainty in architecture)[15], yet taking temporary Nash equilibrium into consideration, of course.

In everyday use, architecture mostly refers to the process of design in regards to physical structures, in particular buildings. This is the classic definition by Vitruvius, Alberti, etc (Forty 2000, p. 240f.). In recent years, however, the architecture schools focused on a much wider approach, taking into consideration the whole experience of space (Bachelard 1958, Böhme 2006) and time. Architecture is creating space, particularly environments. This process includes designing, constructing, reshaping/refactoring and defining spaces – regardless of their size and structure, including feelings and emotions (Pallasmaa 2005). Therefore, we probably have to take the mentioned quote by Le Corbusier back into consideration and see the act of ordering(Law 1992), although post-structural, as an ideal. One that can actually help create dynamic solutions and systems of systems (Boehm 2006), like Foucault and Habermas developed in their dance-like debate (Conway 1999). If we want, we can make use of an appropriate domain language to ease configuration and standardization, hence reusability – this domain-specific language could be called A Pattern Language.

I see the task of architecture as the defense of the authenticity of human experience

Juhani Pallasmaa, Encounters

---

[1] I am aware of the fact that those personalities have not been the first prophets. Morville (2005, p.26) mentiones Kevin Lynch as inventor of patterns in building architecture, Pree (1995, p. 62) accredits Bruce Anderson of the transformation to IT.

[2] Shorthand for the 1994s  “Design patterns” book authors: Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides

[3] See for example Foote 1998, p. xi “So patterns people can play not only Adam and Eve, and Carolus Linnaeus, but Luther Burbank as well” (Carolus Linnaeus was the inventor of a taxonomic system for biology) or Meyer 1996.

[4] According to AMS 1996, p. 1159 „ Information sciences originated in cybernetics” or, more in depth, Mahoney 1989

[5] Both terms derived from linguistics and mathematics rather than from architecture hence from the earlier times of computer science as Kohanski (1998, p. 89ff.) shows.. See Irons (1961) for one of the first descriptions of the definition of a compiling language or, for a more in-depth reference list Wexelblat (1981)

[6] The term was mentioned before to show the common use it arguably reached since its introduction. However, several other suggestions have been made in the similar directions, see Lakoff (1980) or Wake (2000)

[7] in Chomsky’s terms. Another famous discussion, see e.g. Good (2006) p. 57 and Livingston (2002) p. 341: “metaphysical interpretation according to which beings are submitted to a unified regime of explanation insofar as they are rule-bound fails to accomplish its goal, because it conceals its own origin in more ordinary uses of the concept of a rule”

[8] Romanticised in terms of referring to medieval, spiritual times. Coyne (1999, p. 6) shows that “Although antagonistic, the rationalist and romantic legacies are not so far from each other”

[9] Self-organisation is the other, even more cybernetic, Pandora’s box: Artificial Intelligence. See  “What computers cannot do” by Dreyfus (1972). Another approach is “crowd sourcing”, again taken from the architectural concept of desire paths (Morville 2005, p. 26).

[10] Actually, Alexander points this out, too, e.g. “each of these parts, if taken by itself, will produce chaos” (p. 163), the network/relationship meta-concept is also referred to in the “Software Pattern Series” e.g. Zimmer (1995, p. 355) and as a constraint in Perry and Wolf’s (1992, p. 43) definition of architectural style as “particular codification of [...] elements”

[11] This can also be applied to the current discussion about generic vs. domain-specific languages in IT context. Wittgenstein argues any language is complete as long as it fulfils the assigned task. A language is valid even if not Turing-complete

[12] In 032c magazine, Maak 2004, p. 113, Yona Friedman’s architecture is compared to Koolhaas’ analysis of chaotic structures

[13] Weizman 2007, p. 186 describes the “movement itself that produces space around it”

[14] A very common example of the divine solution problem is the story of J2EE (Scalability vs. transparency (Bass et a. (2003) p. 420)) as a programming pattern or MVC as an architectural pattern (see Coplien 1995, p. 59). An upcoming discussion in the same area is “convention over configuration”, again merely a problem of defining a language. I am, however, fully aware of the fact that a system change and feedback are always possible as Brown (1998), Fowler (1999) and Feathers (2002) have shown in far more details than I could annotate here.

[15] In information architecture, scholars become more aware of the field of uncertainty in agility as Kalbach (2009) shows.