ThoughtWorks emergent design & evolutionary architecture NEAL FORD software architect / meme wrangler ThoughtWorks 3003 Summit Boulevard, Atlanta, GA 30319 housekeeping ask questions anytime download slides from download samples from index.html?S_TACT=105AGX02&S_CMP=EDU agenda things to think about Emergent, a. [L. emergens, p. pr. of emergere.] 1. Rising or emerging out of a fluid or anything that covers or conceals; issuing; coming to light. [1913 Webster] 2. Suddenly appearing; arising unexpectedly; calling for prompt action; urgent. [1913 Webster] spectrum of design big design up front Project Planning/Estimation Requirements Use Cases/ Functional Specs Design Specifications Code Test Fix/Integrate $ who thought this was a good idea? emerging design finding abstractions & patterns Patterns patterns == nomenclature patterns describe effective abstractions idiomatic patterns good abstractions disappear abstractions leak the simpler the substrate, the easier the abstraction leaky abstractions All non-trivial abstractions, to some degree, are leaky. joel spolsky file system in java javascript libraries o/r mapping ActiveRecord in ruby on rails ! abstracting too early speculation without facts yagni business processes change radically and often experience helps spike solutions emergent design "what is software design? design?" what Jack C. Reeves fall 1992, c++ journal software "engineering" "The final goal of any engineering activity is some type of documentation" "When the design effort is complete, the design documentation is turned over to the manufacturing team." what is the design document in software? the source code source == design " is cheap to build. It does not qualify as inexpensive; it is so cheap it is almost free". manufacturing == build process " design is easy to create, at least in the mechanical sense." "Given that software designs are relatively easy to turn out, and essentially free to build, an unsurprising revelation is that software designs tend to be incredibly large and complex." source == design " is cheaper and simpler to just build the design and test it than to do anything else." "The overwhelming problem with software development is that everything is part of the design process." "Coding is design, testing and debugging are part of design, and what we typically call software design is still part of design." "Software may be cheap to build, but it is incredibly expensive to design." things that obscure emergent design complexity essential complexity inherent complexity accidental complexity all the externally imposed ways that software becomes complex essential vs. accidental complexity examples Hunting Season EJB / Biztalk Essential Field Level Security Accidental technical debt technical debt technical debt Code Base Complexity principal effort for new features interest negotiating repayment you must convince someone technical debt exists... ...start a conversation about repayment demonstration trumps discussion convincing metrics rampant genericness genericness often result of over engineering "if we build lots of layers for extension, we can easily build more onto it later" increases software entropy accidental complexity generic obfuscation emergent design enablers testing as a design tool confidence against unanticipated side effects regression testing understandable (executable) documentation executable intent protection between API boundaries test driven design more about design than testing design will emerge from tests atomic understanding of intent better abstractions less accidental complexity case study: perfect numbers perfect number: of the factors == number (not including the number) of the factors - # == # ? whole number square roots Classifier design implications vs. perfect number finder classifier tdd vs test-after test after doesn't expose design flaws early example: wrong abstraction level tdd forces you to think about every little thing encourages refactoring what's not right refactoring collective code ownership fix broken windows whenever you see them regularly fix obsolescent abstractions prudently refactor aggressively code should get stronger with age expressiveness matters if code is design, readable design matters complex languages hurt readability most comments don't help not executable always (potentially) out of date idiomatic pattern without closures without closures with closures (groovy) with closures (ruby) abstraction styles imperative structured / modular object-oriented functional anti-objects collaborative diffusion "The metaphor of objects can go too far by making us try to create objects that are too much inspired by the real world. " " antiobject is a kind of object that appears to essentially do the opposite of what we generally think the object should be doing." architecture Evolution, n. [L. evolutio an unrolling: cf. F. ['e]volution evolution 1: a process in which something passes by degrees to a different stage (especially a more advanced or mature stage) emergent evolutionary application architecture describes the coarse-grained pieces that compose an application framework level architecture the unit of reuse in java is the library when was the last time you downloaded a single class? JSR 277, the java module system...abandonware JSR 294 (superpackage)...IN JAVA 7! implemented by ivy & maven enterprise architecture concerns itself with how the enterprise as a whole (which usually means the applications running inside a large organization) consumes applications enterprise architecture == city planning application architecture == building plan extant definitions "The RUP, working off the IEEE definition, defines architecture as `the highest level concept of a system in its environment.' The architecture of a software system (at a given point in time) is its organization or structure of significant components interacting through interfaces, those components being composed of successively smaller components and interfaces.'" post on the XP mail list technical definition "A better definition would be: In most successful software projects, the expert developers working on that project have a shared understanding of the system design. This shared understanding is called "architecture." This understanding includes how the system is divided into components and how the components interact through interfaces.'" Ralph Johnson, rebutting the original post process definition Architecture is about the important stuff. Whatever that is. Martin Fowler's definition "the important stuff " vague but descriptive many arguments about architecture revolve around misunderstanding what is important what's important to business analysts differs from important stuff for an enterprise architect differences can in fact be mutually exclusive "SOA favors flexibility over performance" Stuff that's hard to change later. Martin Fowler, in conversation There should be as little of that stuff as possible. architectural considerations politics of architecture build or buy business processes are not commoditizable "you can buy this generic business process software..." "...we just need to tweak it with a few customizations" myth radically unique across similar businesses software can provide strategic business advantage standards-based vs. standardized flash-back to java web development before j2ee standards helped developers tremendously... ...but vendors hate it the price of commodity software quickly approaches $0 contrast j2ee & sql big enterprise package software touts standards-based ESB: standards-based but not standardized held together with highly proprietary glue even the open source ESBs suffer from this not likely to change consider the impact on your overall architecture postlude design is about code other artifacts aid in creating code all artifacts besides code are transient code hygiene matters fix broken windows pay back technical debt last responsible moment How can I make my decision reversible? Do I need to make this decision now? What can I do to allow me to defer the decision? ThoughtWorks ?'s please fill out the session evaluations samples at NEAL FORD software architect / meme wrangler ThoughtWorks This work is licensed under the Creative Commons Attribution-Share Alike 3.0 License. 3003 Summit Boulevard, Atlanta, GA 30319 ThoughtWorks resources Emergent Design & Evolutionary Architecture series: