Software Architecture Clues

Editor's Notes

• #2: Sometimes we just need to get a clue, and this talk has a bucket full. The idea of a clue bucket came up when we were working on a project and grasping around for ideas, and I said “I need a clue bucket so I can just reach in a get a clue.”
• #9: We develop, through direct and indirect experience, "a considerable body of contextual sense" -- that is, "a knowledge of what is reasonable within a given context" (Rechtin, 1991). Engineering heuristics, or rules of thumb, are "statements of common, or contextual, sense that aid in concept development, problem solving, decision making, or judgment"(Rechtin, 1991). They can be descriptive (what is the situation) or prescriptive (what to do about it) (Rechtin, 1991). "Heuristics offer plausible approaches to solving problems, not infallible ones."— Rebecca Wirfs-Brock Image created from “15 Fundamental Laws of Software Development” by Matthew Jones, https://exceptionnotfound.net/fundamental-laws-of-software-development/
• #10: Let’s talk first about creating a good architecture – one that fellow architects would recognize as having the properties of a good architecture
• #11: Let’s talk first about creating a good architecture – one that fellow architects would recognize as having the properties of a good architecture
• #12: Grady Booch identified these four “fundamentals remain fundamental“ to architectural design. mne·mon·ic — scars Taking the first of those (Separation of Concerns), and to illustrate surfacing and illuminating heuristics, here's a story from the Tao. It is the story of the dextrous butcher. In one translation, the master cook tells us:
• #13: Chuang Tzu — Taoist thinker ‘Cook Ting was cutting up an ox for Lord Wen-hui. As every touch of his hand, every heave of his shoulder, every move of his feet, every thrust of his knee — zip! zoop! He slithered the knife along with a zing, and all was in perfect rhythm, as though he were performing the dance of the Mulberry Grove or keeping time to the Ching-shou music. “Ah, this is marvelous!” said Lord Wen-hui. “Imagine skill reaching such heights!” Cook Ting laid down his knife and replied, “What I care about is the Way, which goes beyond skill. When I first began cutting up oxen, all I could see was the ox itself. After three years I no longer saw the whole ox. And now — now I go at it by spirit and don’t look with my eyes. Perception and understanding have come to a stop and spirit moves where it wants. I go along with the natural makeup, strike in the big hollows, guide the knife through the big openings, and following things as they are. So I never touch the smallest ligament or tendon, much less a main joint. “A good cook changes his knife once a year — because he cuts. A mediocre cook changes his knife once a month — because he hacks. I’ve had this knife of mine for nineteen years and I’ve cut up thousands of oxen with it, and yet the blade is as good as though it had just come from the grindstone. There are spaces between the joints, and the blade of the knife has really no thickness. If you insert what has no thickness into such spaces, then there’s plenty of room — more than enough for the blade to play about it. That’s why after nineteen years the blade of my knife is still as good as when it first came from the grindstone. “However, whenever I come to a complicated place, I size up the difficulties, tell myself to watch out and be careful, keep my eyes on what I’m doing, work very slowly, and move the knife with the greatest subtlety, until — flop! the whole thing comes apart like a clod of earth crumbling to the ground. I stand there holding the knife and look all around me, completely satisfied and reluctant to move on, and then I wipe off the knife and put it away.” “Excellent!” said Lord Wen-hui. “I have heard the words of Cook Ting and learned how to care for life!”’ Translated by Burton Watson  (Chuang Tzu: The Basic Writings, 1964) It goes according to natural laws, Striking apart large gaps, Moving toward large openings, Following its natural structure. … Nevertheless, every time I come across joints, I see its tricky parts, I pay attention and use caution, What do we extract, that helps guide us in architecting? Two heuristics jump out: follow the natural structure when we come to the tricky parts, slow down
• #14: software doesn't have natural topology. Or does it? The system boundary is an obvious candidate: separate interactions at the boundary from the core of the system. In the founding classic of system architecture, Eberhardt Rechtin presents heuristics gleaned from his amazing career as a system architect in aerospace, and master teacher of system architects in the seminal program he created at USC. One of these heuristics (is a "turtles all the way down" sort of thing, but applies also at the system level): "Design things to make their performance as insensitive to the unknown or uncontrollable external influence as practical." That, for me anyway, has echoes of fort design and Ambrose Bierce' Devil's Dictionary definition of abatis: "rubbish in front of a fort, to prevent the rubbish outside from molesting the rubbish inside"
• #15: A fort visually, and in intent, has echoes of Alistair Cockburn's hexagonal architecture (pattern). Here, adapters at the system boundary, shield the (core) application (code) from interactions with the outside, keeping business logic uncontaminated by the peculiarities of external agents or systems and their states, interactions and interaction paradigms, and keeping business logic from leaching into user, or other system, interface code. Moreover, ports and adapters are a first mechanism of address for cost of change, partitioning the system boundary into plug-and-play access points.
• #16: One place to go, in looking for the natural topology, to find the natural shape to follow in creating internal system boundaries, is the "problem" domain — that is, the domain(s) being served by the system(s) we're evolve-building. This brings us to bounded contexts and heuristics and guidance in Domain-Driven Design to suggest system and component or microservice boundaries. http://martinfowler.com/bliki/BoundedContext.html
• #17: Another place to go, in looking for abstractions, is back to 1989 and Ward Cunningham and Kent Beck's CRC cards, but repurposed for components, along with a heuristic that takes the form of a neat antimetabole (thanks to Jan van Til's paraphrase of a Tom Graves point): "The responsibility of architecture is the architecture of responsibility." Which points us in the direction of identifying responsibilities related to system capabilities and properties, and the arrangement of responsibilities. Separating responsibilities along the lines of concerns. [This is not the only way we use the term "separation of concerns' in software design, of course.]
• #18: The dual to separation of concerns is a coalescence or cohesion of concerns? That is, not only are we looking for what to pull apart, but what to keep together to form crisp and resilient abstractions. We use abstraction in different ways — levels of abstraction, abstractions as elements of code we work with, the activity of abstracting. J. Edsger Dijkstra (The Humble Programmer, 1972) noted: http://vanderburg.org/blog/2011/01/31/cohesion.html
• #19: We use abstraction in different ways — levels of abstraction, abstractions as elements of code we work with, the activity of abstracting. J. Edsger Dijkstra (The Humble Programmer, 1972) noted: “The purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise.” Here, we're talking about heuristics for identifying architectural elements (or components, or microservices if those are the architectural elements of the system in question). A crisp abstraction has a clear, unifying purpose or cohesive identity; it has, in effect, a single responsibility at the level of abstraction of the abstraction. That is, there is high cohesion among its sub-responsibilities, and all contribute to the element's distinguishing purpose, or single responsibility. A single, if you like, defining job-to-be-done. This suggests a focus on responsibilities as an avenue to system (de)composition. We identify responsibilities and assign to them components, working in either (and both!) direction: start with a first cut notion of components the system will need, and identify and allocate responsibilities to them; or start with responsibilities evident from what the system needs to do, and factor to find components. A related heuristic takes the form:
• #20: We take a guess as a starting point, and improve on it: Run thought experiments and model (use cases or user stories; focus on one property, then another, etc.) to flush out responsibilities we overlooked in our initial guess. Change perspective, and review the responsibilities and abstractions in the view of, for example, thinking through allocation to the physical system topology, or a system change. Be disciplined in seeking alternatives — for example, by emphasizing a different requirement. Do this quickly and cheaply, to get a good enough starting point to build the next increment of code, and iterate on the design. Update the responsibilities lists as we do so.
• #21: These lists of responsibilities are a powerful and largely overlooked/underused tool in the architect's toolbelt. If the responsibilities don't cohere within an overarching responsibility, or purpose, that should trip the architect's boundary bleed detectors. We may need to refactor responsibilities, and discover new components in the process. We think that "naming things" is the big problem of software development, but "places to put things" is the proximate corollary.
• #22: Heuristics don't take away the need to think, to reason and try things out. They help us identify what to think about, as we do so, and may suggest how to go about it (better). In another of the foundational classics of our field, David Parnas (On the Criteria To Be Used in Decomposing Systems into Modules, 1972) proposes: "that one begins with a list of difficult design decisions or design decisions which are likely to change. Each module is then designed to hide such a decision from the others. " The other heuristic in his criteria for decomposing (already quoted above) is: "[begin] with a list of difficult design decisions [..] Each module is then designed to hide such a decision from the others ." As an aside, we also recommend identifying key architectural challenges when we're determining architectural strategy, to think about principles we might craft to help shape our approach to the challenge. And this approach is again useful, when identifying abstractions, or (re)assessing already identified abstractions for goodness. In a modular approach, parts of the system that are unstable, due to uncertainty and experimentation to resolve that uncertainty, can be shielded from other, better understood and more stable parts of the system.
• #23: That's two heuristics, and thanks to Robert Martin, the second has primary place in the SOLID set of principles that have been a touchstone of software design, and object-oriented design in particular: Single Responsibility Principle (SRP): a class should have one and only one reason to change Stuart Brand, writing about pace or shearing layers in building architecture in How Buildings Learn, observed that different parts of a structure change at different rates (with the building site, and then the structure, being least likely to change, given the cost and difficulty of doing so; and the stuff inside being the most likely to change). This observation gives rise to the heuristic: keep things that change together, together
• #24: Jeremy Campbell, GRAMMATICAL MAN: Information, Entropy,Language and Life Systems evolve – we’re constantly adapting them. Complexity from accrual Complexity from mess Accidental complexity Versus inherent complexity
• #25: Several things in the quote are striking: An idea of boundary But the idea was real. Important. Ambiguous. Two-faced. What was inside it, and what was outside it, depended on what side you were on.
• #26: But what about the boundaries within the boundary? These abstractions we use to give our system internal form, must be invented. They are conceits. In every sense of the word, perhaps. At least, when asked about the granularity of microservices, I point to the WELC master going "about so big." Obviously a joke, the joke being on us, the right answer of course being "the right size, and no bigger." Which is not an answer, but where can we look, for a better answer? One place to go, in looking for the natural topology, to find the natural shape to follow in creating internal system boundaries, is the "problem" domain -- that is, the domain(s) being served by the system(s) we're evolve-building. And indeed, we're broadly using bounded contexts in Domain-Driven Design to suggest microservice boundaries. And when we get to the tricky parts, like "customer" or "product" that have potentially overlapping, yet different, meanings in different domains, we slow down, and move more carefully. A customer, after all, experiences themselves to be one person through all touchpoints with "the system," and doesn't want to feel like the system cleaved them brutalistically. So we separate domains, but the architect is noting this point of articulation, this tricky part, where we have to move more carefully.
• #27: What is often the case with something we might call common sense -- even if we mean the contextual sense variety -- is that it looks obvious once noted. And yet it time and again that a system design ends up with a component that has a high center of gravity or something, and attracts way more responsibility than other components (common enough it goes by a name, "god class"). "A single responsibility at the level of abstraction of the abstraction" is a judgment call (or set of judgment calls). One way to check our judgment is to apply the heuristic: consider the distribution of responsibilities for balance.
• #28: The last of the fundamentals organized by the SCARS mnemonic, reminds us to strive to simplify. In response to H.D. Thoreau's "simplify, simplify," Ralph Waldo Emerson reportedly retorted 'One "simplify" would have sufficed.'
• #29: Eb Rechtin, on the other hand, offers us the heuristic: "Simplify, simplify, simplify. " Another heuristic (Rechtin, 1991) might illuminate: "Communicate, communicate, communicate. " That is, we need to simplify, and simplify again, and again, maintaining a discipline of repeatedly simplifying. And we need to communicate the need, and underscore the need, to simplify. Complexity is something we both embrace (we want more than hammers in our toolsets) and eschew (complexity costs -- attention, errors, ..., resources), so we need to harness complexity and reduce accidental, gratuitous, and unnecessary complexity. “Everything that needs to be said has already been said. But since no one was listening, everything must be said again. — André Gide
• #30: Jeremy Campbell, GRAMMATICAL MAN: Information, Entropy,Language and Life Systems evolve – we’re constantly adapting them. Complexity from accrual Complexity from mess Accidental complexity Versus inherent complexity
• #33: There's considerable discipline in applying Occam's Razor (also Ockham's razor or Ocham's razor; Latin: lex parsimoniae "law of parsimony"): Occam's razor is the problem-solving principle that the simplest solution tends to be the right one. That is, given a solution, it takes discipline and discernment to look for a simple(r) solution. Simplicity--the art of maximizing the amount of work not done--is essential. – Agile Manifesto, 12 Principles http://agilemanifesto.org/principles.html
• #34: Following the spirit of Bill Venner's conversation with Ward Cunningham on "do the simplest thing that could possibly work," we form the heuristic: "do the simplest thing that is good enough."
• #36: We're looking to simplify our system design, and if we only consider the system internals, the obvious can be hidden in plain sight -- the simplest way to reduce complexity, is to not add to it: Steve Jobs: “The hardest thing is, when you think about focusing, focusing is saying, yes, no, “he said, walking backwards as his argument changed direction, his hands splayed out wide. “Focusing is about saying no. Focusing is about saying no,” he repeated, as scattered applause breaks out. “You’ve got to say no, no no and when you say no you piss off people" http://observer.com/2011/10/steve-jobs-is-dead-has-died-passed-away/ "A primary cause of complexity is that software vendors uncritically adopt almost any feature that users want." — Niklaus Wirth "I wasn't the one pushing things in the wrong direction, but I should have been the one to stop it." -- Chad Fowler
• #37: Everything you use, must save you time and pay for itself – Martin Thompson “Inside every large program is a small program struggling to get out.” - Tony Hoare "A programming language is low level when its programs require attention to the irrelevant." - Alan Perlis
• #38: spent some time in the rain forests on the Queensland coast. One of the natural wonders of this area are the huge strangler vines. They seed in the upper branches of a fig tree and gradually work their way down the tree until they root in the soil. Over many years they grow into fantastic and beautiful shapes, meanwhile strangling and killing the tree that was their host. This metaphor struck me as a way of describing a way of doing a rewrite of an important system. Much of my career has involved rewrites of critical systems. You would think such a thing as easy - just make the new one do what the old one did. Yet they are always much more complex than they seem, and overflowing with risk. The big cut-over date looms, the pressure is on. While new features (there are always new features) are liked, old stuff has to remain. Even old bugs often need to be added to the rewritten system. An alternative route is to gradually create a new system around the edges of the old, letting it grow slowly over several years until the old system is strangled. Doing this sounds hard, but increasingly I think it's one of those things that isn't tried enough. The most important reason to consider a strangler application over a cut-over rewrite is reduced risk.
• #39: Booch's fundamentals yield a system where responsibilities have a clear and well-understood location in the system; architectural elements provide clear realms of test, experiment, and change; and elements are loosely coupled.
• #41: protect by not naming the source!
• #42: Everything you use, must save you time and pay for itself – Martin Thompson “Inside every large program is a small program struggling to get out.” - Tony Hoare "A programming language is low level when its programs require attention to the irrelevant." - Alan Perlis
• #53: Abstraction depends on context When I heard the statement “A lock is a lock because of the key”, it was one of the exhilarating insights I have listened to. To put in perspective, if you happen to lose the key to the lock, the lock is nothing but a dead weight. So when we design abstractions for lock, we are also parallelly designing abstractions for key. So our prior knowledge is playing a role there. We are always designing abstraction in pair most of the time sub-consciously. The abstraction we derive is nothing but interactions between the two components of the coupling. As coupling changes the abstraction should also change. But since only a part of the coupling is exhibited in code, the other part is often left to interpretations and assumptions. https://shripad-agashe.github.io/2 018/03/19/abstractions/
• #54: Interfaces -- Whether Remote or in-Process is a shaping influence om the interface although you can encapsulate many things behind object boundaries, you can't encapsulate the remote/in-process distinction. An in-process function call is fast and always succeeds (in that any exceptions are due to the application, not due to the mere fact of making the call). Remote calls, however, are orders of magnitude slower, and there's always a chance that the call will fail due to a failure in the remote process or the connection. https://martinfowler.com/articles/distributed-objects-microservices.html
• #57: https://twitter.com/KentBeck/status/713044232722718720 in response to NPM -- Left pad (11 LOC) “Every package that you use adds yet another dependency to your project. Dependencies, by their very name, are things you need in order for your code to function. The more dependencies you take on, the more points of failure you have. Not to mention the more chance for error: have you vetted any of the programmers who have written these functions that you depend on daily?” http://www.haneycodes.net/npm-left-pad-have-we-forgotten-how-to-program/
• #59: "Always design a thing by considering it in its next larger context" -- Eliel Saarinen
• #63: What do we do — selective attention and bubbles? move to different PoV — value of the process! not to be a big framework thing but scaffolding; and reminder of where we can go (next) to take a different look, etc. Also, a matter of discipline — self-discipline, and discipline of engineering — probe, criticism, come up with alternatives, model, reason, come up with counter arguments
• #64: We see things not as they are, but as we are” — Anais Nin
• #71: Virginia Plan — Madison’s notebook On May 29, 1787, Virginia delegate Edmund Randolph proposed what became known as "The Virginia Plan." Written primarily by fellow Virginian James Madison, the plan traced the broad outlines of what would become the U.S. Constitution: a national government consisting of three branches with checks and balances to prevent the abuse of power. In its amended form, this page of Madison's plan shows his ideas for a legislature. It describes 2 houses: one with members elected by the people for 3-year terms and the other composed of older leaders elected by the state legislatures for 7-year terms. Both would use population as a basis for dividing seats among the states. The Virginia Plan went through several revisions before it was finalized.
• #72: We the People of the United States, in Order to form a more perfect Union, establish Justice, insure domestic Tranquility, provide for the common defense, promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity, do ordain and establish this Constitution for the United States of America.
• #74: Commitment to making other people successful
• #75: Commitment to making other people successful
• #78: • good: technically sound; • right: meets stakeholders goals and fits context and purpose; and • successful: actually delivers strategic outcomes. “Much of the essence of building a program is in fact the debugging of the specification.” — Fred Brooks