Chapter 12. The Semantic Design Manifesto

Should I, after tea and cakes and ices / Have the strength to force the moment to its crisis?

TS Eliot, “The Lovesong of J. Alfred Prufrock”

We have come to the end that is hopefully a beginning.

What here are we proclaiming? A definitive answer? No. A different path forward? Yes.

Proclamations of that order seem to require a manifesto.

The Manifesto

Any manifesto will reject the supposed values, claims, methods, and models of the past, proposing to replace them with new ones. An exciting age of expansion and prosperity is heralded, but only for the true-believing radicals who can see the promise of the proposed One True Light and Way. Astonishing advances have been made in the reach and power of software in the past 50 years. So we do not wish to trumpet quite such claims, which doubtless will ultimately prove facile and reduce to a fascism of the mind.

But perhaps something has occurred in the history of software that could be called the architecture of our concept of software itself. Within this field of signs, directives, meanings, and metaphors, we shape our words, and thereafter our words shape us.

Our practices have countless times failed to achieve our aims. This is obvious in our collective landscape, littered with growing project failures. Seven out of ten software projects fail by not meeting budget, timeline, or feature requirements. More than eight out of ten big data projects fail. One in six software projects fail so spectacularly that they threaten the very existence of the company. Project failures abound, costing three times, ten times, twenty times as much as proposed; they take longer, do less, and if they don’t destroy the company, they leave unhappy customers and unhappy makers. Projects churn, with large portions thrown away and redone. And the problem has grown worse over the past twenty years, not better.

Received architecture frameworks, Zachmann, TOGAF, DODAF have not saved us in this, having done more to relegate architects to the Ivory Tower of Irrelevance than to realize the promise of project success.

Agile in its merriment has not saved us in this. But its religion has had a (possibly purposeful) side effect of relegating architecture as something unnecessary, or as a facet of the hated enemy other-method, Waterfall.

We see a major contributing factor as a failure of comprehension of the complexities and contradictions in the world. The world is an infinite conjunct of propositions and their predicates. Software and systems design demands we bound a context, to represent our ideas in the system, translating them to instructions. When these concepts and our language falter, we wait, as one waits for Godot, forever, for the “requirements.”

There are no requirements. There is only fecund imagination, and the subsequent work of reduction into rigorous concepts. Rigor here does not mean rigid: quite the opposite. Any further conceptual demarcation in the system’s design and naming that fails to embrace context, complexity, contradiction, and change will act as the kernel of its eventual upset, across many possible vectors.

We identify this is a key contributor to our history of failure. And this is what we here address.

We can do better.

How?

With X.

We call this method, this conceptual model for design “semantic software design,” “deconstructive design,” an “Architecture X.”

Why “Architecture X”? “X” is provisional, temporary. X refers to the horizontal axis, the flatness of the extending horizon on which rhizomes work, rather than falsely hierarchical Y-based systems; it is used to represent the in-between of time. “x” in the art and fashion worlds is used to signify a collaboration between two or more artists, as in “Jane x Jill”; in other fields, “x” is the unknown value, which we embrace in seeing that our software is a representation of a concept of the world—our work suffers when we disavow its complexities and paradoxes, as by extension software presents the world view of its designers. “x” in mathematics represents the independent variables—they are the inputs or causes; that is, potential reasons for variation, which can confound the system. “x” is one sign we use for the multiplying operator—x is generative and fecund, not reductive; x is a kiss. X marks the spot. X crosses boundaries of disciplines, and its cultural allure alights in cross-pollination.

Yet, and “so,” X is not X. Because X represents, in one usage, the unknown, we might easily replace the term “Architecture X.” We can be united in our ideals, but recognize a panoply of names, each foregrounding some different facets, each true and useful; we might here be doing “Generative Architecture” or “Oblique Architecture” following Brian Eno, “Active Architecture,” “Dadaist”—provocative against the architecture of the day. Or the X is for “x-perimental.” Or rather it is a “deconstructed” architecture following Derrida, a “thousand plateaus” following Deleuze. So we emerge from Architecture X into Semantic Software Design.

Because it is in fact no architecture at all, precisely not architecture. There are no architects in fashion houses, but rather designers. Is it so different? Music has composers. Is this more estranged from our work than making a concrete building in legally demarcated physical space? We are precisely semanticists, whose only tools are language and logic. We have no material but these, and imagination, creativity. Our work is in producing, challenging, and inhabiting the semantic field we demarcate to produce a properly structured concept of our software. This is the piece that’s missing from our failures.

We assert: an image of thought called architecture has been formed historically, and even though we don’t agree on job descriptions and practices, this effectively stops people from thinking. The X architect must create. We reject architecture as categorizing and classifying. No one with a P&L has any use for such endless classification of the animals. This is not Ivory Tower architecture: quite the contrary—Semantic Design, Deconstructive Design, X Architecture is street-fighting architecture, firmly rooted and working in the gritty, real world of customer demands, annual budget cycles, arbitrary strategies, mergers and acquisitions, our own independent flights into the unknown.

We recognize that that real world is richly complex, abounding with contradiction and deferral, and if we refuse or ignore or are blind to that, we will continue to make weak systems and fill them with accidental complexity later, making them very expensive and difficult to maintain. So we embrace the tools of deconstruction. Software is, to paraphrase Samuel Beckett, not about something—it is something. The code represents the design; the code is not the product. The code is a design factory, which emits into production only once invoked. So our work is further complicated in that the design is the primary object of our design. Recognizing this double action is critical. Recognizing that our true materials for construction are semantics is critical.

What you are creating is not architecture, not anything like a single building that won’t be changed over decades and hundreds of years: very far from. You are designing the concept. You design the design in semantic space.

So Architecture X is not architecture. We use the term “architecture” here under erasure. As X Architects, X Designers, Concepters, we not only read but hear, and hear "Ex-Architects,” and we do not in fact practice architecture at all. We practice semantics and design (“of signs”). We leave architecture as a metaphor, adopted decades ago at a conference in Germany in 1968, as one proposed option among many, to help find a language to talk about how to talk about how to talk in software. Perhaps it has served us to some extent: one needs to put a stake in the ground. But we assert, no longer. “Architecture” itself is not what we do, but there our job titles sit in fields in the totalitarian wisdom of HR databases. Yet it is a metaphor that, like all language, can’t help but generate ancillary metaphors, like “blueprint” and “plumbing,” which lead us further away from our efficacy.

X Architects/Semantic Designers see that our language prescribes a space, demarcates our words and as such our thoughts and expressions of our work. As such, we recognize across the spectra of writing and speech the sound quality that presents us: ex-architects. We abandon this metaphor while we recognize its pervasiveness, and deconstruct it from within, in search of a better model for our work.

Our work, hardly fully captured by the term “architect,” is making software, designing software, designing whole systems, designing data models and infrastructure and datacenters and managing teams and projects, thinking, mentoring, researching, changing the structure of organizations, recommending, coding, reviewing, deciding what and how and when and why and writing, presenting; we are the philosophers of the organization, its fools, in the classical sense: we advise the king. We support mergers and acquisitions. We make broad strategies and tactical fixes. We debug, we curate. We arrange the semiotics of what names and labels and words mean what in relation to each other. We design. “De” = “of” + sign. Design = “of the sign.” Comprehensively, not merely software, but across all these vectors. So little of the work of making software has anything to do with the coding.

And we design from nothing, no cloth, no marble—nothing but our concept. There is no ground we are given as in other fields. We invite our constraints like few others. Our design subject is systems, of all kinds—systems of thought, organizations, datacenters, and software. And the work we apply to them is design work. We have forgotten how to think and lost our words. We mustn’t, because a key job is naming the things in the semantic space: the infinite Turing tape of propositions and their predicates.

But it is frivolous and impotent to merely replace one word with another and keep doing the same things. We care less for the worry over the word as the concern to shift our focus, to rethink what our work actually is, to improve the outcome. “Scrum Master” is a metaphor borrowed from the game of rugby—what in the world has that to do with software development? Yet fresh-out-of-college HR recruiters happily repeat it, echoing the phrase into the internet as if it were a thing. As indeed it is. So it does happen.

X Architecture, X Design, this notion of not-architecture but concepting, is a recognition of the world as an infinite conjunct of propositions, and that systems require demarcating a semantic field in a way that paradoxes take hold at the boundaries, and these hurt the unprepared system. We take a holistic view, expanding the work of designer: everything in the organization, the software, the data, the infrastructure, is a design candidate, and structures are riddled with binary oppositions, each featuring a privileged term, creating hierarchies, which create semantic glitches that we tend to gloss over, but which the software cannot. It is this that ultimately undermines the software in inadequacy or complication. We therefore deconstruct such structural binary oppositions, foreground the idea of the concept to be designed, and invite a collage of other lenses from other fields, to make better software.

The Four Ideals

We design software with these basic ideals:

  • Our work is to design concepts, before designing software, rather than classify in taxonomies and falsely overlay simplicity in a single frozen layered picture. The concept is of the system.

  • Our view is comprehensive: the organization, the project, the integrations, the documentation, the conversation, the data, the infrastructure, the metrics, as well as the software applications are systems as objects of design. We recognize the impossibility of this, as we work beyond monoliths, or any totalitarian idea of completeness and stability.

  • Our mode is de-centered, deconstructed: it embraces imagination, diversity, context, complexity, contradiction, and change. Meaning in the system is not rigid, but deconstructed; it is generative. Requirements change and overlap in gaps of incompleteness and contradiction, so we embrace uncertainty and design for it. We employ lateral thinking, rhizomatic root systems over arborescent hierarchies, rather than thinking in traditional structures, in binary oppositions, and hierarchies of technology versus business; this ensures our concepts can continue to create themselves and to change in evolution: they are autonomous, learning, unfolding, multiplicative modes.

  • Our focus is on diverse customers: we are highly focused on outcomes, the “difference that makes a difference” to customers, the result, over the process, over our own activity. We recognize the diversity of voices in the many “users” of the system, and make our systems accessible, foregrounding our algorithmic and design bias so we can subvert it.

The Key Practices

We follow these practices that are consistent with our ideals. Some are new, some are old, some gain power by their inclusion and juxtaposition here.

Concept design practices

  • You focus on the concept: what is the world you are making that creates a context for the design that becomes the production factory?

  • How do you invite, not reject constraints, to find grounding? How specific can you be while still saying only true things in all the names throughout the design?

  • How can you invite nonbuilding metaphors to help you as a bricolage?

  • You are engaging and alighting the curiosity and intellect and cross-pollinating vectors of thought in your team. You are encouraging and nurturing their thinking processes. That is the first element. Not to dictate and prescribe, but to inspire and alight according to the principles of X.

  • Use Design Thinking when thinking about what to make.

  • Use lateral thinking when thinking about how to implement it. This is about solving problems using an indirect and creative approach via reasoning that is not immediately obvious. It involves ideas that may not be obtainable using only traditional step-by-step logic. Lateral thinking is more concerned with the “movement value” of statements and ideas. Edward de Bono defines four types of thinking tools: 1) idea-generating tools intended to break current thinking patterns—routine patterns, the status quo; 2) focus tools intended to broaden where to search for new ideas; 3) harvest tools intended to ensure more value is received from idea generating output; and 4) treatment tools that promote consideration of real-world constraints, resources, and support.

  • Use Strategic ideas, as in Technology Strategy Patterns, to ensure your concept aligns with the business vision: raise your visor and innovate at this level of the idea. We’re not interested in arguing over frameworks. They don’t matter.

  • Use Oblique Strategies to challenge your own conventional or default view. Pick one in the standup each day, or send them to the team. You’re activating critical and lateral thinking and imagination.

  • Use deconstruction: look at the multiplicity of structures that start to arise in the system, find the binary oppositions that support those structures (Master/Slave, Center/Margin, Speech/Writing, Production/Development—development is production to the developers, Functional/nonfunctional Requirements), determine which of the terms is the privileged one, and show how the traces undermine that privileging. Then design something without marginalizing one of the terms.

  • We do DevOps, as it is a deconstruction of that historical binary opposition.

  • The system has many small pieces, with high cohesion following the Single Responsibility Principle.

  • Keep a design scrapbook or lookbook as they are called in fashion houses. You will need a multiplicity of views and perspectives represented in different formats for varying time horizons, executives, customers, and developers.

  • We do not freeze in time, which has perpetually frustrated software designers. Instead we admit that this never works, and instead foreground Becoming. What we design is not the system as is, but a process of change, flight, or movement within an assemblage. As Deleuze and Guattari explain:

    The process of “becoming-” is not one of imitation or analogy, it is generative of a new way of being that is a function of influences rather than resemblances. The process is one of removing the element from its original functions and bringing about new ones. Hans is also taken up in an assemblage: his mother’s bed, the paternal element, the house, the cafe across the street, the nearby warehouse, the street, the right to go out onto the street, the winning of this right, the pride of winning it, but also the dangers of winning it, the fall, shame.... These are not phantasies or subjective reveries: it is not a question of imitating a horse, “playing” horse, identifying with one, or even experiencing feelings of pity or sympathy. Neither does it have to do with an objective analogy between assemblages. The question is whether Little Hans can endow his own elements with the relations of movement and rest.”

    For architects, this means we resist the demands to freeze things in time. Things are not themselves: they are only ever becoming-things. So we design for movement, design for becoming, not now-ness, not false decideability and stasis. We know that we don’t know the most important things, how it will be used, and we design an empty center to hold that not-knowing up front, on purpose, with care: this makes us foreground pluggability throughout the system.

Comprehensive view practices

  • There is only one substance. Therefore everything which exists must be considered on the same plane, the same level, and analyzed by way of their relations, rather than by the “essence” of the “entity”, as if it exists in a vacuum: that false assumption damages our designs. Despite its name, Relational Data Models privilege the essence, and relegate the relation to a second-class citizen. Deconstruction loves the relation, and see things not in an special essence, but in their relations of differences. We find ways in the design to foreground the relations and the differences.

  • Graph databases are therefore an excellent tool, as are pub/sub, eventing streams that turn the database inside out, and modeling services as contextual agents, not essences. That is, we don’t design the One True “Profile” service to rule them all. We instead abstract up and design the Persona service, wherein the single person with one tax ID has many different relevant modes of being in the world, and designing for that multiplicity aggregates nuances that improve the richness of the system.

  • The elements of the architecture (business, application, data, infrastructure, as well as their representations) are not viewed as separate but in a unity, altogether, and their impact and forcefulness and intensities on each other are examined and considered and designed for.

  • Make design decisions in empathetic thought of customer personas in different extremes, both valid.

  • Design with the entire picture in mind: business, application, data, and infrastructure as systems. When designing, think of the monitoring as you implement an algorithm; consider the business implications of infrastructure choices; and consider the changing requirements when locking down a data modeling.

  • We recognize our duty and responsibility and joy to design all the things, not just focus on the software: when subsystems are in harmony with each other and the broader concept, the product will be optimal across many concerns (-ilities, project concerns). Design the software together with the business design and the infrastructure and data design.

  • The making of the software is one small piece of the puzzle; overindexing on software and thinking it will run separately when magical fairies whisk it away to the cloud is a recipe for failure.

  • In considering infrastructure, we design pipelines.

  • We radically automate the testing: we never manually test, but have test automation engineers. The tests are not ancillary to code: they are code, and are first-class citizens.

  • Write automation tests against the infrastructure, too.

  • Design in extremes such that if you are plotting out this event type, get two different points of view and put one stake in the ground and then a second one on a different extreme (how does ecommerce work for a sleep room, how does it work for a coffee mug).

  • Code reviews are not about today’s code and policing programming but about encouraging the concept and broadening its understanding and application. It is about mentoring, sharing, mapping to the principles and not scrutinizing junior developers but more like pair programming so others learn it and make less SPOKs and learn the principles and refactor the design.

  • We do Pair Architecting: share a screen and review the design together, in a plain text file to get the words right. You’re agreeing on metaphors that carve our semantic space and it should be MECE.

  • Radical immutability in the architecture whereas the artifact that is built and deployed is totally immutable. Focusing on making things immutable everywhere possible is the key to allowing movement and change and mutability, which is the hallmark of a manageable, monitorable, extensible, cost-effective system

  • Write a Design Definition Document that compiles the views across business, application, software, and infrastructure stakes. This might be called a Petits Recits.

  • Why wait for a week to demo code? When code is done, invite the team to demo it immediately, or in a very short amount of time, in an event-driven way, when your story is done—don’t wait until the end of the sprint if you use those (though Kanban is more consistent). Make a party of it.

Decentered, deconstruction practices

  • Defer the implementation: do not write precisely the requirements as given. First create a context in which those imagined requirements could come to life; then implement them as merely what happens to be the first known requirements (not even the “default”—that’s always a false privileging). See what you’re doing now, abstract up to the category, and make that first as the context, then implement whatever the stated requirement is, knowing it will change. Give names that recognize this.

  • Design machine learning capabilities throughout the system. This is generative architecture, active design: embrace that. Yes, make your machine learning for customer product recommendations, but also for data clean up and rogue monitoring. Design learning into the system to make it truly organic. This suggests making feedback loops and machine learning pipelines. You don’t decide the implementation up front even necessarily: let the system pick based on what it learns. You’re recommending products to your customers without being asked: your system can, in an adversarial manner, learn, pick a champion, and deploy it in an ultra-dynamic manner. The radical pluggability supports this idea. You are designing the system that can do better, move, grow. We don’t make frozen pictures of software or frozen software.

  • Do not design with hierarchies or enums in the data model. These will almost always be proven to be false deconstructed in the world, so your software must reflect more fluidity.

  • Assume your component will eventually be Decorated or have a Strategy injected. Instead of directly implementing any business logic, write it as the presumed first Strategy implementation.

  • Prefer peer-to-peer protocols and systems without Master/Slave, such as Cassandra.

  • Interfaces over inheritance.

  • Do multivariate testing, canary deploys, multivariate deploys.

  • Do not privilege production over staging: have a multiplicity of stagings, and automate learning which ones work best. Why have one production? Have many running at once. Use multi-armed bandits to explore and exploit. The world of X production is richly multidimensional.

  • Do automation at the beginning of the project, not at the end.

  • Define metrics at the beginning of the project, not at the end. Code toward success metrics.

  • Break production on purpose: run Chaos Monkey to ensure you’re breaking production. This makes your services more resilient.

  • Design and use pipelines for Continuous Integration and Continuous Deployment. Design them up front, when your software is just Hello World, not at the end. This way you test them throughout the project. They are software, too. Make pipelines for your database (FlywayDB). Make a machine learning Pipeline for Continuous Learning.

  • We know that we don’t know, so we design for change, not fake frozen pictures, we design it to be pluggable.

  • Design the configuration first, not last as we typically do, when things are most rigid: consider how this will be changed and design it for that change, not for fixing on arbitrary and changing requirements.

  • Don’t treat exceptions as exceptions (ancillaries, losers in the binary opposition) but rather as one of the valid paths requests will take through the system in its cyclomatic complexity. This will make your services more robust and resilient. You’ll design Dead Letter Offices, consider retries and compensations and useful messages to quickly improve, and narrow your design appropriately. Accept exceptions, and invite them. If you don’t give them power, they will take yours.

  • Do not assume the single data tool: model your associations for change and becoming, with extensibility as the feature. This will mean there are many data models: the different database implementations, in which one may be suited for reads and one for writes under different services with different scalability needs and usage patterns. There are also different views of the same data: yes, use materialized views but also caches and denormalized patterns, which may differ.

  • Use the Specification pattern (see “Specifications”) for decoupling external search criteria from the entity.

  • Design for the team, the first user of the software, to maximize flow. You as a designer are making space for the rest of the team so they can do their work in parallel to maximize velocity and ownership. Design for the team too and then Conway’s Law will work well, and the software will be well organized and developers can be accountable and have pride of ownership.

Diversity of customer practices

  • Outcomes over activity.

  • Value creation over process obedience.

  • Employ use cases with an outside-in approach.

  • Commit to the Value Chain and make it efficient towards outcomes. Do not focus on policing developers or preventative architecture review boards to ensure compliance with an arbitrary and academic committee that does not own an P&L.

  • Your internal colleagues are your customers too: developers are the first users of the system; the Network Operations Center monitoring crew are users of the system; testers are users of the system. Everything you do in the design to support this diverse customer set will pay off.

  • There are nearly 200 countries on earth, with thousands of languages. Externalize user strings and design for internationalization and localization from the beginning.

  • If this is a web system, design it for a watch. If this is a phone app, design it for a game console, or for voice. Foreground varying UIs by creating a separate UI Package service. There is no “The UI.”

  • Use eventing liberally. We default to synchronous request/response models, as if we know the meaning, we know what should always happen. Instead, foreground asynchronous. This improves scalability and description of the system. But it also does something for you where you don’t have to decide the meaning: you allow the “import” of the event to be deferred. This is powerful because the business changes their mind frequently, the system evolves, different customers need different things, things means something different to diverse audiences. Any reaction in your application should not be hardcoded. Use event handlers.

  • Allow for a multiplicity of voices using polyglot persistence and polyglot programming: make them right for the job and there will be different kinds for different jobs in your application. This is the majesty of AND, not the tyranny of OR. This dramatically improves performance and prevents lock-in of contracts, licensing, and thinking.

Opening

Though perhaps counterintuitive when they were born, some of our most cherished and time-tested tools and ideas embody some form of these ideals, particularly in a field in which we didn’t imagine why we needed to keep source code private, and we often just shipped for free with hardware products until the Copyright Act of 1976. This rich history includes even (again) our language, such as recursive definitions, as in, “GNU stands for GNU’s Not Unix”:

  • BitTorrent, Cassandra database, and the World Wide Web itself, as peer networks

  • Unix variants

  • The Apache Foundation and open source projects

  • Wikis

  • Eclipse, Emacs

  • Chaos engineering

  • Servant leadership

  • Quantum computing

  • Blockchain

So in a sense, what we propose is a radical departure from the status quo, that few would recognize as the corporate enterprise architect job, and is shocking once fully understood. Yet in a sense we have intuited these things in our history, and in this way made some of the most important contributions to the field. Semantic Design, X Architecture, is scary-radical, such that the standard-issue business person paying even some attention might find it astonishing, preposterous. Yet it bears traces of some of the best work in our collective history, and looks backwards as it looks forward, and in this sense is nothing new, nothing to fear, tut tut. But practicing it as a framework, as a method, as a collection of job strategies as we put forth here will help us build the future better.

We have practiced this art, this craft, this design method, this strategy, this way, and it has proven to work well. Nothing is perfect. We will invite new problems in this, surely. But in making software this way, we find we can do more with less, we move quickly, our designs are better, our projects more successful, the software more sturdy, flexible, scalable, harmonious, even beautiful and delightful. Our customers do better. Our businesses do better. We do better.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.136.18.48