The Value Chain for Designing and Building, Called Engineering
We took a couple of weeks away from our business training since Luke was busy with school work and had suffered a cold and a mild bout with the flu bug. As he regained his health and acquired a chunk of free time, we returned to our preparation for his summer lemon business.
My next educational objective was to illustrate how value chains and data models can be designed with systematic discipline while still allowing for huge creativity and expressiveness. My 34 years of IT work are laced with ad hoc, extemporaneous improvisations that tried to reinvent value chain and data modeling.
Highly intelligent computing professionals can tend to acquire a case of overconfidence. This leads them to ignore the structure and discipline of their own industry and launch off down some appealing and creative, but damaging, pathways. Additionally, industry data models and pattern libraries are often underused for the same “not invented here” syndrome. I caught this affliction for decades, but I wanted Luke to respect the past century of engineering discipline and not lapse into thinking he is smarter than the cumulative wisdom of thousands of his predecessors.
To make this point, I started shopping around for engineering projects we could visit, but first Luke hopped online to read a quick definition from Wikipedia:
Engineering is the discipline, art, skill, profession, and technology of acquiring and applying scientific, mathematical, economic, social, and practical knowledge, in order to design and build structures, machines, devices, systems, materials and processes.
I would add that engineering design generally solves the design problem from general to specific. This is called forward engineering. In some cases, you have an existing product where you need to reverse engineer the design. Reverse engineering goes from specific to general. Engineering also ensures that the general design is faithfully embodied in the specific design. Of course, the specific design adds some new aspects to the general design while being faithful to the general design’s intent.
The Value Chain for House Designing
I have a friend, Ray, who is a structural engineer. He designs and builds houses, and he suggested I let Luke observe his next project. At first, this seemed like a waste of time, but Ray pointed out that houses are concrete things that are easily understood. Also, housing has a long history of treating engineering as a discipline. Therefore, if Luke can understand the process of designing houses, he can be both structured and creative in designing various products that need lemons.
I was doubtful that Luke would benefit, but since I was curious myself how the process worked, I said yes. Luke was resistant to the idea, but he likes Ray’s wacky sense of humor, so he said, “Sure, let’s go; that Ray is a real kick in the pants.”
Ray started with a discussion of the steps in his process, shown in the table below from general on top to specific on the bottom.
He starts out with a general scope statement for the project. This is the most general state of Ray’s design. Then he adds enough details in the owner’s view step to get clear agreement with the customer on a cost and timeframe for the project. Ray uses a cardboard house to give the owner something concrete to look at. Next, he sits at his computer and works with 3-D geometry to design the architect’s view. This view describes internal structure that was not relevant to the owner’s view. Ray’s builder, who will do the actual work of construction, will take Ray’s blueprints, partition them for each subcontractor, and then add some details to produce the builder’s view. Ray’s builder subcontracts most of the labor to specialists in plumbing, electrical, sheet rock, framing, etc. The subcontractors will add material suppliers and specific SKUs to the builder’s view to create the subcontractor’s view. When the project is complete, the family will move into their “finished house.”
Level 1 – Defining the Scope
First, we watched Ray talk with the Smith family as they discussed what they wanted in their new house. The husband wanted a pool and a workshop, the wife wanted a 1,500 square foot kitchen, and the kids wanted a media room that sounded a lot like Disneyland. Luke viewed the parents as being unreasonable, while he found the kids’ needs for their Disneyland media room to be perfectly rational.
Ray locked the Smiths in on a budget and then gradually established priorities for allocation of the budget. Everyone was partially disappointed in the end, but the final compromise was true to the budget and priorities. The purpose of defining the Level 1 deliverable of scope was to ensure the price and features of the house meet the new owner’s general needs and are within his budget.
Level 2 – Designing the Owner’s View
Next, Ray built a 3-D computer model of the house and a miniature cardboard house with little trees and fake grass. The cardboard house had a roof that could be removed, making the interior rooms easily visible. In the 3-D computer model house, they could walk through the rooms and look around. Ray sat down with the Smith family and walked them through both the cardboard house and the 3-D model house. Each family member had some comments on likes and dislikes.
Ray updated the two model houses with the changes requested by the Smiths, updated the cost and time estimates, and then repeated the process. After the second review with the Smiths, everyone agreed to the price, time, and features of the new house. Their previous disappointment had now turned to excitement about the prospects of a new house. The purpose of the Level 2 deliverable for the owner’s view was to add enough details to the scope statement to ensure the owner fully understood what he would be getting for his money.
Level 3 – Designing the Architect’s View
A few weeks later, Ray invited us over to see the architectural blueprints he had drafted. They were huge sheets of paper with front views, side views, top views, and landscape views. The Level 3 deliverable of the architect’s view was created in part to communicate to the Smith family, but it was mostly created to communicate to Jones & Sons Contractors, who would be the builders for this project. They needed a clear and detailed picture of what they were going to build.
Level 4 – Designing the Builder’s View
About a month later, we got a call from Jones & Sons Contractors to come over and see the builder’s blueprints. This time there were many more large rolls of blueprints. Jones had a set of blueprints for each subcontractor who would be working on the house. There were separate blueprints for foundation, framing, plumbing, wiring, flooring, and several others we chose not to view. Luke noticed that Ray had called for a metal I-beam in the large open space, but Jones’ blueprints had called for a glue-laminated beam in place of the I-beam. Jones explained that the I-beam manufacturers were all on strike, so the cost and time for them would be prohibitive. He had substituted a glue-laminated beam since they are half the cost and can be picked up at any time.
Jones explained that this Level 4 deliverable of the builder’s view was created to communicate clearly to the subcontractors exactly what materials they should use when they start working. The subcontractors would be the people who would pick up the materials, take them to the site, and assemble the materials to create the house.
Level 5 – Designing the Sub-Contractor’s View
Shortly after visiting Jones, we got a call from John at Valley Framing subcontractors. John invited us over to see what he had created using Jones’ blueprints. John had long lists of materials, with quantities for each item and the supplier addresses where the items could be picked up. This was the most detailed description of a house I have ever seen. He had pictures of nailing patterns to use at key stress points, as well as the fabrication sequence for the roof joists. John explained that this Level 5 deliverable of the subcontractor’s view was to communicate to the people who would pick up all the materials, transport them to the site, and swing the hammers that built the house.
Level 6 – Appreciating the Finished House
Ray called us about four months later to visit the finished house. The inspectors were there at the time, doing the last inspections before issuing the occupancy permit for the new owners. Luke was very impressed with the final result; having seen the process from inception to completion made it even more memorable for him. The purpose of the Level 6 deliverable was to give the customer their finished goods.
The Value Chain for Software Engineering
Since the house project observation was so educational for Luke, I decided to try it again, but this time with software instead of houses. Luke had expressed an interest in helping me with the database design for the computing system I would build to assist him in his lemon business. I called some old IT friends and requested a few visits to show us a real IT software project in action. After eliminating the IT projects that looked too chaotic, we found a nice small project that was on track enough to spend some time with Luke. My friend Rachel offered to let us sit in on design sessions over the next three months to observe the process.
Level 1 – Defining the Scope
Our first visit with Rachel’s IT project was a review of the “return on investment” calculations for various scopes of resources, time, and features. This conversation reminded me of the Smith family, where the kids wanted a media room with all the features of Disneyland. The business wanted huge features with minimal time and resources to make their return on investment look earth-shaking. The IT staff held their ground, insisting that in the real world of IT the resources and time estimates should be an order of magnitude greater.
After an hour and a half of heated debate, everyone settled on a single scope statement that included a modest and realistic return on investment with reasonable amounts of resources, time, and features. The purpose of this Level 1 deliverable of scope is to assure that the timeframe, price, and features of the system meet the business owner’s needs and budget.
Level 2 – Designing the Owner’s View
For our next visit, Rachel referred us to Mike, who was the project’s Conceptual Modeler. Mike walked us through the conceptual value chain model and the conceptual data model. They were both crystal clear as to what would happen in the business after the new system was delivered. Mike explained to Luke that conceptual data models are simple models that focus on the major aspects of the project that are of importance to the business owners who are sponsoring the project. All the details of how computers and software work are hidden in conceptual models since those details are not required for the business owner who is paying for the new system.
Luke crinkled his eyebrows and looked at the ceiling as he recalled the cardboard house that Ray produced for the family shopping for a new house. Yes, the conceptual model serves the same purpose as the cardboard house did in the house project. During the review, there were some IT people present and some business people who had been developers of software in a previous incarnation. Both groups tended to discuss server performance issues and network latency issues. Mike would gently reel them in by reminding them that these conceptual models are not a computing system design and therefore cannot have slow response times.
When the group returned to the topic of the conceptual models, they did uncover some missing pieces and a few things that were out of scope. The scope statement was ambiguous about these things. The conceptual models removed all the ambiguity from the short and simple scope statement from Level 1. The purpose of this Level 2 deliverable of the owner’s view is to add enough details to the scope statement to ensure the business owner clearly and fully understands what he will be getting for his money and when he will be getting it.
Level 3 – Designing the Logical Information System Designer’s View
Our next call from Rachel was to introduce us to James, the project’s Logical Modeler. He was holding a design session with the IT architects and a few key developers. James was running into the same issues as Mike. The meeting contained several technology experts, and during the session, they kept discussing technologies, supportability, availability, and performance. Since the logical model is independent of all technology considerations, James would gently reel them in and return to the topic at hand, which was logical value chain and logical data modeling.
It was interesting to see how James would take the processes and entities from the conceptual models, elaborate on them to develop finely detailed processes and entities with attributes, and then abstract them into higher-level processes and entities. He showed us how these abstractions provide the flexibility to easily change the system as the business evolves.
Luke got lost at this phase. He understood the scope statement and the conceptual models, since they were at a low level of abstraction, but the logical models were beyond his young mind’s ability to comprehend. Abstraction in design would need several additional learning sessions before Luke would understand. The purpose of this Level 3 deliverable of the logical model is to abstract the conceptual models into easily changeable structures and to partially communicate to the business owners about the system you will be building. But mostly, it is to communicate to the designers and builders of the physical system.
Level 4 – Designing the Physical Information System Designer’s View
Rachel sent an email to us to schedule the meeting with Louise, the project’s Physical Modeler. Louise knew the database technology backwards and forwards. In our session, she took the logical models and structured them to match the technologies to be used in building the system. Focus on technology, supportability, availability, and performance was encouraged, since these physical models must leverage the strengths of the technologies while avoiding their weaknesses. The purpose of the Level 4 deliverable of the physical model is to enable building the data structures that will store data about the business in a specific database technology.
Level 5 – Building the Developer’s and Tester’s View
Rachel did not want Luke and me to bother the developers and testers as they did detail design and coding work, so we browsed the source code tree to view the code and test cases that were developed. Luke was totally bored with this exercise, but he did perk up when the database schema diagram was reviewed. The purpose of the Level 5 deliverable is to provide the software, installation instructions, and hardware requirements for those who will install the finished hardware and software solution.
Level 6 – Appreciating the Finished System
Rachel let us visit the business site to watch the users of the new system as they took support calls from people who bought products and needed some help getting the full value from the products.
Luke was able to identify portions of the user interface that corresponded to pieces of the database schema diagram. That evening, after a wrestling and tickling match on the living room floor, Luke stopped for a moment, looked up at me, and said, “Dad, I want to learn how to data model.” I tried not to look shocked and euphoric as I pseudo-calmly replied that we would start tomorrow. Just before bedtime, I introduced Luke to the Zachman Framework, developed by one of the many brilliant IBM research and development employees in the 1960s and 1970s. The Zachman Framework looks at how people design and build things.
He was pretty confused during my lecture—and sleepy—so I related the Zachman Framework to the house and software projects we had recently observed. Since all six levels of the Zachman Framework for engineering align well with the six levels of the house and software projects, Luke finally got it well enough that we could turn out the lights and tell stories until the sound of his sleeping breath signaled that dad was officially on break for this day.
Key Points
- To deliver successful projects for houses, software, or any product or service, we need a balance of structure and creativity.
- The structure comes from following the engineering value chain, where we design the general deliverable before designing the specific deliverable.
- The creativity comes from delivering unique designs, using innovative building processes, and inventing methods for accurate, timely measurements of progress through the engineering value chain.
- And, all of these engineering processes are happening in feedback loops that trigger re-design, re-build, and re-measurement.
The next morning, we started with an introduction of what a model is as Luke’s first data modeling lesson. A model is a microcosm of some aspect of the real world. Luke looked confused, so I clarified that a microcosm is a small pattern that matches a large pattern. For example, the real world contains some large patterns within the scope of a railroad. We built a model railroad that has the same patterns, but on a smaller scale. I remember that the model railroad in our basement was operated by Luke until the tracks were worn smooth. Our model railroad in the basement is a microcosm of the real world of trains. Good models can simulate most real world events with clear and common sense relationships between real world events and the corresponding model entities.
A data model represents the persons, places, things, events, or measurements required by a business (or any aspect of the real world). The data model contains the minimum set of relationship rules required to maintain consistency between the data model entities. Each entity has one or more uniqueness constraints so that we can differentiate one thing from another thing. Each entity also has a set of attributes that describe the entity. Data models are precise and cover lots of information in a single picture.
The combinatorial pathways (unique SQL statements) for navigating through a medium-sized model are in the millions. Correspondingly, the real world business event combinatorial pathways through a medium-sized business are in the millions as well. A good data model represents the set of information pathways that match the real world business execution pathways. The data model will contain both “currently used pathways” and “waiting to be used pathways.” When the business asks for new pathways, they should already exist in the pool of “waiting to be used pathways.” This means new business needs can be met without logical data model or physical database schema changes. Good data models are isomorphic (similar in structure and function) to the real world “scope” or “aspect” being modeled. Good data models also contain abstraction in the right places and at the right level based on the stage of the data engineering lifecycle (conceptual, logical, physical). Good models also accommodate the needs of the business for flexibility when business changes trigger computing system changes.
Luke was clearly daydreaming during our discussion, so we went to the basement and played with the model train set while I replayed some of the concepts. This seemed to help. It was refreshing to dust off the old train set and get it running again. It reminded me of the days when Luke’s imagination turned our model railroad into a real steam engine thundering down the tracks. Watching children use their imaginations to turn inanimate toys into living adventures makes my adult analytical brain feel humbled and a bit bored. Luke was ready now to move on to the idea of tabular data storage since the idea of models was mostly clear to him.
Key Points
- Models are good ways to organize our thinking about the real world without having to interact with real world objects.
- Models are easy to analyze since time and space are compressed into a few diagrams and definitions.
- Models make explicit the interdependences of entity/entity, entity/process, and process/process.
- It is better to find problems in models than it is to build real world things and only then find problems with the things we paid to build.
- We can use models to:
- Simulate real world events to remove inefficiencies and tune models.
- Develop a single vocabulary and a value chain for all employees, and partners, and customers.
- Serve as a baseline for computing system construction (IT).
- Serve as a baseline for business processes execution instructions (people).