CHAPTER 13
Model-Driven Everything

We have noticed a shift toward model-driven development (and away from application code-driven development) often co-occurs with the shift toward Data-Centric approaches. This isn’t surprising if you consider the mindsets and thought patterns behind the two concepts. When someone embarks on a Data-Centric journey, they begin thinking differently about their data. They see different relationships between data and applications, and between data and architecture. This has been evident in the case studies discussed thus far.

Model-driven development

The concept of model-driven development has its roots in parametric Computer Aided Design (CAD) and Computer Aided Software Engineering (CASE) systems. Parametric CAD describes an approach to designing physical parts; the approach is especially productive when designing intermediate parts embedded within a complex manufactured artifact.

When you look at a complex assembled product, such as a car, you might think that all the parts are independently designed and assembled. The average automobile contains over 30,000 parts53. Some common parts (e.g., standard screws and bolts) can be used across many different variations in the drive train, but many unique or specialty parts must be modified with each redesign.

One common intermediate part in an automobile is the motor mount, which attaches its motor to its frame. The size, shape, and fit of the motor mount piece depends on the motor and the frame; all three must fit together perfectly. As such, if the company decides the following year to extend the chassis by 3 inches or to put in a more powerful (and heavier) motor, last year’s motor mount will no longer properly fit when building the new model Historically, this meant that every redesign sent engineers back to a literal drawing board, forcing them to very precisely redesign each intermediate part to the new specifications.

But that was then, and this is now. Instead of hand-designing every intermediate part, automotive designers have figured out how to build and utilize a digital models of many of the intermediate parts.

The engineers working on our motor mount could construct a digital model of a typical motor mount with relative ease. Within this model the engineers could adjust many input variables, including frame length, motor weight and torque. Based on these new parameters, the model would produce as output a revised design for the motor mount, optimized to perfectly fit the current year’s model.

CASE was the software industry’s first attempt to emulate such parametric CAD modeling in the 1980s. Early CASE systems were primarily code generation systems. Developers noticed that many software modules were incredibly repetitive. CASE systems allowed the developers to describe the differences and generate the software equivalent of boilerplate.

Early systems were limited in scope (they could only generate simple form-based systems typically) and were often just a way to get a first draft of the application code, but they were a start.

The movement largely went underground through the 1990s as the majority of application implementation was package based, and therefore, model-driven approaches didn’t apply. While it was underground, progress continued to be made.

While I was at a healthcare dot com in the 1990s, we designed, built, and patented the first fully model-driven application architecture.54 Many other variations have emerged since then, to the point that Gartner has now noticed the movement.

Low-code and No-code

There are now at least a dozen firms with model-driven offerings. Gartner coined the complementary terms “low-code” and “no-code”55 , and the terms typically appeal to different audiences. “Low-code” systems generate code from the inputs the designer gives it; this approach appeals more to software developers looking for a productivity edge. “No-code” systems build the general functionality once and then supply the variation via input variables. “No-Code” systems tend to appeal to business analysts who wish to deliver business functionality without necessarily ever compiling anything.

Salesforce really popularized the low-code/no-code concept. Some of the leading firms offering either low-code or no-code systems include:

• Salesforce Lightning
• Out Systems
• EnterpriseWeb
• AppSyngery (from the Sokil case study)
• Appian
• Microsoft Power Apps
• Zoho Creator
• FluidOps
• Mendix

The maturity of the model-driven approach is a great boon to adopters of the Data-Centric approach; more and more of the functionality of the system can be delivered without writing additional code. We have built systems with no application code (and very little architectural code), so we know that this approach can be married with a Data-Centric outlook.

The following sections will offer a high-level glimpse of the possibilities opened by model driven systems. For those interested in how to implement this in an enterprise architecture, look for more detail in this volume’s companion book, The Data-Centric Architecture.

Declarative code

We’ve become quite familiar with declarative queries due to the popularity of relational databases and SQL. The thing that marked the big difference between SQL and its predecessors was that with prior technology, the query or the transversal was intimately tied with how the data was physically stored. A programmer would use their knowledge of the storage structure (e.g., linked lists or hash tables) to optimize data access. The problem was that this knowledge became embedded in the application code. This approach is often called “procedural” because the knowledge of how to access the data is locked up in application code. The “how” is thoroughly enmeshed with the “what.”

With “declarative queries,” the query writer describes what they want to do, and the query manager (software) works out how to do it. The power here is that the query manager can learn over time. The declarative query need not be rewritten every time the query manager comes up with a more performant way of executing it. It is this aspect that allowed Relational Databases to go from woefully inferior query performance to mostly superior query performance.

In the same way that we can convert queries from procedural to declarative, we can do the same thing with applications. Right now, almost all application code is procedural, and therefore, a great deal of the “how” is mixed in with the “what.” The “model” in model-driven development is a declarative description of what the application needs to do independent of how it does it.

An application needs to do a great deal of work; however, almost all of what an application does can be reduced to a model, and therefore, be made declarative.

Model-driven constraints and validation

If you study traditional application software, the first depressing thing you realize is that most of the code just shuttles data back and forth from database schema or API (application programming interfaces) to screen fields and forms. We tend to think of the “high value” code in an application system as being the code that captures the sophisticated business logic. The code that just “shuttles” data back and forth between the database, the API, the Object Layer, the browser, and any other intermediary structure is relatively “low value” code. It is necessary, due to the architecture of the system, but it adds very little net business benefit.

After this “shuttling” code, the next most common category of code is that which performs validation and constraint management. Most of this is incredibly trivial. This type of code often checks whether entered data is of the appropriate syntactic type (e.g., numbers, characters, or dates), whether data has the right number of characters (e.g., in 5-digit zip codes), matches a pattern (e.g., the XXX-XX-XXXX pattern of U.S. social security numbers), or matches a predefined list of valid values (e.g., countries, states, or genders).

“Sophisticated” validation includes cross-field validation (is the “from” date before the “to” date), completeness (are all the required fields present), and cardinality (ensuring that each project has at least one and not more than two sponsors).

The sad thing, when you look very long and very hard at a traditional application system, far less than 1% of the code could be considered to be “high value” providing benefit to the business. This “business logic” is typically marbled in amongst the million or more lines of “low value” code, making it hard to find and modify the high-value code.

The good news is that most of the low-value code can be model-driven, using standards such as SHACL from the W3C.

Model-driven Constraints

As we mentioned, most of the type of constraints described above are now covered by a W3C specification called SHACL. SHACL stands for the SHApes Constraint Language. SHACL defines “shapes” that portions of the graph must conform to. A shape might declare that a person in a given database repository may have up to two biological parents. This constrains the shape of the graph.

What we didn’t mention is that SHACL is a model driven language. The constraints are expressed in data, triple formatted data. A SHACL engine then interprets the shape models and determines if the data conforms before allowing an update to persist to the database.

Model-driven UI

80-90% of the user interfaces in most applications could easily be generated from a model without any application code. There is a need to write or purchase some infrastructure code that can interpret the model, but once this is done, it executes all models without additional code.

Our experience is that there will be a handful of high use and high-value interfaces that you may want to have hand-coded, so rather than shoot for 100%, you should have a more modest ambition. Keep in mind if you do implement 90% of your user interfaces as models, you will have eliminated 90% of your code, 90% of the defects, and 90% of your future maintenance cost.

In order to implement a model-driven user interface, you will need a mechanism to establish use cases. If you have adopted a model-driven constraint approach, this will provide most of the structure you need for the UI generation. You will need to add:

• Grouping (which fields in a form you would like grouped together)
• Sequencing (within a group and between groups, what you would like the order to be)
• Entry or tab sequence (where the cursor goes on entry)
• Default values (what value is supplied if the user doesn’t supply anything)
• Actions supported (edit, cancel, delete, as well as any custom actions).

These days most UIs will be web-based, much of the styling and aesthetics will be driven by style sheets, and default designs will use Responsive Web Design (RWD). This means that UI will adapt gracefully for different form factors, such as a cell phone.

Model-driven identity management

Identity management concerns assigning unique IDs to items and detecting whether an item has been encountered in the past and therefore, can reuse the previously assigned ID. It also involves detecting whether other systems have assigned different identifiers to the same item and maintaining alias lists.

This can be achieved in a model-driven manner. It is a matter of expressing the identity rules in executable models. We have had clients who refer to this as their “match spec,” and we have adopted that term. Some match spec implementations supply confidence factors—how certain are we that these two records represent the same item?

For each class of item, at least one match spec is prepared. At the time of addition or maintenance, the architecture can determine whether the attributes of a given item (e.g., Person, Organization, Product, or Task) match those of a previously captured item. If they do match, their identities can be merged. There are several approaches to implementing the merge, ranging from ID rewriting to keeping lists of aliases.

Model-driven security

Application security is primarily focused on authorization—what are the rules governing who can see or modify what information? Unfortunately, most existing applications also deal with “authentication,” which is determining the identity and credentials of the user accessing the system. The reason I say this is unfortunate is that authentication clearly belongs at the architecture/ infrastructure level. You should be able to log in once and have your credentials forwarded to the various applications you use. Many firms are making progress on this, most call it their “single sign-on” project. The reason it has been so slow to adopt is that many applications were designed to run on a stand-alone basis and integrating with various enterprise authentication schemes was an afterthought.

We will assume that an authentication scheme is in place. Authorization is harder than it first sounds and is very poorly handled now. We believe a model-driven approach to authorization can improve things greatly, and at the same time, make access management simpler and more consistent.

The shortcomings of current authorization schemes are due to the way the application-centric mindset has carved up the problem. Currently, authorization is managed on an application-by-application basis. Each application defines a set of “roles” that determine what people can do. Typically, there is some sort of admin or superuser role that can do almost anything. There are roles for people who can change other roles, a handful of function-specific roles (e.g., Accounts Receivable and Accounts Payable), and a general read-only role, without access to sensitive information. One way or another, users must be assigned to these roles on an application-by-application basis (and removed from these roles in the event they leave the firm or are assigned to different duties).

There are four reasons this current approach falls short:

1. It is very redundant; assignment to roles and permissions must be executed in many, many places.
2. It is locally idiosyncratic; each application carves up the possible roles differently.
3. It is inconsistent; in most enterprises, the same data exists in many systems. Customer data is typically in dozens of systems (go ask your GDPR implementation project if you don’t believe me). Rules, regulations, and policies about the protection of data are not application specific. In healthcare, the Healthcare Insurance Portability and Accountability Act (HIPAA)—the standards that govern what kind of medical information can be shared—does not suggest you should implement different rules in your Electronic Medical Records system from your Lab System, your Scheduling System, Pharmacy System, or from your Data Warehouse, and yet that is what people do.
4. It is often not granular enough. I had the misfortune to sit in on some of the early planning sessions for HL7’s (a healthcare interoperability standard) approach to managing authorization. I watched as they attempted to create roles not only for Physicians Assistants, Nurses, and Doctors (which makes sense at first glance, at least), but then they went on to define roles for all the known medical specialties (and there are a lot). As complex as this was (there were over 100 roles), it wasn’t granular enough to address the issues. The core idea in healthcare is what is the relationship of the person accessing the data to the patient (for patient data). It is more important to know whether this is the patient’s physician, or a physician in that physicians’ practice than it is to know what their specialty.

Given all this, there is a great deal that a generalized, centralized authorization system must do. However, we think this is what makes the model-driven approach even more important.

We need a model-driven way of assigning people to logical roles, rather than relying on people self-selecting roles or being assigned to them by some sort of admin.

The Workers Compensation Insurance company we described was able to assign many users to roles based on their business relationship with the company. If you had a claim, then you were in the injured worker role, which allowed you access to information about your claim (but no others). If you paid insurance premiums, then you were in the role of the an employer, and you had certain access to claim information regarding your employees. If you were an employee of the Insurance company, you were in yet another role.

These roles both granted and denied access (which is one of the reasons self-assignment doesn’t work). And they are based on things that change over time (getting injured at work, for instance). If you are both an employee of the insurance company and are injured, you are in a special category (inside claims).

While there is some complexity in this, we believe that a model-driven approach is the only viable way to get consistency and coverage of authorization.

Chapter Summary

Model-driven development is not essential in the Data-Centric approach, but the two have some great synergy. When you update your architecture to embrace the Data-Centric approach, you no longer rely on monolithic application software for most of your data management functions. This shift means you have a clean slate for solving common data management issues anew. The model-driven approach is a Data-Centric way to implement common data management functions.

If you adopt a Data-Centric approach, you will often find your model-driven methods easier and more powerful. The “model” part of the model driven approach can stored in the same database, integrated with the Data-Centric model. This makes impact analysis easy. With a traditional system when you change part of the data model you have no idea what code might be affected. When you co-locate your model-driven models with the Data-Centric data model, all the impacts are available through a simple query.

Moving much of what was traditional application logic into the model makes the Data-Centric aspect of the system even more apparent. Moving 90+% of the code to the data models exposes the true algorithms rather than leaving them hidden, marbled in amongst millions of lines of repetitive code.

By merely adjusting how we design and build enterprise applications from an application-centric point of view to a Data-Centric outlook, we can achieve massive reductions in cost and complexity. In the next chapter, we explore how Data-Centric methods can “hitch a ride” on some of the currently emerging technology trends.