Chapter 3
Organizing the four data governance arenas
Typically, the main function of data governance frameworks is to control and check what people are doing. Such an approach takes a rigorous set of procedures, checks, roles and rules. There are multiple problems with this approach: it includes clear signals of mistrust, it will easily become a bureaucracy, and it will move the focus from the data toward the control system. An overdesigned data governance framework contains rigorously detailed roles, placed in a structure on at least three levels and with decision bodies on all levels. It is characteristically filled out exhaustively with precise criteria of who appoints who and when to escalate to each level. Maybe it is the word “governance” in it that makes people motivated to invent a domestic system of justice.
Such a data governance system of justice is only about preparing for failure. Instead of getting ready to face an organized crime, let us increase the chances of doing things better.
In this chapter we will delve into how to organize data governance work, in “just enough” size, and, to strive for improvements. After all, diplomatic data governance is about initiating, guiding and encouraging people toward better behavior regarding data that benefits the entire company and its stakeholders.
Roles overview
Below are the most significant roles in a diplomatic data governance structure. Each role is defined from one arena, but any role can appear, interact and communicate in any other arena.
As mentioned, a person can play more than one role, and, occasionally more than one person can share a role. More roles than these can be active, take part and contribute in these arenas in various activities. We will have CIO’s, enterprise architects, data base designers, and so on, regardless if we run data governance, or not. They would do their job in quite a similar way anyway, albeit somewhat affected when we introduce data governance. Some of those roles may also hold a data governance role.
Below are descriptions for both accountabilities and responsibilities, so let’s start by figuring out the fundamental difference between them.
“Responsibility” means making sure that things are happening and that results are achieved. Responsibilities typically apply close to the sharp end, where things are happening. “Accountability” includes a funding aspect which implies justifying the cost for the activities or what they may imply. As the data diplomacy approach addresses behaviors, the focus here is on responsibilities.
Roles in the managerial arena:
- Data governance lead – the executive data governance officer that directs data governance operations and strategy, and coordinates people and activities to meet data governance goals.
- Enterprise data architect – the overall data design ward coordinating data governance’s and data architecture’s results.
Roles in the change arena:
- Entity manager – a business representative that is responsible for the data design in a data domain.
- Entity accountable – a business manager that enables entity managers to be active.
- Domain business data architect – a data designer that facilitates entity managers’ work.
Roles in the operational arena:
- Operational data worker – a person that has a responsibility of accomplishing correct data as a part of her or his job function.
- Data content accountable – a business manager that is accountable for data quality as a part of his or her job function.
Roles in the IT arena:
- Solution data architect – a person that designs technical data solutions based on business data design and other requirements.
The following sections deal with these roles and describes what they do, how they act and how they interact.
Organizing the Data Governance Entirety arena
Organizing the data governance entirety means organizing the roles, procedures, decision-making and other prerequisites in all arenas, included in this pie chart.
Data governance must have a leader. This is an Admiral without an armada. This leader is preferably already within the organization thus aware of the core business, has a network and is aware of the organization’s natural behavior. This leader needs to be supported by a knowledgeable consigliere10, an advisor, who could be a coaching consultant, whereas the data governance lead should preferably be an employee. To be practical, these two roles could be referred to as the data governance lead being the Admiral, and the enterprise data architect as the advisor. The rest of the people active in this area are already hired as managers and co-workers, data architects and various IT professionals.
Having just two main roles who act for the entirety is quite lean. If the workload for these becomes too heavy, as it can become in larger global companies, the data governance organization can have multiple instantiations. That would mean having more than one pairs of Admirals and consiglieres, for instance for separate business units which do not share very much data.
Taking into consideration the mission and the task there is, however, no need for more active heads in these roles. They should not do more than necessary, and they should not be loaded with work that is not about the data governance.
The role of data governance lead
The data governance lead is the driver of the data governance initiative. Having this role, you set the level of ambition for the data governance and how it should evolve. The data governance lead is a promoter and encourager of data governance without interfering in detail matters.
The data governance lead paves the way for data governance work. Typically, the data governance lead is heavily involved in finding and including individuals to be engaged in the business data design and in the business data quality.
Once individuals are found and engaged, they need a context to work in. Paving the data governance way is a full-time job. Fortunately, not everything needs to be done at once. The data governance lead is one of few roles that could work full-time with data governance.
Maintaining data quality in everyday work should, as mentioned, be regarded as part of the daily work. Thus, it cannot have any separate control, but should be integrated into the existing management systems. No new managers are needed for this, but rather it is about adding or adjusting how the management is conducted. Thus, a data governance lead needs to ensure that this happens.
The diplomatic approach with governance thus means coaching leaders to improve their data governance. Not bringing in new governance.
Undoubtedly, the best opportunities for improving the data assets are when something is subject to change, when there is gravity. Therefore, it is crucial that data governance has an impact on the change processes. From the earliest phases and throughout the change process, there must be clear elements on how to improve the business data whereas other changes are formed and launched. Data governance has no better arena for significant progress than here. Thus, a data governance lead needs to ensure that this is enabled.
The diplomatic approach thus means helping improvement in respect to data.
Data governance lead funding
To achieve benefits, awareness and, in the long run, a lasting change, a dedicated data governance department is not required. The important thing is: what is done and where it is done.
Successful data governance work can be organized in network form; a network of knowledgeable people who are spread out but who can be linked together in some way for common matters. Data governance can also be formed as a set of guidelines describing the way of working and provided with knowledge support. There needs however to be some kind organizational mechanism to have access to the members in the network when needed. There is however still a need for someone to drive the process as culture does not change by itself.
Some believe that the data governance lead should not belong to the IT department, while others claim that the role is definitely a strategic IT role. But it doesn’t really matter where a data governance lead is placed. Home is where the work is11. The most important thing is that the manager of the data governance lead values the data governance work and ensures that she/he get the best conditions.
There are those data governance leads who have a budget, own resources and an allocation in the line organization, including a formal mandate and a marked accountability. There are both advantages and disadvantages to having it that way.
The benefit of this approach is that there are resources and power when needed. It is also favorable for the coordination of ongoing activities. Additionally, it is easier to build up competence when keeping the active data governance people together.
But one should pay attention to the risks of referring data management and its resources to a single place in the organization. The other parts of the organization can perceive that they do not have to worry about data architecture and data quality as there is a special department for it. Another risk is that the organizational boundaries can become obstacles when there is a need to make some sort of correction or adjustment; it is seldom popular to have one department interfere in the responsibilities of another department. A special data quality department that also has a mandate to enter wherever it is needed can easily be perceived as a regulator, rather than a partner. The biggest risk, however, is that such a department is peripheral and isolated instead of being a natural part of ongoing operations. Such a peripheral group can easily be questioned and become extinct in the next reorganization.
The role of the enterprise data architect
While the data governance lead is the figure that drives, engages and promotes data management activities throughout the organization, the enterprise data architect is the one who organizes what the data management should work with. An enterprise data architect identifies information areas, subject areas, entity areas, data domains, or whatever they are referred as where you live, as a world map of the organization’s data assets.
An enterprise data architect is aware of how the organization’s data is handled and has a general knowledge about which data is vital. This means knowing what the current situation looks like, including strong and weak points. Like the data governance lead, or perhaps through her or him, the enterprise data architect has knowledge of the coming challenges that the organization will face and what data is required to meet these challenges.
As indicated when introducing the Business Data Design arena, we need a set of business-oriented data architects. It is necessary to organize them in order to meet the business’s challenges. One way to divide the data architecture work is to specialize in different data areas, such as Customer Data, Product Data, or Financial Data. Another is to work with different business areas, which can, for example, be different types of offers or customer segments or product lines. A third way is to act as an internal consulting organization and act as a knowledge resource when needed; having data architects with broad and generic skills but however not too specialized.
No matter how this work is divided, an architect or team of architects needs to provide an overall picture. Doing so is normally the responsibility of the enterprise data architect or the enterprise architecture team.
The Diplomatic Enterprise Data Architecture
The enterprise data architect not only needs an enterprise data model. A person in this role also needs to know how to use such a model. An enterprise data model is not, as some people think, “The Model”, you know, the overall blueprint that should be implemented throughout the entire enterprise, using force, if needed. Those attempts occurred during the information-engineering era in the 1980’s. The reason that was stopped was because such work was endless. And, there was no receiver, customer or user for such an enterprise-wide data model. Except for the person who created it, of course.
The enterprise data architect is not the person that architects the entire enterprise’s data assets – he or she is the person who makes sure these data assets are architected.
The person who is acting as the enterprise data architect may occasionally step into a data design role and for instance support a project. But the primary role of the enterprise data architect is to coordinate and coach the data architecting efforts throughout the enterprise. This implies that, ideally, the enterprise data architect has both architecting skills and leadership skills. It is perhaps not the best architect that is the best lead for enterprise data architecture.
The enterprise data model is a great tool for the enterprise data architect. In a start-up phase, it is a good idea to organize the contents of existing data models, according to the enterprise model areas, by splitting them up to fit into the areas. This helps clarify the enterprise data model and enables any previous work to be reused. The data architects who once drew these models will feel honored and be glad to have their previous results reused.
An enterprise data model can be expressed as a data model, which is a diagram depicting the structure of how data is connected. But the easiest way is to have a list of data areas, sometimes referred as subject areas. Using graphics is of course preferable. The small example in Figure 3-3 below is a set of data areas, each formed as a box.
There is no particular order of the data areas in the diagram but there are a few principles outlining how they are organized in the picture.
- The data areas that reflect master data (slowly changing data that is commonly used) are placed along the edges.
- Product master data are placed on the left, master data about parties are placed at the top and geographical master data at the bottom (with a few examples). Some use colors to reflect this.
- The data areas that reflect the business events (data keeping daily business processes together) are centered in the model.
These principles are merely good practice and each organization should develop its own layout standards. A benefit from using a common layout standard when drawing larger business data models is that it facilitates readability. Especially in a large organization, this kind of consistency can help people more quickly understand different models.
Each data area can contain a data model. As data governance and data design proceed, the granularity level on this model can increase since after a while we will need higher resolution in what we are governing.
The enterprise data architect requirements should set the level of ambition for the work on business data design. This implies that all those who architect data in the organization have a role in supporting the overall enterprise architecture.
Organizing the Data Design arena
Because many organizations do not run business-oriented and business-represented data design in a structured way, it is worth spending some time thinking about how to organize the data design and change arena. This section is centered around the entity manager, a central character in business data design.
The entity manager – finding the hero
The group of people who participate in data requirement work and in business data design meetings is usually temporary. When the work is done the group evaporates. Their sense of ownership for the result is thus short of duration and fragile. This may not come as a surprise as data modeling is hardly what they have spent 10, 000 hours on doing. They just go back to their “real” work.
The trick is to find and grab a candidate entity manager in this group who may take over a lasting responsibility for the business data design results.
An entity manager is a role within data governance that takes responsibility for the definition, structure and designation of business data. This person is not an IT professional. Rather, this person works in the actual business and probably has done so for a while. A perfect entity manager is
- well-experienced in her/his field,
- has first-hand insight into the challenges that the organization is facing, or is about to face,
- has opinions about how things are done, and ideas how things should be done,
- is broad- and open-minded,
- has been in a “misery zone”, meaning, has experiences from data-related things that went seriously bad,
- is trusted by others,
- can cooperate with others.
Such a person may be talkative, but he/she could also be an undiscovered superstar that has not yet had a breakout at the proper arena. But the odds are that this is most likely a heavily occupied person.
Then it remains which entities this person should manage. Again, follow the principle of managing quality where the data is created. A set of entities whose data is created in the same part of the organization where the person works, can form an entity area. This grouping can be influenced by, but should not be completely ruled by, the enterprise data model.
“My passion for data structures and data quality brought me into entity management which turned into a career path.”
Joachim Bondeson, Process Manager and Entity Manager at Volvo Penta.
The example in Figure 3-4 below depicts “one entity is managed by one entity manager”. That is of course convenient but not always possible. It works quite well for kinds of data that originate at one place in the organization or in a well-standardized process, such as data about products, physical assets, employees, suppliers and ledger accounts. But for data that is captured in many places and in many different ways, such as data about customers, work orders and logistics parameters. it is not as easy to find just one person to manage that single entity. In those cases, entity management can be staffed with a small set of representatives each covering their area.
An entity manager manages mostly metadata (definitions, relationships, etc.). The operational data itself, that is, the data, is usually managed by other people in the organization (refer to the Data Content arena described shortly). One exception is entities that reflects category data, such as “Target Group” in the example model in Figure 3-4. In these cases, the category value set is managed by the entity manager as the value set constitutes the definition of the entity.
Entity accountable
Having an entity manager, it is quite easy to find the entity accountable: just appoint the closest director above the entity manager. Here, the role is deliberately referred to as entity accountable and not Entity Owner, as an employee does not own anything that in fact belongs to the company.
The mistake often made is to first appoint an entity accountable who in turn appoints entity managers, without either of these understanding what it is about. The result is rarely successful.
Entity managers are best found, not appointed. People in this role need to already have a high level of understanding of the data, the processes that create it, and the uses to which it is put. As noted earlier, an entity manager who has been tempered in a ‘misery zone’, through experiences of things that have gone wrong with data, will be better prepared for this work than one who has not gone through that kind of experience.
The most important task for an entity accountable is to allow entity managers to engage in enterprise-wide data matters and spend working hours on these. Simply find the proper entity manager and appoint her or his boss as an entity accountable.
Later, in a more mature state, the entity accountable may be asked to do some measures and guidance about business data directions. But this is not required in the early phases as it is usually about first getting the basic data to work as it should. So otherwise, don’t bother the entity accountable very much.
Designing business data properly takes knowledge, not rank.
Business domain data architects
“Architects refers to the art and science of building things (especially habitable structures) and to the result of the process of building – the buildings themselves.” 12
DAMA DMBoK2, Data Architecture Chapter
According to the DAMA Data Management Body of Knowledge (DMBOK2), architecture refers to both the ideas and to the results. That applies very well on the perception of data architecture. The DMBOK2 states that data architecture encompasses three essential components:
- data architecture results (drawings and specifications),
- data architecture activities (fulfilling intentions) and
- data architecture behavior (mindset, skills and collaboration).
This harmonizes very well with the diplomatic approach of what to expect from a data architect. In order for the business data design to become a reality, it will require all the usual architectures within systems engineering. Typically, it is the architect who, as in art and science, is the author of the design of data processing. The diplomacy aspects appear in the third bullet above, the challenge to stand back in the art process and adopt an interpretive approach to the creation of business data.
The entity manager is supported by a diplomatic data architect who needs to have the ability to interpret the entity manager’s perception of how the business data is organized and how it should be developed into understandable business data models. And to do this without influencing the result too much.
Finding suitable business-oriented data architects is just as important as finding the right entity managers. In addition to being capable of changing from designer to the supporter of the method, the architect must also be:
- Communicative – this work is mostly about communication.
- Interested – being able to listen to what is said, and almost said.
- Experienced in data modeling – being able to translate thoughts into graphic structures quickly.
- A comprehensive thinker – able to keep the whole organization in mind – that is avoiding data design from becoming too local.
- Enduring and patient – it takes time for business representatives to enter business data design mode.
- Committed to the goals – by guiding the business representatives into accomplishing a feasible and yet audacious enough data design.
A data architect with these characteristics will naturally influence the work. And that is okay. Facilitation does not solely mean reflecting what is being expressed. It also means influencing a wider and forward-looking perspective. The diplomacy part is in the ability to balance between leading a group to a solution rather than imposing the architect’s own solution. In the beginning, the person playing the enterprise data architect role may act as the business data architect and thereby set the standards for others. When the business data design process has matured and more and more areas are modeled, it is a good idea to divide the work between some facilitating data architects. These architects can divide the work into domains within which they develop their specialist knowledge.
How to cooperate in the Business Data Design arena
Five roles have been presented, so far:
- The data governance lead
- The enterprise data architect
- The entity manager
- The entity accountable
- The business domain data architect.
They all have work to do in the Business Data Design arena. How to cooperate can be determined using a set of principles, described below.
The initial main task for the data governance lead in the Business Data Design arena is to scout for possible entity managers. This takes a lot time spent on tracking and enquiries. Possible candidates are preferably invited to contribute in business data design in areas that still lack a person with entity responsibility. The selection of people for such work is therefore almost strategic, at least tactically important. The data governance lead will need to use, and indeed also extend, his or her network in the organization. It is time-consuming work, as each new person she/he meets needs to be presented and convinced of the benefits of doing business data design; including business representatives.
Once up and running, the data governance lead needs to ensure the entity manager involvement in various development initiatives. This is one of the main tricks of having engaged and active entity managers: keep them busy and they will remain in your business data improvement movement.
Active entity managers prevent having data architects as bottlenecks.
The data governance lead needs to make sure that the entity managers are not too busy and turn into bottlenecks themselves. The entity manager might also undertake other kinds of roles like a product owner (a project or system role), a project steering committee member, and also digging into the data as such, thus becoming too busy.
A rule of thumb is that an entity manager should not hold roles in more than two arenas.
The key to reach enterprise-wide data consistency and coherence is progress in a coordinated way. This can be done in a non-bureaucratic way.
Check with others before closing a data design process. Correcting later will take more effort. Bad design consumes more efforts than getting things right from start.
But everybody is busy, and in particular the entity managers are busy as they are people who are often in high demand to start with. Any coordination effort needs to be lean and brief. The following way has proved to meet this.
The trick is to keep track of these relationships. Both entity relationships and human relationships.
No business data is autonomous. No data model can therefore be autonomous. No system data contents can be autonomous. Use the business data model to understand who to coordinate with. Data crosses borders, so do people. Because data entities are related to each other, entity managers must be connected to each other as they manage their data. Thus, each entity relationship to another entity area implies a connection to another entity manager.
As soon as a data design changes or a completely new business data design emerges, these related entity managers should be:
- informed, if we want to use their data in a new way. In particular, if we want to process it further,
- committed, if their data needs to look different, or, contain something more, or less, to fulfill our new design,
- activated, if our new design will mean changes for other entities.
As an example, if the entity manager in charge of “Product Offer” entities intend to introduce Target Groups as a sub-division of the Customer Segments, a logged email informing the entity manager of the Customer Segments would be sufficient, if the formalities require so. But if the Target Groups require a new Customer Segment data value, it will require a committed entity manager to do so, or to reject such a request.
The data architects, if they have split the responsibilities by domains, need to relate in a similar way. As they are a little bit more committed to data models than the entity managers, they may discover the need to initiate and promote cooperation and relationships between the entity managers.
So how big is this data network? The ratio between the number of entity managers and the number business domain data architects is on average 10:1, having approximately ten or so entity managers supported by one business domain data architect. And there may be from one to around 20 data architects in a Business Data Design arena where at least one should be the enterprise data architect.
Consequently, when fully organized, we are talking possibly hundreds of entity managers in a large organization. That is why entity managers need help from the business domain data architects to determine who to coordinate with and when to connect with their counterparts who have responsibility for other entities.
The business domain data architects need help from the enterprise data architect who is responsible for setting the scope of what the entire network should focus on. The data governance lead needs to keep track of the entirety, such as how the coordination works, how the people who need to be engaged are activated, and how to ensure engagement for emerging data areas.
This might seem a little complicated. Once this is working, it is quite simple. Start with understanding what kind of data is being targeted when a business data design initiative is about to begin. Include responsible and affected entity mangers in the work and select what business domain data architect should facilitate. Go from there according to the principles above.
Making business data design decisions
With a network of hundreds of people, of course, the question arises: “who decides?” However, it is not that complicated. Two things are needed: a decision-making body and a decision preparation process.
Let the decision to adopt a data design be close to the data source. Thus, engage the entity managers in the decision-making. Do all the work in the preparation process so that people understand the context and the best option, and the decision meeting is only formal.
The decision-making body for data design is, in its simplest and most effective form, a collection of entity managers. The implication from this is that the entity accountable is delegating to the entity manager to make the data design decisions. The decision-making group can be built up by a business data domain, that is, a few areas from the Enterprise Data Model.
Any governance body set up to guide business data design will need a mandate to make these design decisions on behalf of the entire organization. This mandate needs to come from the highest possible level, such as a strategic decision-making body with at least C-level participation. The delegation is typically proposed and prepared by the data governance lead as a part of the “paving the way” work.
The boundaries and interface between the delegated data governance decision bodies can be defined and influenced by the enterprise data architect. The trick is to avoid stove-piping between these decision bodies is to use the entity manager’s co-operation principles mentioned in the previous section. The decision body should follow the data, not the organization.
Someone important should be the chair. Possibly, a CIO. Not the data governance lead, at least not initially, as this may be perceived as invasive; the data governance lead would then both be paving the way, and striving for his/her own power. Maybe the chief enterprise architect would do given that he/she is in a position of authority in business matters and is a person of standing.
Note that these decision bodies are not staffed with the entity accountable but with the people are used to be included in data design.
The data governance lead sets the agenda and prepares the meeting and makes sure that the proposals are ready for decisions. Each decision proposal is submitted by an entity manager and is prepared with support from his/her regular supporting domain data architect.
The main idea of making the preparation work brief and lean is to have the proper involvement and participation in the data design process. As mentioned earlier, those who are affected by a change in the data design have either participated in the modeling, or, have had the opportunity to validate the result. Compliance and alignment to existing decided models is settled during the design work. The support data architect does this in the background while modeling.
Apart from the data design, the decision preparation process needs to include the following steps:
- Clarifying what is being changed, such as an existing data design, or a decided business data model (if it has not yet been implemented).
- Producing an understandable description of the design proposal.
- Having the design checked by issuing the proposal to concerned parties, such as related entity managers and data consumers.
- Resolving technical implications such as what systems, data hubs and integration that will be affected.
- Checks against parallel ongoing data design and implementation work.
These steps do not have to be carried out in this particular order. They can be rearranged, depending on the change being addressed. First, there needs to be an as-is and a to-be description, which are steps 1 and 2. Producing an understandable design proposal description can be done using a “Data Model Story” (see Chapter 4). Resolving technical implications should not be the responsibility of the entity manager. Rather, an IT-oriented architect will determine where, when and how the business data design proposal will be implemented. The design check with parallel ongoing design work is easiest to do while designing the proposal.
When all this is done, the entity manager can bring the prepared proposal to decision. The decision meeting will then be just a formality. The design has already been worked out, verified and validated in various contexts. As a formality, the chair checks that all steps above have been taken and then asks all delegates individually in the decision-making body if they grant the proposal. If so, it is passed, and the decision can be communicated and put into work.
The chair does not need to engage completely in the decision material. It is enough just to trust the entity managers and that the process is working. Decisions may include:
- A business data design, such as an improved CRM data model, a changed customer discount rule set, a product designation convention, a customer loyalty data structure, or, a unified data model for product construction and product assembly.
- Changes in values in governed Category Data Entities (reference data values), such as the name and definition of a new Customer Segment or the data naming of a new product line.
Category Data values (often reference data), such as customer categories, can sometimes be hard to distinguish whether there is an entity in its own right, or, a value in a category entity. The difference is usually a matter of generalization, or convenience in separating responsibilities among the entity managers. The values can be defined and decided in the Business Data Design arena, or, in the Data Content arena.
Organizing the Data Content arena
When organizing the Data Content arena, one cannot rely only on the business data models; capturing, using and processing data in daily operations does not really correspond to normalized data. Rather, data content needs to be understood in terms of workflow, business process and data flow. Data models can however be used to clarify what kind of data is being managed within these flows. As mentioned previously the best way to get high quality data is to get first-hand information.
Capture data once near its spot-of-origination.
That means get as close to reality as possible. This is also valid when defining the data conceptually; go to the spot where the data is naturally created and let the people in that environment determine how the data should be defined. In this way, information for data definitions can also be sourced first-hand. It is preferable to assign responsibility for data quality in the context of where data is captured or created.
Since each organization is unique, the organization of operational data content also becomes unique.
When applying a Non-Invasive Data Governance approach13, one avoids appointing people into a data governance organization top-down. Rather, one strives to find those who are already doing important data work, formalize that work, and acknowledge the contribution of those that do such work by assigning them a data management role, if needed. Not only is this non-invasive and non-coercive, but it is also diplomatic.
Avoiding “stewards”
The “data steward” is a frequently used term within data governance. Traditionally, it is a role that cares for data, from various perspectives, toward the expectations on data governance, such as improving data quality. It is, however, omitted here of several reasons.
First, the “steward” part of role name is indicating that it is someone that is wiping someone else’s mess, similar to a steward on a Caribbean cruise ship. Even regarding the data stewardship as an administrative role indicates that it is a role dealing with a separate thing from the business operations.
Secondly, it is a role that is designed for data governance and is not a term that neither the business or the IT sides can relate to.
Moreover, what the role implies is not easy to adopt. If you are a line manager for an office dealing with customer’s claims, you are probably not recognizing your work as a data steward. The data in your office is just a reflection of the work you are dealing with, not a separate thing. Keeping track of the office’s performances and how well it meets the expectations include, or should include, data quality measures combined with other indicators. As stated already in the introduction, everybody is a data worker. Stewarding data may be what we already are doing; simply do your work as it is expected and the data you create will be of high quality and available in time.
Roles do not improve things. Activities do and responsibilities makes activities happen.
If there are problems with the data, it is not only about correcting it, which a traditional data steward would be dedicated to. Rather, it is about preventing it from being incorrect again. Picture two teams that have similar tasks and the same prerequisites, but they perform their tasks differently; one team exceeds expectations and the other one performs low. We may very well be thinking of two teams that are resolving customer’s claims. Let us say that the second team is late when delivering data, does not comply to standards formats, and often omits data. Monitoring the data quality, data standards compliance, or data delivery accuracy may be a way to monitor the team’s performances, together with other measurements. How would we improve the performance in the second team? Probably by letting them learn from the high-performing team, combined with coaching activities, and investigate if there are any obstacles preventing them from performing.
Examples of organizing customer data and product data
Below are two examples of organizing the accountability and the operational responsibility for data correctness. In both cases, the business data models help to visualize this accountability.
In this first example, a company’s market structure corresponded to the organizational structure and thus the operational responsibilities, as shown in the data model in Figure 3-12.
Each market unit has a responsibility to develop its own market. Account managers at the market units, who find, establish and maturate customer relationships, undertake this development. As these account managers are very close to the natural spot of customer data origination, they become responsible for customer data quality. In this real example, this formed the basis for data content responsibilities, as the data structure was already coherent with the organizational structure.
As shown in Figure 3-13, the regional manager became accountable for data correctness and completeness while the account managers became responsible for accomplishing it. The middle manager was given a coordinating role.
As one of the regions was covering the European Union, this distribution of data responsibilities was driven by GDPR14 compliance reasons. The managers’ data responsibility included not only a customer master data database table, but all data regarding the customers.
The CRM data model for this customer concerned more than just the GDPR scope. As shown in Figure 3-14, almost all sales data related to customers was included in the data responsibility. And, not shown in this figure, much more data related to this was included, such as quoted products, customer contract contents, and so on.
Consequently, the account managers for larger accounts needed to delegate the data content management to others on the account team. Doing that was purely a local decision and completely born out of practical reasons.
According to their enterprise data model, there were many more entities related to the Customer entity, such as the Customer’s Invoice and Claim. As that data was not originated close to the account managers, the responsibility for that data resided elsewhere.
In the second data content organization example, the data model did not completely follow the organizational structure. Parts of the product data structure could however be used to address data governance roles, but the data flow and the data life cycle were the key to making data more coherent throughout the organization.
The product line managers were assigned accountability for product data correctness and completeness. The product managers who were reporting to product line managers, were made responsible for all product data concerning their product models throughout the product’s entire life cycle. They became product data suppliers, and also product data coordinators, working with their product representatives from other domains, such as pricing, logistics, and aftermarket. All of these people could very well have been called ‘operational data workers’.
This organization was primarily established to gain efficiency and worked very well. The most important coordination work the product manager did was to follow the product data flow across organizational and geographical borders. The main task for the product line managers, being accountable for product data, was to make sure that this cross-functional data flow was established and in motion.
Business processes completed with master data
The following example describes a situation where a company had data governance for master data and a responsibility structure for managing the business processes. These were organized separately.
There was an early decision taken to let data governance cover nothing more than master data. The idea was to start there, gain experience and then let this knowledge area grow. This scope was prioritized because the master data for products and customers had been a neglected area for ages.
In parallel, work was done to organize responsibility for the business processes. When both data governance and process work were well established and were quite mature, an interesting finding was made; they were incredibly well suited to each other.
The process management work did not pay much attention to master data. It was simply just expected that the data would be available whenever a process was about to be executed. This meant, in turn, that those who worked with master data had well-defined customers for their outcomes. The process management work focused on the data that was used within each process boundary; for example, the invoicing process focused on invoice data, not the master data about customers and sales items. Within the master data work, the master data was centered in terms of covering the entire organization. Within the processes, fast data was of interest, and within master data the static and slow-moving data was the focus.
Business process management completed with management of master data is a very successful marriage.
Simple enough, the data governance within the process management requires correctness and completeness in order to achieve high process performance, because the customers and the owners required it. The achievement of having high correctness of master data was there because the processes required it. One thing that made this marriage successful was the process managers habit of inviting proper master data entity managers when remodeling the business processes.
Making decisions about data
As a principle of data diplomacy, no new forums or decision-makers are to be organized just because data governance is established. As the responsibility structure for data should follow the existing organizational structure, existing decision forums can be used in most cases.
Let us continue with the customer data accountability example in the previous section. The top sales managers, on a vice president level, became data content accountable. In fact, they already had that accountability, as they are accountable for the co-workers in their regions who were acquiring accurate data and making that data available for those who need it. Nobody looked at the organization targeting certain managers to be appointed as data accountable.
Instead, each manager at a certain level such as vice presidents was approached by a data governance lead to clarify “What data are you accountable for?”
They already had VP Sales meetings every second week where data matters could be resolved, if needed. Decisions or statements, in this example, include:
- Comment on changes proposed in the Business Data Design arena.
- Decisions and harmonization regarding rollouts of data design changes.
- Decisions regarding compliance with regulatory requirements affecting data.
- Decisions on accessibility and dissemination of data.
- Addressing data design change requests to the Business Data Design arena (which in practice means taking part in the design process at the appropriate time and to the appropriate level).
Many of the data decisions made and data improvement actions taken in these meetings happened even before data governance was established. The main difference is that data matters received more attention. Also, that data change matters became directed to the Business Data Design arena. Some of the operational data workers and those who are data accountable are preferably included in the business data design work.
“Data content management at my job works a little like lubrication in our production. It is silent if everything works but it will crack and squeak if something is wrong.”
Johan Lindholm, data governance lead at a utility company.
Organizing the Data Processing arena
The data governance, business data design, and data content roles meet in the Data Processing arena. The entity managers act as subject matter experts in various projects, either as active parts in the work or though project steering committees.
Business domain data architects communicate the business data design models and guide solution architects and business analysts where to find the model documentation that they need.
The data governance lead and the enterprise data architect are engaged in IT planning to secure the data change aspects in roadmaps and project portfolios. They also work to improve the IT processes to make business data improvement a natural part of the IT work.
All these roles should be prepared to spend time on training and coaching. There are constantly new project members that need to become familiar with the processes that support good data design.
Putting the “I” into IT
The following sections contain suggestions for improvements in common IT activities and processes.
Plugging business data activities into IT processes where data governance roles cooperate is diplomatic. Plugging in gates to check other people’s results is coercive.
The “I” in IT stands for “information”. Information is concretized by data. The activities described below put the “I” back into the IT, in case anyone had forgotten what IT is all about.
There are a few obvious reasons for including business data-centric activities into IT work:
- To benefit from the business data design that has already been done, have it translated and developed into system requirements, which can become the system capabilities that the business needs.
- To ensure that previous data design work is reused, and others can reuse what you find.
- To create opportunities to achieve shared, consistent and comparable data in ongoing system work.
The suggested data-centric activities are put early in the common IT processes. It is the job of the data governance lead to integrate these activities into these processes. And it is the job of the CIO to sponsor this kind of change.
Project portfolio planning
IT works take budgeting, regardless of who is paying. Portfolio planning is usually an optimization of, or even a trade-off, between funding, business demands, enterprise architecture roadmaps and data improvement strategy. Project portfolios are often prepared by portfolio managers and decided by a CIO.
Using a diplomatic approach, the data governance lead has an agenda of what data will be improved, and the enterprise architect has an idea of what systems should manage what and be transferred where. The enterprise data architect strives for a “normalized” entirety and meaning, aiming for reducing redundant data and promoting data reuse.
If these roles agree on a common picture, they can schedule business data improvement initiatives into projects and activities.
Being two and well prepared increases the chance getting the data design schedule integrated into the project portfolio intact.
It is of great importance to address business data design activities into initiatives and projects. Not only to secure the room for business data design, but also to plan and secure the critical resources for the initiatives and projects. This may need the data governance lead to act.
Modeling business capabilities is a common way for enterprise architects to organize system landscapes and to form and scope projects in roadmaps and portfolio planning. These business capabilities usually just appear in enterprise architects’ minds, sometimes in a miraculous way. There is however a methodology to identify and model business capabilities in a data-driven way, described in Enterprise Architecture Made Simple: Using the Ready Set Go Approach to Achieving Information Centricity15. When the business data is accurately represented in high-level data models, it can be used to influence systems planning and portfolio management.
Project initiation
The data governance lead needs to set the activities for business data modeling in important projects prior to the project’s start. Otherwise, the project manager must be convinced to bring such activities into the project plan – which can be a problem as a project manager is eager to exclude everything that is not a part of the expected project delivery. What’s more, a business data model is usually not a project deliverable, whereas the system is.
Lastly, when introducing prerequisites for a project and communicating how projects are run, the business design and various data governance activities must be settled in the project plan. Once a project has started it is usually too late. Again, getting it right from the start helps for a smooth delivery.
Concept and feasibility studies
As mentioned earlier, the business data design is best performed in the early phases of an IT project. Perhaps, it is even carried out prior to the formal project start. While modeling, reuse existing business data model material; for example, models that are approved and checked into data governance model management, and models that have turned up in parallel initiatives. The whole purpose with this is to move forward in a common direction, and to progress at a common pace, when needed.
Project coordination is expensive and takes time. Coordinating verbally between people is probably the most expensive way of doing it. What is at the forefront of people’s minds and the risks being considered are usually topics easily spoken about. But if team members look at each other’s data models and sample data, risky overlaps, gaps or confusing representations become immediately obvious.
So, what is it that makes data models and data examples particularly capable to be coordinated? Simply that they reflect easily what a system will handle. If a system does something without any business data being involved, then it is probably not very important.
Data architects usually undertake such coordination work, and by doing that, they become busy bottlenecks. It is therefore better to put entity managers to work by letting them use their networks to coordinate on data matters.
A great way to minimize time-consuming coordination is to hold business data models in workshops, having the interested parties in the same room. Read about conducting business data modeling in workshops in Chapter 4.
Requirement work
During requirement work the business data models are more detailed. This can be done in a project requirement phase or in a sprint. There is a section in Chapter 4 which describes how to detail a business data model based on requirements.
A detailed business data model needs to be aligned with the approved business data models. Being diplomatic, this is done simultaneously as a business design data model is being detailed. This can lead to changing existing data structures which in turn lead to change requests toward approved, or existing, data models. If the appropriate entity manager is included directly, the change request process can be settled in a few hours. Minor changes can even be set by a small email conversation. Keep these things lean.
Somewhere during the detail requirement work, a solution data architect takes over the responsibility of improving the business data, from a business-oriented data architect. It may be the same person having two roles, but this is not preferable.
The business data modeler is an interpreter, a facilitator and the business’ advocate. The solution data architect is a systems designer in the IT community. These are two different jobs that require different personalities.
The business data modeler and the systems designer are not enemies. They are a dynamic duo.
In-house built systems
When building a system, or in another way taking responsibility for the design, it is necessary to verify whether the data design solution meets business expectations, as described in the business data model. This is especially important when the system is used to manage data that is crucial for the business, such as data affecting the customer’s perception of value, or data needed to meet regulatory requirements. This can be done by merely comparing the data storage solution with the business data design model. As mentioned previously, a solution data design does not need to be equivalent to a business data design; it just needs to be able to handle the business data.
In the best of all worlds, this check-up should not be needed. However, solution architects bring many other perspectives into the solution data architecture, and when doing that, the business data design is transformed to fit what is about to be built. It is very often about improvements, and those just need a quick check. But, of course, there is always a risk that something might be missed in the transformation.
The heaviest reason for checking a solution design is that a solution architect typically has a project and system perspective, not an enterprise entirety perspective.
The most secure way of mapping business and solution data models is to use example data. Just looking at mock-ups won’t give the answer.
The business data design model can be based on designing generic system interfaces, such as Application Programming Interfaces (APIs) in hubs and standardized integration formats. Letting these be influenced by the business data design is a way of distributing the data structure and vocabulary.
Systems acquisitions
Acquiring a commercial off the shelf (COTS) application represents challenges similar to reusing an existing system, such as a solution from a company within the group has built, a group standard application, or what an authority imposes. In these cases, there is really no need for a detailed breakdown of requirements. Similarly, there is no need for an exhaustive requirement data model, detailed as if it would be the basis for implementation. The business data model would most likely do and should provide enough information to determine the implications of using such a system, as it reveals the data structure and the vocabulary. The way to go is the same as checking a solution design in the section above.
It is amazing how many organizations choose to incorporate a system without first checking if it has a data structure that fits.
Pre-launch work
Sometimes, data migration surprises come as a cold shower at the end of IT projects. When this happens, it is an indication of something having been neglected, for instance the consequences of existing data from a developed data structure. Or, the project has not recognized the low quality of the data, something which is possible to assess prior to the project. The quality shortage may even have contributed to the running of the project. The concerned entity manager or the data content accountable should have been connected to the project in some way and have reported the status of the quality.
A part of the diplomatic approach is to be frank about how things really are and trust people to take necessary action.
At some point prior to a system launch, the business data design model turns from a “to be” state to “as-is” state, or rather, “as became”. That might include a formal data governance decision. There are those who make decisions on a “to be” business data model, for setting a new standard for a data structure. Others wait until implementation to decide whether a model is valid. If others depend on an approved structure, they need to wait, which could slow them down. But, then again, if a model decision is taken too early, it may change when implemented, and those who are depending on it will need to adapt as well.
Maintenance work
The diplomatic approach to systems maintenance entails simply being awake and aware of coming business data design changes. That means keeping track of what design activities go on and how these could affect future change requests. Sometimes this involves the entity manager and the domain business data architect. An IT maintenance incident may include a quick visit in the Business Data Design arena, and, even in the Data Content arena, if the change implies modifications in data values.