Chapter 14
MANAGING THE DATA RESOURCE
Getting started and keeping it going.
The previous chapters summarized the contents of Data Resource Simplexity and described the theory, concepts, principles, and techniques for data resource integration. Data resource integration develops a comparate data resource within a common data architecture. Data culture integration creates a cohesive data culture within a common data culture.
Chapter 14 briefly describes the management problems that must be addressed and overcome to achieve complete data resource integration and data culture integration. It provides a direction for establishing a formal data management profession responsible for managing data as a critical resource of the organization. All of the theory, concepts, principles, and techniques described in earlier chapters will be relatively useless unless formal data resource management can be established.
HALTING AND RESOLVING DISPARITY
The theme of Data Resource Simplexity was to stop the headlong charge over the event horizon of disparity. That charge, which continues today, has severely impacted both public and private sector organizations. If that charge continues, which it appears to be doing, the service to citizens and customers will reach a critical point where business survival becomes an issue.
The theme of Data Resource Integration is to repair the damage that has been caused by that headlong charge toward disparity. The best way to repair the damage is to recognize data as a critical resource of the organization and to begin formally managing data accordingly. The best way to gain respect for data resource management is to step up to the task of resolving the disparity that has been created and to ensure that it never happens again.
Managing Data
Understanding and resolving an organization’s disparate data resource has been adequately described in the previous chapters. However, situations with data from outside the organization must be handled, such as multiple data models, multiple data documentation sources, universal models and generic architectures, data standards, application acquisition, data registries, and acquired data. Each of these situations is described below.
Multiple Data Models
Data models, regardless of their source from within or without the organization, often conflict and do not adequately represent the organization’s perception of the business world. Many organizations have modeled their critical data numerous times using different techniques and perceptions based on the modeler, not on the organization. None of these data models should be used as a common data architecture for the organization, or even as an initial common data architecture. The best approach is to document each data model as a data product, make cross-references to a common data architecture, and use the insight to designate a preferred data architecture.
Multiple Data Documentation Sources
The existing data documentation in most organizations, whether developed by the organization or obtained from outside the organization, resides in an assortment of formal and informal dictionaries. The dictionaries may come from a variety of different sources and exist in a variety of different forms. Virtually none of these dictionaries are useful for formally managing the data resource. The best approach is to document each of these dictionaries as a data product, make cross-references to a common data architecture, and use the insight to designate a preferred data architecture. The result is a formal documentation of the organization’s data resource.
Universal Models and Generic Architectures
Universal data models and generic data architectures seldom provide any real insight into how an organization perceives the business world where they operate. Using models and architectures as a common data architecture does not help development of a data resource that supports the organization’s business information demand. The best approach is to document these data models and data architectures as data products, make cross-references to a common data architecture, and use the insight to designate a preferred data architecture.
Data Standards
Data standards cannot resolve the existing data disparity, and it’s even questionable if they can prevent further data disparity. Seldom do they provide any real insight into how the organization should build a data resource to support its business activities based on its perception of the business world. In many situations, data standards are simply a reporting requirement between different organizations. The best approach is to document these data standards as a data product, cross-reference them to a common data architecture, and use the insight to designate a preferred data architecture. Data transformations can then be specified to meet the reporting requirements.
Application Acquisition
Application acquisition is a prominent trend today, but a purchased application can substantially increase data disparity and could warp the organization’s perception of the business world. The proactive approach is to document the data architecture of an application before the application is purchased, cross-reference it to a common data architecture, and determine how different it is from the preferred data architecture. Then an informed decision can be made as to whether or not to purchase that application. If the application does not have a complete data model, the application is in question and should not be considered.
If the application has already been purchased, the reactive approach is to document the data architecture of the application, cross-reference it to a common data architecture, and then determine how that application should be used to support the organization’s business. Data should not be indiscriminately dumped into an application (a brute-force-physical approach) without formally determining how those data should be managed.
Data Registries
Data registries are often viewed as a standard data model that should be used. However, the data model in those data registries are, at best, the perception of one organization and how that organization perceives the business world. The data may not represent how another organization perceives their business world. Again, the best approach is to document a data registry as a data product, cross-reference it to a common data architecture, and use the insight to designate a preferred data architecture.
Acquired Data
Data are often acquired from outside the organization, or through mergers and acquisitions. These data are typically quite disparate and seldom support an organization’s perception of the business world. These data should be documented as data products, cross-referenced to a common data architecture, and used to designate a preferred data architecture. Then the data can be transformed accordingly to become part of the organization’s comparate data resource to adequately support the business.
Approach
Several approaches to halting and resolving data disparity, include stopping hype-cycles, resolving the lexical challenge, changing attitudes, and stop warping the business. Each of these approaches is described below.
Stop the Hype-Cycles
The current trend of one hype-cycle after another must be stopped to achieve formal data resource management. Hype-cycles are a form of silver bullets, and usually become tarnished bullets that often make the disparate data situation worse. Hype-cycles have not helped data resource management become professional, and lead to much of the lack of respect for formal data management.
Some organizations attempt to make hype-cycles persistent, through declaration, standards, certifications, and so on. However, hype-cycles seldom lack any real foundation in theories, concepts, principles, and techniques. They are typically current centric, trendy, and often have profit motives. In due time, they all fail and are replaced by another hype-cycle.
The best approach is to establish formal data resource management based on sound theories, concepts, principles, and techniques. Formal data resource management becomes persistent and pervasive, and leads to a formal data management profession. Standards, certifications, and initiatives are built around formal data resource management, and can change as technology changes. Organizations can then create initiatives to implement formal data resource management, and can use any theme or logo they desire to carry that initiative within their organization.
Resolve the Lexical Challenge
The current burgeoning lexical challenge in data resource management must be resolved. Formal terms must be established based on roots, prefixes, and suffixes in a language (English, German, and so on), must have a comprehensive and denotative definition, and must be used consistently throughout the data management profession. The data management professionals themselves must be actively involved in creating a formal lexicon of terms and must be willing to use those terms.
Many formal terms have been presented in Data Resource Simplexity and the current book to help achieve the goal of a formal lexicon for data resource management. A combined Glossary has been provided in the current book and is readily available to anyone wanting to use formal terms. Hopefully, professional organizations will endorse these terms, begin using them, and will promote their use.
I ran across an interesting term a while ago—integrating data integration. Actually, it was about data integration, not data resource integration. However, the approach was to combine all of the individual data integration efforts in an organization into a single data integration effort. The idea was good—with respect to disparate data integration. The point that was being made was exactly the same as the data resource integration described in the current book. A formal approach is needed to integrate all of the disparate data in an organization and create a comparate data resource.
Change the Attitudes
Many people have attitudes about data resource management that must be changed. After Data Resource Simplexity was published, I had several people approach me and really get on my case about my comments on attitudes. They tore me up one side and down the other. Who did I think I was? Attitudes—really? People standing around me were waiting to see how I would respond. After listening to these outbursts, I calmly replied, I rest my case.
An enlightening situation occurred a number of years ago with a fellow manager in another organization. He commented that his programmers were very good at programming, but were not so good with the business. He made the statement that they were super compiler writers and amateur human beings. I wouldn’t be quite that severe, but many of the data management professionals today are certainly super technicians that are weak on business understanding.
I see similar situations with business professionals’ attitudes toward data management professionals. What went wrong with data resource management? Why do data management professionals create a disparate data resource that doesn’t meet business needs? How did the business allow a critical resource to be mismanaged?
These attitudes must be changed and teamwork must prevail. Business professionals and data management professionals must bring their collective knowledge and skills together in a team approach to building and maintaining a comparate data resource. A good analogy is building an airplane. Professionals on passenger comfort, wiring, hydraulics, aerodynamics, structure, instrumentation, and so on, must come together to build an airplane. No independent actions.
Stop Warping the Business
Business professionals and data management professionals, alike, must stop warping-the-business to fit an application or data model. I’ve seen golf-course decisions by business professionals where an application was acquired, sometimes against the recommendations of data management professionals, and data management professionals were left with implementation. I’ve seen data management professionals play with the technology to the detriment of the business.
Similar situations are brute-force-physical development, paralysis-by-analysis, and suck-and-squirt techniques. I recently ran across a brute-force-physical data warehouse that had low quality and didn’t meet business needs. In one meeting, the database technician asked what the business needed and one business manager explained the data needed. The database technician said, Oh, I can just stuff another column in the table to do that. No edits, no integrity, no definition, no understanding of the data hierarchy, and so on.
An analogy is an airplane that doesn’t have enough power. Let’s just bolt another engine on each wing, or cut the tail smaller to reduce drag, and so on. All brute-force-physical approaches create disparate data and ultimately impact the business.
None of these approaches is appropriate for the business or for formal data resource management. The data resource must be formally designed, built, and implemented to meet business needs. The effort must include both business professionals and data management professionals working together toward a common goal.
No Quick Fixes
Business professionals and data management professionals must stop looking for quick fixes. Quick fixes do not exist, and every attempt at a quick fix only delays a real fix, often making the situation worse. Quick fixes can’t understand the data, resolve data disparity, and create a comparate data resource. Only people can understand disparate data and build a comparate data resource.
Many ETL products offer a quick solution to disparate data. When I describe the data integration process to ETL vendors, they claim their product covers that process. Then I go into the real detail and usually get the answer something like Oh, we can do that, but you have to write the specs. The ETL products don’t, and can’t, develop the specs because they can’t understand the disparate data or perceive the comparate data resource needed to support the business.
The best approach is to recognize that only people can understand the disparate data and develop a comparate data resource. Only hard work—real hard-thinking kind of work—can understand the disparate data, perceive the comparate data, and develop the specs for data resource integration. Only people can perform that kind of hard work, and products can provide the support.
ON GAINING RESPECT
Respect is a common theme in most organizations, both for business professionals and data management professionals. Respect comes from looking at the situations that result in a lack of respect and taking an initiative to resolve those situations and earn respect.
Most Frequent Complaint
The most frequent comment I hear from data management professionals is that they don’t get any respect from the business. I agree, but the question is why don’t they have any respect? What have they done, or not done, to have a lack of respect? Data Resource Simplexity and the current book provide many examples.
The next most frequent comment I hear from data management professionals is that someone needs to get to the executives and convince them that data management and the data resource deserve more respect. Many of my consulting engagements start with a request that I meet with executives and convince them that the data resource needs to be properly managed. However, the problem isn’t necessarily with the executives.
Stop Telling and Start Listening
Data management professionals must stop telling the business and start listening to the business. All too often I hear data management professionals tell the business what’s wrong and the resolution. Seldom do data management professionals really listen to the business.
I’ve heard many people mention creating an elevator pitch, so that when they have a couple of minutes with an executive or business professional they can throw that pitch and gain some respect. The bottom line is that most of those elevator pitches are not within the other person’s problem set, are poorly received, and don’t work. Here we go, data management professionals telling me what to do again.
A far better approach is to take the opportunity and ask an executive or business professional what problems they are facing. Engage them in their problem set, rather than adding something new to their problem set. In many situations their problem has roots in a data problem, which can be pursued to a successful conclusion. If not, at least that executive or business professional had an opportunity to discuss their problems, and the data management professional may just have learned something about the business.
Stop Demanding and Start Earning
Data management professionals must stop demanding respect and start earning it. Demanding respect seldom achieves any true benefits, and often alienates people. Earning respect provides lasting benefits for both data management professionals and the business. The proper approach is to command respect, not demand respect.
All of the concepts, principles, and techniques in Data Resource Simplexity and the current book, if followed, will go a long way toward earning respect. Real hard-thinking kind of work to stop further data disparity and resolve the existing data disparity will allow both business professionals and data management professionals to command respect.
Other IT Component Disparity
All four components of the information technology infrastructure have disparity. The worst disparity is with business activities, and the next worst disparity is with the data resource. The least disparate is usually the platform resource. The business activities are the most dynamic component because they must reflect changes in the business world. The data resource is the next most dynamic because it must maintain historical data as well as current data. The platform resource is the least dynamic.
Stabilizing the data resource through data resource integration assists in stabilizing the business activities through business activity integration. Many well-intentioned business activity integration initiatives, sometimes known as business process reengineering, have failed because of a disparate data resource. Formal data resource integration, the use of data brokers, and forward and reverse data transformation provide a stable environment for business activity integration.
Therefore, the best approach is to resolve the data resource disparity to provide some stability in the organization. Then tackle the business activity disparity based on a stable data resource. Any platform disparity can then be resolved to handle a stable data resource and stable business activities.
Take The Initiative
I’m often asked how long it will take to resolve a disparate data resource and create a comparate data resource. The reasonable answer is that it will take ten years to substantially resolve a disparate data resource if the organization is diligent. However, in all likelihood the entire disparate data resource will never be completely resolved. A point of diminishing returns is reached where it’s not feasible to expend the effort to resolve the remaining disparate data.
I’ve seen organizations take their data redundancy from a factor of ten to a factor of four and be ecstatic about the results. I’ve seen other organizations take their data redundancy from a factor of eight to a factor of three and not be satisfied until the disparity is reduced further. The choice is up to the organization and their perception of the point of diminishing returns.
Generally, when data redundancy reaches a factor of two or three, the point of diminishing returns has been reached. The organization can concentrate on maintaining the comparate data resource and prevent further data disparity. The remaining data redundancy will disappear over time without much additional effort.
I often ask people what it’s like after they have resolved data disparity and created a comparate data resource. The response is always something like they can’t believe how easy it is to maintain the data resource and support the business information demand. They fully understand the power of a comparate data resource and its support for the business. They wish they had managed the data resource properly from the beginning, and can’t understand how they ever let a disparate data resource evolve.
Getting Started
I’m frequently asked what’s a good starting point for starting data resource integration. The starting point varies for each individual organization and is based on specific problems with the data resource. The best starting point is where the organization is feeling the most pain with data quality or data availability.
Finding the pain points is relatively easy if one listens to the business problems—remember to stop talking and start listening. Engage business people in a discussion about the problems they have and how those problems might be related to the data resource. Meet business professionals in their offices or a neutral place and start discussing their problems. You may be surprised at what you learn.
A real incentive to getting started is readily sharable data. When the pain of not meeting business needs exceeds the pain of crossing organizational boundaries and sharing data, people will begin integrating the data resource. When the pain of existing low quality data exceeds the pain of creating a comparate data resource, people will begin interacting and creating a comparate data resource.
One approach to getting started is to sell the benefits of disaster reduction or disaster avoidance. Disparate data create a risk for an organization by being large and costly to maintain and by not supporting a rapidly changing business. That risk could lead to a disaster if an organization misses opportunities or fails to fully utilize an opportunity. Developing an integrated data resource can prevent a disaster or reduce the impact of a disaster by reducing the risk.
Talk to business professionals about impact avoidance and cost avoidance. I used to say that long term benefits has lost its meaning and short term benefits was more appropriate. I’m now realizing that even short term benefits has begun to lose its meaning. I now talk about impact avoidance and cost avoidance, which seems to have a much better reception.
Getting started may be a perception problem. Creating a comparate data resource is intangible to most people. They can’t visualize the benefits of developing a comparate data resource. The perception is that if a process does not produce code, it does not provide any benefit to the organization.
Building a comparate data resource is often perceived as an esoteric process where the rubber meets the sky. Past data management practices have enforced that perception. The perception needs to be overcome by selling the disadvantages and benefits of a comparate data resource.
Paint a Vision
The initiative must provide a very vivid vision about what’s wrong with a disparate data resource and how a comparate data resource can support the current and future business information demand. A vision must describe the future and focus people on the benefits of a comparate data resource. It must include creative ideas from a wide variety of people, including visionaries and knowledge workers.
The vision must bring order out of chaos, comparity out of disparity, and orderliness out of disorder. It must be so powerful, so compelling, so overwhelmingly beneficial in its simplicity that the organization can only respond with Yeah, that’s what I need.
Avoid Excuses
Many organizations use the excuse that data resource integration is too difficult and too expensive. They can’t afford the resources or the perceived impact on the business. They seem to be in a comfort zone, in spite of the pain they are feeling due to low quality data and the business information demand that is not being met.
One thing I’ve noticed over the years is that when business is good and revenue is up, organizations don’t see a need for data resource integration. When business is bad and revenue is down, organizations don’t have the resources available for data resource integration. There never seems to be a point where the business is just right and the revenue is just right for data resource integration.
Organizations must avoid the excuses, recognize the pain that is caused by disparate data, and establish an initiative for data resource integration. The result will be much like the person who’s eyesight slowly diminishes and finally gets their first pair of glasses. They never realized things could look so clear.
No organization can afford to shut down business for several months, let alone several years, to develop a comparate data resource. Any organization that shuts down that long will likely go out of business. Data resource integration must be done on-the-fly, while conducting normal business activities.
Going Out of Business
I’m often asked if an organization will go out of business due to a disparate data resource. The answer is possibly yes for private sector organizations. I’ve been involved with several private sector organizations that have either lost their customer base or lost control of their customer base due to improper management of the data resource. These organizations were in grave danger of going out of business, had they not regained control of their customer data.
The answer is probably not for public sector organizations. Most business functions in public sector organizations are legislatively mandated. Those business functions can only be eliminated by legislative mandate. However, a disparate data resource can result in poor management of a business function and a waste of resources, which brings public scrutiny in these tough economic times.
Therefore, the best approach for both public and private sector organizations is to properly manage the data as a critical resource so that it adequately supports the current and future business information demand.
Techniques Exist
One comment I frequently hear is that no techniques are available to stop the creation of disparate data and create a comparate data resource. People say the profession has not matured enough to handle these complex problems. Well, the profession will never mature with that attitude.
The techniques are available and have been proven. They are easy to use and produce results. They are relatively easy to implement, although real hard-thinking work is needed to produce results. They provide the easiest route to follow that assures success.
The techniques provide a success motivation that becomes contagious. They lead to involvement, which leads to commitment, which leads to acceptance, which leads to success. One only needs to start using the techniques—maybe as early as tomorrow.
Keeping it Going
I used to wonder how I would keep a data resource integration effort going after the current project was completed. How would I keep the momentum going? How would I get the next project started?
In the majority of situations, I’ve been pleasantly surprised. The issue was not going out and finding another data resource integration project, it was choosing which one of those waiting in line would be the next. People continually approached me to ask if they could be the next project for data resource integration.
Once the initiative was started and people saw the benefit of integrating disparate data and creating a comparate data resource, they wanted their data integrated. I frequently started one team in one subject area, got it moving, and then started another team in another subject area. Often I took a key person from a successful project and put them on a new project to keep it moving.
Business Involvement
Business professionals should, and will eventually, become more involved in managing the data resource that supports their business activities. They are the people knowledgeable about the business and the rapid changes that occur in the business. They will become more data management enabled to design, build, maintain, and use a comparate data resource.
Data management is likely to go to the business, rather than remain in IT. The same skills and techniques will be required, regardless of where the data management function is placed. However, placing that function in the business will ensure that it more readily supports the business.
I’ve had a relatively easy time teaching business professionals the skills and techniques for managing data, and a relatively difficult time getting traditional data modelers and architect to understand the business. I’ve had a relatively easy time getting business professionals to understand data quality with respect to the business, and a relatively difficult time getting traditional data modelers and architects to really understand data quality.
I’ve had a relatively easy time getting business professionals to take both a people orientation and a business orientation. I’ve found it easier to get business professionals involved in a teamwork approach. I’ve found it easier to get business professionals involved in a business survival orientation.
I’ve found it easier to convince business professionals that quality is not free, but it’s far less expensive if built in from the beginning, than adding it later. I’ve found it easier to convince business professionals that data quality improvement and data resource integration has a point of diminishing returns.
SUMMARY
The task of managing data as a critical resource of the organization requires patience and understanding. It’s difficult and challenging, but it’s far from impossible. The situation will get worse before it gets better, but it will get better. The benefits of a comparate data resource more than pay for the data resource integration effort.
The approach and the benefits vary from organization to organization, from project to project, and from year to year. The process for starting and maintaining an initiative to create a comparate data resource varies from organization to organization. The secret is to start on a pathway toward success and learn along the way. Follow the concepts, principles, and techniques described in Data Resource Simplexity and the current book.
Seize the opportunity to stop further data disparity and create a comparate data resource that supports business survival. Seize the opportunity to begin formal data resource management and manage data as a critical resource of the organization that supports the current and future business information demand.
Don’t delay! Start today!
QUESTIONS
The following questions are provided as a review of managing the data resource, and to stimulate thought about what’s involved in formally managing a data resource.
- How are data from outside the organization managed?
- What needs to change to establish formal data resource management?
- How can data management professionals start earning respect?
- How can resolving data resource disparity help resolve disparity in the other IT components?
- How does an organization get started with data resource integration?
- Why is a vivid vision necessary for starting data resource integration?
- How is an initiative to integrate the data resource maintained once it has been started?
- Why is business involvement in data resource management necessary?
- Why is data resource management likely to become a business function rather than an IT function?
- How can data management professionals help establish a formal data management profession?