Chapter 17
People, Process and Politics
Abstract
People, process, and politics are at the heart of the stickiest business intelligence (BI) project issues. As with any relationship, the interactions between IT and business are vulnerable to misinterpretations, poor communication, and disagreements that can bog down a BI project. The groups need to learn how to communicate and engage in give-and-take. It's important to identify the roles and responsibilities clearly, and understand that some roles span both business and IT. When they start building the team, IT and the business should work together to sponsor and govern design, development, deployment, and ongoing support. With sponsorship and governance in place, they can build the project management and project-development teams. BI training should involve both business and IT people, and the content and methods should be tailored to specific needs. Training is far more than just learning tools; without the concepts and fundamentals of BI and DW, the tools won't have context. Data governance is a people-centric program that is critical for transforming data into actionable information. BI projects would be easy without the challenges of sorting through people, process, and political issues. These challenges are often harder than the technology issues.
Keywords
BI training; Business and IT; Data governance; People; Politics; Process; Project management team; Project team• The technology trap
• The business and IT relationship
• Roles and responsibilities for IT and the business group
• Building the BI team
• Project sponsorship and governance
• Building the project management team
• Building the project development team
• Training types and methods for IT and business people
• Data governance
The Technology Trap
For those of us in high tech, it’s easy to fall into the trap of concentrating too hard on the products, technology, and architectural aspects of a business intelligence (BI) project. We might feel more comfortable with technology because it’s something we can control. It’s not just IT people who feel this way; in today’s society, people often assume technology is going to magically solve their problems.
However, the problem with our comfort level with products is that often the critical success factors of a solution lie with the other three P’s: people, policies, and politics, as shown in Figure 17.1. These are three things that are a lot harder to predict and control than technology. Technology will do whatever people make it do; however, people, the processes they create, and the politics that drive them can be weak links in a project if they’re not understood or handled properly.
Successful BI projects focus on business needs, not IT needs. It doesn’t matter how big a data warehouse (DW) is or what features a BI tool has. What matters is if the business people get the right information to do their jobs. As we know, when they don’t get what they want, they end up building data shadow systems to get the information they need, and this just causes more problems.
Meeting Expectations
When you fall into the technology trap, you’re so focused on technology that you tend to forget about peoples’ expectations. And one of the biggest reasons BI/DW projects are considered failures is because they don’t meet expectations.
The project may have proceeded exactly as the IT group, systems integrator, or software vendor said it would, but in the end, the business group is still disappointed. They didn’t get what they thought they would. More specifically, they could be faced with one of these typical outcomes:
• Unpopular solution—Maybe the business expected that many more business people would have found the resulting BI reports/dashboard/analytics easier to use or more helpful. So the shortfall is that only a small portion of the expected business people are customers of the BI solution.
• Difficult analytics—Perhaps the business people expected a lot more analytics built for them rather than reports or “roll your own” analysis (self-service).
• Information shortfall—Or, maybe some of the critical information that the business was expecting has not become available or is not clean enough to use with the BI solution.
Who do you blame? There could be various guilty parties: the software vendors or systems integrators (SI) who were a little over zealous with their commitments, the IT people who were a little inexperienced or did not do enough due diligence with the vendor or SI promises, and the business people who were naïve enough to believe the claims proclaimed in PowerPoint slideshows from sales people. If it looks too good to be true, it probably is. BI projects would be easy if all it took was buying the right tool.
For your BI effort, do your due diligence, understand what it will realistically take to deploy what the business wants, and above all, set the proper expectations. You might have to tell business people something they don’t want to hear, but the push-back in the beginning is nothing compared to the push-back at the end if the project fails to meet expectations. Remember, it does not matter how much you think has been delivered, it only matters what the business, your customer, thinks.
Getting People, Process, and Politics under Control
So, if you’re going to launch and sustain BI programs that meet the business group’s expectations, you’ve got to put the people, policies, and politics on your front burner. The areas to focus on include:
1. The relationship between business and IT groups
3. Organizational structure, not just of the project team, but also the people involved in the project’s use and maintenance
4. Training
5. A data governance program on the business side
6. Program and project management
7. Centers of excellence (COEs)
The rest of this chapter will focus on first five areas, with Chapter 18 discussing project management and Chapter 19 discussing COEs.
The Business and IT Relationship
As with any relationship, the interactions between IT and business are vulnerable to misinterpretations, poor communication, and disagreements that can bog down a BI project. Becoming aware of these problems is the first step these groups can take to resolve them, perhaps even avoid them altogether.
Users or Customers?
IT has historically created BI solutions for people working in internal business groups. While this is still the main customer base for BI projects, it’s now common to have external customers such as workers in far-flung business groups, prospects, partners, suppliers, and other stakeholders. So, how do we refer to these people?
Other than the term “power user,” which describes a highly-technical member of a business group, I’ve never liked calling people “users.” When it comes to IT and BI, this is more than just semantics. Instead of thinking of business users, which is rather abstract, try to think of them as business customers and/or partners, or at the very least, as people. It implies that they are more important to you, and that satisfying their needs is more critical.
Adopting a customer or partner mindset, as opposed to a user one, will help drive the interaction between IT and the business group and encourage the formation of more balanced, respectful relationships.
Who’s in Charge?
In a worst-case scenario, IT and the business are completely at odds with one another. IT says they are in control of all things related to technology and don’t want business groups acting without them. Business groups, on the other hand, feel they either do not need IT or cannot afford to wait for them. Should one side be in charge?
Like the parent of a young adult, IT needs to realize that control is no longer possible or desirable. IT needs to resist hovering over the business and, in the context of using technology, embrace business groups as their customers as opposed to their children. Customers come back if you provide great service, and rarely return if made to feel foolish.
The business group, on the other hand, has to understand that IT sometimes has to take a little more time on projects in order to better serve them, and the rest of the enterprise, in the long run. They may be juggling requests from many different business groups. Business groups that are being courted directly by software vendors also need to understand that although new technology can seem alluring, it’s not always a quick solution to their problems.
Communication Shortcomings
“What we’ve got here is failure to communicate.”
Captain in the 1967 film Cool Hand Luke
One of the key ingredients to setting and meeting business expectations (assuming the IT group can deliver the technology solution) is communication.
For communication to be effective, it must be with the right people and involve two-way discussions. The most common communication shortcomings in BI projects are:
• Overreliance on business power users. Power users are the business people most involved in using data and technology for reporting and analysis. It is natural that they are the business people with whom IT most closely works. However, they may provide a barrier to two-way feedback. Plus, they are also the people who develop the data shadow systems that are competing with the BI solution.
• Wrong (and unmotivated) people. The IT governance committees may not have the right people to make or influence decisions on the steering committee. It might also have the wrong people to get the tasks done in the working committee. The wrong people tend to send the wrong messages. When the committee is staffed with people who are involved and encouraged, they’ll be more likely to provide honest feedback. Too often, these committees become more about process than communication.
• A bogged-down feedback loop. The formal processes created for providing BI feedback and enhancement requests may have become so bureaucratic or time-consuming that business people give up in frustration. If all the work goes into the request process rather than fulfilling the request, people will seek answers elsewhere, such as in data shadow systems or listening to software vendors pitch the latest prebuilt BI solution to replace the current BI efforts. If your business customers stop asking for things, then it is likely they are getting their requests satisfied elsewhere.
The best communication is open and frank. When a project is taking longer than expected, discuss this fact and provide clear reasons in the language that the business people understand—not technical gobbledygook. Projects can be slowed down when the data integration effort is snagged by the state of the data or when the reports the business group initially specified are a lot more involved than anticipated. These are not uncommon occurrences, but they need to be openly discussed and managed.
The business and IT groups should share the responsibility for any tradeoffs or project extensions. In addition, they should communicate and manage slow creep of little snags or changes. Many projects keep moving forward with the IT group doing a great job of handling these small issues, only to be hit with a death by a 1000 needles. The business will not know your truly Herculean efforts unless they are kept informed. It is important that communication is both verbal and written so that a history is kept for reference. If they find out afterward, when you have missed your deadlines and gone over budget, it only sounds like excuses.
Don’t provide the opportunity for business people to think your BI project hasn’t met expectations, or worse, is a failure. Make effective use of two-way communications to ensure you meet expectations and avoid unwelcome surprises.
Roles and Responsibilities
When it comes to business and IT groups, each has its main role, and then there are areas where the roles can overlap. For the most part, IT’s role is to create the infrastructure, integrate the data, and give business people what they need to perform analytics. IT handles everything in the “back office.” The business group handles the “front office,” that is, using the data IT provides to do business analysis. New technology and self-service BI is now blurring these distinctions—something we discuss in Chapter 14.
The Business Group’s Role in the Front Office
In the front office, the business group is focused on the content and use of data. They have to analyze data, sometimes huge amounts of it, to keep the company running smoothly. They’re examining metrics like profitability, return on investment, inventory levels, and market share. It is this group, not IT, that has to define what a BI solution is going to accomplish in a business context and identify its business value. Sometimes this core purpose of BI is lost to IT when they are too focused on technology and generating reports.
The data and applicable business rules used in the enterprise are the core of a BI project. Without rules and definitions, data is just a bunch of characters and numbers. The business group defines these requirements, which can include:
• Business events or transactions, such as sales, employee pay, and website traffic.
• Hierarchies and groupings of dimensions such as geographies, sales, organizations, and product categories.
• Metrics that are used to monitor and measure the business.
• Business rules that may be used to select, filter, aggregate, and calculate data and metrics.
Although IT may physically define these requirements when implementing them during the project, it is the business people who are responsible for defining what they really mean in a business context.
IT’s Role in the Back Office
In the back office, IT is focused on the technical aspects of the data so they can deliver it to the business. This work may or may not require a data warehouse. IT’s role includes:
• Determining (after discussing with the business) what data is needed for BI.
• Doing source system analysis or data profiling to determine where the data is located and how to get it.
• Data modeling to enable integration and BI.
• Developing the data and information architecture.
• Physically integrating the data.
• Designing and developing the infrastructure to support all of these efforts.
Roles that Span Both Business and IT
There are some roles that may be filled by either IT or the business group. It often depends on the technical skills of the business people. It’s more likely if the business group has power users who have IT skills and are already creating customized BI applications for their coworkers. Typically, if the business group has data shadow systems, these are the people who built them. Some organizations tend to foster more power users, and therefore are more likely to create or customize their own applications rather than always relying on IT
A business or system analyst might be in either an individual business group or a centralized IT group. This is the person who typically works with the business to determine the business and data requirements for BI applications, and creates specifications that IT can use to develop them. Having the analyst in either group can work, but those in the business group tend to have a better understanding of the business operations and needs. It depends on the business structure, and which group has the people with the skills to do this work. What is most important is that someone is in this role, and it’s not just assigned to IT developers along with their other tasks.
With the trend toward self-service BI, business customers can modify or create BI applications without always relying on IT As BI becomes more pervasive, even more people learn BI skills. Self-service BI has evolved to include data discovery, easy-to-use dashboards, and, even better, spreadsheet integration, the latter of which is a familiar and popular tool with business people. As the trend of self-service BI evolves, the IT and business roles will continue to overlap.
Building the BI Team
A successful BI project team is like a four-legged table—each leg holds up its share of the weight. Remove one and the project wobbles. The four legs of a team are:
• Project sponsorship and governance
• Project management
• Development team (core team)
• Extended project team
See Figure 17.2 to see where each function fits. While large companies may be able to assign these roles to different people, smaller companies always have people performing multiple roles. The typical size BI project team I have worked with over the years is 6–12. There has been a wide range, as I have worked with a teams with dozens of people and teams of one! Regardless of the size, these are the roles that should be done, but as any small team knows, you prioritize and do the best you can.
Project Sponsorship and Governance
IT and the business should work together to sponsor and govern design, development, deployment, and ongoing support. Although business sponsors obviously need to come up with the money, they also have to commit business resources to work with the BI team throughout the project.
If there is just one BI project on the table, a steering committee will usually suffice. If there are several projects, additional committees may be required, as explained below.
Oversight of One BI Project—the Steering Committee
With just one project, the company’s oversight efforts can start small—with the BI project itself. In this situation, a BI steering committee is a very effective organizational mechanism to guide and support the project. This committee is composed of the business and IT management that the BI project team reports to. It typically meets biweekly or monthly during a major BI project. The committee hears the status from the BI project’s management and discusses issues and concerns.
Its purpose is to keep the BI project on target by talking with the project team, interacting with groups outside the project team when necessary to resolve issues or enlist support, and communicating about the project to the rest of the enterprise. For an individual project, this is a very successful model that monitors the project team and keeps them from being isolated, thereby avoiding project failure.
Oversight of Multiple BI Projects—or a Complete BI Program
With multiple BI projects, oversight gets a bit more complicated. You’ll need to expand your organizational and governance mechanisms so the BI steering committee includes business representation from the business sponsors of the individual BI projects and, ideally, from all lines of business (LOBs), business functions, and geographies that are stakeholders any of the projects.
The BI steering committee should be composed of the people who have BI budgetary control. This group will be responsible not only for the individual project’s success, but the overall BI program’s success. Therefore, they will deal with monitoring projects as well as prioritization and funding across projects.
BI Working Committee
A BI working committee is often formed as a bridge between the individual BI projects and the BI steering committee. It can be composed of the individuals managing the BI projects, often their management and their peers from business units who do not have active BI projects. The purpose of this committee, which often meets biweekly or monthly, is cross-project management and issue resolution. This group interacts with the BI steering committee and the individual BI project teams. With a proactive BI working committee, the BI steering committee may shift the frequency of their meetings to monthly or quarterly.
Program Management Office
Truly proactive enterprises will also form program management offices (PMOs) to manage cross-function efforts. In these enterprises, you can think of the BI PMO as being a supersized BI working committee. The PMO is set up to manage an effort that is cross-functional in nature, and within that charter it assists in prioritizing projects, distributing funding, communicating to the various stakeholders, measuring ROI and marketing BI efforts throughout the enterprise. In addition, the PMO sponsors the creation of an overall BI strategy and architecture that ensures that individual BI projects work toward a common goal and infrastructure. It endorses the methodologies and standards to use within BI projects. This framework assists in the reuse of data, functions, and applications across projects, ultimately improving the time-to-market of BI solutions within the enterprise in a cost-effective and flexible manner.
A word of caution: people are usually not indifferent with regard to PMOs—you either love them or hate them. They can be viewed as an excellent cross-functional organization or a bureaucratic entity that thwarts anything constructive. If your situation is the former, then getting a PMO dedicated to the BI program is an absolute necessity from an organizational sponsorship and funding perspective. If it’s the latter, then various business units will circumvent the PMO by funding their own BI efforts, often disguised within various business initiatives. This then creates information stovepipes and all the problems they cause (as discussed in Chapter 16).
BI Center of Excellence
An organizational entity that some companies have implemented, and industry analysts are promoting, is the BI competency center or BI center of excellence (COE). The COE is created to gain a more cohesive approach for IT to work within a BI program perspective. (Chapter 19 is devoted to COEs.)
Often, the IT personnel working on a BI project are analysts working for a business unit, either directly as members of that unit or indirectly through business unit funding. Meanwhile, the IT personnel developing the enterprise data warehouse are located in another group, often a central IT group.
The IT analysts and their business unit partners sometimes view the DW group as slow to respond to needs, and this fosters stand-alone (stovepipe) BI applications. The BI COE should include the IT analysts developing the BI applications in the various business units as well as the DW group. This enables the IT analysts to develop a cross-functional viewpoint and gets them to work with the DW and infrastructure efforts. Some proponents of BI COEs suggest combining the IT analysts only. That is good, but not as strong as also including the DW group and getting their unique data perspective into the equation. Data is the glue that binds a BI program and a COE together.
Project Management Team
Project management includes managing daily tasks, reporting status, and communicating to the extended project team, steering committee, and affected business people. We discuss project management in detail in Chapter 18.
The project management team needs extensive business knowledge, BI expertise, DW architecture background, and people management, project management, and communications skills. The project management team leadership includes three functions or members:
• Project development manager
• Business advisor
• BI/DW project advisor
Project Development Manager
The project development manager is responsible for deliverables, managing team resources, monitoring tasks, and reporting status and communications. This role requires a hands-on IT manager with a background in iterative development (Chapter 18). The person must understand the changes caused by this approach and the impact on the business, project resources, schedule, and the trade-offs.
The responsibilities include:
• Creating project plans and overseeing scope management processes.
• Organizing the working environment for both core and extended team members.
• Monitoring and reporting on project status.
• Managing project risks and client expectations.
• Promoting communication and coordination at all organizational levels.
• Working with business and IT to identify and obtain resources to fulfill project staffing requirements.
• Helping ensure that milestones are met and quality is delivered.
Business Advisor
The business advisor works within the sponsoring business organization(s). The responsibilities include:
• Monitoring the extended team’s involvement and ensuring they meet their commitments.
• Identifying issues, concerns, or risks that could potentially impede deliverables or quality.
• Serving as the business advocate on the project team and the project advocate within the business community.
Often, the business advocate is a project co-manager who defers daily IT tasks to the IT project manager, but oversees the budget and business deliverables.
BI/DW Project Advisor
The project advisor, who might even be an outside consultant, has enough expertise with architectures and technologies to guide the project team on their use. Responsibilities include ensuring that architecture; data models; databases; extract, load and transform (ETL) code; and BI tools are all being used effectively and conform to best practices and standards.
It’s especially important to have a project advisor when the project development manager is not very experienced with BI data or technology architectures, or when that person’s primary role is managing the people, not the deliverables.
Project Development Team (Core Team)
The project development team performs four core functions (Figure 17.3) and is typically split into corresponding sub-teams:
• Business Analysis—This sub-team may be composed of either business people who understand IT systems, sometimes referred to as “BI power users” or IT people who understand the business. In either case, the team represents the business and their interests. They are responsible for gathering and prioritizing business needs, translating them into data and IT systems requirements, interacting with the business on the data quality and completeness, and ensuring the business provides feedback on how well the solutions deployed meet their needs.
• Architecture—This sub-team designs and develops the overall BI architecture, selects the appropriate technology, creates the data models, maps the overall data workflow from source systems to BI analytics, and oversees the ETL and BI development teams from a technical perspective. This sub-team is responsible for the successful deployment of the four architectures: information, data, technology, and product.
• Data integration (DI) Development—This sub-team receives: the business, data, and data quality requirements from the business analysis sub-team; data architecture and technology from the architecture sub-team; and target data models to be used by BI analytics to design, develop, and deploy the supporting DI processes. Currently, most development teams are primarily using ETL functionality even if their DI tool offers more capabilities. Often, a system analyst who is an expert in the source systems (such as SAP or Oracle applications) is part of the team to provide knowledge of the data sources, customizations, and data quality. As business requirements get more demanding, real-time integration and complex event processing functionality become part of this team’s expanding role.
• BI Application Development—This sub-team designs and creates the reports or business analytics that the business customers will interact with to do their jobs. This is typically a very iterative process requiring much interaction with the business people using the BI applications. This sub-team is responsible for not only meeting the business requirements, but also selecting and deploying the appropriate analytical styles supporting the business workflow.
Table 17.1 shows how the individual roles fit into the four sub-teams described above. A role does not necessarily mean an individual person or job; sometimes one person assumes two or more roles, or there are several people in one role. The ideal size of the project team is dependent on the scope of the BI requirements, complexity of source systems, data volumes, data quality, analytical functionality, and diversity of the BI consumers, along with the skills and experience of the team members. Small firms may have a one-person team, while a large enterprise, in such industries as financial services, may have many dozens of people engaged in a significant BI implementation.
The individual roles are detailed below:
Principal Architect—BI, DW, and/or Technology
The principal architect is experienced with the technology and applications used to build BI systems. This person may need knowledge and experience with databases, DI and ETL, BI and analytics, data design, and technical infrastructure. The responsibilities include:
• Applying knowledge of technology options, technology platforms, design techniques, and approaches across life-cycle phases to design an integrated, quality, and cost-effective solution addressing business requirements.
• System integration of many diverse components and technologies used in the design, construction, testing, deployment, and operation of BI solutions.
• Designing technology infrastructure to support performance, availability, and architecture requirements.
• Supervising the technical aspect of the BI development project team—staff, work deliverables, and issues. The principal architect is the de facto technical leader of the project team.
• Providing input and recommendations on technical issues to the project manager.
• Reviewing and participating in testing of data design, tool design, data extracts, networks, and hardware.
There needs to be a primary architect driving the overall systems design. This role may be referred to as a DW or a BI architect depending on the orientation of the project team and its name, i.e., DW or BI group. If the project team is large enough, there may actually be multiple architecture roles with the primary architect managing the overall system and the BI and DW-specific functions split between multiple architects who specialize in specific functionality.
Table 17.1
Data Architect
The data architect determines the source systems’ data availability and understands the nature and quality of the data. The role requires an understanding of business processes and data requirements, as well as how to translate the business requirements into an actionable data model. The person in this role should be experienced in the use of various techniques to develop quality data models (e.g., joint application development (JAD) facilitation, interviewing, iterative development/prototyping of conceptual data models). The responsibilities include:
• Reviewing the source systems to understand what is available and if its quality meets analytics requirements.
• Designing the data models for extensibility, scalability, simplicity, consistency, and integrity.
• Working with the principal architect to design and implement databases that support the agreed upon BI architecture and DI workflows.
• Working closely with the DBA to help ensure that the physical design meets the business requirements, meets enterprise technical standards, conforms to security requirements, and adheres to industry standards.
Data Modeler or Designer
The data modeler or designer has a strong understanding of logical modeling techniques: entity relationship diagrams (ERD) and dimensional data modeling. The person should be proficient in using a data modeling tool and have a thorough understanding of physical implementation issues, data strategies, design implications, and performance requirements. He or she also understands the database engine, database structure, and the implications of the physical design. Although a relational database is still the primary database technology used, the data designer needs to be familiar with other technologies that may potentially be used, including columnar, massive parallel processing (MPP), NoSQL, online analytical processing (OLAP), and in-memory, in addition to BI appliances that may use a variety of these technologies along with specialized hardware and/or specific logical and physical data architectures. The role includes working closely with the DBA(s) managing the development and production databases, as well as the data architect.
There are two distinct roles in designing databases for BI and DI:
Logical data design. This is the techniques used to represent the data in its business context supporting business relationships, transformations, and rules. This is sometimes referred to as defining business subject areas. Logical design also enables the DI workflow and processes used to conform dimensions, standardize facts and metrics, and transform data to business information.
Physical data design. Logical design needs to be physically implemented, and also leverage database, DI, and BI technologies. If the project team follows the data architecture discussed in Chapter 6, then the data designer will need to physically model separate schemes for DI and BI. Designing relational databases would involve best practice designs for tables, indexes and partitioning, while columnar, in-memory, and other data storage technologies would involve very different physical designs. The variety of data storage technologies are used either to improve access speed, expand the breadth and depth of analytics, or enable various analytical styles such as data visualization, predictive analytics, OLAP, and data discovery.
The responsibilities of the data modeler or designer include:
• Defining, designing, and constructing data structures or databases.
• Working with the data architect to ensure that data structures contain all required data elements.
• Recommending database optimization and physical design changes.
• Conforming to corporate database security and database backup procedures.
Database Administrator (DBA)
The DBA is responsible for the physical databases used by the BI solution across the entire data workflow, from source systems through information consumption by BI consumers. This may include multiple databases and database technologies such as relational, columnar, MPP, NoSQL, OLAP, in-memory, and other emerging data storage technologies.
The responsibilities include:
• Designing, testing, deploying, maintaining, and securing databases used in BI solutions.
• Implementing technology and security standards for databases and collaborating with associated infrastructure resources.
• Supporting the development and testing database environments.
• Supporting production databases (this role is often in an infrastructure oriented group).
• Providing expertise to architects, developers, and data modelers on:
• Physical database design
• Database configuration, performance, and tuning
• Database utilization
• Database security and privacy
DI Developer
The DI developer is responsible for gathering and integrating data from the source systems to the data structures supporting business analytics. The required DI workflow and supporting DI code will likely require data movement, transformation, and integration with several databases. This work includes designing, developing, testing, and deploying DI code for data profiling, data validation, data cleansing, and data transformation. This person will work with the source data analyst to understand the source system’s business rules, both current and historical, to ensure they are accounted for in the DI processes.
Responsibilities include:
• Designing the system components for the DI or conversion of data from source systems to the target application.
• Constructing the extract, transform, transfer, and audit components of a data warehousing system or data conversion programs through the use of extract development tools or custom developed procedures.
• Defining and managing the manual data load procedures.
• Documentation of above. Many DI tools generate documentation but typically this is very technically oriented and needs to be supplemented with the business, data and quality requirements implemented in the DI code.
DI Leader
The DI leader is responsible for the overall processes that transfer data from the legacy systems to the data warehouse including data validation, data cleansing, data transformation and calculations. This individual will manage all DI developers (see above), as well as participate in the DI work. If the DI sub-team is small, then this individual is likely the senior DI developer and performs hands-on development work while coordinating any others in the group. As the sub-team expands, it is likely this individual will take on a more managerial role with less or even no hands-on development.
BI Developer
The BI analyst has BI tool experience. The analyst may work on BI and reporting tools, OLAP tools, data mining tools, or a variety of other tools for specific types of users. The analyst will ensure data security, user friendly reports, “drill-down” features, as well as a flexible design of data hierarchies and a logical, easy-to-use interface and web enabling of user interfaces for the people who will ultimately use the solution. Additionally, the analyst must ensure the presentation tool provides all functionality supported by the conceptual data model, and that the tool takes advantage of the physical database design features. The responsibilities include:
• Verifying the correctness of the data relationships, mapping, and definitions.
• Verifying the correctness and completeness of the conceptual data model.
• Working closely with business people and the data architect to translate business information requirements into flexible applications.
• Understanding the usage, nuances, and architecture of the presentation tool being used
• Defining, designing, and constructing reports.
• Defining, designing, and constructing system functions.
• Mapping report layouts to data warehouse objects and application software functions and features.
BI Leader
The BI leader is responsible for the design, development, testing, and deployment of all business BI consumer access of the data via reporting or analytical applications. This individual will manage all BI developers (see above), as well as participate in the BI development work. If the BI development sub-team is small, then this individual is likely the senior BI developer and performs hands-on development work while coordinating any others in the group. As the sub-team expands it is likely this individual will take on a more managerial role with less or even no hands-on development.
Business Analyst
The business analyst serves as the advocate for the business with the BI development project team, and as the liaison between business and IT. He or she gathers business needs and translates them into data and IT systems requirements. He or she determines if the business is satisfied with the BI solutions, ensuring that their feedback reaches the development team.
The responsibilities include:
• Gathering business requirements from business stakeholders and working with them to establish priorities.
• Working with principle architect or DI developers to correlate business requirements to data requirements mapped to their source systems.
• Documenting business requirements, including data needs and process flows.
• Translating business requirements into preliminary specifications for both DI and BI development sub-groups.
• Testing and validating DI and BI applications in regards to meeting business requirements.
• Communicating project status, issues and concerns related to business requirements with business stakeholders.
• Coordinating end user acceptance testing (UAT) of BI applications with business stakeholders.
Source Data Analyst
The source data analyst determines the data availability in the source systems. This role often involves reviewing existing load routines, validation programs, and report routines. He or she understands the nature and quality of the data and should provide a data dictionary of the source data (if an accurate data dictionary is not already available). Responsibilities include:
• Reviewing the source system(s) so as to understand the data they contain.
• Determining what data is available from the source system, and its quality.
• Creating/validating a data dictionary of the source system.
Business Analysis Leader
The business analysis leader is responsible for the interaction between the business stakeholders and the BI development project team. He or she will manage all business analysts (see above), as well as participate in the business analysis work. If the business analysis sub-team is small, then he or she is likely the senior business analyst and performs hands-on work while coordinating any others in the group. As the sub-team expands, it is likely that he or she will take on a more managerial role with less or even no hands-on business analysis.
Extended Project Team
The extended project team handles several functions required by the project that occur at discrete times and that are performed by people outside of the core development team. The extended team encompasses three primary roles:
• Players—A group of business customers are signed up to “play with” or test the BI analytics and reports as they are developed to provide feedback to the core development team. BI solutions developed in an iterative and interactive fashion, often leveraging agile development techniques consistently yield higher business value and adoption than the “old school” waterfall methodology. This is a virtual team that provides focused attention during specific periods of the project. Since business peoples’ time is valuable, it is necessary to schedule these times and get business management to commit to allocating people to these tasks. Communication of project status and any scheduling changes must be managed by the BI project leader.
• Testers—A group is gathered, similar to the virtual team just mentioned, to perform more extensive QA testing of the BI analytics, ETL processes and overall systems testing. You may have project members test other members’ work, such as the ETL team test the BI analytics and vice versa. Other groups that are often asked to participate are the source system experts to verify DI processes, and BI consumers, particularly “power users,” to validate BI applications. These tests include usability, but more importantly are used to reconcile data from source systems and compare results with any legacy reporting systems or data shadow systems.
• Operators—IT operations is often separated from the development team, but it is critical that they are involved from the beginning of the project to ensure that the systems are developed and deployed within your company’s infrastructure. Key functions are database administration, systems administration, and networks. In addition, this extended team may also include help desk and training resources if they are usually provided outside of development.
Training
One of the biggest mistakes project teams make is in refusing to acknowledge the need for training. This is just one of the ways companies undermine their own efforts to take BI to its full potential.
Types of Training
Getting the right kind of training is important. It’s not enough to just learn the software products. Often, software vendors offer courses in addition to their product-specific training, or devote a portion of a class on BI or ETL tools to DW/BI concepts. Although this is useful, it is generally not detailed enough to really give people an understanding of what is involved in developing a BI project, and although the people teaching vendor-sponsored classes are very knowledgeable in their tools, they may not have many years of relevant experience to draw upon in their teaching.
Vendor-agnostic foundational training—It’s important to start out with fundamental BI training that explains concepts, architectures, and best practices. This training should include the topics of DI, data modeling, business analysis, project methodology, and analytics. This type of training works best when done in-house with all the business customers who will be using the BI solution.
Tool-specific training—Those who understand the concepts of BI are ready to learn the particular tools their group will be using. This is often taught by the software vendors. On-the-job training can be very effective in relation to tools training if a group has a tool guru who would be able to mentor less experienced peers and create templates or standards for them to follow.
Training with use cases—The best way to expand the use of BI and self-service BI, especially with business consumers, is to train with use cases involving the reports and data they would use in their own jobs. For example, business analysts in the sales group might see use cases related to sales campaigns, ideally with dashboards accessing previous campaigns. Financial analysts might see use cases with analysis of budget forecast to actual expenditures. If business people train with analytics and data they understand, then learning how and when to use analytic tools is much easier to grasp and remember.
So how do you justify getting BI training, especially if you have been involved in your company’s BI program? Pitch your training as necessary to increase the business return on investment (ROI) from your company’s BI efforts or, even better, explain how you be able to better support the latest business initiative that requires analytics. Your investment of time and expense in really understanding BI and data warehousing will be rewarded when you improve your company’s corporate performance management (CPM), BI, or DW efforts in the future.
Training for the IT Group
Unfortunately, the IT group tends to skip formal training, or limits it to just the specific tools they’re using because they think that prior experience building transactional applications is enough. This is a big mistake, as BI applications are very different. IT people really need to understand not just the mechanics of the tools, but how they’re actually used in BI projects.
Working on a BI project can be overwhelming if you don’t have a solid grounding in the basics. It’s difficult to focus on the goals of the project when you’re bogged down by unanswered questions; it’s even more dangerous if you don’t even know what questions to ask. By arming yourself with foundational training on the concepts, fundamentals, and best practices, you can hit the ground running.
Also, it takes time to become proficient in ETL/BI tools and in BI itself. Too often, project managers plan tasks as if rookies are just as productive as those who have years of experience.
The types of training that IT people tend to need include:
• DW/BI fundamentals and architecture
• Tools used by individuals:
• ETL
• BI
• Database
• Database and SQL programming
Training for the Business Group
Simply sending business people to generic BI vendor training is not effective, although it often happens. The problem is that people generally do not learn how to use a tool effectively if it does not have the context in which they will use it. Business people need to understand what data is available and how that data was transformed before they can use BI effectively and correctly.
A better approach for training the business group includes customized training that uses their own data and BI application. It should take place when both the BI application and the associated data are available (usable, but not necessarily final), allowing the business people to see real-life examples and put things into context. Although this customized training may cost more and be harder to arrange up front, it pays big dividends down the line, and should be considered a requirement.
The content of business training should include:
• BI analytics fundamentals and capabilities
• Types of analytics available and how to use them
• What data is accessible for analytics
• What filters, business transformations and rules are used in BI applications
Keep these guidelines in mind for training business groups:
Class size—If you have many business users to train, it’s best to split them into smaller groups of 6–12 people. Ensure that each person has hands-on access to the BI applications and data during class. The class should be a mixture of slides, demos, and hand-on workshops.
It’s a good idea to split classes by business unit so those who use similar data are in the same class and can support one another. Include the group’s power users even if they don’t think they need training; participation in the class will allow them to support colleagues during in-class workshops, and encourages supportive relationships after the class.
Remember who the audience is—Business group training is different from IT training. The same BI tool is going to look different depending on your perspective. IT professionals tend to focus on a software product’s bells and whistles. Business users, on the other hand, are usually just looking to do their jobs, and don’t respond to training that doesn’t show how they can use a tool to accomplish what they’re doing now with spreadsheets.
Use real data—The training should use the company’s own data, not generic information. After the initial training session, encourage business people to try out the BI technology and get completely comfortable using it with actual data well before their bosses start asking them for reports. Just have someone ready to hold their hands if needed. Detailed follow-on training should then be made available for the users who really need it.
Customize it—Before starting a training program, get a complete understanding of the software features that business people actually need for their BI usage, and then train them on that functionality. It’s OK to give them an overview of the entire BI tool, but focus on what they specifically will be using to do their jobs. Get power users to look at the tool and give you feedback on which functions are most useful and which ones aren’t needed. Then you should follow the “less is more” approach when it comes to training the rest of the business. If you overwhelm them with mind-numbing talk about features that they don’t need, the useful capabilities may be forgotten.
Timing—Most classes should be 2–4 hour sessions with the number of sessions depending on the complexity and variety of BI applications, such as dashboards, they need to learn. Training shouldn’t take place too early in the project, because you want business people to be able to use the tools soon afterward, before they forget them.
Bear in mind that effective training takes time. If projects are late, business customer training is often cut or rushed. This becomes a big problem when business people then become frustrated with the BI environment and request constant hand-holding from IT. Also, the work to develop customized business training is often underestimated. It typically it takes 8 hours of development time for 1 hour of training.
Market the project—Among the benefits of business group training is that it’s a great marketing tool for the BI efforts. It’s an opportunity to connect with BI customers, create and reinforce positive feelings about using the BI solution, and get feedback that may be invaluable in developing future BI applications or improving existing ones.
Training Methods
There are several training approaches that can work, depending on your situation:
Custom class (for the business group)—As discussed above, vendor-neutral, tailored training on the basics of BI is the best way to get the business group started. Although some lecture slides may be necessary for foundational training, it is critical that each business person attending training has hands-on access to the BI tools.
On the job (for the IT group)—Learning a tool on the job can be effective, especially if experienced consultants are on hand to help out. It’s important, however, that people receive foundational BI training first, or they won’t fully understand what they’re doing with the tool. Developers who have experience with comparable products are in a good position to learn new tools on the job, because they’re already familiar with the concepts and how other tools work.
Vendor (for the IT group)—Vendor training is useful if no team members have extensive experience with the products. Not everyone needs this training, but it’s a good idea to make sure someone on the team has it, and can then mentor the others. There may be a wider selection of courses on mature BI products, so take advantage of the opportunity to send people to training targeted to their needs, e.g., send the BI developer to development classes and someone else to administration-type classes.
Train the trainer (for the business group)—This approach can work if the business group has power users. In this case, power users get trained by or along with the IT group, then go back to their business groups to teach them. This helps ensure that different business groups get consistent training across the enterprise.
Lunch and learn (for the business group)—If you are simply modifying or expanding an existing BI implementation, the lunch-and-learn approach can work well. Invite business people to lunch or breakfast sessions, introduce them to BI concepts if it’s new for them, or jump right into training them on what’s new in their BI implementation. It’s a great way to get feedback, provide ongoing support, and expand the internal customer base.
Data Governance
More and more companies are recognizing that they’re accumulating ever-increasing amounts of data, but not necessarily gaining business insights from it. The missing link is the transformation of data into information that is comprehensive, consistent, correct, and current. That is a problem technology cannot solve for you; it requires people.
The answer is establishing a data governance program to help your organization truly treat its data as a corporate asset and maximize its value. The data governance program will enforce consistent definitions, rules, business metrics, policies, and procedures for areas such as:
• Data creation
• Data movement, transformation, and integration
• Business metrics and data definitions
• BI data models
• Master or reference data
• Use cases for specific BI tools and self-service BI
• Data change management
• Monitoring governance
• Information access and delivery
• Information consumption (reporting and analysis)
Data governance helps ensure that your data is trustworthy and provides business value. The process of governance, however, is becoming more challenging as organizations rely more on data that is unstructured and from the cloud, as well as Big Data.
It is critical that data governance manages not only data creation, but also data consumption. Too often enterprises concentrate solely on creation only to have business and IT people alter data in their spreadmarts or BI applications. It is seemingly pointless to spend all one’s efforts to ensure consistent data creation, only to have the data altered, and thus become inconsistent when business people conduct their analytics.
Who’s Responsible, Business or IT?
Because data governance is all about how you handle a key corporate asset (data) and not just about technology, the business group, not IT, should be the key driver in data governance efforts. IT often regards data governance sponsorship as business executives writing a check and putting people on a governance committee (see below). While that is, in fact a great first step, the business must also be committed to it.
People from the business side need to create the data definitions, business rules, and key performance indicators (KPIs) for a data governance program; achieve agreement on them across an organization; enforce usage and compliance; and ensure that the definitions, rules, and KPIs are updated on an ongoing basis as business needs evolve and change.
The reality is that in the vast majority of cases, data governance tasks are merely tacked on to the already overloaded schedules of business managers instead of being made a priority, with other responsibilities correspondingly getting taken off their to-do lists. Without a real business-resource commitment, data governance will take a back seat to the daily firefight and will never be implemented effectively. It is critical that both business and IT commit to data governance on an ongoing basis.
Getting Started with Data Governance
Ideally, data governance is, eventually, an enterprise-wide effort involving multiple business groups and IT. In most companies, though, it starts with a project or two and then grows. If you’re responsible for getting a governance effort launched, start by creating the organization, then focus on defining processes and data.
Start Small, Build for the Future
A typical approach would be to start small and take a tactical approach with a handful of projects that have data issues. The processes you build may be focused on these projects, but they should also be reusable for future projects. The budget, too, should be set with the goal of sustaining the governance effort over the long haul.
A significant trap that many data governance efforts fall into is trying to solve all of an organization’s data problems in the initial phase of the project. Or, companies start with their biggest data problems, issues that span the entire enterprise, and are likely to be very political. It’s almost impossible to establish a data governance program while at the same time tackling data problems that have taken years to build up. This is a case in which you need to “think globally and act locally.” In other words, data problems need to be broken down into incremental deliverables. “Too big, too fast” is a sure recipe for disaster.
Narrow Down the Scope
Another issue that can arise is that the program is too high-level and substantive data issues are never really dealt with; or, the opposite problem, it attempts to create definitions and rules for every data field in every table in every application that an enterprise has—with the result being that the effort gets bogged down in minutiae. There needs to be a happy compromise between those two extremes that enables the data governance initiative to create real business value.
Creating a Data Governance Organization
Before creating the data governance organization, it’s important to ensure everyone understands the scope of the data governance effort and their roles and responsibilities. A guaranteed way to stall a data governance initiative and lead the business to lose interest is to prematurely organize the management framework and then realize you need a do-over.
Be mindful of how many people are on the committees. Often, the more people on each committee, the more politics comes into play and the more watered-down governance responsibilities become. To be successful, try to limit the size of committees to between 6 and 12 people and make sure that committee members have the required decision-making authority.
Data Governance Board
Just like a corporate board of directors, the data governance board’s purpose is the overall oversight across an enterprise. The members of this board need to have the responsibility and budgetary authority to sponsor data governance and commit the resources to implement it.
Data Governance Task Force
The data governance task force is the working hands-on group. Their mission is to define, implement, and enforce data and reporting governance. The group’s responsibilities include:
• Defining the mission and scope of the data governance effort
• Creating a road-map of projects
• Defining priorities and determining which projects from the road-map will be tackled first
• Defining data and reporting governance processes
• Communicating with data stakeholders in the business and IT groups, up to the executive level if necessary, ensuring that they understand the governance concepts and issues
• Working on initial governance projects
• Monitoring and enforcing governance processes on future projects
The group is led by its data governor, who oversees the governance initiatives and coordinates with the BI and DI COEs. The other committee members include data owners and data stewards. See Table 17.2 for an overview of their responsibilities.
Working with COEs
Ideally, this group works in concert with the BI COE and DI COE (described in Chapter 19). Working together, these teams help provide the enterprise with consistent information and useful business analytics. See Figure 17.4.
Table 17.2
Data governor
• Establishes and prioritizes data governance requirements
• Coordinates multiple governance projects
• Enables change management in processes and organization
• Coordinates with business, BI COE and DI COE
Data owner
• Assumes business ownership (or co-ownership) of key business metrics and data subject areas
• Defines data and reporting governance processes
• Resolves business metric and data disputes
• Advocates for governance in the business group
Data steward
• Partners with business stakeholders
• Defines business metrics and data definitions
• Defines data and reporting governance processes
• Works on data governance projects
• Resolves data and reporting governance issues
• Must understand business systems analysis, data structures used in DW and applications, and how to query data and use BI tools
Keeping Data Governance on Track
What gets organizations in trouble is how they actually go about implementing governance programs. Below are a few things to keep in mind to help you become better positioned for governance success.
Use it or lose it—In order for the data governance effort to produce business value, people have to actually use the data definitions, business rules, and KPIs. The governance process needs to be a complete feedback loop in which data is defined, monitored, acted upon, and changed when appropriate.
Keep communicating—Similarly, ongoing communication about governance initiatives should not be forgotten. Without it, business users are more likely to revert to their old habits, causing the data governance program to lose momentum.
Deal with change—If data governance is to be successful, both business and IT processes need to be changed; however, the accompanying need for change management is seldom addressed. People and process issues, and the internal politics resulting from them, are challenges that need to be tackled.
Put technology in its place—Software is not a quick fix, letting you avoid people, process, and political issues. Companies that buy master data management, DI, or data quality software—or a mix of the three—to support their data governance programs sometimes fall prey to the combination of vendor hype and high price tags and forget that it’s the internal interactions that make or break a governance effort.
Keep an eye on data shadow systems—A very common governance mistake is to focus on an enterprise’s transactional and BI systems, assuming that all the important data can be found there. But often, key information is located in data shadow systems scattered throughout an organization. For example, the “real” BI reports and analytical findings used by business workers often end up in Microsoft Excel spreadsheets. (See Chapter 16.)
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.