4. Technology
by Michelle C. Dy, Michael W. Hawkins, Mark Humphries
Data Warehousing: Architecture and Implementation
- Copyright
- Preface
- 1. Introduction
- 1. The Enterprise IT Architecture
- The Past: Evolution of Enterprise Architectures
- The Present: The IT Professional's Responsibility
- Business Perspective
- Technology Perspective
- Architecture Migration Scenarios
- Migration Strategy: How Do We Move Forward?
- In Summary
- 2. Data Warehouse Concepts
- 1. The Enterprise IT Architecture
- 2. People
- 3. The Project Sponsor
- How Will a Data Warehouse Affect our Decision-Making Processes?
- How Does a Data Warehouse Improve My Financial Processes? Marketing? Operations?
- When Is a Data Warehouse Project Justified?
- What Expenses Are Involved?
- What Are the Risks?
- Risk-Mitigating Approaches
- Is My Organization Ready for a Data Warehouse?
- How Do I Measure the Results?
- In Summary
- 4. The CIO
- How Do I Support the Data Warehouse?
- How Will My Data Warehouse Evolve?
- Who Should Be Involved in a Data Warehouse Project?
- Steering Committee
- User Reference Group
- Warehouse Driver
- Warehouse Project Manager
- Business Analyst(s)
- Warehouse Data Architect
- Metadata Administrator
- Warehouse DBA
- Source System Database Administrators (DBAs) and System Administrators (SAs)
- Conversion and Extraction Programmer(s)
- Technical and Network Architect
- Trainer
- What Is the Team Structure Like?
- What New Skills Will My People Need?
- How Does Data Warehousing Fit into My IT Architecture?
- How Many Vendors Do I Need to Talk to?
- What Should I Look for in a Data Warehouse Vendor?
- How Does Data Warehousing Affect My Existing Systems?
- Data Warehousing and Its Impact on Other Enterprise Initiatives
- When Is a Data Warehouse Not Appropriate?
- How Do I Manage or Control a Data Warehouse Initiative?
- In Summary
- 5. The Project Manager
- How Do I Roll Out a Data Warehouse Initiative?
- How Important Is the Hardware Platform?
- What Technologies Are Involved?
- Do I Still Use Relational Databases for Data Warehousing?
- How Long Does a Data Warehousing Project Last?
- How Is a Data Warehouse Different from Other IT Projects?
- What Are the Critical Success Factors of a Data Warehousing Project?
- In Summary
- 3. The Project Sponsor
- 3. Process
- 6. Warehousing Strategy
- Strategy Components
- Determine Organizational Context
- Conduct Preliminary Survey of Requirements
- Conduct Preliminary Source System Audit
- Identify External Data Sources (If Applicable)
- Define Warehouse Roolouts (Phased Implementation)
- Define Preliminary Data Warehouse Architecture
- Evaluate Development and Production Environment and Tools
- In Summary
- 7. Warehouse Management and Support Processes
- 8. Data Warehouse Planning
- Assemble and Orient Team
- Conduct Decisional Requirements Analysis
- Conduct Decisional Source System Audit
- Design Logical and Physical Warehouse Schema
- Produce Source-to-Target Field Mapping
- Select Development and Production Environment and Tools
- Create Prototype for This Rollout
- Create Implementation Plan of This Rollout
- Warehouse Planning Tips and Caveats
- In Summary
- 9. Data Warehouse Implementation
- Acquire and Set Up Development Environment
- Obtain Copies of Operational Tables
- Finalize Physical Warehouse Schema Design
- Build or Configure Extraction and Transformation Subsystems
- Build or Configure Data Quality Subsystem
- Build Warehouse Load Subsystem
- Set Up Warehouse Metadata
- Set Up Data Access and Retrieval Tools
- Perform the Production Warehouse Load
- Conduct User Training
- Conduct User Testing and Acceptance
- In Summary
- 6. Warehousing Strategy
- 4. Technology
- 10. Hardware and Operating Systems
- 11. Warehousing Software
- 12. Warehouse Schema Design
- OLTP Systems Use Normalized Data Structures
- Dimensional Modeling for Decisional Systems
- Two Types of Tables: Facts and Dimensions
- A Schema Is a Fact Table Plus Its Related Dimension Tables
- Facts Are Fully Normalized, Dimensions Are Denormalized
- Dimensional Hierarchies and Hierarchical Drilling
- The Time Dimension
- The Granularity of the Fact Table
- The Fact Table Key Concatenates Dimension Keys
- Aggregates or Summaries
- Dimensional Attributes
- Multiple Star Schemas
- Core and Custom Tables
- In Summary
- 13. Warehouse Metadata
- 14. Warehousing Applications
- 5. Where to Now?
- 15. Warehouse Maintenance and Evolution
- Regular Warehous Loads
- Warehouse Statistics Collection
- Warehouse User Profiles
- Security and Access Profiles
- Data Quality
- Data Growth
- Updates to Warehouse Subsystems
- Database Optimization and Tuning
- Data Warehouse Staffing
- Warehouse Staff and User Training
- Subsequent Warehouse Rollouts
- Chargeback Schemes
- Disaster Recovery
- In Summary
- 16. Warehousing Trends
- Continued Growth of the Data Warehouse Industry
- Increased Adoption of Warehousing Technology by More Industries
- Increased Maturity of Data Mining Technologies
- Emergence and Use of Metadata Interchange Standards
- Increased Availability of Web-Enabled Solutions
- Popularity of Windows NT for Data Mart Projects
- Availability of Warehousing Modules for Application Packages
- More Mergers and Acquisitions Among Warehouse Players
- In Summary
- 15. Warehouse Maintenance and Evolution
- 6. Appendices
- A. R/OLAPXL® User's Manual
- Welcome to R/OLAPXL!
- Installation
- Tutorial
- User's Guide
- Working with R/OLAPXL Columns
- Adding a R/OLAPXL Column
- Moving a R/OLAPXL Column
- Deleting R/OLAPXL Columns
- Using Derived Columns
- Using Functions
- Drilling Up and Down Using R/OLAPXL Columns
- Ad Hoc Drills
- Hierarchical Drills
- Drilling Up Hierarchically
- Drilling Down Hierarchically
- Drilling To Hierarchically
- Constraining Your R/OLAPXL Worksheet
- Constraint Sets in R/OLAPXL
- Setting R/OLAPXL Options
- Rerun Report After Drilling
- Start Cell
- Suppress Repeating Values
- Cascading User Formulas in the Report Body
- Running a R/OLAPXL Report
- Using Subtotals
- Subtotals and Column Suppressing
- Subtotals and Sorting
- Sorting R/OLAPXL Reports
- Sorting and Column Suppressing
- Sorting and Subtotals
- Using Pivot Tables with your R/OLAPXL Reports
- Pivot Table Data Sources
- Commands for R/OLAPXL Pivot Table Reports
- New or View R/OLAPXL Pivot Report
- View R/OLAPXL Pivot Report Source
- Refresh R/OLAPXL Pivot Report
- Other Pivot Table Commands
- Viewing R/OLAPXL SQL Statements
- The R/OLAPXL Toolbars
- Macro Programming
- R/OLAPXL Messages
- B. Warehouse Designer® User's Manual
- C. Online Data Warehousing Resources
- D. Tool and Vendor Inventory
- E. Software License Agreement
- A. R/OLAPXL® User's Manual
A quick browse through the Process section of this book makes it quite clear that a data warehouse project requires a wide array of technologies and tools. The data warehousing products market (particularly the software segment) is a rapidly growing one; new vendors continuously announce the availability of new products, while existing vendors add warehousing-specific features to their existing product lines.
Understandably, the gamut of tools makes tool selection quite confusing. This section of the book aims to lend order to the warehousing tools market by classifying these tools and technologies. The two main categories, understandably, are:
Hardware and operating systems. These refer primarily to the warehouse servers and their related operating systems. Key issues include database size, storage options, and backup and recovery technology.
Software. This refers primarily to the tools that are used to extract, clean, integrate, populate, store, access, distribute, and present warehouse data. Also included in this category are metadata repositories that document the data warehouse. The major database vendors have all jumped on the data warehousing bandwagon and have introduced, or are planning to introduce, features that allow their database products to better support data warehouse implementations.
In addition, this section of the book focuses on two key technology issues in data warehousing:
Warehouse schema design. We present an overview of the dimensional modeling techniques, as popularized by Ralph Kimball, to produce database designs that are particularly suited for warehousing implementations.
Warehouse metadata. We provide a quick look at warehouse metadata—what it is, what it should encompass, and why it is critical in data warehousing.
-
No Comment