Index

4GL technology, 3–4

A

access. See data access
access query, 340
active data warehouse, 14–16
active metadata repository, 99–100
ad hoc transactions, 340
Address data, 196
administration
architectural, 348–351
Archival Sector, 348–349
budget, 334, 358–359
data base, 352–353
data models, 347–348
management, of DW 2.0 environment, 358–361
managing consultants, 359–361
metadata, 41, 351–352
Near Line Sector, 349–351
prioritization/conflicts, 358
resource allocation, 359
scheduling and milestones, 359
stewardship, 353–355
systems/technology, 355–358
aggregate data
data models, 159
granularity managers, 235
airline metadata example, 106–109
analysis
creating new, 273–277
creating new, from DW 2.0 data, 276–277
analytical productivity and response time, 243–244
analytical response time, 241
applications
active data warehouse, 14–16
from application to corporate data, 216, 219–221
evolution of data warehousing, 2–3, 5–6, 9–10
Interactive Sector, 29
monitoring, 172
OLAP, 20
transaction monitor and response time, 171–172
transaction processing. See OLTP
useful, 51–52
architecture
architectural administration, 348–351
building “real” data warehouse, 21–22
creating new analysis, 273–277
flow of data through, 203
new paradigm of DW 2.0, 24, 25
Archival Sector, 76–86
adding, 264–265
architectural administration, 348–349
data access, 33–34, 80–81, 84–85
flow of data, 48–49, 209
life cycle of data, 27–30
metadata, 31–33, 105–106
passive indexes, 81–83, 344
performance, 80–81
processes, 343–344
reasons for, 30–31
searching, 80–85
time-variant data, 192–193, 199–200
volumes of data, 50–51, 80, 85
workload, 80
archiving data. See also Archival Sector
business perspective, 26
with metadata, 32
ASCII/EBCDIC conversions, 223
assets and ROI, 128
ATM transactions, 28
attacks on data, sensing, 185–187
audience and star schemas, 18–19
audit trail
correcting/resetting data, 330–331
ETL processing, 223–224

B

backflow of data
from exploration facilities, 152–154
from Integrated Sector, 212–213
bad data
balancing entry for, 330
introduction, 329–330
making corrections, 330–331
resetting values, 330
balancing entry for bad data, 330
bank account information, 116
banking transactions, 193, 217
barrier security, 172
Basel II taxonomy, 104, 109
batch mode
ETL processing, 217–218
exception-based flow of data, 212
beverage sales data, 143–144
BI (business intelligence) universed, 97–98
big bang approach, 8–9
blather, 38
brittleness and star schemas, 18
Brooks, Fred, 115
budget, 334, 358–359
business
BI universe, 97–98
changing requirements, 47–48, 114–115
corporate data and Integrated Sector, 62
corporate data model, 162–163
corporate information factory, 12, 13, 134
enterprise knowledge coordination stream, 129–133
impact of data warehousing, 11
metadata, 44, 102–103, 352
representation and data models, 157–158
“Business Intelligence Road Map” (Moss), 125–126
business perspective
cost justification, 282
data models, 166
DW 2.0 environment, 24, 26, 90–92
ETL processing, 227–228
evolution of data warehousing, 5–6, 14
flow of data, 213–214
granularity managers, 238
metadata, 109–110
migration, 269–270
monitoring DW 2.0 environment, 178
performance, 258–259
security, 187–188
statistical processing, 155–156
system of record, 319–320
technology infrastructure, 121–122
time-variant data, 200
unstructured data, 310

C

caching data, 54, 71, 356
calculations and data mapping, 317
capacity/disk storage
active data warehouse, 15
costs, 335–336
DBMS activities, 2
evolution to DW 2.0 environment, 10
history of data warehousing, 1
Near Line Sector, 73
optimization, 336
planning and performance, 247–248, 356–357
captured text, 87
CDC (changed data capture), 226
changes
in business requirements, 47–48, 114–115
CDC, 226
growth of data, 334
infrastructure. See technology infrastructure
mitigating business change, 119
propagation of, 20
rapid business changes, 114
states of data, 215, 219–221
treadmill of change, 114–115
click-stream data, 232
CMSM (cross-media storage manager), 74–75, 211–212
Coca Cola sales data, 143–144
code
checking automatically generated, 257–258
ETL, 225
concepts data models, 291
consultants, managing, 359–361
continuity of data, 198
continuous time span data
beginning/ending sequence of records, 197–198
features of, 194–196
nonoverlapping records, 197
overview, 63
sequence of records, 196
conversions, data, 62, 221–223
corporate data
from application data to, 216, 219–221
enterprise knowledge coordination stream, 129–133
Integrated Sector, 62
model, 162–163
corporate information factory, 12, 13, 134
correcting/resetting data, 330–331
costs/cost justification
active data warehouse, 16
business perspective, 24
creating new analysis, 273–277
creating new analysis from DW 2.0 data, 276–277
DW 2.0 implementation, 271, 273
economics and evolution of data warehousing, 10–11
factoring cost of DW 2.0, 277–278
factors affecting, for new analysis, 276
first-generation vs. DW 2.0, 281–282
historical information, 280–281
macro-level justification, 271–272
micro-level justification, 272–273
Near Line Sector, 72–73
perspective of business users, 282
real economics of DW 2.0, 279
reality of information, 278
storage, 335–336
time value of information, 279–280
value of integration, 280
credit card data, 11, 295
cross-media storage manager (CMSM), 74–75, 211–212
currency conversions, 62, 221–223
customer metadata, 101–102
customer profiles, 11

D

data access
Archival Sector, 33–34, 80–81, 84–85
business perspective, 26
in DW 2.0 environment, 33–34
Integrated Sector, 66–67, 69–70
Interactive Sector, 59
Near Line Sector, 74
probability of, 30–31, 209–210
security, 181
unstructured data, 89–90
volumes. See volumes of data
data base
administration, 352–353
DBMS, 1–2, 223, 332–337
relational, 309
data correction stream, 133
data flow. See flow of data
data integration
Integrated Sector, 58
introduction, 7–8
data item set (dis) level, 159–160
data mapping, 219, 223, 316–319
data marts
convenience of, 324–325
as data warehouse, 15, 20–21
described, 13
vs. exploration facilities, 152
moving data, 327–328
performance, 251
source data, 323
system of record, 319
transforming data from, 325
data models
business representation, 157–158
concepts, 291
corporate, 162–163
corporate, and seven streams approach, 131
data item set level, 159–160
ERD level, 159–160
granular vs. summarized data, 159
intellectual road map, 157
Interactive Sector, 161–162
levels of, 159–161
logical, 291–294
perspective of business users, 166
physical model, 159–160, 290–292
scope of integration, 158–159
top-down modeling, 294–296
transformation, 163–164
types used in DW 2.0, 289–294
unstructured data, 164–166
data profiling
enterprise knowledge coordination stream, 129–133
inconsistencies, 294–296
and mapping stream, 133
tools and reverse-engineered data model, 288–289
data quality
checking, 170–171, 174–175, 224
data model types, 289–294
data profiling inconsistencies, 294–296
data profiling tools, 288–289
DW 2.0 environment, 285–286
reverse-engineered data model, 288–289
TIQM, 134–137
tool set, 287–288
TQdM, 134
data quality editor, 63
data quality monitor, 170–171, 174–175, 224
data warehouse
active approach, 14–16
bad data, 329–330
building “real” vs. variations, 21–22
business appeal of DW 2.0, 24
business impact, 11
business perspective, 26, 90–92
changing business requirements, 47–48, 114–115
compared to data mart, 21
data mart. See data marts
defined, 7
different development approach, 8–9
diversity of, 40–41
DW 2.0 technology foundation, 45–46
environment. See environment, DW 2.0
exploration warehouse, 13
federated approach, 14–15, 16–18
first-generation. See history of data warehousing
house/city analogy, 261–262
integrating data, 7–8
new paradigm of DW 2.0, 24, 25
shaping factors of DW 2.0, 23–24
speed of data movement into/through, 331
star schema approach, 15, 18–19
suboptimal, 19
useful applications, 51–52
variations, 14–15
volumes of data, 8, 50–51
data warehouse monitor
falling probability of data access, 209–210
features of, 176–177, 326–327
overview, 171
security, 185
data warehouse utility (DWU), 332–337
Dataupia, 332, 336
date formats, 62, 222
Date of Birth data, 196
DBMS (data base management systems)
conversions, 223
data warehouse utilities, 332–337
purpose of, 1–2
DDL (data definition language), 290
Decision Support Systems (DSS) processing, 41, 70–71
default values and ETL processing, 223
demographics, 51
development
data warehouse approach, 8–9
ETL programs, 224–225
PCs and 4GL technology, 3–4
devolution, 19
dis (data item set) level, 159–160
discontinuity of data, 197
discrete data
continuity, 198
time-variant data, 194
disk storage. See capacity/disk storage
diversity and metadata, 41
dividing data and technology infrastructure, 121
domain checking, 63, 174, 224
domains in data quality tool set, 287
dormant data
monitoring, 176–177
removing, 245–246
Dow Jones Industrial average, 194
DSS (Decision Support Systems) processing, 41, 70–71
dump, data, 184–185
DW 2.0. See data warehouse; environment, DW 2.0
DWU (data warehouse utility), 332–337

E

EBCDIC/ASCII conversions, 223
edited text, 87, 302
ELT (extract/load/transform) processing
described, 226–227
Interactive Sector, 58
managing, 351
perspective of business users, 227–228
email
blather, 38
as unstructured data, 299
view of customer, 51
encryption
features of, 181–182
limiting, 184
end-user perspective. See business perspective
English, Larry, 134, 137
enterprise knowledge coordination stream, 129–133
enterprise reference model stream, 130
enterprise-wide metadata
features of, 101–102
local metadata, 43–45, 97–98, 103
metadata in DW 2.0 environment, 97–98
environment, DW 2.0
access of data, 33–34
components of, 11–13
cost justification, 271
data access, 33–34
data flow, 48–49
data warehouse, 6
data warehouse monitor, 171
DW 2.0 landscape, 290
ETL processing, 215–216
evolution, 9–11
management administration, 358–361
metadata, 31–33, 40–44, 96–99
migration to unstructured, 267–269
monitoring, 169, 246–247
performance, 239
preparing unstructured data, 38–40
referential integrity, 52
reporting, 53, 206
responding to business changes, 47–48
spider web, 4–5, 6
structured/unstructured data, 34–35, 86–90
technology foundation, 45–46
textual data, 34–35
transaction monitor, 169–170
transaction monitor and response time, 171–172
volumes of data, 50–51
ERD (entity relationship level), 159–160
ETL (extract/transform/load) processing
from application to corporate data, 216, 219–221
audit trail, 223–224
batch mode, 217–218
CDC, 226
changing states of data, 215, 219–221
code creation/parametrically driven, 225
compared to ELT, 226–227
complex transformations, 221
creating programs, 224–225
data flow in DW 2.0, 48, 205
data quality monitor, 170–171, 174–175, 224
default values, 223
domain checking, 174, 224
in DW 2.0 environment, 215–216
Integrated Sector, 29, 63, 67–68
Interactive Sector, 27–29, 58–59
introduction, 12
mapping, 219
metadata, 223
migration shock absorber, 267
online mode, 216–217
real-time processing, 218
rejected data, 225–226
source/target, 218–219
system of record, 218
technology to prepare data, 308
throughput, 222–223
unstructured processing, 87–88
evolution of data warehousing. See history of data warehousing
exception-based flow of data, 210–213
exploration facilities
backflow of data, 152–154
data marts compared to, 152
features of, 147
frequency of analysis, 147
project-based data, 13, 150–151
refreshing exploration data, 149
sources for exploration processing, 149
using data internally, 155
exploration processing, 146
exploration warehouse, 13, 24
extensibility
nonextensibility and data marts, 20
star schemas, 18
external taxonomies, 104–105, 304–305
extract/load/transform (ELT) processing, 226–227
extracts
ETL. See ETL (extract/transform/load)
processing
proliferation and data marts, 20

F

federated data warehouse
described, 16–18
variations of data warehouses, 14–15
filtering data, 232–234
Find it domain, 287
firewalls, 182
first-generation data warehousing. See history of data warehousing
Fix it domain, 287
flow of data
Archival Sector, 48–49, 209
bulk batch mode, 212
in DW 2.0 environment, 48–49
exception-based, 210–213
falling probability of data access, 209–210
Integrated Sector, 48–49, 205–207
Interactive Sector, 48–49, 203–205
Near Line Sector, 48–49, 207–209
performance, 241–242
perspective of business users, 213–214
role of ETL, 205
staging request, 213–214
throughout architecture, 203
triggers, 206–207
foreign keys, 174–175
freezing data, 145–146
frequency of analysis, 147
frequent flyer programs, 11

G

gender data, 63, 174, 196, 223
general/specific text, 39–40
glossaries, 304
granularity
in data warehouse, 7
devolution, 19
in federated data warehouse, 17–18
granular vs. summarized data, 159
Integrated Sector, 65, 70–71
Interactive Sector, 60
in star schemas, 19
granularity managers
aggregate data, 235
compared to ELT, 231–232
eliminating data, 234
filtering data, 232–234
functions of, 234–236
home-grown vs. third-party, 236
metadata as by-product, 237–238
parallelizing, 237
perspective of business users, 238
raising level of granularity, 232
recast data, 235
summarizing data, 234
growth of data, 334

H

hardware/software selection, 256
heuristics
analysis and statistical processing, 145–146
freezing data, 145–146
processing, 243, 341
highway analogy for workload, 64, 66
historical data
data warehouse, 7
federated data warehouse, 17
Integrated Sector, 65, 67, 70
Interactive Sector, 60
value of information, 280–281
historical record, 120
history of data warehousing
from business perspective, 5–6, 14
capacity/disk storage, 1
data warehouse environment, 6
DBMS, 1–2
DW 2.0 compared to first-generation, 23–24
early progression of systems, 2
forces shaping, 9–11
master files, 5
online applications, 2–3
PCs and 4GL technology, 3–4
spider web environment, 4–5, 6
homographs
resolution, 303–304
taxonomies, 105
house/city analogy, 261–262

I

Improving Data Warehouse and Business Information Quality (English), 137
indexing
passive indexes for archival data, 81–83, 344
performance, 245
information factory development
stream, 133
infrastructure
stream, 133
technology. See technology infrastructure
integrated data
in data warehouse, 7–8
evolution of data warehousing, 10
federated data warehouse, 17
integrating text, 301–307
scope of, and data models, 158–159
value of integration, 280
Integrated Sector, 62–71
changes to data, 67
continuous time span data, 63
corporate data, 62
data access, 66–67, 69–70
data key reconciliation, 62
data quality editor, 63
DSS processing, 70–71
ETL processing, 29, 63, 67–68
flow of data, 48–49, 205–207
granularity, 65, 70–71
historical data, 65, 67, 70
life cycle of data, 27–30
performance, 65–66
processes, 341–342
profile data, 63
queries/searches, 65–67
reasons for, 30–31
referential integrity, 68–69
subject-oriented detailed data, 62–63
summary data, 63
time relativity, 192–193
transactions, and time-variant data, 193–194
volumes of data, 50–51, 65
workload, 64
integrity of data
active data warehouse, 15
comparisons, 144–145
referential, 52
star schemas, 19
statistical comparison, 144–145
Interactive Sector, 55–61
access of data, 33–34
data access, 59
data models, 161–162
ETL processing, 58
flow of data, 48–49, 203–205
granularity, 60
historical data, 60
life cycle of data, 27–30
metadata, 31–33
performance, 57, 254–255
protecting, 254–255
reasons for, 30–31
referential integrity, 58
searches, 58
time relativity, 192
volumes of data, 50–51, 57
workload, 56
internal taxonomies, 104–105
intersector/intrasector referential integrity, 52
inventory management, 11, 127–128
IT (information technology)
reducing IT response time, 115
technology infrastructure, 112–113

K

Kalido, 121
keys
reconciliation, 62
restructure/creation, 223

L

LDM (logical data model), 291–294
legacy data. See also ETL (extract/transform/load) processing
creating new analysis, 273–275
as data source, 315
federated data warehouse approach, 16–18
life cycle of data, 27–30
operational/legacy systems environment, 313
licenses and DWU, 336
life cycle of data
reasons for sectors, 30–31
sectors, described, 27–30
linkages, 87, 309
local metadata, 43–45, 97–98, 103
logic
data mapping, 318
LDM, 291–294
transactions, 339–341
logical data model (LDM), 291–294

M

macro-level cost justification, 271–272
maintaining metadata, 106
management administration of DW 2.0 environment, 358–361
mangled characters, monitoring, 175
mapping, data, 219, 223, 316–319
maps, level of detail, 159–160
master files, 5
meltdowns, 173
metadata
active/passive repositories, 99–100
administration, 41, 351–352
in Archival Sector, 31–33, 105–106
building infrastructure, 266
business, 44, 102–103, 352
business perspective, 26
as by-product of granularity manager, 237–238
card catalog analogy, 95
creating enterprise, 265–266
in DW 2.0 environment, 31–33, 40–44, 96–99
end-user perspective, 109–110
enterprise-wide, 101–102
ETL processing, 223
infrastructure and performance, 248
in Interactive Sector, 31–33
local, 43–45, 97–98, 103
maintaining, 106
repositories, 98–99
reusability of data and analysis, 96, 249
stop words, 105, 302
structure of, 96–97
system of record, 102–103
taxonomies, 104–105
technical, 44, 102–103, 352
transformation process, 341
unstructured data, 41, 103, 104–105
using, airline example, 106–109
methodology
seven streams approach, 129–139
spiral, 123–129, 137–139
waterfall, 123–126
micro-level cost justification, 272–273
migration
adding Archival Sector, 264–265
adding components incrementally, 262–264
building metadata infrastructure, 266
creating enterprise metadata, 265–266
ETL as shock absorber, 267
in perfect world, 262
perspective of business users, 269–270
swallowing source systems, 266–267
to unstructured environment, 267–269
milestones and scheduling, 359
money data, 17, 162, 220–221
Monitor it and report it domain, 287
monitoring DW 2.0 environment
application monitoring, 172
by architectural administrator, 350
data quality, 170–171, 174–175, 224
data warehouse monitor, 171, 176–177, 185, 326–327
domain checking, 174, 224
dormant data, 176–177, 245–246
mangled characters, 175
meltdowns, 173
null values, 175
outlying range, 175
overview, 169
peak-period processing, 172–174
performance, 246–247
perspective of business users, 178
queue monitoring, 171
sniffing, 176
transaction monitor, 169–170
transaction monitor and response time, 171–172
transaction queue monitoring, 171
transaction record monitoring, 172
unmatched foreign keys, 174–175
Moss, Larissa, and spiral methodology, 125–128
Move it domain, 287
Mythical Man Month, The (Brooks), 115

N

Name data, 196
NAME data, 308
Near Line Sector, 71–76
architectural administration, 349–351
CMSM, 74–75, 211–212
cost and performance, 72–73
data access, 74
data storage, 73
flow of data, 48–49, 207–209
life cycle of data, 27–30
processes, 342–343
reasons for, 30–31
security, 187
time relativity, 192–193
volumes of data, 50–51, 76
workload, 73–74
nonextensibility and data marts, 20
nonoverlapping records, 197
normalization of textual data, 38–40
normalized data, 38
null values, monitoring, 175

O

ODS (operational data store), 13
offline data and security, 182–184
OLAP (online application processing), 20
OLTP (online transaction processing)
DWU, 336
federated data warehouse, 16–17
long queue time, 243
ODS, 13
performance, 239
SLA, 254
O’Neill, Bonnie, 287
online applications/processing
active data warehouse, 14–16
evolution of data warehousing, 9–10
history of data warehousing, 2–3, 5–6
OLAP, 20
transaction processing. See OLTP
online mode for ETL, 216–217
online response performance, 239–241
operational application systems
environment, 218, 313–314, 316
operational/legacy systems environment, 313
organizational charts, 116–117, 120
outlying range, 175

P

paradigm of DW 2.0, 24, 25
parallelization
batch, and performance, 249
granularity managers, 237
transaction processing, 249–250
partitioning data, 255–256
passive indexes for archival data, 81–83, 344
passive metadata repository, 99–100
password flooding attacks, 186
patient’s records, 51, 52
PCs and 4GL technology, 3–4
peak-period processing, 172–174
performance
analytical productivity and response time, 243–244
analytical response time, 241
Archival Sector, 80–81
batch parallelization, 249
capacity planning, 247–248
checking automatically generated code, 257–258
data marts, 251
data models and Interactive Sector, 161–162
in DW 2.0 environment, 239
end-user education, 246
exploration facilities, 252
facets to, 244–245
federated data warehouse, 16–17
flow of data, 241–242
hardware/software selection, 256
heuristic processing, 243
indexing, 245
Integrated Sector, 65–66
Interactive Sector, 57, 254–255
metadata infrastructure, 248–249
monitoring environment, 246–247
Near Line Sector, 72–73
OLTP, 239
online response time, 240–241
parallelization for transaction processing, 249–250
partitioning data, 255–256
perspective of business users, 258–259
physically grouping data, 257
protecting Interactive Sector, 254–255
queues, 242–243
reducing IT response time, 115
removing dormant data, 245–246
separating farmers/explorers, 256–257
separation of transactions into classes, 253–254
service level agreements, 254
transaction monitor and response time, 171–172
unstructured data, 88–89
workload management, 250–251
physical model, 159–160
physically grouping data, 257
pointers and unstructured processing, 87
preprogrammed complex transactions, 340
prioritization/conflicts and administration, 358
probability of data access
for different sectors, 30–31
elevated, 210
falling probability, 209–210
processing in DW 2.0 environment, 339–344
in DW 2.0 sectors, 341–344
transaction types, 339–341
profile data, 63, 131
project-based data, 13, 150–151
proliferation and star schemas, 19
protecting Interactive Sector, 254–255

Q

quality monitor, 170–171, 174–175, 224. See also data quality
queries. See also data access; searches
access, 340
ad hoc, 340
Interactive Sector, 58
nonreplicable, in federated data warehouse, 17
statistical processing, 141–143
queue monitoring, 171
queues, 242–243

R

random data access, 33–34
range checking, 63, 175, 224
rationalization of textual data, 39–40
reading text for analytical processing, 299–300
reality of information, 278
real-time ETL processing, 218
reasonability checking, 224
recast data, 235
reconciliation
data marts, 20
encoded values for data mapping, 318
keys, 62
referential integrity
described, 52
Integrated Sector, 68–69
Interactive Sector, 58
refreshing exploration data, 149
rejected data, 225–226
relational data base, 309
reports, 53, 206, 278
repositories for metadata, 98–100
resetting data values, 330
resource allocation, 359
response time
analytical productivity, 243–244
IT, reducing, 115
online response time, 240–241
transaction monitor, 171–172
transaction monitoring, 171–172
return on investment (ROI), 128
reusability
data and analysis, 96
metadata, 249
spiral methodology, 127–128
revenue metadata, 101–103
reverse-engineered data model, 288–289
road maps
“Business Intelligence Road Map” (Moss), 125–126
DW/BI project road map, 137–138
intellectual road map, 157

S

sales data
in Integrated Sector, 62–63
semantically stable data, 117
sales territories, 116–117
SAN technology, 333, 334
SAP technology, 333
Sarbanes Oxley taxonomy, 109, 306–307
scheduling and milestones, 359
scope creep, 348
screening data, 38
SDLC (systems development life cycle), 123
searches. See also data access; queries
Archival Sector, 80–85
direct/indirect, 306–307
indexing, 81–83, 245, 344
Integrated Sector, 65–67
Interactive Sector, 58
sectors. See also Archival Sector;
Integrated Sector; Interactive
Sector; Near Line Sector
data access, 33–34
metadata, 31–33
reasons for different sectors, 30–31
types of sectors, 27–30, 55
security
barrier security, 172
data access, 172
data warehouse monitor, 185
direct dump of data, 184–185
drawbacks, 182
encryption, 181–182, 184
firewall, 182
moving data offline, 182–184
Near Line Sector, 187
password flooding attacks, 186
perspective of business users, 187–188
protected unstructured data, 187
sensing attacks, 185–187
Self Organizing Map (SOM), 165
semantic relationships
enterprise metadata, 101
mitigating business change, 119
mixing stable/unstable data, 118
separating stable/unstable data, 118
stable data, 117
temporal data, 116–117
temporal/static data, 115–116
semantically temporal/static data, 116–117
semistructured data/value, 307–308
sequence of records, 196
sequential data access, 33–34
service level agreements (SLAs), 254, 259, 349–350
settling data, 331
seven streams approach
data correction stream, 133
data profiling and mapping stream, 133
DW/BI project road map, 137–138
enterprise knowledge coordination stream, 129–133
enterprise reference model stream, 129
information factory development stream, 133
infrastructure stream, 133
overview, 129
summary, 137–139
total information quality management stream, 134–137
shared data mart data, 327–328
SLAs (service level agreements), 254, 259, 349–350
slivers and spiral methodology, 127
snapshots of data, 119–120
sniffing and data warehouse monitor, 176
software
creating ETL, 224–225
disruption and DWU, 336
performance, 256
SOM (Self Organizing Map), 165
source data system of records, 316
sources
best source data from operational environment, 316
data mapping, 219, 223, 316–319
data marts, 323
ETL processing, 218–219
exploration processing, 149
swallowing source systems, 266–267
system of record, 313–319
specific/general text, 39–40
speed of data movement, 331
spellings, alternate, 105, 305
spider web environment
history of data warehousing, 4–5
transition to data warehouse environment, 6
spiral methodology, 123–129, 137–139
stable/unstable data, semantically features of, 117
mixing/separating, 118
staging request, 213–214
star schemas, 15, 18–19
states of data, changing, 215, 219–221
static/temporal data, 115–117
statistical analysis, 141–143
statistical comparison, 144–145
statistical processing
active data warehouse, 16, 141
backflow of data, 152–154
data marts and exploration facilities, 152
in DW 2.0 environment, 141–142, 341
exploration facilities, 147
exploration processing, 146
freezing data, 145–146
frequency of analysis, 147
heuristic analysis, 145–146, 341
integrity of comparisons, 144–145
perspective of business analyst, 155–156
project-based data, 150–151
queries, 141–143
refreshing exploration data, 149
sources for exploration processing, 149
using exploration data internally, 155
using statistical analysis, 143
stemming, 301, 305
stewardship, 353–355
stop words, 105, 302
storage. See capacity/disk storage
Strauss, Derek, 287
structured data
data flow, 48–49
linkages, 87, 309
metadata, 41
vs. unstructured data, 34–35
volumes of data, 50–51
subject-oriented detailed data, 62–63
subjects
subject area definitions, 101–102
system of record, 102–103
summary data
granular vs. summarized data, 159
granularity manager, 234
in Integrated Sector, 63
synonyms
replacement/concatenation, 303
taxonomies, 105
system of record
best source data from operational environment, 316
data mapping, 316–319
data marts, 319
ETL processing, 218
metadata, 102–103
operational/legacy systems environment, 313
perspective of business users, 319–320
systems development life cycle (SDLC), 123
systems/technology administration, 355–358

T

targets
data mapping, 219, 223, 316–319
ETL processing, 218–219
taxonomies
external, 104–105, 304–305
features of, 104–105
stop words, 105, 302
synonyms, 105
unstructured processing, 87
technical metadata, 44, 102–103, 352
technology
administration, 355–358
for different sectors, 34
evolution of data warehousing, 9
federated data warehouse, 17
responding to business changes, 47–48
seven streams approach, 129–139
spiral methodology, 123–129, 137–139
technology infrastructure
creating snapshots of data, 119–120
dividing data, 121
end-user perspective, 121–122
features of, 112–113
getting off treadmill, 115
historical record, 120
mitigating business change, 119
mixing semantically stable/unstable data, 118
overview, 111–112
rapid business changes, 114
reducing IT response time, 115
semantically stable data, 117
semantically temporal data, 116–117
semantically temporal/static data, 115–116
separating semantically stable/unstable data, 118
treadmill of change, 114–115
temporal/static data, 115–117
terminology
handling text, 307
normalization of, 38–40
text across languages, 305
textual analytical processing, 300–301
textual data
alternate spellings, 105, 305
analytics, 35
data access, 89–90
direct searches, 306
in DW 2.0 environment, 34–35
ETL technology, 308
evolution of data warehousing, 10
external glossaries/taxonomies, 304–305
homographic resolution, 303–304
indirect searches, 306–307
integrating text, 301–307
NAME data, 308
normalization, 38–40
performance, 88–89
perspective of business users, 310
relational data base, 309
semistructured data/value, 307–308
simple editing, 87, 302
specific/general, 39–40
stemming, 305
stop words, 105, 302
synonym replacement/concatenation, 303
synonyms, 105
terminology, 307
text across languages, 305
themes, 104, 165–166, 304
unstructured processing, 86–90
workload, 88
themes, 104, 165–166, 304
throughput, ETL, 222–223
time capsules, 79
time, performance. See performance
time value of information, 279–280
time-variant data
Archival Sector, 192–193, 199–200
beginning/ending sequence of records, 197–198
continuity, 198
continuous time span data, 63, 194–196
discrete data, 194, 198
end-user perspective, 200
key structure, 192
nonoverlapping records, 197
sequence of records, 196
structure of DW 2.0 data, 191–192
time relativity in Interactive Sector, 192
time-collapsed data, 198–199
transactions in Integrated Sector, 193–194
TIQM (total information quality management) stream, 134–137
top-down modeling, 294–296
total information quality management (TIQM) stream, 134–137
total quality data management (TQdM), 134
TQdM (total quality data
management), 134
transaction monitor
application monitoring, 172
overview, 169–170
queue monitoring, 171
record monitoring, 172
response time, 171–172
transaction processing. See also OLTP
ad hoc transactions, 340
in DW 2.0 environment, 339–344
Interactive Sector, 56–57
life cycle of data, 27–30
logic, 339–341
parallelization and performance, 249–250
performance, 253–254
preprogrammed transactions, 340
simple/complex transactions, 339–340
transaction types, 141–142
transformation
of data. See ETL (extract/transform/load) processing
of data mart data, 325
of data models, 163–164
process, 340–341
transparency
CMSM, 74–75
data base, 336
triggers for data flow, 206–207

u

unstable/stable data, semantically
features of stable data, 117
mixing/separating, 118
unstructured data
alternate spellings, 105, 305
data access, 89–90
data flow, 48–49
data models, 164–166
direct searches, 306
DW 2.0 environment, 299
ETL technology, 308
evolution of data warehousing, 10
external glossaries/taxonomies, 304–305
homographic resolution, 303–304
indirect searches, 306–307
integrating text, 301
linkages, 87, 309
metadata, 41, 103, 104–105
migration to unstructured environment, 267–269
NAME data, 308
performance, 88–89
perspective of business users, 310
preparing, for DW 2.0 environment, 38–40
processing, 86–90
reading text, 299–300
relational data base, 309
screening for blather, 38
semistructured data/value, 307–308
simple editing, 87, 302
stemming, 305
stop words, 105, 302
vs. structured data, 34–35
synonym replacement/concatenation, 303
terminology, 307
text across languages, 305
textual analytical processing, 300–301
themes, 104, 165–166, 304
volumes of data, 50–51
workload, 88

V

volumes of data
Archival Sector, 50–51, 80, 85
business perspective, 26
for different sectors, 31
in DW 2.0 environment, 8
filtering data, 232–234
Integrated Sector, 50–51, 65
Interactive Sector, 50–51, 57
Near Line Sector, 50–51, 76

w

waterfall methodology, 123–126
workload
Archival Sector, 80
highway analogy, 64, 66
Integrated Sector, 64
Interactive Sector, 56
Near Line Sector, 73–74
unstructured data, 88
workload management, 250–251

z

Zachman framework, 131–133, 289–292
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.136.84.88