Contents

About the Authors

About the Technical Reviewer

Acknowledgments

Introduction

 Chapter 1: Big Data Introduction

Why Big Data

Problem

Solution

Aspects of Big Data

Problem

Solution

How Big Data Differs from Traditional BI

Problem

Solution

How Big Is the Opportunity?

Problem

Solution

Deriving Insight from Data

Problem

Solution

Cloud Enabled Big Data

Problem

Solution

Structured vs. Unstructured Data

Problem

Solution

Analytics in the Big Data World

Problem

Solution

Big Data Challenges

Problem

Solution

Defining a Reference Architecture

Problem

Solution

Need for Architecture Patterns

Problem

Solution

Summary

 Chapter 2: Big Data Application Architecture

Architecting the Right Big Data Solution

Problem

Solution

Data Sources

Problem

Solution

Ingestion Layer

Problem

Solution

Distributed (Hadoop) Storage Layer

Problem

Solution

Hadoop Infrastructure Layer

Problem

Solution

Hadoop Platform Management Layer

Problem

Solution

Problem

Solution

Security Layer

Problem

Solution

Monitoring Layer

Problem

Solution

Analytics Engine

Co-Existence with Traditional BI

Search Engines

Problem

Solution

Real-Time Engines

Problem

Solution

Visualization Layer

Problem

Solution

Big Data Applications

Problem

Solution

Problem

Solution

Problem

Solution

Summary

 Chapter 3: Big Data Ingestion and Streaming Patterns

Understanding Data Ingestion

Problem

Solution

Multisource Extractor Pattern

Problem

Solution

Protocol Converter Pattern

Problem

Solution

Multidestination Pattern

Problem

Solution

Just-in-Time Transformation Pattern

Problem

Solution

Real-Time Streaming Pattern

Problem

Solution

Problem

Solution

Problem

Solution

ETL Tools for Big Data

Problem

Solution

Problem

Solution

Problem

Solution

Problem

Solution

Summary

 Chapter 4: Big Data Storage Patterns

Understanding Big Data Storage

Façade Pattern

Problem

Solution

Data Appliances

Problem

Solution

Storage Disks

Problem

Solution

Data Archive/Purge

Problem

Solution

Data Partitioning/Indexing and the Lean Pattern

Problem

Solution

Problem

Solution

HDFS Alternatives

Problem

Solution

NoSQL Pattern

Problem

Solution

Polyglot Pattern

Problem

Solution

Big Data Storage Infrastructure

Problem

Solution

Typical Data-Node Configuration

Problem

Solution

Summary

 Chapter 5: Big Data Access Patterns

Problem

Solution

Understanding Big Data Access

Stage Transform Pattern

Problem

Solution

Connector Pattern

Problem

Solution

Near Real-Time Access Pattern

Problem

Solution

Lightweight Stateless Pattern

Problem

Solution

Service Locator Pattern

Problem

Solution

Rapid Data Analysis

Problem

Solution

Secure Data Access

Problem

Solution

Problem

Solution

Problem

Solution

Summary

 Chapter 6: Data Discovery and Analysis Patterns

Problem

Solution

Problem

Solution

Problem

Solution

Data Queuing Pattern

Problem

Solution

Index based Insight Pattern

Problem

Solution

Constellation Search Pattern

Problem

Solution

Machine Learning Recommendation Pattern

Problem

Solution

Converger Pattern

Problem

Solution

Challenges in Big Data Analysis

Problem

Solution

Log File Analysis

Problem

Solution

Sentiment Analysis

Problem

Solution

Data Analysis as a Service (DaaS)

Problem

Solution

Summary

 Chapter 7: Big Data Visualization Patterns

Introduction to Big Visualization

Problem

Solution

Big Data Analysis Patterns

Problem

Solution

Problem

Solution

Mashup View Pattern

Problem

Solution

Compression Pattern

Problem

Solution

Zoning Pattern

Problem

Solution

First Glimpse Pattern

Problem

Solution

Exploder Pattern

Problem

Solution

Portal Pattern

Problem

Solution

Service Facilitator Pattern

Problem

Solution

Summary

 Chapter 8: Big Data Deployment Patterns

Big Data Infrastructure: Hybrid Architecture Patterns

Traditional Tree Network Pattern

Problem

Solution

Resource Negotiator Pattern for Security and Data Integrity

Problem

Solution

Spine Fabric Pattern

Problem

Solution

Federation Pattern

Problem

Solution

Lean DevOps Pattern

Problem

Solution

Big Data on the Cloud and Hybrid Architecture

Problem

Solution

Big Data Operations

Problem

Solution

Summary

 Chapter 9: Big Data NFRs

“ilities”

Problem

Solution

Security

Parallel Exhaust Pattern

Problem

Solution

Variety Abstraction Pattern

Problem

Solution

Problem

Solution

Real-Time Streaming Using the Appliance Pattern

Problem

Solution

Distributed Search Optimization Access Pattern

Problem

Solution

Anything as an API Pattern

Problem

Solution

Security Challenges

Problem

Solution

Operability

Problem

Solution

Big Data System Security Audit

Problem

Solution

Big Data Security Products

Problem

Solution

Problem

Solution

Problem

Solution

Problem

Solution

Problem

Solution

Problem

Solution

Summary

 Chapter 10: Big Data Case Studies

Case Study: Mainframe to Hadoop-Based NoSQL Database

Problem

Solution

Case Study: Geo-Redundancy and Near-Real-Time Data Ingestion

Problem

Solution

Case Study: Recommendation Engine

Problem

Solution

Case Study: Video-Streaming Analytics

Problem

Solution

Case Study: Sentiment Analysis and Log Processing

Problem

Solution

Case Study: Real-Time Traffic Monitoring

Problem

Solution

Case Study: Data Exploration for Suspicious Behavior on a Stock Exchange

Problem

Solution

Case Study: Environment Change Detection

Problem

Solution

Summary

 Chapter 11: Resources, References, and Tools

Big Data Product Catalog

Problem

Solution

Hadoop Distributions

Problem

Solution

In-memory Hadoop

Hadoop Alternatives

Problem

Solution

Hadoop SQL Interfaces

Problem

Solution

Ingestion tools

Problem

Solution

Map Reduce alternatives

Problem

Solution

Cloud Options

Problem

Solution

Table-Style Database Management Services

Problem

Solution

NoSQL Databases

Problem

Solution

In-Memory Big Data Management Systems

Problem

Solution

DataSets

Problem

Solution

Data Discovery

Visualization

Problem

Solution

Analytics Tools

Data Integration Tools

Problem

Solution

Summary

 Appendix A: References and Bibliography

Index