0%

Book Description

Monitoring, the practice of observing systems and determining if they're healthy, is hard--and getting harder. In a perfect world, your anomaly detection system would warn you about new behaviors and data patterns in time to fix problems before they happened, and would be completely foolproof, never ringing the alarm bell when it shouldn't. Such a system doesn't exist (yet), but that shouldn't make you lose sight of the fact that better anomaly detection is possible and can provide tremendous operational benefits.

This report demystifies the topic and clarifies some of the fundamental choices that you have to make when constructing anomaly detection mechanisms. You'll learn why some approaches to anomaly detection work better than others in certain situations, and why a better solution for some challenges may be within reach after all. Authors Preetam Jinka and Baron Schwartz introduce the various types of monitoring systems, explain the logic behind them, and help you to navigate the labyrinth of current anomaly detection by outlining the tradeoffs associated with different approaches so you can make judgments as you reach each fork in the road.

Table of Contents

  1. Foreword
  2. 1. Introduction
    1. Why Anomaly Detection?
    2. The Many Kinds of Anomaly Detection
    3. Conclusions
  3. 2. A Crash Course in Anomaly Detection
    1. A Real Example of Anomaly Detection
    2. What Is Anomaly Detection?
    3. What Is It Good for?
    4. How Can You Use Anomaly Detection?
    5. Conclusions
  4. 3. Modeling and Predicting
    1. Statistical Process Control
      1. Basic Control Chart
      2. Moving Window Control Chart
      3. Exponentially Weighted Control Chart
      4. Window Functions
    2. More Advanced Time Series Modeling
    3. Predicting Time Series Data
    4. Evaluating Predictions
    5. Common Myths About Statistical Anomaly Detection
      1. The Data Doesn’t Need to Be Gaussian
      2. Sample Distribution Versus Population Distribution
    6. Conclusions
  5. 4. Dealing with Trends and Seasonality
    1. Dealing with Trend
    2. Dealing with Seasonality
    3. Multiple Exponential Smoothing
    4. Potential Problems with Predicting Trend and Seasonality
    5. Fourier Transforms
    6. Conclusions
  6. 5. Practical Anomaly Detection for Monitoring
    1. Is Anomaly Detection the Right Approach?
    2. Choosing a Metric
    3. The Sweet Spot
    4. A Worked Example
    5. Conclusions
  7. 6. The Broader Landscape
    1. Shape Catalogs
    2. Mean Shift Analysis
    3. Clustering
    4. Non-Parametric Analysis
    5. Grubbs’ Test and ESD
    6. Machine Learning
    7. Ensembles and Consensus
    8. Filters to Control False Positives
    9. Tools
      1. Graphite and RRDTool
      2. Etsy’s Kale Stack
      3. R Packages
      4. Commercial and Cloud Tools
  8. A. Appendix
    1. Code
3.147.55.42