Science of Anomaly Detection
7 likes6,937 views
The document discusses anomaly detection using Hierarchical Temporal Memory (HTM) algorithms. It defines anomalies as deviations from what is standard or expected. It describes challenges with temporal anomalies and how HTM is well-suited to detect different types of anomalies, including subtle patterns humans may miss. The document outlines HTM's anomaly detection process, including how it uses raw anomaly scores, historical comparison, and anomaly likelihood. It also discusses evaluating and benchmarking streaming anomaly detection algorithms.
![Historical Comparison
Compute normal distribution over history
Compute probability for each point relative to the
distribution
m = xP(x)å s = E[(X -m)2
]](https://image.slidesharecdn.com/numentaworkshoppresentation2014-150223131836-conversion-gate01/85/Science-of-Anomaly-Detection-10-320.jpg)
Science of Anomaly Detection
- 1. Science of Anomaly Detection Numenta Workshop October 17, 2014 Scott Purdy Engineering Manager
- 2. What is an anomaly? Something that deviates from what is standard, normal, or expected.
- 3. Types of Anomalies Spatial (static) anomalies Temporal anomalies
- 4. Challenges with Temporal Anomalies Tolerance to noise Continuous learning
- 5. • How anomaly detection fits into HTM theory • How we do anomaly detection – HTM Learning Algorithms – Anomaly score processing • Evaluating anomaly detection algorithms Outline
- 6. What does anomaly detection have to do with Hierarchical Temporal Memory? Applications of HTM: • Prediction • Classification • Anomaly Detection No changes to the algorithm
- 7. Anomaly Detection with HTM HTM Algorithms Encoder SDR Prediction Raw anomaly score Time average Historical comparison Anomaly likelihood Data How do we turn a data stream into anomaly scores?
- 8. Raw Anomaly Score Raw anomaly score is the fraction of active columns that were not predicted. rawAnomalyScore = At -(Pt-1 Ç At ) At Pt = Predicted columns at time t At = Active columns at time t
- 9. Load Balancer Example 0 0.5 1 1.5 2 2.5 3 3.5 Latency(s) Latency 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 RawAnomalyScore Raw Anomaly Score
- 10. Historical Comparison Compute normal distribution over history Compute probability for each point relative to the distribution m = xP(x)å s = E[(X -m)2 ]
- 11. Anomaly Likelihood 0 0.2 0.4 0.6 0.8 1 1.2 Raw Anomaly Score Mean and Standard Deviation mean std dev 0 0.05 0.1 0.15 0.2 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Probability Probability Distribution
- 12. Anomalies in Load Balancer Latency
- 15. Subtle Anomalies Patterns that humans can’t see but are important
- 16. HTM Learning Algorithms Compared to Other Techniques Anomaly Type Sudden In predictable data In noisy data Human can’t see HTM Learning Algorithms Yes Yes Yes Yes Thresholds Yes No No No Various Statistical Yes Maybe Yes No Time Series Analysis Yes Yes No No Distance-based Yes Maybe No No Supervised Methods N/A N/A N/A N/A See “The Science of Anomaly Detection” White Paper at numenta.com
- 17. Benchmarking Streaming Anomaly Detection • No training/test set • No parameter tuning per data sample • Need real data samples in addition to artificial • We haven’t found any streaming anomaly detection benchmarks so far
- 18. Numenta Anomaly Benchmark (NAB) • Work in progress • High velocity, streaming data • Currently 21 real data samples and 11 artificial samples • Hand-labeled, requiring multiple labelers to agree • Open source, configurable – Currently runs HTM Learning Algorithms and Etsy Skyline algorithms • Follow progress at http://github.com/numenta/nab • Please participate!
- 19. Next Steps Read the white paper http://numenta.com/#technology Scott Purdy spurdy@numenta.com NAB http://github.com/numenta/nab Algorithm code http://numenta.org @numenta @scottmpurdy