H2O Design and Infrastructure with Matt Dowle
3 likes1,310 views
This document provides an overview of H2O, an open source machine learning platform that allows for distributed, in-memory analytics of large datasets. It discusses how H2O works, including how it uses a map-reduce style to parallelize machine learning algorithms across multiple nodes. The document demonstrates starting an 8-node H2O cluster on Amazon EC2 and importing a 23GB dataset in under a minute, significantly faster than with other tools. It also summarizes how H2O's distributed fork-join framework executes tasks across nodes and shares data through its distributed data structures.
H2O Design and Infrastructure with Matt Dowle
- 1. H2O.ai Machine Intelligence H2O Design and Infrastructure R Summit and Workshop, Copenhagen 27 Jun 2015 Matt Dowle
- 2. H2O.ai Machine Intelligence 2 Overview 1. What exactly is H2O 2. How it works
- 3. H2O.ai Machine Intelligence 3 I'll start an 8 node cluster live on EC2 now Click
- 4. H2O.ai Machine Intelligence 4 4 mins to start up, 2 slides
- 5. H2O.ai Machine Intelligence 5 H2O Machine learning e.g. Deep Learning In-memory, parallel and distributed 1. Data > 240GB needle-in-haystack; e.g. fraud 2. Data < 240GB compute intensive, parallel 100's cores 3. Data < 240GB where feature engineering > 240GB Speed for i) production and ii) interaction Developed in the open on GitHub Liberal Apache license Use from R, Python or H2O Flow … simultaneously I now work here
- 6. H2O.ai Machine Intelligence 6 8-node cluster on EC2 is now ready LIVE 15MIN DEMO
- 7. H2O.ai Machine Intelligence 7 To use from R # If java is not already installed : $ sudo add-apt-repository -y ppa:webupd8team/java $ sudo apt-get update $ sudo apt-get -y install oracle-java8-installer $ sudo apt-get -y install oracle-java8-set-default $ java -version $ R > install.packages(“h2o”) That's it.
- 8. H2O.ai Machine Intelligence 8 Start H2O > library(h2o) > h2o.init() H2O is not running yet, starting it now... Successfully connected to http://127.0.0.1:54321 R is connected to H2O cluster: H2O cluster uptime: 1 sec 397 ms H2O cluster version: 2.8.4.4 H2O cluster total nodes: 1 H2O cluster total memory: 26.67 GB H2O cluster total cores: 32
- 10. H2O.ai Machine Intelligence 10 library(h2o) h2o.importFile("/dev/shm/test.csv") # 50 seconds library(data.table) fread("/dev/shm/test.csv") # 5 minutes library(readr) read_csv(“/dev/shm/test.csv”) # 12 minutes 23GB .csv, 9 columns, 500e6 rows Parallel Single thread Single thread
- 11. H2O.ai Machine Intelligence 11 h2o.importFile also ● compresses the data in RAM ● profiles the data while reading; e.g. stores min and max per column, for later efficiency gains ● included in 50 seconds ● accepts a directory of multiple files
- 12. H2O.ai Machine Intelligence 12 hex <- h2o.importFile(conn, path) summary(hex) hex$Year <- as.factor(hex$Year) myY <- "IsDepDelayed" myX <- c("Origin", "Dest", "Year", "UniqueCarrier", "DayOfWeek", "Month", "Distance", "FlightNum") dl <- h2o.deeplearning(y = myY, x = myX, training_frame = hex, hidden=c(20,20,20,20), epochs = 1, variable_importances = T) Standard R
- 13. H2O.ai Machine Intelligence 13 How it works Slides by Cliff Click CTO and Co-Founder
- 14. 14 Simple Data-Parallel Coding ● Map/Reduce Per-Row: Stateless ● Example from Linear Regression, Σ y2 ● Auto-parallel, auto-distributed ● Fortran speed, Java Ease double sumY2 = new MRTask() { double map( double d ) { return d*d; } double reduce( double d1, double d2 ) { return d1+d2; } }.doAll( data );
- 15. 15 Simple Data-Parallel Coding ● Map/Reduce Per-Row: Statefull ● Linear Regression Pass1: Σ x, Σ y, Σ y2 class LRPass1 extends MRTask { double sumX, sumY, sumY2;// I can have State? void map( double X, double Y ) { sumX += X; sumY += Y; sumY2 += Y*Y; } void reduce( LRPass1 that ) { sumX += that.sumX ; sumY += that.sumY ; sumY2 += that.sumY2; } }
- 16. 16 Non-blocking distributed KV ● Uniques ● Uses distributed hash set class Uniques extends MRTask { DNonBlockingHashSet<Long> dnbhs = new ...; void map( long id ) { dnbhs.add(id); } void reduce( Uniques that ) { dnbhs.putAll(that.dnbhs); } }; long uniques = new Uniques(). doAll( vecVistors ).dnbhs.size(); Setting dnbhs in <init> makes it an input field. Shared across all maps(). Often read-only. This one is written, so needs a reduce.
- 17. 17 Limitations ● Code runs distributed... ● No I/O or Machine Resource allocation ● No new threads, no locks, no System.exit() ● No global / static variables ● Instead they become node-local ● "Small" global read state: in constructor ● "Small" global writable state: use reduce() ● "Big" state: read/write distributed arrays (Vecs) ● Runs one (big step) to completion, then another...
- 18. 18 Strengths ● Code runs distributed & parallel without effort ● Millions & billions of rows; 1000's of cores ● Single-threaded coding style ● No concurrency issues ● Excellent resource management ● "No knobs needed" for GC or CPUs or network ● No "data placement", no "hot blocks" or "hot locks"
- 19. 19 How Does It Work? (Code)
- 20. Distributed Fork / Join ● T = new MRtask().doAll(data); JVM Task JVM JVM JVM JVM JVM JVM JVM
- 21. Distributed Fork / Join ● T = new MRtask().doAll(data); ● Log tree fan-out copy... JVM Task JVM Task JVM Task JVM JVM JVM JVM JVM
- 22. Distributed Fork / Join ● T = new MRtask().doAll(data); ● Log tree fan-out copy... JVM Task JVM Task JVM Task JVM Task JVM Task JVM Task JVM Task JVM
- 23. Distributed Fork / Join ● T = new MRtask().doAll(data); ● Log tree fan-out copy... JVM Task JVM Task JVM Task JVM Task JVM Task JVM Task JVM Task JVM Task
- 24. Distributed Fork / Join ● Within a single node, classic Fork/Join ● Divide & Conquer JVM Task data data data data
- 25. Distributed Fork / Join ● Within a single node, classic Fork/Join ● Divide & Conquer JVM Task Task data data data data
- 26. Distributed Fork / Join ● Within a single node, classic Fork/Join ● Divide & Conquer JVM Task Task Task Task data data data data
- 27. Distributed Fork / Join ● Within a single node, classic Fork/Join ● Divide & Conquer JVM Task Task Task Task data data data data
- 28. Distributed Fork / Join ● Within a single node, classic Fork/Join ● Divide & Conquer, parallel map over local data JVM Task Task Task Task data data data data map() Task.B Task.B Task.B Task.B
- 29. Distributed Fork / Join JVM Task.B Task.B Task.B Task.B reduce() reduce() ● Within a single node, classic Fork/Join ● Parallel, eager reduces
- 30. Distributed Fork / Join JVM Task.B Task.B reduce( ) ● Within a single node, classic Fork/Join ● Log-tree reduce
- 31. Distributed Fork / Join JVM ● Within a single node, classic Fork/Join ● Reduce to the same top-level instance Task.B
- 32. Distributed Fork / Join ● Reductions back up the log-tree JVM Task.B JVM Task.B JVM Task.B JVM Task.B JVM Task.B JVM Task.B JVM Task.B JVM Task.B reduce() reduce() reduce()
- 33. Distributed Fork / Join ● Reductions back up the log-tree JVM Task.B JVM Task.B JVM Task.B JVM Task.B JVM JVM JVM JVM reduce()
- 34. Distributed Fork / Join ● Final reduction into the original instance JVM Task.B JVM Task.B JVM JVM JVM JVM JVM JVM
- 35. Distributed Fork / Join ● Final reduction into the original instance JVM Task.B JVM JVM JVM JVM JVM JVM JVM
- 36. 36 How Does It Work? (Data)
- 37. 37 JVM 4 Heap JVM 1 Heap JVM 2 Heap JVM 3 Heap Distributed Data Taxonomy Distributed Parallel Execution Vec Vec Vec Vec Vec ● All CPUs grab Chunks in paralle ● F/J load balances ● Code moves to Da ● Map/Reduce & F/J handles all sync ● H2O handles all comm, data mana
- 38. 38 Distributed Data Taxonomy Frame – a collection of Vecs Vec – a collection of Chunks Chunk – a collection of 1e3 to 1e6 elems elem – a java double Row i – i'th elements of all the Vecs in a Frame
- 39. 39 Summary
- 40. 40 Distributed Coding Taxonomy ● No Distribution Coding: ● Whole Algorithms, Whole Vector-Math ● REST + JSON: e.g. load data, GLM, get results ● R, Python, Web, bash/curl ● Simple Data-Parallel Coding: ● Map/Reduce-style: e.g. Any dense linear algebra ● Java/Scala foreach* style ● Complex Data-Parallel Coding ● K/V Store, Graph Algo's, e.g. PageRank
- 41. 41 Summary: Writing (distributed) Java ● Most simple Java “just works” ● Scala API is experimental, but will also "just work" ● Fast: parallel distributed reads, writes, appends ● Reads same speed as plain Java array loads ● Writes, appends: slightly slower (compression) ● Typically memory bandwidth limited – (may be CPU limited in a few cases) ● Slower: conflicting writes (but follows strict JMM) ● Also supports transactional updates
- 42. 42 Summary: Writing Analytics ● We're writing Big Data Distributed Analytics ● Deep Learning ● Generalized Linear Modeling (ADMM, GLMNET) – Logistic Regression, Poisson, Gamma ● Random Forest, GBM, KMeans, PCA, ... ● Come write your own (distributed) algorithm!!!
- 43. H2O.ai Machine Intelligence 43 Further articles from H2O Efficient Low Latency Java and GCs A K/V Store For In-Memory Analytics: Part 1 A K/V Store For In-Memory Analytics, Part 2 H2O Architecture http://h2o.ai/about/