SlideShare a Scribd company logo

H2O Deep Learning at Next.ML

Sri Ambati, profile picture
Sri Ambati

The document outlines a workshop on scalable data science and deep learning using H2O, focusing on various applications including Higgs boson classification, MNIST digit recognition, and text classification. It provides detailed insights into H2O's architecture, features, and hands-on sessions on machine learning models, techniques for data preparation, and the development of predictive analytics. Additionally, it includes live demonstrations of deep learning capabilities with performance metrics and highlights the advantages of H2O's open-source platform.

Software
Related topics:
Data Science Insights • Deep Learning
1 of 48
Downloaded 561 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Scalable Data Science and Deep Learning with H2O Next.ML Workshop San Francisco, 1/17/15 Arno Candel, H2O.ai http://tiny.cc/h2o_next_ml_slides
Who am I? PhD in Computational Physics, 2005
 from ETH Zurich Switzerland ! 6 years at SLAC - Accelerator Physics Modeling 2 years at Skytree - Machine Learning 13 months at H2O.ai - Machine Learning ! 15 years in Supercomputing & Modeling ! Named “2014 Big Data All-Star” by Fortune Magazine ! @ArnoCandel
H2O Deep Learning, @ArnoCandel Outline Introduction (10 mins) Methods & Implementation (20 mins) Results and Live Demos (20 mins) Higgs boson classification MNIST handwritten digits Ebay text classification h2o-dev Outlook: Flow, Python Part 2: Hands-On Session (40 mins) Web GUI: Higgs dataset R Studio: Adult, Higgs, MNIST datasets 3
H2O Deep Learning, @ArnoCandel Teamwork at H2O.ai Java, Apache v2 Open-Source #1 Java Machine Learning in Github Join the community! 4
H2O Deep Learning, @ArnoCandel H2O: Open-Source (Apache v2) Predictive Analytics Platform 5
H2O Deep Learning, @ArnoCandel 6 H2O Architecture - Designed for speed, scale, accuracy & ease of use Key technical points: • distributed JVMs + REST API • no Java GC issues 
 (data in byte[], Double) • loss-less number compression • Hadoop integration (v1,YARN) • R package (CRAN) Pre-built fully featured algos:
 K-Means, NB, PCA, CoxPH,
 GLM, RF, GBM, DeepLearning
H2O Deep Learning, @ArnoCandel Wikipedia:
 Deep learning is a set of algorithms in machine learning that attempt to model high-level abstractions in data by using architectures composed of multiple 
 non-linear transformations. What is Deep Learning? Input:
 Image Output:
 User ID 7 Example: Facebook DeepFace
H2O Deep Learning, @ArnoCandel What is NOT Deep Linear models are not deep (by definition) ! Neural nets with 1 hidden layer are not deep (only 1 layer - no feature hierarchy) ! SVMs and Kernel methods are not deep (2 layers: kernel + linear) ! Classification trees are not deep (operate on original input space, no new features generated) 8
H2O Deep Learning, @ArnoCandel 1970s multi-layer feed-forward Neural Network (stochastic gradient descent with back-propagation) ! + distributed processing for big data (fine-grain in-memory MapReduce on distributed data) ! + multi-threaded speedup (async fork/join worker threads operate at FORTRAN speeds) ! + smart algorithms for fast & accurate results (automatic standardization, one-hot encoding of categoricals, missing value imputation, weight & bias initialization, adaptive learning rate, momentum, dropout/l1/L2 regularization, grid search, 
 N-fold cross-validation, checkpointing, load balancing, auto-tuning, model averaging, etc.) ! = powerful tool for (un)supervised machine learning on real-world data H2O Deep Learning 9 all 320 cores maxed out
H2O Deep Learning, @ArnoCandel “fully connected” directed graph of neurons age income employment married single Input layer Hidden layer 1 Hidden layer 2 Output layer 3x4 4x3 3x2#connections information flow input/output neuron hidden neuron 4 3 2#neurons 3 Example Neural Network 10
H2O Deep Learning, @ArnoCandel age income employment yj = tanh(sumi(xi*uij)+bj) uij xi yj per-class probabilities
 sum(pl) = 1 zk = tanh(sumj(yj*vjk)+ck) vjk zk pl pl = softmax(sumk(zk*wkl)+dl) wkl softmax(xk) = exp(xk) / sumk(exp(xk)) “neurons activate each other via weighted sums” Prediction: Forward Propagation activation function: tanh alternative:
 x -> max(0,x) “rectifier” pl is a non-linear function of xi: can approximate ANY function with enough layers! bj, ck, dl: bias values
 (indep. of inputs) 11 married single
H2O Deep Learning, @ArnoCandel age income employment xi Automatic standardization of data
 xi: mean = 0, stddev = 1 ! horizontalize categorical variables, e.g. {full-time, part-time, none, self-employed} 
 ->
 {0,1,0} = part-time, {0,0,0} = self-employed Automatic initialization of weights ! Poor man’s initialization: random weights wkl ! Default (better): Uniform distribution in
 +/- sqrt(6/(#units + #units_previous_layer)) Data preparation & Initialization Neural Networks are sensitive to numerical noise,
 operate best in the linear regime (not saturated) 12 married single wkl
H2O Deep Learning, @ArnoCandel Mean Square Error = (0.22 + 0.22)/2 “penalize differences per-class” ! Cross-entropy = -log(0.8) “strongly penalize non-1-ness” Training: Update Weights & Biases Stochastic Gradient Descent: Update weights and biases via gradient of the error (via back-propagation): For each training row, we make a prediction and compare with the actual label (supervised learning): married10.8 predicted actual Objective: minimize prediction error (MSE or cross-entropy) w <— w - rate * ∂E/∂w 1 13 single00.2 E w rate
H2O Deep Learning, @ArnoCandel Backward Propagation 
 ! ∂E/∂wi = ∂E/∂y * ∂y/∂net * ∂net/∂wi = ∂(error(y))/∂y * ∂(activation(net))/∂net * xi Backprop: Compute ∂E/∂wi via chain rule going backwards wi net = sumi(wi*xi) + b xi E = error(y) y = activation(net) How to compute ∂E/∂wi for wi <— wi - rate * ∂E/∂wi ? Naive: For every i, evaluate E twice at (w1,…,wi±∆,…,wN)… Slow! 14
H2O Deep Learning, @ArnoCandel H2O Deep Learning Architecture K-V K-V HTTPD HTTPD nodes/JVMs: sync threads: async communication w w w w w w w w1 w3 w2 w4 w2+w4 w1+w3 w* = (w1+w2+w3+w4)/4 map:
 each node trains a copy of the weights and biases with (some* or all of) its local data with asynchronous F/J threads initial model: weights and biases w updated model: w* H2O atomic in-memory
 K-V store reduce:
 model averaging: average weights and biases from all nodes, speedup is at least #nodes/log(#rows) arxiv:1209.4129v3 Keep iterating over the data (“epochs”), score from time to time Query & display the model via JSON, WWW 2 2 431 1 1 1 4 3 2 1 2 1 i *auto-tuned (default) or user-specified number of points per MapReduce iteration 15
H2O Deep Learning, @ArnoCandel Adaptive learning rate - ADADELTA (Google)
 Automatically set learning rate for each neuron based on its training history Grid Search and Checkpointing
 Run a grid search to scan many hyper- parameters, then continue training the most promising model(s) Regularization
 L1: penalizes non-zero weights
 L2: penalizes large weights
 Dropout: randomly ignore certain inputs Hogwild!: intentional race conditions Distributed mode: weight averaging 16 “Secret” Sauce to Higher Accuracy
H2O Deep Learning, @ArnoCandel Detail: Adaptive Learning Rate ! Compute moving average of ∆wi 2 at time t for window length rho: ! E[∆wi 2]t = rho * E[∆wi 2]t-1 + (1-rho) * ∆wi 2 ! Compute RMS of ∆wi at time t with smoothing epsilon: ! RMS[∆wi]t = sqrt( E[∆wi 2]t + epsilon ) Adaptive annealing / progress: Gradient-dependent learning rate, moving window prevents “freezing” (unlike ADAGRAD: no window) Adaptive acceleration / momentum: accumulate previous weight updates, but over a window of time RMS[∆wi]t-1 RMS[∂E/∂wi]t rate(wi, t) = Do the same for ∂E/∂wi, then obtain per-weight learning rate: cf. ADADELTA paper 17
H2O Deep Learning, @ArnoCandel Detail: Dropout Regularization 18 Training: For each hidden neuron, for each training sample, for each iteration, ignore (zero out) a different random fraction p of input activations. ! age income employment married single X X X Testing: Use all activations, but reduce them by a factor p (to “simulate” the missing activations during training). cf. Geoff Hinton's paper
H2O Deep Learning, @ArnoCandel 19 Application: Higgs Boson Classification Higgs
 vs
 Background Large Hadron Collider: Largest experiment of mankind! $13+ billion, 16.8 miles long, 120 MegaWatts, -456F, 1PB/day, etc. Higgs boson discovery (July ’12) led to 2013 Nobel prize! http://arxiv.org/pdf/1402.4735v2.pdf Images courtesy CERN / LHC HIGGS UCI Dataset: 21 low-level features AND 7 high-level derived features (physics formulae) Train: 10M rows, Valid: 500k, Test: 500k rows
H2O Deep Learning, @ArnoCandel 20 Live Demo: Let’s see what Deep Learning can do with low-level features alone! ? ? ? Former baseline for AUC: 0.733 and 0.816 H2O Algorithm low-level H2O AUC all features H2O AUC Generalized Linear Model 0.596 0.684 Random Forest 0.764 0.840 Gradient Boosted Trees 0.753 0.839 Neural Net 1 hidden layer 0.760 0.830 H2O Deep Learning ? add
 derived
 ! features Higgs: Derived features are important!
H2O Deep Learning, @ArnoCandel MNIST: digits classification Standing world record:
 Without distortions or convolutions, the best-ever published error rate on test set: 0.83% (Microsoft) 21 Train: 60,000 rows 784 integer columns 10 classes Test: 10,000 rows 784 integer columns 10 classes MNIST = Digitized handwritten digits database (Yann LeCun) Data: 28x28=784 pixels with (gray-scale) values in 0…255 Yann LeCun: “Yet another advice: don't get fooled by people who claim to have a solution to Artificial General Intelligence. Ask them what error rate they get on MNIST or ImageNet.”
H2O Deep Learning, @ArnoCandel 22 H2O Deep Learning beats MNIST Standard 60k/10k data No distortions No convolutions No unsupervised training No ensemble ! 10 hours on 10 16-core nodes World-record! 0.83% test set error http://learn.h2o.ai/content/hands-on_training/deep_learning.html
H2O Deep Learning, @ArnoCandel POJO Model Export for Production Scoring 23 Plain old Java code is auto-generated to take your H2O Deep Learning models into production!
H2O Deep Learning, @ArnoCandel Parallel Scalability (for 64 epochs on MNIST, with “0.83%” parameters) 24 Speedup 0.00 10.00 20.00 30.00 40.00 1 2 4 8 16 32 63 H2O Nodes (4 cores per node, 1 epoch per node per MapReduce) 2.7 mins Training Time 0 25 50 75 100 1 2 4 8 16 32 63 H2O Nodes in minutes
H2O Deep Learning, @ArnoCandel Goal: Predict the item from seller’s text description 25 Train: 578,361 rows 8,647 cols 467 classes Test: 64,263 rows 8,647 cols 143 classes “Vintage 18KT gold Rolex 2 Tone in great condition” Data: Bag of words vector 0,0,1,0,0,0,0,0,1,0,0,0,1,…,0 vintagegold condition Text Classification
H2O Deep Learning, @ArnoCandel Out-Of-The-Box: 11.6% test set error after 10 epochs! Predicts the correct class (out of 143) 88.4% of the time! 26 Note 2: No tuning was done
 (results are for illustration only) Train: 578,361 rows 8,647 cols 467 classes Test: 64,263 rows 8,647 cols 143 classes Note 1: H2O columnar-compressed in-memory store only needs 60 MB to store 5 billion values (dense CSV needs 18 GB) Text Classification
H2O Deep Learning, @ArnoCandel MNIST: Unsupervised Anomaly Detection with Deep Learning (Autoencoder) 27 The good The bad The ugly Download the script and run it yourself!
H2O Deep Learning, @ArnoCandel 28 How well did Deep Learning do? Let’s see how H2O did in the past 10 minutes! Higgs: Live Demo (Continued) <your guess?> reference paper results Any guesses for AUC on low-level features? AUC=0.76 was the best for RF/GBM/NN (H2O)
H2O Deep Learning, @ArnoCandel H2O Steam: Scoring Platform 29 Higgs Dataset Demo on 10-node cluster Let’s score all our H2O models and compare them! http://server:port/steam/index.html Live Demo
H2O Deep Learning, @ArnoCandel 30 Live Demo on 10-node cluster: <10 minutes runtime for all H2O algos! Better than LHC baseline of AUC=0.73! Scoring Higgs Models in H2O Steam
H2O Deep Learning, @ArnoCandel 31 Algorithm Paper’s l-l AUC low-level H2O AUC all features
 H2O AUC Parameters (not heavily tuned), 
 H2O running on 10 nodes Generalized Linear Model - 0.596 0.684 default, binomial Random Forest - 0.764 0.840 50 trees, max depth 50 Gradient Boosted Trees 0.73 0.753 0.839 50 trees, max depth 15 Neural Net 1 layer 0.733 0.760 0.830 1x300 Rectifier, 100 epochs Deep Learning 3 hidden layers 0.836 0.850 - 3x1000 Rectifier, L2=1e-5, 40 epochs Deep Learning 4 hidden layers 0.868 0.869 - 4x500 Rectifier, L1=L2=1e-5, 300 epochs Deep Learning 5 hidden layers 0.880 0.871 - 5x500 Rectifier, L1=L2=1e-5 Deep Learning on low-level features alone beats everything else! Prelim. H2O results compare well with paper’s results* (TMVA & Theano) Higgs Particle Detection with H2O *Nature paper: http://arxiv.org/pdf/1402.4735v2.pdf HIGGS UCI Dataset: 21 low-level features AND 7 high-level derived features Train: 10M rows, Test: 500k rows
H2O Deep Learning, @ArnoCandel Coming very soon: h2o-dev New UI: Flow New languages: python, Javascript 32
H2O Deep Learning, @ArnoCandel h2o-dev Python Example 33
H2O Deep Learning, @ArnoCandel Part 2: Hands-On Session 34 Web GUI Import Higgs data, split into train/test Train grid search Deep Learning model Continue training the best model ROC and Multi-Model Scoring R Studio Connect to running H2O Cluster from R Run ML algos on 3 different datasets More: Follow examples from http://learn.h2o.ai 
 (R scripts and data at http://data.h2o.ai)
H2O Deep Learning, @ArnoCandel H2O Docker VM 35 http://h2o.ai/blog/2015/01/h2o-docker/ H2O will be at
 http://`boot2docker ip`:8996
H2O Deep Learning, @ArnoCandel Import Higgs data 36 Enter
H2O Deep Learning, @ArnoCandel Split Into Train/Test 37
H2O Deep Learning, @ArnoCandel Train Grid Search DL Model 38 Enter Enter Enter Enter
H2O Deep Learning, @ArnoCandel Continue Training Best Model 39 Scroll right Enter
H2O Deep Learning, @ArnoCandel Inspect ROC, thresholds, etc. 40
H2O Deep Learning, @ArnoCandel Multi-Model Scoring 41
H2O Deep Learning, @ArnoCandel Control H2O from R Studio 42 http://learn.h2o.ai/ R scripts in github 1) Paste content of
 http://tiny.cc/h2o_next_ml into R Studio 2) Execute line by line with Ctrl-Enter to run ML algorithms on H2O Cluster via R 3) Check out the links below for more info http://h2o.gitbooks.io
H2O Deep Learning, @ArnoCandel Snippets from R script 43 Install H2O R package & connect to H2O Server Run Deep Learning on MNIST
H2O Deep Learning, @ArnoCandel 44 H2O GitBooks Also available: GBM & GLM GitBooks at http://h2o.gitbooks.io H2O World learn.h2o.ai R, EC2, Hadoop Deep Learning
H2O Deep Learning, @ArnoCandel H2O Kaggle Starter R Scripts 45
H2O Deep Learning, @ArnoCandel Re-Live H2O World! 46 http://h2o.ai/h2o-world/ http://learn.h2o.ai Watch the Videos Day 2 • Speakers from Academia & Industry • Trevor Hastie (ML) • John Chambers (S, R) • Josh Bloch (Java API) • Many use cases from customers • 3 Top Kaggle Contestants (Top 10) • 3 Panel discussions Day 1 • Hands-On Training • Supervised • Unsupervised • Advanced Topics • Markting Usecase • Product Demos • Hacker-Fest with 
 Cliff Click (CTO, Hotspot)
H2O Deep Learning, @ArnoCandel You can participate! 47 - Images: Convolutional & Pooling Layers PUB-644 - Sequences: Recurrent Neural Networks PUB-1052 - Faster Training: GPGPU support PUB-1013 - Pre-Training: Stacked Auto-Encoders PUB-1014 - Ensembles PUB-1072 - Use H2O at Kaggle Challenges!
H2O Deep Learning, @ArnoCandel Key Take-Aways H2O is an open source predictive analytics platform for data scientists and business analysts who need scalable and fast machine learning. ! H2O Deep Learning is ready to take your advanced analytics to the next level - Try it on your data! ! Join our Community and Meetups! https://github.com/h2oai h2ostream community forum www.h2o.ai @h2oai 48 Thank you!

More Related Content

PDF
H2ODeepLearningThroughExamples021215
Sri Ambati
 
PDF
H2O Distributed Deep Learning by Arno Candel 071614
Sri Ambati
 
PDF
MLconf - Distributed Deep Learning for Classification and Regression Problems...
Sri Ambati
 
PDF
How to win data science competitions with Deep Learning
Sri Ambati
 
PDF
H2O Open Source Deep Learning, Arno Candel 03-20-14
Sri Ambati
 
PDF
Deep Learning in the Wild with Arno Candel
Sri Ambati
 
PDF
Alex Tellez, Deep Learning Applications
Sri Ambati
 
PDF
Deep Learning through Examples
Sri Ambati
 
H2ODeepLearningThroughExamples021215
Sri Ambati
 
H2O Distributed Deep Learning by Arno Candel 071614
Sri Ambati
 
MLconf - Distributed Deep Learning for Classification and Regression Problems...
Sri Ambati
 
How to win data science competitions with Deep Learning
Sri Ambati
 
H2O Open Source Deep Learning, Arno Candel 03-20-14
Sri Ambati
 
Deep Learning in the Wild with Arno Candel
Sri Ambati
 
Alex Tellez, Deep Learning Applications
Sri Ambati
 
Deep Learning through Examples
Sri Ambati
 

What's hot (20)

PPTX
Deep Learning with Python (PyData Seattle 2015)
Alexander Korbonits
 
PDF
Webinar: Deep Learning with H2O
Sri Ambati
 
PDF
San Francisco Hadoop User Group Meetup Deep Learning
Sri Ambati
 
PDF
Ted Willke, Senior Principal Engineer & GM, Datacenter Group, Intel at MLconf SF
MLconf
 
PPTX
Introduction to Deep Learning
Adam Rogers
 
PDF
Introduction to Deep Learning with Python
indico data
 
PDF
Deep Learning and Reinforcement Learning
Renārs Liepiņš
 
PDF
Distance oracle - Truy vấn nhanh khoảng cách giữa hai điểm bất kỳ trên đồ thị
Hong Ong
 
PDF
Using Deep Learning to do Real-Time Scoring in Practical Applications - 2015-...
Greg Makowski
 
PPTX
An introduction to Deep Learning
David Rostcheck
 
PDF
Machine Learning and Deep Learning with R
Poo Kuan Hoong
 
PDF
Deep Learning And Business Models (VNITC 2015-09-13)
Ha Phuong
 
PDF
[DSC 2016] 系列活動：李宏毅 / 一天搞懂深度學習
台灣資料科學年會
 
PPTX
Machine Learning, Deep Learning and Data Analysis Introduction
Te-Yen Liu
 
PDF
Deep Learning Cases: Text and Image Processing
Grigory Sapunov
 
PDF
Python for Image Understanding: Deep Learning with Convolutional Neural Nets
Roelof Pieters
 
PDF
Applying your Convolutional Neural Networks
Databricks
 
PPTX
Diving into Deep Learning (Silicon Valley Code Camp 2017)
Oswald Campesato
 
PDF
Using Deep Learning to do Real-Time Scoring in Practical Applications
Greg Makowski
 
PDF
Language translation with Deep Learning (RNN) with TensorFlow
S N
 
Deep Learning with Python (PyData Seattle 2015)
Alexander Korbonits
 
Webinar: Deep Learning with H2O
Sri Ambati
 
San Francisco Hadoop User Group Meetup Deep Learning
Sri Ambati
 
Ted Willke, Senior Principal Engineer & GM, Datacenter Group, Intel at MLconf SF
MLconf
 
Introduction to Deep Learning
Adam Rogers
 
Introduction to Deep Learning with Python
indico data
 
Deep Learning and Reinforcement Learning
Renārs Liepiņš
 
Distance oracle - Truy vấn nhanh khoảng cách giữa hai điểm bất kỳ trên đồ thị
Hong Ong
 
Using Deep Learning to do Real-Time Scoring in Practical Applications - 2015-...
Greg Makowski
 
An introduction to Deep Learning
David Rostcheck
 
Machine Learning and Deep Learning with R
Poo Kuan Hoong
 
Deep Learning And Business Models (VNITC 2015-09-13)
Ha Phuong
 
[DSC 2016] 系列活動：李宏毅 / 一天搞懂深度學習
台灣資料科學年會
 
Machine Learning, Deep Learning and Data Analysis Introduction
Te-Yen Liu
 
Deep Learning Cases: Text and Image Processing
Grigory Sapunov
 
Python for Image Understanding: Deep Learning with Convolutional Neural Nets
Roelof Pieters
 
Applying your Convolutional Neural Networks
Databricks
 
Diving into Deep Learning (Silicon Valley Code Camp 2017)
Oswald Campesato
 
Using Deep Learning to do Real-Time Scoring in Practical Applications
Greg Makowski
 
Language translation with Deep Learning (RNN) with TensorFlow
S N
 
Ad

Viewers also liked (20)

PDF
Transform your Business with AI, Deep Learning and Machine Learning
Sri Ambati
 
PDF
Deep Learning Computer Build
PetteriTeikariPhD
 
PDF
Deep Learning - The Past, Present and Future of Artificial Intelligence
Lukas Masuch
 
PDF
Passive stereo vision with deep learning
Yu Huang
 
PDF
Deep Learning and the state of AI / 2016
Grigory Sapunov
 
PPTX
New pedagogies for deep learning
Danielgroenewald78
 
PPTX
Squeezing Deep Learning Into Mobile Phones
Anirudh Koul
 
PPTX
What Deep Learning Means for Artificial Intelligence
Jonathan Mugan
 
PPTX
Deep learning at nmc devin jones
Ido Shilon
 
PPTX
Best Deep Learning Post from LinkedIn Group
Farshid Pirahansiah
 
PDF
Donner - Deep Learning - Overview and practical aspects
Vienna Data Science Group
 
PDF
Deep learning - Part I
QuantUniversity
 
ODP
Start a deep learning startup - tutorial
Mostapha Benhenda
 
PDF
Indoor Point Cloud Processing - Deep learning for semantic segmentation of in...
CubiCasa
 
PPTX
Spark machine learning & deep learning
hoondong kim
 
PDF
Introduction to Machine Learning and Deep Learning
Terry Taewoong Um
 
PDF
Deep learning - Conceptual understanding and applications
Buhwan Jeong
 
PDF
Deep Water - GPU Deep Learning for H2O - Arno Candel
Sri Ambati
 
PPTX
Deep Learning in Computer Vision
Sungjoon Choi
 
PDF
Deep Learning - Convolutional Neural Networks
Christian Perone
 
Transform your Business with AI, Deep Learning and Machine Learning
Sri Ambati
 
Deep Learning Computer Build
PetteriTeikariPhD
 
Deep Learning - The Past, Present and Future of Artificial Intelligence
Lukas Masuch
 
Passive stereo vision with deep learning
Yu Huang
 
Deep Learning and the state of AI / 2016
Grigory Sapunov
 
New pedagogies for deep learning
Danielgroenewald78
 
Squeezing Deep Learning Into Mobile Phones
Anirudh Koul
 
What Deep Learning Means for Artificial Intelligence
Jonathan Mugan
 
Deep learning at nmc devin jones
Ido Shilon
 
Best Deep Learning Post from LinkedIn Group
Farshid Pirahansiah
 
Donner - Deep Learning - Overview and practical aspects
Vienna Data Science Group
 
Deep learning - Part I
QuantUniversity
 
Start a deep learning startup - tutorial
Mostapha Benhenda
 
Indoor Point Cloud Processing - Deep learning for semantic segmentation of in...
CubiCasa
 
Spark machine learning & deep learning
hoondong kim
 
Introduction to Machine Learning and Deep Learning
Terry Taewoong Um
 
Deep learning - Conceptual understanding and applications
Buhwan Jeong
 
Deep Water - GPU Deep Learning for H2O - Arno Candel
Sri Ambati
 
Deep Learning in Computer Vision
Sungjoon Choi
 
Deep Learning - Convolutional Neural Networks
Christian Perone
 
Ad

Similar to H2O Deep Learning at Next.ML (20)

PDF
H2O.ai's Distributed Deep Learning by Arno Candel 04/03/14
Sri Ambati
 
PPTX
Online learning, Vowpal Wabbit and Hadoop
Héloïse Nonne
 
PDF
Arno candel scalabledatascienceanddeeplearningwithh2o_reworkboston2015
Sri Ambati
 
PDF
ArnoCandelScalabledatascienceanddeeplearningwithh2o_gotochg
Sri Ambati
 
PDF
W2 - Multilayer Neural Network Lecture notes university of sydney
mayanksharmadav
 
PPTX
Le Song, Assistant Professor, College of Computing, Georgia Institute of Tech...
MLconf
 
PDF
The Back Propagation Learning Algorithm
ESCOM
 
PDF
Convolutional neural networks for image classification — evidence from Kaggle...
Dmytro Mishkin
 
PDF
qconsf 2013: Top 10 Performance Gotchas for scaling in-memory Algorithms - Sr...
Sri Ambati
 
PDF
Top 10 Performance Gotchas for scaling in-memory Algorithms.
srisatish ambati
 
PDF
Ling liu part 02：big graph processing
jins0618
 
PDF
Arno candel scalabledatascienceanddeeplearningwithh2o_odsc_boston2015
Sri Ambati
 
PDF
Scalable Data Science and Deep Learning with H2O
odsc
 
PDF
Martin Takac - “Solving Large-Scale Machine Learning Problems in a Distribute...
diannepatricia
 
PDF
My Postdoctoral Research
Po-Ting Wu
 
PDF
Discovering Your AI Super Powers - Tips and Tricks to Jumpstart your AI Projects
Wee Hyong Tok
 
PPTX
Artificial Neural Network
Dessy Amirudin
 
PDF
Productive Use of the Apache Spark Prompt with Sam Penrose
Databricks
 
PDF
Functional Programming with Immutable Data Structures
elliando dias
 
PPTX
Digit recognizer by convolutional neural network
Ding Li
 
H2O.ai's Distributed Deep Learning by Arno Candel 04/03/14
Sri Ambati
 
Online learning, Vowpal Wabbit and Hadoop
Héloïse Nonne
 
Arno candel scalabledatascienceanddeeplearningwithh2o_reworkboston2015
Sri Ambati
 
ArnoCandelScalabledatascienceanddeeplearningwithh2o_gotochg
Sri Ambati
 
W2 - Multilayer Neural Network Lecture notes university of sydney
mayanksharmadav
 
Le Song, Assistant Professor, College of Computing, Georgia Institute of Tech...
MLconf
 
The Back Propagation Learning Algorithm
ESCOM
 
Convolutional neural networks for image classification — evidence from Kaggle...
Dmytro Mishkin
 
qconsf 2013: Top 10 Performance Gotchas for scaling in-memory Algorithms - Sr...
Sri Ambati
 
Top 10 Performance Gotchas for scaling in-memory Algorithms.
srisatish ambati
 
Ling liu part 02：big graph processing
jins0618
 
Arno candel scalabledatascienceanddeeplearningwithh2o_odsc_boston2015
Sri Ambati
 
Scalable Data Science and Deep Learning with H2O
odsc
 
Martin Takac - “Solving Large-Scale Machine Learning Problems in a Distribute...
diannepatricia
 
My Postdoctoral Research
Po-Ting Wu
 
Discovering Your AI Super Powers - Tips and Tricks to Jumpstart your AI Projects
Wee Hyong Tok
 
Artificial Neural Network
Dessy Amirudin
 
Productive Use of the Apache Spark Prompt with Sam Penrose
Databricks
 
Functional Programming with Immutable Data Structures
elliando dias
 
Digit recognizer by convolutional neural network
Ding Li
 

More from Sri Ambati (20)

PDF
H2O Label Genie Starter Track - Support Presentation
Sri Ambati
 
PDF
H2O.ai Agents : From Theory to Practice - Support Presentation
Sri Ambati
 
PDF
H2O Generative AI Starter Track - Support Presentation Slides.pdf
Sri Ambati
 
PDF
H2O Gen AI Ecosystem Overview - Level 1 - Slide Deck
Sri Ambati
 
PDF
An In-depth Exploration of Enterprise h2oGPTe Slide Deck
Sri Ambati
 
PDF
Intro to Enterprise h2oGPTe Presentation Slides
Sri Ambati
 
PDF
Enterprise h2o GPTe Learning Path Slide Deck
Sri Ambati
 
PDF
H2O Wave Course Starter - Presentation Slides
Sri Ambati
 
PDF
Large Language Models (LLMs) - Level 3 Slides
Sri Ambati
 
PDF
Data Science and Machine Learning Platforms (2024) Slides
Sri Ambati
 
PDF
Data Prep for H2O Driverless AI - Slides
Sri Ambati
 
PDF
H2O Cloud AI Developer Services - Slides (2024)
Sri Ambati
 
PDF
LLM Learning Path Level 2 - Presentation Slides
Sri Ambati
 
PDF
LLM Learning Path Level 1 - Presentation Slides
Sri Ambati
 
PDF
Hydrogen Torch - Starter Course - Presentation Slides
Sri Ambati
 
PDF
Presentation Resources - H2O Gen AI Ecosystem Overview - Level 2
Sri Ambati
 
PDF
H2O Driverless AI Starter Course - Slides and Assignments
Sri Ambati
 
PPTX
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 
PDF
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
Sri Ambati
 
PPTX
Generative AI Masterclass - Model Risk Management.pptx
Sri Ambati
 
H2O Label Genie Starter Track - Support Presentation
Sri Ambati
 
H2O.ai Agents : From Theory to Practice - Support Presentation
Sri Ambati
 
H2O Generative AI Starter Track - Support Presentation Slides.pdf
Sri Ambati
 
H2O Gen AI Ecosystem Overview - Level 1 - Slide Deck
Sri Ambati
 
An In-depth Exploration of Enterprise h2oGPTe Slide Deck
Sri Ambati
 
Intro to Enterprise h2oGPTe Presentation Slides
Sri Ambati
 
Enterprise h2o GPTe Learning Path Slide Deck
Sri Ambati
 
H2O Wave Course Starter - Presentation Slides
Sri Ambati
 
Large Language Models (LLMs) - Level 3 Slides
Sri Ambati
 
Data Science and Machine Learning Platforms (2024) Slides
Sri Ambati
 
Data Prep for H2O Driverless AI - Slides
Sri Ambati
 
H2O Cloud AI Developer Services - Slides (2024)
Sri Ambati
 
LLM Learning Path Level 2 - Presentation Slides
Sri Ambati
 
LLM Learning Path Level 1 - Presentation Slides
Sri Ambati
 
Hydrogen Torch - Starter Course - Presentation Slides
Sri Ambati
 
Presentation Resources - H2O Gen AI Ecosystem Overview - Level 2
Sri Ambati
 
H2O Driverless AI Starter Course - Slides and Assignments
Sri Ambati
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
Sri Ambati
 
Generative AI Masterclass - Model Risk Management.pptx
Sri Ambati
 

Recently uploaded (20)

PDF
System and Network Administration Chapter 2
jaramharo
 
PDF
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
rohnjim07
 
PPTX
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
hci7se
 
PPTX
ISO 45001 Occupational Health and Safety Management System
EHA Soft Solutions
 
PPTX
Odoo POS Development Services by CandidRoot Solutions
CandidRoot Solutions Private Limited
 
PDF
System and Network Administraation Chapter 3
jaramharo
 
PDF
Navsoft: AI-Powered Business Solutions & Custom Software Development
Navsoft
 
PDF
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
itskinga12
 
PDF
Softaken Excel to vCard Converter Software.pdf
markwillsonmw004
 
PDF
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
William Chong
 
PDF
2025 Textile ERP Trends: SAP, Odoo & Oracle
Aetos Technologies
 
PDF
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
Andre Kraus
 
PDF
How to Choose the Right IT Partner for Your Business in Malaysia
Asian Business Services & Technologies
 
PPTX
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
PDF
Internet Downloader Manager (IDM) Crack 6.42 Build 41
ghrom2211g
 
PPTX
history of c programming in notes for students .pptx
Vijayakumari829030
 
PPTX
ai tools demonstartion for schools and inter college
harshavardhanreddyka11
 
PDF
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
Actowiz Solustions
 
PPTX
Agentic AI Use Case- Contract Lifecycle Management (CLM).pptx
mhxyzzp
 
PDF
top salesforce developer skills in 2025.pdf
CloudMetic
 
System and Network Administration Chapter 2
jaramharo
 
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
rohnjim07
 
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
hci7se
 
ISO 45001 Occupational Health and Safety Management System
EHA Soft Solutions
 
Odoo POS Development Services by CandidRoot Solutions
CandidRoot Solutions Private Limited
 
System and Network Administraation Chapter 3
jaramharo
 
Navsoft: AI-Powered Business Solutions & Custom Software Development
Navsoft
 
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
itskinga12
 
Softaken Excel to vCard Converter Software.pdf
markwillsonmw004
 
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
William Chong
 
2025 Textile ERP Trends: SAP, Odoo & Oracle
Aetos Technologies
 
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
Andre Kraus
 
How to Choose the Right IT Partner for Your Business in Malaysia
Asian Business Services & Technologies
 
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
Internet Downloader Manager (IDM) Crack 6.42 Build 41
ghrom2211g
 
history of c programming in notes for students .pptx
Vijayakumari829030
 
ai tools demonstartion for schools and inter college
harshavardhanreddyka11
 
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
Actowiz Solustions
 
Agentic AI Use Case- Contract Lifecycle Management (CLM).pptx
mhxyzzp
 
top salesforce developer skills in 2025.pdf
CloudMetic
 

H2O Deep Learning at Next.ML

  • 1. Scalable Data Science and Deep Learning with H2O Next.ML Workshop San Francisco, 1/17/15 Arno Candel, H2O.ai http://tiny.cc/h2o_next_ml_slides
  • 2. Who am I? PhD in Computational Physics, 2005
 from ETH Zurich Switzerland ! 6 years at SLAC - Accelerator Physics Modeling 2 years at Skytree - Machine Learning 13 months at H2O.ai - Machine Learning ! 15 years in Supercomputing & Modeling ! Named “2014 Big Data All-Star” by Fortune Magazine ! @ArnoCandel
  • 3. H2O Deep Learning, @ArnoCandel Outline Introduction (10 mins) Methods & Implementation (20 mins) Results and Live Demos (20 mins) Higgs boson classification MNIST handwritten digits Ebay text classification h2o-dev Outlook: Flow, Python Part 2: Hands-On Session (40 mins) Web GUI: Higgs dataset R Studio: Adult, Higgs, MNIST datasets 3
  • 4. H2O Deep Learning, @ArnoCandel Teamwork at H2O.ai Java, Apache v2 Open-Source #1 Java Machine Learning in Github Join the community! 4
  • 5. H2O Deep Learning, @ArnoCandel H2O: Open-Source (Apache v2) Predictive Analytics Platform 5
  • 6. H2O Deep Learning, @ArnoCandel 6 H2O Architecture - Designed for speed, scale, accuracy & ease of use Key technical points: • distributed JVMs + REST API • no Java GC issues 
 (data in byte[], Double) • loss-less number compression • Hadoop integration (v1,YARN) • R package (CRAN) Pre-built fully featured algos:
 K-Means, NB, PCA, CoxPH,
 GLM, RF, GBM, DeepLearning
  • 7. H2O Deep Learning, @ArnoCandel Wikipedia:
 Deep learning is a set of algorithms in machine learning that attempt to model high-level abstractions in data by using architectures composed of multiple 
 non-linear transformations. What is Deep Learning? Input:
 Image Output:
 User ID 7 Example: Facebook DeepFace
  • 8. H2O Deep Learning, @ArnoCandel What is NOT Deep Linear models are not deep (by definition) ! Neural nets with 1 hidden layer are not deep (only 1 layer - no feature hierarchy) ! SVMs and Kernel methods are not deep (2 layers: kernel + linear) ! Classification trees are not deep (operate on original input space, no new features generated) 8
  • 9. H2O Deep Learning, @ArnoCandel 1970s multi-layer feed-forward Neural Network (stochastic gradient descent with back-propagation) ! + distributed processing for big data (fine-grain in-memory MapReduce on distributed data) ! + multi-threaded speedup (async fork/join worker threads operate at FORTRAN speeds) ! + smart algorithms for fast & accurate results (automatic standardization, one-hot encoding of categoricals, missing value imputation, weight & bias initialization, adaptive learning rate, momentum, dropout/l1/L2 regularization, grid search, 
 N-fold cross-validation, checkpointing, load balancing, auto-tuning, model averaging, etc.) ! = powerful tool for (un)supervised machine learning on real-world data H2O Deep Learning 9 all 320 cores maxed out
  • 10. H2O Deep Learning, @ArnoCandel “fully connected” directed graph of neurons age income employment married single Input layer Hidden layer 1 Hidden layer 2 Output layer 3x4 4x3 3x2#connections information flow input/output neuron hidden neuron 4 3 2#neurons 3 Example Neural Network 10
  • 11. H2O Deep Learning, @ArnoCandel age income employment yj = tanh(sumi(xi*uij)+bj) uij xi yj per-class probabilities
 sum(pl) = 1 zk = tanh(sumj(yj*vjk)+ck) vjk zk pl pl = softmax(sumk(zk*wkl)+dl) wkl softmax(xk) = exp(xk) / sumk(exp(xk)) “neurons activate each other via weighted sums” Prediction: Forward Propagation activation function: tanh alternative:
 x -> max(0,x) “rectifier” pl is a non-linear function of xi: can approximate ANY function with enough layers! bj, ck, dl: bias values
 (indep. of inputs) 11 married single
  • 12. H2O Deep Learning, @ArnoCandel age income employment xi Automatic standardization of data
 xi: mean = 0, stddev = 1 ! horizontalize categorical variables, e.g. {full-time, part-time, none, self-employed} 
 ->
 {0,1,0} = part-time, {0,0,0} = self-employed Automatic initialization of weights ! Poor man’s initialization: random weights wkl ! Default (better): Uniform distribution in
 +/- sqrt(6/(#units + #units_previous_layer)) Data preparation & Initialization Neural Networks are sensitive to numerical noise,
 operate best in the linear regime (not saturated) 12 married single wkl
  • 13. H2O Deep Learning, @ArnoCandel Mean Square Error = (0.22 + 0.22)/2 “penalize differences per-class” ! Cross-entropy = -log(0.8) “strongly penalize non-1-ness” Training: Update Weights & Biases Stochastic Gradient Descent: Update weights and biases via gradient of the error (via back-propagation): For each training row, we make a prediction and compare with the actual label (supervised learning): married10.8 predicted actual Objective: minimize prediction error (MSE or cross-entropy) w <— w - rate * ∂E/∂w 1 13 single00.2 E w rate
  • 14. H2O Deep Learning, @ArnoCandel Backward Propagation 
 ! ∂E/∂wi = ∂E/∂y * ∂y/∂net * ∂net/∂wi = ∂(error(y))/∂y * ∂(activation(net))/∂net * xi Backprop: Compute ∂E/∂wi via chain rule going backwards wi net = sumi(wi*xi) + b xi E = error(y) y = activation(net) How to compute ∂E/∂wi for wi <— wi - rate * ∂E/∂wi ? Naive: For every i, evaluate E twice at (w1,…,wi±∆,…,wN)… Slow! 14
  • 15. H2O Deep Learning, @ArnoCandel H2O Deep Learning Architecture K-V K-V HTTPD HTTPD nodes/JVMs: sync threads: async communication w w w w w w w w1 w3 w2 w4 w2+w4 w1+w3 w* = (w1+w2+w3+w4)/4 map:
 each node trains a copy of the weights and biases with (some* or all of) its local data with asynchronous F/J threads initial model: weights and biases w updated model: w* H2O atomic in-memory
 K-V store reduce:
 model averaging: average weights and biases from all nodes, speedup is at least #nodes/log(#rows) arxiv:1209.4129v3 Keep iterating over the data (“epochs”), score from time to time Query & display the model via JSON, WWW 2 2 431 1 1 1 4 3 2 1 2 1 i *auto-tuned (default) or user-specified number of points per MapReduce iteration 15
  • 16. H2O Deep Learning, @ArnoCandel Adaptive learning rate - ADADELTA (Google)
 Automatically set learning rate for each neuron based on its training history Grid Search and Checkpointing
 Run a grid search to scan many hyper- parameters, then continue training the most promising model(s) Regularization
 L1: penalizes non-zero weights
 L2: penalizes large weights
 Dropout: randomly ignore certain inputs Hogwild!: intentional race conditions Distributed mode: weight averaging 16 “Secret” Sauce to Higher Accuracy
  • 17. H2O Deep Learning, @ArnoCandel Detail: Adaptive Learning Rate ! Compute moving average of ∆wi 2 at time t for window length rho: ! E[∆wi 2]t = rho * E[∆wi 2]t-1 + (1-rho) * ∆wi 2 ! Compute RMS of ∆wi at time t with smoothing epsilon: ! RMS[∆wi]t = sqrt( E[∆wi 2]t + epsilon ) Adaptive annealing / progress: Gradient-dependent learning rate, moving window prevents “freezing” (unlike ADAGRAD: no window) Adaptive acceleration / momentum: accumulate previous weight updates, but over a window of time RMS[∆wi]t-1 RMS[∂E/∂wi]t rate(wi, t) = Do the same for ∂E/∂wi, then obtain per-weight learning rate: cf. ADADELTA paper 17
  • 18. H2O Deep Learning, @ArnoCandel Detail: Dropout Regularization 18 Training: For each hidden neuron, for each training sample, for each iteration, ignore (zero out) a different random fraction p of input activations. ! age income employment married single X X X Testing: Use all activations, but reduce them by a factor p (to “simulate” the missing activations during training). cf. Geoff Hinton's paper
  • 19. H2O Deep Learning, @ArnoCandel 19 Application: Higgs Boson Classification Higgs
 vs
 Background Large Hadron Collider: Largest experiment of mankind! $13+ billion, 16.8 miles long, 120 MegaWatts, -456F, 1PB/day, etc. Higgs boson discovery (July ’12) led to 2013 Nobel prize! http://arxiv.org/pdf/1402.4735v2.pdf Images courtesy CERN / LHC HIGGS UCI Dataset: 21 low-level features AND 7 high-level derived features (physics formulae) Train: 10M rows, Valid: 500k, Test: 500k rows
  • 20. H2O Deep Learning, @ArnoCandel 20 Live Demo: Let’s see what Deep Learning can do with low-level features alone! ? ? ? Former baseline for AUC: 0.733 and 0.816 H2O Algorithm low-level H2O AUC all features H2O AUC Generalized Linear Model 0.596 0.684 Random Forest 0.764 0.840 Gradient Boosted Trees 0.753 0.839 Neural Net 1 hidden layer 0.760 0.830 H2O Deep Learning ? add
 derived
 ! features Higgs: Derived features are important!
  • 21. H2O Deep Learning, @ArnoCandel MNIST: digits classification Standing world record:
 Without distortions or convolutions, the best-ever published error rate on test set: 0.83% (Microsoft) 21 Train: 60,000 rows 784 integer columns 10 classes Test: 10,000 rows 784 integer columns 10 classes MNIST = Digitized handwritten digits database (Yann LeCun) Data: 28x28=784 pixels with (gray-scale) values in 0…255 Yann LeCun: “Yet another advice: don't get fooled by people who claim to have a solution to Artificial General Intelligence. Ask them what error rate they get on MNIST or ImageNet.”
  • 22. H2O Deep Learning, @ArnoCandel 22 H2O Deep Learning beats MNIST Standard 60k/10k data No distortions No convolutions No unsupervised training No ensemble ! 10 hours on 10 16-core nodes World-record! 0.83% test set error http://learn.h2o.ai/content/hands-on_training/deep_learning.html
  • 23. H2O Deep Learning, @ArnoCandel POJO Model Export for Production Scoring 23 Plain old Java code is auto-generated to take your H2O Deep Learning models into production!
  • 24. H2O Deep Learning, @ArnoCandel Parallel Scalability (for 64 epochs on MNIST, with “0.83%” parameters) 24 Speedup 0.00 10.00 20.00 30.00 40.00 1 2 4 8 16 32 63 H2O Nodes (4 cores per node, 1 epoch per node per MapReduce) 2.7 mins Training Time 0 25 50 75 100 1 2 4 8 16 32 63 H2O Nodes in minutes
  • 25. H2O Deep Learning, @ArnoCandel Goal: Predict the item from seller’s text description 25 Train: 578,361 rows 8,647 cols 467 classes Test: 64,263 rows 8,647 cols 143 classes “Vintage 18KT gold Rolex 2 Tone in great condition” Data: Bag of words vector 0,0,1,0,0,0,0,0,1,0,0,0,1,…,0 vintagegold condition Text Classification
  • 26. H2O Deep Learning, @ArnoCandel Out-Of-The-Box: 11.6% test set error after 10 epochs! Predicts the correct class (out of 143) 88.4% of the time! 26 Note 2: No tuning was done
 (results are for illustration only) Train: 578,361 rows 8,647 cols 467 classes Test: 64,263 rows 8,647 cols 143 classes Note 1: H2O columnar-compressed in-memory store only needs 60 MB to store 5 billion values (dense CSV needs 18 GB) Text Classification
  • 27. H2O Deep Learning, @ArnoCandel MNIST: Unsupervised Anomaly Detection with Deep Learning (Autoencoder) 27 The good The bad The ugly Download the script and run it yourself!
  • 28. H2O Deep Learning, @ArnoCandel 28 How well did Deep Learning do? Let’s see how H2O did in the past 10 minutes! Higgs: Live Demo (Continued) <your guess?> reference paper results Any guesses for AUC on low-level features? AUC=0.76 was the best for RF/GBM/NN (H2O)
  • 29. H2O Deep Learning, @ArnoCandel H2O Steam: Scoring Platform 29 Higgs Dataset Demo on 10-node cluster Let’s score all our H2O models and compare them! http://server:port/steam/index.html Live Demo
  • 30. H2O Deep Learning, @ArnoCandel 30 Live Demo on 10-node cluster: <10 minutes runtime for all H2O algos! Better than LHC baseline of AUC=0.73! Scoring Higgs Models in H2O Steam
  • 31. H2O Deep Learning, @ArnoCandel 31 Algorithm Paper’s l-l AUC low-level H2O AUC all features
 H2O AUC Parameters (not heavily tuned), 
 H2O running on 10 nodes Generalized Linear Model - 0.596 0.684 default, binomial Random Forest - 0.764 0.840 50 trees, max depth 50 Gradient Boosted Trees 0.73 0.753 0.839 50 trees, max depth 15 Neural Net 1 layer 0.733 0.760 0.830 1x300 Rectifier, 100 epochs Deep Learning 3 hidden layers 0.836 0.850 - 3x1000 Rectifier, L2=1e-5, 40 epochs Deep Learning 4 hidden layers 0.868 0.869 - 4x500 Rectifier, L1=L2=1e-5, 300 epochs Deep Learning 5 hidden layers 0.880 0.871 - 5x500 Rectifier, L1=L2=1e-5 Deep Learning on low-level features alone beats everything else! Prelim. H2O results compare well with paper’s results* (TMVA & Theano) Higgs Particle Detection with H2O *Nature paper: http://arxiv.org/pdf/1402.4735v2.pdf HIGGS UCI Dataset: 21 low-level features AND 7 high-level derived features Train: 10M rows, Test: 500k rows
  • 32. H2O Deep Learning, @ArnoCandel Coming very soon: h2o-dev New UI: Flow New languages: python, Javascript 32
  • 33. H2O Deep Learning, @ArnoCandel h2o-dev Python Example 33
  • 34. H2O Deep Learning, @ArnoCandel Part 2: Hands-On Session 34 Web GUI Import Higgs data, split into train/test Train grid search Deep Learning model Continue training the best model ROC and Multi-Model Scoring R Studio Connect to running H2O Cluster from R Run ML algos on 3 different datasets More: Follow examples from http://learn.h2o.ai 
 (R scripts and data at http://data.h2o.ai)
  • 35. H2O Deep Learning, @ArnoCandel H2O Docker VM 35 http://h2o.ai/blog/2015/01/h2o-docker/ H2O will be at
 http://`boot2docker ip`:8996
  • 36. H2O Deep Learning, @ArnoCandel Import Higgs data 36 Enter
  • 37. H2O Deep Learning, @ArnoCandel Split Into Train/Test 37
  • 38. H2O Deep Learning, @ArnoCandel Train Grid Search DL Model 38 Enter Enter Enter Enter
  • 39. H2O Deep Learning, @ArnoCandel Continue Training Best Model 39 Scroll right Enter
  • 40. H2O Deep Learning, @ArnoCandel Inspect ROC, thresholds, etc. 40
  • 41. H2O Deep Learning, @ArnoCandel Multi-Model Scoring 41
  • 42. H2O Deep Learning, @ArnoCandel Control H2O from R Studio 42 http://learn.h2o.ai/ R scripts in github 1) Paste content of
 http://tiny.cc/h2o_next_ml into R Studio 2) Execute line by line with Ctrl-Enter to run ML algorithms on H2O Cluster via R 3) Check out the links below for more info http://h2o.gitbooks.io
  • 43. H2O Deep Learning, @ArnoCandel Snippets from R script 43 Install H2O R package & connect to H2O Server Run Deep Learning on MNIST
  • 44. H2O Deep Learning, @ArnoCandel 44 H2O GitBooks Also available: GBM & GLM GitBooks at http://h2o.gitbooks.io H2O World learn.h2o.ai R, EC2, Hadoop Deep Learning
  • 45. H2O Deep Learning, @ArnoCandel H2O Kaggle Starter R Scripts 45
  • 46. H2O Deep Learning, @ArnoCandel Re-Live H2O World! 46 http://h2o.ai/h2o-world/ http://learn.h2o.ai Watch the Videos Day 2 • Speakers from Academia & Industry • Trevor Hastie (ML) • John Chambers (S, R) • Josh Bloch (Java API) • Many use cases from customers • 3 Top Kaggle Contestants (Top 10) • 3 Panel discussions Day 1 • Hands-On Training • Supervised • Unsupervised • Advanced Topics • Markting Usecase • Product Demos • Hacker-Fest with 
 Cliff Click (CTO, Hotspot)
  • 47. H2O Deep Learning, @ArnoCandel You can participate! 47 - Images: Convolutional & Pooling Layers PUB-644 - Sequences: Recurrent Neural Networks PUB-1052 - Faster Training: GPGPU support PUB-1013 - Pre-Training: Stacked Auto-Encoders PUB-1014 - Ensembles PUB-1072 - Use H2O at Kaggle Challenges!
  • 48. H2O Deep Learning, @ArnoCandel Key Take-Aways H2O is an open source predictive analytics platform for data scientists and business analysts who need scalable and fast machine learning. ! H2O Deep Learning is ready to take your advanced analytics to the next level - Try it on your data! ! Join our Community and Meetups! https://github.com/h2oai h2ostream community forum www.h2o.ai @h2oai 48 Thank you!