CoreML
- 2. Who are we ? • Software Engineers at VentureDive • Graduated from NED University of Engineering and Technology • Doing iOS development for last 2 years
- 3. Machine Learning Machine learning is USING DATA to answer QUESTIONS
- 4. Steps for Machine Learning • Data collection and processing • Model Training and Testing • Inference / prediction
- 5. What is a Model • A function that take some inputs and give one or more outputs on the basis of the knowledge it gains from the data on which it was trained
- 6. About CoreML • Apple’s Machine learning framework • iOS 11 • Integrate trained machine learning model into your app • MacOS , iOS , tvOS and watchOS • Predictive Purpose
- 7. Applications • Real-Time image detection • Next word predictions • Entity Recognition • Music tagging • Semantic Analysis • Style Transfer • Face Detection
- 8. Examples
- 9. Apple Uses Machine Learning • Photos App (People Detection) • Keyboard (Next word prediction) • Smart Watches (Handwriting detection)
- 10. Let’s Dive Deep in CoreML
- 11. Architecture
- 12. Model Types
- 13. Pros • Runs locally • Privacy • No server costs • Hardware Optimized • Real time machine learning
- 14. Cons • Unsupervised and Reinforcement learning is not supported • Training of model can’t be done
- 15. • DEMO 1
- 16. Summary • CoreML makes it easy to integrate machine learning models into your app (Drag and drop and you get you get code interface ) • Hardware optimized • Compatible with popular format • CoreML has rich data type support
- 17. Possible I/O • Numeric (House price prediction) • Categorical • Arrays (video and gesture ) • Dictionaries (text based application) • Images(flower classier)
- 18. Obtaining Model • Pre-trained model (models with a .mlmodel file extension) • Pre-trained (open source models) + convert —> .mlmodel • Collect data + trained on open source + convert — > .mlmodel
- 19. Obtaining Model
- 20. Converting Trained Models to Core ML • Core ML Tools (including Keras, Caffe, scikit-learn, libsvm, and XGBoost) • TensorFlow converter • MXNet converter
- 21. Core ML Tools
- 22. Core ML Tools
- 23. Core ML Tools • Converters (set of converters one for each popular training) • Build your own converters using CoreML Bindings and Converter Library • CoreML Specification (Read and write api in cormel model format directly OR Write your own model)
- 24. DEMO 2 • Caffe: an open framework for deep learning developed by Berkeley Vision and Learning Center • Python 2.7. • import coremltools • deploy.prototxt – describes the structure of the neural network • oxford102.caffemodel – the trained data model in Caffe format. • class_labels.txt – contains a list of all the flowers that the model is able to recognize. • coreml_model = coremltools.converters.caffe.convert(('oxford102.caffemodel', 'deploy.prototxt'), image_input_names='data', class_labels=‘class_labels.txt') • coreml_model.save('Flowers.mlmodel')
- 25. Abstract Overview Convolutional Neural Network • Convolutional Neural Networks (ConvNets or CNNs) are a category of Neural Networks • Very effective in areas such as image classification • Each layer is not connected to every other neuron in the next layer , only part of the image is connected called receptive field • There are a few distinct types of Layers (e.g. CONV/FC/RELU(Activation)/POOL are by far the most popular), • Layers are basically series of operations (called convolutional blocks)
- 26. Abstract Overview Convolutional Neural Network
- 27. CONV
- 28. POOLING • Makes the representation smaller and more manageable • Spatially Downsamples , gives more relevant part
- 29. Activation and Regularization • Activation is helpful to train nonlinear function (ReLU) • Regularization help CNN to avoid overfitting and generalize classification(L1,L2,DropOut) • Gradient Descent for back propagation (local minima) • Dense layer: A linear operation in which every input is connected to every output by a weight (so there are n_inputs * n_outputs weights - which can be a lot!). Generally followed by a non-linear activation function
- 30. DEMO 3 • Train model in keras • Convert keras model into mlmodel • Epoch = No of times an entire data sets are fed to the network (To help neural n/w to get better) • Batch : every single time to get my neural network train
- 31. CNN
- 32. DEMO 4 • Convert tensor flow model into mlmodel • import tfcoreml (google support) • https://github.com/tf-coreml/tf-coreml • https://github.com/tf-coreml/tf-coreml/issues/91 • https://github.com/tf-coreml/tf-coreml/issues/80
- 33. Downloading and Compiling a Model on the User's Device • Downloading and compiling models on the user's device – rather than bundling them with your app – is beneficial for specific use cases • If your app has a variety of features supported by different models, downloading the models individually in the background could save on bandwidth and storage, by not forcing the user to download every model. Likewise, different locales or regions might use different Core ML models. • The model definition file (.mlmodel) must be on the device before it's compiled. • let compiledUrl = try MLModel.compileModel(at: modelUrl) • let model = try MLModel(contentsOf: compiledUrl)
- 34. Custom Layers • The field of neural networks is constantly evolving, with new architectures and layers rapidly being created to solve new problems • Models that use custom layers require these OS versions or later: iOS 11.2, • If you acquire a third-party model that uses custom layers, the Xcode UI for your model will look slightly different than it would for a model with no custom layers • Very familiar with the architecture of your neural network and the intended behavior of your custom layer • add_custom_layers=True, which tells the converter to detect custom layers • http://machinethink.net/blog/coreml-custom-layers/ •
- 35. Custom Layers
- 36. DEMO 5 • Simple and efficient tools for data mining and data analysis • In statistics, linear regression is a linear approach for modelling the relationship between a scalar dependent variable y and one or more explanatory variables (or independent variables) denoted X. • Price = 500 - 0.2*(Age of car) • coreml_model = sklearn.convert(lm) • coreml_model.save('linear_model.mlmodel')