Recreating "The Clock" with Machine Learning and Web Scraping
0 likes244 views
This document discusses machine learning development lifecycles and tools. It notes that ML development differs from traditional development due to large datasets and computational needs. Training models can be very expensive requiring GPUs, TPUs, and significant computing resources. Kubernetes has become popular for deploying ML workloads but adds complexity with additional components required for networking, load balancing, and managing state. The document recommends starting simply by running Kubernetes locally before exploring more advanced ML tools that integrate with Kubernetes like Kubeflow, TensorRT inference server, and Pachyderm for data versioning. It stresses the importance of observability tools like metrics, logs, and traces for understanding errors in complex distributed ML systems.
Recreating "The Clock" with Machine Learning and Web Scraping
- 1. Recreating “The Clock” with Machine Learning and Web Scraping Kirk Kaiser @burningion
- 3. Dedicated hardware for inference Availability of large, labeled datasets Original gains with CNN to detect objects in images There is (almost) nothing new in universal turing machines. Why is machine learning suddenly a thing?
- 5. OpenAI & Dota 2
- 7. Build, clean, and label a dataset Build a model Deploy and Evaluate How does it work? ???
- 8. How to build a deep learning model model build a deep learning model
- 9. Actual model used for a project:
- 10. Build, clean, and label a dataset Build a model Deploy and Evaluate How does it work? Real world data is filled with garbage and messy Models tend to be brittle and opaque, difficult to debug How do you measure ‘performance’?
- 11. ML Development Lifecycle is Different from Traditional Development
- 12. Suddenly the machine you’re running on matters. (GPUs and TPUs necessary for training quickly on massive datasets.)
- 13. Driver updates can speed up your code 300% (?!)
- 14. Training cost to reproduce GPT-2 text generation model (via OpenGPT-2) from scratch ~$50,000 on GCP* * https://blog.usejournal.com/opengpt-2-we-replicated- gpt-2-because-you-can-too-45e34e6d36dc Training Computational Needs Can Be Staggering Computational costs for running AlphaGo Zero estimated to cost ~$35 million* * https://www.yuzeh.com/data/agz-cost.html
- 15. Cell phones are all adding acceleration to native chips Bandwidth and latency limitations on cameras forces inference computation to edge Inference Gets Pushed Out to the Edge
- 16. Gather, clean, and label data Once a model is deployed, can then be used to bootstrap better data Datasets Become Integral Part of Your Code
- 20. And Containers Usually Means Orchestration
- 21. Moving from managing and deploying to individual servers to pushing code to a “sea of infinite computation”
- 22. But there is a tradeoff...
- 23. – etcd, default database to manage kubernetes state, recommends 5 (!) server instances to durability – Growing list of subpackages to control networking layers, load balancing (Istio, Envoy) – Completely rethink development, testing, deployment as kubernetes grows beyond single dev machine Kubernetes comes with extra complexity
- 25. That complexity is trade off in move from building software as a service to software as a utility.
- 26. More Pieces to Manage and Deploy
- 27. Kubeflow ML toolkit for Kubernetes from Google TensorRT Inference Server Custom Inference server with Optimizations for NVIDIA Hardware Pachyderm Version control for data, and data pipelines Kubernetes Native ML Tools
- 28. Kubeflow supports plenty of options for your ML workflow
- 29. That’s a lot of moving pieces!
- 30. How do I get started?
- 31. Simplify first.
- 32. Run Kubernetes locally, with GPU acceleration.
- 33. @sterlingcrispin
- 34. +
- 36. PyTorch & FFMPEG w/ Hardware Acceleration Dockerfile
- 38. Observability in systems become a necessity to understand what’s happening in software with so many moving pieces.
- 40. See complete units of work, as they pass through your entire system, especially useful in Pipelines with multiple steps. Add tags to be able to see specific customers, organizations, and their direct experience with your systems. Compress Distributed System Complexity with Traces
- 41. Individual Trace
- 42. See bottlenecks in CPU, disk space, memory usage. For GPU / TPU, see hardware level metrics Correlate with logs and traces to isolate errors to software or hardware level issues. See History of Machine State with Metrics
- 45. Ingested logs show the history of work on each individual system component. Correlate with traces and metrics to isolate errors to library, software dependency level See Auditable Trail of Side Effects with Logs
- 47. – Either of these advances take in isolation adds to a platform – Taken together, we have a chance to rethink the way software behaves. (Images as code and APIs.) Object detection alone opens up new platforms
- 48. Bin-e, a self sorting recycling bin
- 49. Dab & T-Pose controlled lights
- 50. Jetson Nano $99 GPU Accelerated Machine
- 52. Thank you!
- 53. Datadog is hiring! Code / Blog post: https://dtdg.co/the-clock