Build AI Cloud with CloudRaft AI Platform
0 likes114 views
The document discusses the integration of AI into daily operations through AI clouds, which support the full AI lifecycle while utilizing cloud native technologies like Kubernetes for enhanced performance. It highlights trends in AI infrastructure, data sovereignty, and the challenges of building AI clouds, including hardware limitations and security concerns. Additionally, the presentation emphasizes the potential of AI clouds to transform enterprises and the importance of addressing related risks.
Build AI Cloud with CloudRaft AI Platform
- 1. Build AI Cloud using Kubernetes CLOUD NATIVE INDORE: TECH TALK Anjul Sahu, CEO, CloudRaft
- 2. About Me Founder & CEO, CloudRaft - an AI & Cloud Native Consulting Organizer, Cloud Native Indore More than 16 years in Industry building large scale systems. Previously worked for Telco, Banks, Product & Startups Passionate about new technology Anjul Sahu CEO, CloudRaft
- 3. 256 GH200 DGX Cluster 1 Exaflop, 144 TB GPU 2023
- 4. In this Presentation Overview What is AI Cloud? Current Trends in AI Infrastructure How Cloud Native helps in running AI Architecture of AI Cloud Cloud Native Projects for AI Challenges Q&A 01 02 03 04 05 06 07
- 5. An AI Cloud simplifies AI implementation for organizations by integrating it into daily operations. AI Clouds cover the AI lifecycle, from creating features and models to operating, monitoring, and sharing them throughout the organization. Platforms supporting the full AI lifecycle are known as AI platforms, and when available in scalable environments, they are termed AI Clouds.
- 6. On-prem , Hybrid or Cloud Support end-to-end lifecycle of AI Self-service Scalable Reliable GPUs & High Performance AI/ML Frameworks Full stack: IaaS, PaaS, SaaS Billing or Chargeback Features of AI Cloud
- 7. Current Trends in AI Infrastructure Data Sovereignty Requirements Enterprise data loss risk, AI Safety and new Govt policies to keep data local Specialized Cloud Eg: CoreWeave, Salad, RunPod, Nebius, Lambda labs etc Cloud Native and Kubernetes is an accelerator for AI 2x Data in every 18 months The demand for data to build better AI/ML models is increasing faster than Moore’s Law, doubling every 18 months GenAI: Bigger Models model size is increasing that means more powerful infrastructure is required 01 02 03 04 05
- 8. AI Runs on GPUs Accelerators AI = matrix multiplications which is massively parallelizable GPUs are great at parallel programming CPU < 32 cores/threads, GPUs> 4000 cores/threads CPU is 10x slower at least Impractical to train or even run any reasonable AI model outside ASICs
- 10. How Cloud Native helps in running AI Workload "Research teams can now take advantage of the frameworks we've built on top of Kubernetes, which make it easy to launch experiments, scale them by 10x or 50x, and take little effort to manage." — CHRISTOPHER BERNER, HEAD OF INFRASTRUCTURE FOR OPENAI
- 11. AI Cloud Reference Architecture
- 12. Cloud Native Projects for AI Distributed Training Model / LLM Observability Vector Databases Data Architectures Governance and Policy General Orchestration ML Serving CI/CD Delivery Workload Observability AutoML Ecosystem is evolving fast... Security
- 13. Challenges in Building AI Cloud Building an AI Cloud is a large investment GPU supply chain issues Skill issues High reliability required for long running distributed training jobs Unknown security threats and AI Risk in the fast evolving ecosystem Sustainability - Each H100 energy consumption is more than avg household Some of the hardware limitations becomes bottlenecks such as storage or the network Why we need AI Cloud? Data Privacy AI is making humans more productive AGI is possible Cost is still less as compared to hyperscalers It is a game changer for many enterprises
- 14. This talk is based on our recent work. And it was not possible without the ground breaking innovations done by Kubernetes, NVIDIA and CNCF foundation See our insights on AI cloudraft.io/blog
- 15. Q & A "Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks." -Stephen Hawking, Theoretical Physicist