08-15-2024 - AI Camp Meetup - Human Pose Estimation in Real-Time Utilizing Edge AI Accelerated Hardware
0 likes366 views
The document discusses human pose estimation using edge AI accelerated hardware, highlighting its application in capturing human movements through a system that identifies key joint positions. It introduces Milvus, an open-source vector database optimized for managing unstructured data, enabling efficient search capabilities and supporting various computational needs. Additionally, the document provides resources and next steps for engaging with the unstructured data community and using AI technologies.
08-15-2024 - AI Camp Meetup - Human Pose Estimation in Real-Time Utilizing Edge AI Accelerated Hardware
- 1. August 15, 2024 Human Pose Estimation in Real-Time Utilizing Edge AI Accelerated Hardware
- 3. Tim Spann Principal Developer Advocate, Zilliz tim.spann@zilliz.com https://www.linkedin.com/in/timothyspann/ https://x.com/paasdev https://github.com/tspannhw https://github.com/milvus-io/milvus Speaker https://proceedings.neurips.cc/pape r/2021/hash/299dc35e747eb77177 d9cea10a802da2-Abstract.html
- 4. SPANN: Highly-efficient Billion-scale Approximate Nearest Neighborhood Search Coincidence? I think not. https://proceedings.neurips.cc/paper/2021/hash/299dc35e747eb77177d9cea10a802da2-Abstract.html
- 6. Agenda Introduction Unstructured data, vector databases, traditional databases, similarity search, Milvus, Vector Database 01 Human Pose Estimation Capture, AI 02 App and Demo Running edge AI application connected to cloud 03
- 7. 7 | © Copyright Zilliz 7 01 This again?
- 8. 8 | © Copyright Zilliz 8 What is Milvus ideal for? • Advanced filtering • Hybrid search • Durability and backups • Replications/High Availability • Sharding • Aggregations • Lifecycle management • Multi-tenancy • High query load • High insertion/deletion • Full precision/recall • Accelerator support GPU, FPGA • Billion-scale storage Purpose-built to store, index and query vector embeddings from unstructured data at scale.
- 9. 9 | © Copyright Zilliz 9 Weʼve built technologies for various types of use cases Compute Types Designed for various compute powers, such as AVX512, Neon for SIMD, quantization cache-aware optimization and GPU Leverage strengths of each hardware type, ensuring high-speed processing and cost-effective scalability for different application needs Search Types Support multiple types such as top-K ANN, Range ANN, sparse & dense, multi-vector, grouping, and metadata filtering Enable query flexibility and accuracy, allowing developers to tailor their information retrieval needs Multi-tenancy Enable multi-tenancy through collection and partition management Allow for efficient resource utilization and customizable data segregation, ensuring secure and isolated data handling for each tenant Index Types Offer a wide range of 15 indexes support, including popular ones like HNSW, PQ, Binary, Sparse, DiskANN and GPU index Empower developers with tailored search optimizations, catering to performance, accuracy and cost needs
- 10. 10 Vector Database : making sense of unstructured data 2024 A vector database stores embedding vectors and allows for semantic retrieval of various types of unstructured data.
- 11. 11 Milvus, OSS vector database since 2019 Originally created by Zilliz, hosted by the Linux Foundation 2024 28K 10000 GitHub Stars Enterprise users 70M Downloads 270 Contributors
- 12. 12 | © Copyright Zilliz 12 02 Human Pose Estimation
- 13. 13 | © Copyright Zilliz 13 What is it? https://paperswithcode.com/task/pose-estimation 1431 papers with code Human Pose Estimation is a computer vision technique that locates and estimates things like eyes, joints in motion. It looks pretty cool and has some interesting applications for medical purposes and robotics. For me, it was one of the cool examples that runs on the AI Kit.
- 14. 14 | © Copyright Zilliz 14 Pose Estimation by Hailo 8L Each person is identified and represented by 17 keypoints Examples nose, eyes, ears, shoulders, elbows, wrists, hips, knees, and ankles. We are tracking eyes. https://github.com/hailo-ai/hailo_model_zoo/blob/master/docs/public_models/HAILO8/HAILO8_pose_estimation.rst https://github.com/tensorboy/centerpose https://softwaremill.com/human-pose-estimation-2023-guide/
- 15. 15 | © Copyright Zilliz 15 Pose Estimation on Hailo 8L Pose Estimation COCO Yolov8s_pose Hailo-8L https://github.com/ultralytics/ultralytics
- 16. 16 | © Copyright Zilliz 16 03 Just Show Me
- 17. 17 | © Copyright Zilliz 17 Show Me The Source Code https://github.com/tspannhw/AIM-RPIAIKit-PoseEstimation https://github.com/hailo-ai/hailo-rpi5-examples lefteye = (f"x: {left_eye_x:.2f} y: {left_eye_y:.2f}") righteye = (f"x: {right_eye_x:.2f} y: {right_eye_y:.2f}") try: imageembedding = extractor(strfilename) milvus_client.insert( COLLECTION_NAME, {"vector": imageembedding, "lefteye": lefteye, "righteye": righteye, "label": label, "confidence": confidence}) except Exception as e: print("An error:", e)
- 18. 18 | © Copyright Zilliz 18 • Vision to Images and Videos • Audio from Cameras and Microphones • Raw Text • Edge Neural Networks and Gen AI • Unstructured Data Processing and Vector DB Edge Unstructured Data
- 19. 19 | © Copyright Zilliz 19
- 20. 20 | © Copyright Zilliz 20
- 21. 21 | © Copyright Zilliz 21 04 Next Steps
- 22. 22 | © Copyright Zilliz 22
- 23. 23 | © Copyright Zilliz 23 Vector Database Resources Give Milvus a Star! Chat with me on Discord! https://github.com/milvus-io/milvus
- 24. 24 | © Copyright Zilliz 24 ● Closer is better ● Empowering AI Robots ● Vector Search Everywhere ● Keep your data and computation close Takeaway:
- 25. 25 | © Copyright Zilliz 25 | © Copyright Zilliz 25 Q&A
- 26. 26 | © Copyright Zilliz 26 ● Edge AI ● Edge Hardware ● Milvus ● Vector Databases Questions
- 27. 27 | © Copyright Zilliz 27 | © Copyright Zilliz 27 RESOURCES
- 28. 28 Unstructured Data Meetup https://www.meetup.com/unstructured-data-meetup-new-york/ This meetup is for people working in unstructured data. Speakers will come present about related topics such as vector databases, LLMs, and managing data at scale. The intended audience of this group includes roles like machine learning engineers, data scientists, data engineers, software engineers, and PMs. This meetup was formerly Milvus Meetup, and is sponsored by Zilliz maintainers of Milvus.
- 29. https://medium.com/@tspann/unstructured-data-processing-with-a-raspberry-pi-ai-kit-c959dd7fff47 Raspberry Pi AI Kit Hailo Edge AI
- 30. https://medium.com/@tspann/from-the-edge-to-the-cloud-and-back-again-01095e95a783 Raspberry Pi AI Kit Hailo Edge AI Pose Estimation
- 32. Extracting Value from Unstructured Data Example • A company has 100,000s+ pages of proprietary documentation to enable their staff to service customers. Problem • Searching can be slow, inefficient, or lack context. Solution • Create internal chatbot with ChatGPT and a vector database enriched with company documentation to provide direction and support to employees and customers. https://osschat.io/chat
- 33. We provide deployment flexibility for different operational, security and compliance requirements BRING YOUR OWN CLOUD Zilliz BYOC Enterprise-ready Milvus for Private VPCs Deploy in your virtual private cloud Zilliz Cloud Milvus Re-engineered for the Cloud Available on the leading public clouds FULLY MANAGED SERVICE Coming Soon! Coming Soon! Milvus Most widely-adopted open source vector database Self hosted on any machine with community support SELF MANAGED SOFTWARE Local Docker K8s
- 34. 34 | © Copyright Zilliz 34 Well-connected in LLM infrastructure to enable RAG use cases Framework Hardware Infrastructure Embedding Models LLMs Software Infrastructure Vector Database
- 35. 3 5 Multi-cloud: Zilliz Cloud is built atop of OSS Milvus AWS, GCP, Azure 2024
- 36. 36 | © Copyright Zilliz 36 T H A N K Y O U