Shift Remote AI: Behind the Scenes Development in an AI Company - Matija Ilijas (Microblink)
0 likes96 views
The document outlines the behind-the-scenes development processes of an AI company, detailing the complexities and resource requirements of AI model development, data management, and deployment. It emphasizes the use of proprietary systems for data annotation and machine learning, as well as the integration of various technologies like Docker, Kubernetes, and C++ for optimized AI inference and cross-platform deployment. Development processes are tailored for efficiency and privacy, catering to both internal clients and end users.
Shift Remote AI: Behind the Scenes Development in an AI Company - Matija Ilijas (Microblink)
- 1. Behind the scenes development in an AI company Matija Ilijaš
- 6. Behind the scenes development • Development that is not necessarily visible to the outside • Can drastically vary in size between seemingly similar companies • Requires more resources and more expertise to scale • Gives more room for engineers to push the limits of a complex system
- 8. Real-time data extraction powered by AI
- 12. +100 million end users
- 18. AI models
- 21. Garbage in, garbage out DATA ANNOTATION ACCURACY RANDOM 50% CROWSOURCED 95% THIRD PARTY SERVICE 98% PROPRIETARY SYSTEM 99,9%
- 22. Data management system • Used by our team of domain experts and data annotators • Enables maximum annotation efficiency with its custom UI • Runs several quality assurance mechanisms to ensure data quality • Performs preannotation with our currently best AI models • Custom built annotation workflow for each product
- 23. Development • Software and UI developers • DevOps, Data and QA engineers • Internal clients: domain experts and data annotators • .NET Core microservices using Docker and Kubernetes • MongoDB, SQL Server on Linux and ElasticSearch • Figma, ReactJS and enterprise-class UI library
- 26. Machine learning engine • Enables our AI development process • Optimizes the data management process with AI model serving • Repository of proprietary neural network layers, architectures and training procedures • Production-scale model training and evaluation with deployment
- 27. Development • Software and DevOps engineers • Experienced with software architecture and microservice development • Python with emphasis on internal Tensorflow integration • Microservices using Docker and Kubernetes • Pipeline definition and execution with Argo • Metadata storage with MongoDB and PostreSQL
- 30. AI inference engine • Optimizing our needs while open source engines are optimizing everyone’s needs • Implemented in C++, completely cross-platform • Maximum control over privacy of the models • Multi-model support optimized in both speed and memory footprint • Our playground for innovation and new products
- 31. Development • C++ developers • Highly optimized neural network inference • Latest C++ standards • Conan C/C++ Package Manager • Build system with Cmake and Jenkins • WebAssembly
- 32. Putting it all together
- 33. Complex AI system • It’s design determines which AI models are needed and how they will interact to solve the problem • Additional processing units are implemented by product developers • Allows control over the parts of the system where crucial or sensitive decisions are made • Implemented and exported as a static C++ library
- 34. Development • C++ developers • Experience in Computer vision • Product development with AI models • Compile-time and meta programming • Latest C++ standard packaged with Conan
- 35. Deployment on all platforms
- 36. Mobile deployment • Custom Camera management • SDKs developed for iOS and Android • Cross-platform auto generated using proprietary code generation tools • Emphasis on simple UX and use of platform specific features • Always exploring new features on platforms
- 37. Web deployment • Custom Web API • AI system static C++ library wrapped in a Docker container • Software as a Service (SaaS) • In-Browser scanning with our AI inference engine powered by WebAssembly