INTERFACE by apidays 2023 - Open Source ML, Omar Sanseviero, Hugging Face
2 likes83 views
The document discusses the landscape of open-source machine learning models, particularly focusing on the Hugging Face Hub, which offers access to over 200,000 community-shared models and 45,000 datasets across various modalities. It outlines challenges in model inference, such as customizability and optimization, and highlights methods for users to fine-tune models for specific use cases using parameter-efficient techniques. Additionally, it emphasizes the importance of building demos for increased accessibility and reproducibility in ML applications.
INTERFACE by apidays 2023 - Open Source ML, Omar Sanseviero, Hugging Face
- 1. Open Source ML - from pretrained models to production Run State of the Art Open Source LLMs in Production
- 3. Models 1 What exists out there?
- 4. The Hugging Face Hub Models Spaces Access over 200k models shared by the community. Build MLApps and Demos to showcase how models work. Datasets Share, access and collaborate on over 45k datasets.
- 5. The Hugging Face Hub Models Spaces Access over 200k models shared by the community Build MLApps and Demos to showcase how models work. Datasets Share, access and collaborate on over 45k datasets. 99k-> 200k 19k->60k 16k->45k
- 6. The Model Hub ● Models across modalities (Computer Vision, NLP, Audio, multimodal, RL, tabular) ● Multiple libraries (PyTorch, Keras, fastai, SpaCy, NeMo, PaddlePaddle, Stanza, timm) ● 180+ supported languages ● Model cards for documentation ○ Metrics reporting ○ CO2 emissions ○ TensorBoard hosting ○ Interactive widgets
- 7. Inference 2 How to do inference of LLMs?
- 8. StarCoder LLaMA Falcon Recent popular models ● Code generation ● 15.5B parameters ● OpenRAILLicense ● 80+ languages ● 1 trillion tokens ● Large ecosystem ● 7B to 65B parameters ● Non-commercial ● 1-1.4 trillion tokens ● Best OS model ● 7B to 40B parameters ● Apache 2.0 ● Multilingual ● 1 trillion tokens
- 9. Challenges Evaluation Existing benchmarks don’t fully capture real world use cases (e.g. multi-turn). Customizability Users want models tuned to their own data or use cases while preserving privacy. Model size LLMs require lots of memory, might not fit into a single machine, require complex parallelism and communication. Optimization Due to model size, latency and throughput are often impacted leading to require optimized models.
- 10. Some things you can do Load in 4-bit or 8-bit mode (bitsandbytes, accelerate) Loading Distribute among GPUs (accelerate) Multi-GPU Use tools optimized for LLMs (text-generation-inference) Inference Libraries Set device_map="auto" or even ooad layers to CPU (slow) Falcon 40B with 45GB (8-bit) or 27GB (4-bit) of RAM Used by HF in production!
- 11. Text-generation-inference (TGI) Tensor Parallelism Token Streaming Metrics and monitoring TGI supports most popular LLMs, such as StarCoder and SantaCoder Falcon LLaMA, Galactica and OPT GPT-NeoX Quantization Optimizations Security
- 12. Some users HuggingChat OpenAssistant nat.dev
- 13. Training 3 How to adjust models to your own use cases?
- 14. Training Fine-tuning PEFT ● $$$ ● Lots and lots of data ● Lots of expertise ● $$ ● Much less data and compute ● $ ● Even less compute Recent popular models overview (Parameter Eicient Fine-Tuning) You can fine-tune Whisper or Falcon-7b in free Collab
- 15. Example: Whisper ● 1% of trainable params, 5x more batch size ● Fine-tune a 1.6B parameter model with less than 8GB GPU VRAM ● The resulting checkpoints were less than 1% the size of the original model Full-Tuning Results in OOM LoRA
- 16. Example: Stable Diffusion “dog” adapter “toy” adapter “toy” + “dog” adapter
- 17. QLoRA 4-bit Quantization 4-bit quantized pretrained LM RLHF Base model with multiple adapters Efficient Fine-tune 65B parameter model on a single 48GB GPU
- 18. Building demos 4 How to build and share my ML apps?
- 19. Why demos? ● Easily present to a wide audience ● Increase reproducibility of research ● Diverse users can identify and debug failure points
- 21. Gradio: typical usage import gradio app = gradio.Interface( classify_image, inputs=“image”, outputs=“label”) app.launch()
- 22. Turning point in usage of ML ML/software engineers anyone who can use a GUI/browser
- 23. CREDITS: This presentation template was created by Slidesgo, and includes icons by Flaticon, infographics & images by Freepik and illustrations by Storyset Thanks! omar@huggingface.co Omar Sanseviero @osanseviero CREDITS: This presentation template was created by Slidesgo, and includes icons by Flaticon, infographics & images by Freepik and illustrations by Storyset and Chunte Lee