Serving models from AWS Lambda
0 likes170 views
This document discusses serving machine learning models from AWS Lambda. It describes using Python and ML libraries like TensorFlow and Keras to build models, convert them to a format compatible with Lambda, and deploy them for low-cost inference. The document also addresses Lambda limits on RAM, package size, and optimization techniques like stripping unnecessary libraries to reduce model sizes and speeds up loading times.
Serving models from AWS Lambda
- 1. Serving models from AWS Lambda Alexey Grigorev 01.08.2019
- 2. AWS Lambda
- 4. How about python&ML stuff? pip install -r requirements.txt -t build cp index.py build cd build && zip -r ../build.zip & cd .. specify output folder this is sent to aws
- 5. How about python&ML stuff? aws lambda update-function-code --function-name ${FUNCTION_NAME} --s3-bucket ${S3_BUCKET} --s3-key ${ZIP_FILE}
- 7. Problem: AWS Lambda Limits ● Limits: ○ 128MB - 3GB of RAM ○ 512MB storage in /tmp ○ Package size 50MB zipped / 250MB unzipped https://docs.aws.amazon.com/lambda/latest/dg/limits.html
- 8. AWS Lambda Limits $ du -sh * | sort -k 2 28K bin 56M build.zip 392K imagehash 32K ImageHash-4.0.dist-info 80M numpy 88K numpy-1.17.0.dist-info 6.8M PIL 48K Pillow-5.4.1.dist-info 32K __pycache__ 28K PyWavelets-1.0.3.dist-info 8.4M pywt 88M scipy 148K scipy-1.3.0.dist-info 28K six-1.12.0.dist-info 32K six.py More than 50MB
- 9. Solution: rm things 32M May 16 10:50 index-index-hasher.zip
- 10. It scalez!
- 12. What if we could use it for serving image models?
- 13. pip install numpy pillow tensorflow==1.7.0 -b build size: 354M I’ll just deploy TF to lambda
- 14. How to cut down the size of TF? !!!
- 15. Yay! original size 354M striping SOs in tensorflow striping SOs in numpy stripped size 177M compressed size 43M
- 16. Keras models in Lambda ● Convert Keras to TF ● Save the model to s3 ● During lambda startup ○ Download the model from s3 ○ Put to /tmp ○ Load the model ○ Don’t delete from /tmp - new lambdas startup will be faster
- 17. Keras → TF Load Keras model Convert to TF and save
- 20. Serve!
- 22. Demo
- 25. More RAM ● 1152MB: 50 sec load, 4 sec inference ● 1344MB: 43 sec load, 3.2 sec inference ● 1536MB: 30 sec load, 3 sec inference
- 26. Price ● 1024MB: ○ (4sec) 40 * 0.000001667 ○ = 0.00006668 USD ○ 1 mln images: 66 USD ● 1344MB: ○ 1 mln images: 66 USD ● 1536MB: ○ 1 mln images: 75 USD https://aws.amazon.com/lambda/pricing/