1.Does ZeRO propose solution for memory- or compute-bounded problem.pdf
0 likes8 views
The document poses a series of questions about zero's functionality in computing, particularly concerning memory and compute limitations, addressing memory fragmentation, and the efficiency of data parallelism. It also inquires about the role of offloading model states in deep learning and seeks clarification on model parallelism versus data or task-type parallelism. Each question challenges the reader to provide justifications and explanations based on technical concepts.
1 of 1
Download to read offline
More Related Content
More from almonardfans (20)
Recently uploaded (20)
1.Does ZeRO propose solution for memory- or compute-bounded problem.pdf
- 1. 1.Does ZeRO propose solution for memory- or compute-bounded problem? Justify your answer. 2. Explain in your own word how ZeRO mitigates memory fragmentation? 3. Why Data Parallelism is communication efficient? Justify your answer. 4. What is the purpose of offloading model states in deep learning (DL) models from GPU to CPU? 5. s Model Parallelism (MP) data- or task-type parallelism? Justify your answer with two arguments.