AI Fundamentals — Hard
Key points
- Model parallelism splits the model itself, data parallelism replicates the model
- Model parallelism is used for large models, data parallelism for processing different data batches
- Model parallelism is about splitting the model, data parallelism is about replicating it
Ready to go further?
Related questions