Machine Learning — Hard
Key points
- Knowledge distillation transfers knowledge through soft probabilities
- The goal is to compress a model while maintaining performance
- Student model mimics the teacher model's behavior
- Soft labels are used instead of hard labels
- Efficient way to compress models without significant loss in performance
Ready to go further?
Related questions