What is the purpose of the temperature parameter in softmax during inference of language models?

Machine Learning Hard

Machine Learning — Hard

What is the purpose of the temperature parameter in softmax during inference of language models?

Key points

  • Temperature parameter adjusts the uncertainty in model predictions
  • Lower temperature leads to more confident, deterministic outputs
  • Higher temperature introduces more randomness and diversity
  • It affects the trade-off between generating novel versus coherent text

Ready to go further?

Related questions