What is the purpose of the Fisher Information Matrix in machine learning?

Question

Machine Learning — Hard

What is the purpose of the Fisher Information Matrix in machine learning?

Accepted Answer

The Fisher Information Matrix in machine learning is a matrix of second-order statistics of the log-likelihood that measures the curvature of the parameter space. It is used in natural gradient descent for more efficient optimization, Elastic Weight Consolidation for continual learning, and as an approximation to the Hessian.