What is a Gaussian Process (GP) and what advantage does it provide over point estimate models?

Question

Machine Learning — Hard

What is a Gaussian Process (GP) and what advantage does it provide over point estimate models?

Accepted Answer

A Gaussian Process defines a prior distribution over functions, ensuring that any finite collection of function values is jointly Gaussian. It provides full posterior distributions over predictions, quantifying uncertainty at each point, unlike point estimate models.