What is the effect of applying one-hot encoding to a high-cardinality categorical feature in a linear model?

Question

Data Science — Hard

What is the effect of applying one-hot encoding to a high-cardinality categorical feature in a linear model?

Accepted Answer

High-cardinality features result in a massive, sparse feature matrix when one-hot encoded. This significantly increases the number of parameters, leading to high variance and overfitting, especially if many categories have low frequency. Furthermore, it creates a 'dummy variable trap' (perfect multicollinearity) unless one category is dropped, which is a common oversight.