What is the primary risk of using Target Encoding on a categorical feature with high cardinality?

Question

Data Science — Hard

What is the primary risk of using Target Encoding on a categorical feature with high cardinality?

Accepted Answer

Target Encoding replaces categories with the mean of the target variable. With high cardinality, categories with few samples become highly correlated with the target, allowing the model to 'memorize' the target values rather than learning generalizable patterns. This is a form of target leakage that leads to extreme overfitting, especially if not mitigated by techniques like smoothing or cross-validation.