Ridge regression

Supporting technique

Ridge regression is a similar technique to LASSO, but it uses a quadratic penalty term where LASSO uses a linear one. Generally speaking, ridge regression performs better when a small amount of regularization is required for a large number of predictor variables, while LASSO performs better when a large amount of regularization is required for a small number of predictor variables. Elastic net is another algorithm that combines the features of both.

Note that ridge regression, unlike LASSO, cannot be used for dimensionality reduction because the procedure never yields zero coefficients.

Tikhonov regularization Weight decay Linear regularization Tikhonov-Miller method
has functional building block
FBB_Classification FBB_Value prediction
has learning style
has relevance
mathematically similar to
typically supports
ALG_Least Squares Regression ALG_Logistic regression