L1 and L2 Regularization Guide: Lasso and Ridge Regression

L1 and L2 Regularization Guide: Lasso and Ridge Regression

This article is about Lasso Regression and Ridge Regression aka L1 and L2 regularization respectively, here we will learn and discuss L1 vs L2 Regularization Guide: Lasso and Ridge Regression.

The key difference between L1 and L2 regularization is the penalty term or how weights are used, L2 is the sum of the square of the weights, while L1 is just the absolute sum of the weights, using these techniques we can to avoid over-fitting.

L1 Regularization or Lasso Regression

In L1 Regularization or Lasso Regression, the cost function is changed by L1 loss function which used to minimize the error, that is the sum of the all the absolute(mod) differences between the actual value and the predicted value.

image.png Cost Function for Lasso Regression

L2 Regularization or Ridge Regression

In L2 Regularization or Ridge Regression, the cost function is changed by L2 loss function which used to minimize the error, that is sum of the all the squared differences between the actual value and the predicted value.

image.png Cost function for Ridge Regression

End Notes:

  1. Lasso Regression is useful for feature selection as regression is performed by removing the slopes whose value after model fitting is approaching to zero, meaning they are less important to the model.

  2. It is very important to choose right value of lambda, otherwise model can lead to under-fit.

Author: Satyampd