# A Quick Overview of Regression Algorithms in Machine Learning

*The media shown in this article are not owned by Analytics Vidhya and is used at the Author’s discretion.*## Introduction

We basically train machines so as to include some kind of automation in it. In machine learning, we use various kinds of algorithms to allow machines to learn the relationships within the data provided and make predictions using them. So, the kind of model prediction where we need the predicted output is a continuous numerical value, it is called a regression problem.

Regression analysis convolves around simple algorithms, which are often used in finance, investing, and others, and establishes the relationship between a single dependent variable dependent on several independent ones. For example, predicting house price or salary of an employee, etc are the most common regression problems.

## Linear ML algorithms

**Linear Regression**

**Multiple linear regression.**This kind of model assumes that there is a linear relationship between the given feature and output, which is its

**limitation**.

**Ridge Regression-The L2 Norm**

Even though coefficient shrinkage happens here, they aren’t completely put down to zero. Hence, your final model will still include all of it.

**Lasso Regression -The L1 Norm**

Both lasso and ridge are regularisation methods

**Let us go through some examples : **

Suppose a data with years of experience and salary of different employees. Our aim is to create a model which **predicts the salary of the employee** based on the year of experience. Since it contains one independent and one dependent variable we can use simple linear regression for this problem.

**Non-Linear ML algorithms**

**Decision Tree Regression**

Let us take a case of house price prediction, given a set of 13 features and around 500 rows, here you need to predict the price for the house. Since here you have a considerable number of samples, you have to go for trees or other methods to predict values.

**Random Forest**

– Pick K random data points from the training set.

– Build a decision tree associated with these data points

– Choose the number of trees we need to build and repeat the above steps(provided as argument)

– For a new data point, make each of the trees predict values of the dependent variable for the input given.

This is similar to guessing the number of balls in a box. Let us assume we randomly note the prediction values given by many people, and then calculate the average to make a decision on the number of balls in the box. Random forest is a model that uses multiple decision trees, which we know, but since it has a lot of trees, it also requires a high time for training also computational power, which is still a drawback.

**K Nearest Neighbors(KNN model)**

The method to find the value can be given as an argument, of which the default value is “Minkowski” -a combination of “euclidean” and “manhattan” distances.

Predictions can be slow when the data is large and of poor quality. Since the prediction needs to take into account all the data points, the model will take up more space when training.

**Support Vector Machines(SVM)**

**multi-dimensional space**, when we have more than one variable to determine the output, then each of the points is no longer a point as in 2D, but are vectors. The most extreme kind of assigning values can be done using this method. You separate classes and give them values. The separation is by the concept of Max-Margin(a hyperplane). What you must note is that SVMs are not at all suitable for predicting values for large training sets. SVM

**fails**when data has more noise.

If training data is much larger than the number of features, KNN is better than SVM. SVM outperforms KNN when there are larger features and lesser training data.

Well, we have come to an end of this article, we have discussed the kinds of regression algorithms(theory) in brief. This is Surabhi, I am B.Tech Undergrad. Do check out my Linkedin profile and get connected. Hope you enjoyed reading this. Thank you.

*The media shown in this article are not owned by Analytics Vidhya and is used at the Author’s discretion.*

## One thought on "A Quick Overview of Regression Algorithms in Machine Learning"

## joe says: January 10, 2021 at 10:28 pm

The graphics do not render correctly.