Unit 3 : Regression

 

Unit 3: Regression

 

Linear Regression: Prediction using Linear Regression, Gradient Descent, Linear Regression with one Variable, Linear Regression with Multiple Variables, Polynomial Regression, Feature Scaling/Selection. Logistic Regression: Classification using Logistic Regression, Logistic Regression vs. Linear Regression, Logistic Regression with one Variable and with Multiple Variables.

 

Linear Regression:

What is Regression?

            Regression is the mathematical method that used to find the value of an unknown variable with the help of another variable. For example X+5=Y , where Y is in between 10  then the value of X be 5

X	5
Y	10

 

What is Linear Regression?

            Linear Regression is also the statistical method that used to find the value of an unknown variable with the help of another variable in a linear way. For example X+5=Y , where Y is in between 10  then the value of X be 5 – 10. Finding relationship between the known and unknown variable and find the value for given sets as well as predicts value for next linear set.

X= 5 => 5+5=10

X=6 => 6+5=11

X=7 => 7+5=12

X=8 => 8+5=13

X=9 => 9+5 = 14

X=10 => 10+5=15

X	5	6	7	8	9	10	?
Y	10	11	12	13	14	15	20

 

Regression is used in all the fields like finance, medicine, weather forecasting

After finding values plot them in a graph and draw a linear line and that will show the gradual rise of prediction’s direction. This line is called as line of regression, the points which plotted on the graph is called as data points, X is dependent variable where Y is independent variable.

Equation of Linear Regression

                                                Y=m*X+b, m is the slope

 

Prediction using Linear Regression, Gradient Descent

How to make prediction using Linear Regression?

Step:1: Collect all the required data

Step:2: Select and fit regression model for the data

Step:3: Verify the model is suitable for the data well?

Step:4: Use the selected equation to find new predictions

 

 

Gradient Decent

            Gradient decent gives maximum minimum – optimum value of m and c in linear regression.

Steps Required in Gradient Descent Algorithm 

• Step 1 we have to initialize the parameters of the model randomly 
• Step 2 Compute the gradient of the cost function of all parameter. w.r.t each parameters partial differentiation will be generated. 
• Step 3 Update the parameters of the model by taking steps in the reverse direction of the model. Here we choose a hyperparameter learning rate which is denoted by alpha. It helps in deciding the step size of the gradient. 
• Step 4 Repeat steps 2 and 3 iteratively to get the best parameter for the defined model 

 

i ← 0

max_iter ← 1000

x, y ← initialize randomly

 

while i < max_iter do

    i ← i + 1

    x_i+1 ← x_i − η ∇x_i

    y_i+1 ← y_i − η ∇y_i

end

 

 

where x-weights and y is bias.

For any programming language to implement gradient decent we need the following four functions

function gradient_descent(X, y, learning_rate, num_iterations):

    Initialize parameters  = θ

    for iter in range(num_iterations):

        predictions = compute_predictions(X, θ)

        gradient = compute_gradient(X, y, predictions)

        update_parameters(θ, gradient, learning_rate)

    return θ

function compute_predictions(X, θ):

    return X*θ

function compute_gradient(X, y, predictions):

    error = predictions - y

    gradient = Xᵀ * error / m

    return gradient

function update_parameters(θ, gradient, learning_rate):

    θ = θ - learning_rate ⨉ gradient

 

1.  gradient_descent – In the gradient descent function we will make the prediction on a dataset and compute the difference between the predicted and actual target value and accordingly we will update the parameter and hence it will return the updated parameter.
2. compute_predictions – In this function, we will compute the prediction using the parameters at each iteration.
3. compute_gradient – In this function we will compute the error which is the difference between the actual and predicted target value and then compute the gradient using this error and training data.
4. update_parameters – In this separate function we will update the parameter using learning rate and gradient that we got from the compute_gradient function. 

 

Linear Regression with one Variable

            Linear Regression with one variable can also be called as simple Linear regression or univariate linear regression. Predicting one output value from one input value. A Linear regression line can be formulate by the equation Y=a+bx

Linear Regression with multiple Variable

            Multi Variable Regression is the algorithm of predicting one output value based on more than one input values for example Predicting CO2 emission based on engine size and no.of cylinders in a car.

 

 

Polynomial Regression,

            Polynomial Regression is a kind of machine learning algorithm used for making predictions. It is a type of linear regression where, the independent variable x and dependent variable y is modeled as n^th degree polynomial. It does not require linear relationship between independent and dependent variables in the dataset. It does not depends on variable x it is dependent the coefficient of x that is b0,b1,b2….bn(which is arranged in a linear order)

 

 

Feature ,Scaling, Selection of Regression

Is the process of identifying

 

Important Questions

1. If r=1 , the angle between the two regression line is ________. [0 / 90/ 270/ 360]

2. Rewrite the order in ascending manner - Step:A: Collect all the required data

Step:B: Select and fit regression model for the data

Step:C: Verify the model is suitable for the data well?

Step:D: Use the selected equation to find new predictions

[ABCD/ CDAB/ DCBA/ ABDA]

3. --------------------is the method for understanding the relationship between independent variable and dependent variable. [Regression/ Regularization/ Synchronization/ Normalization]

4. Rewrite the steps of gradient decent  in ascending order - Step 1 we have to initialize the parameters of the model randomly 

Step 2 Compute the gradient of the cost function of all parameter. w.r.t each parameters partial differentiation will be generated. 

Step 3 Update the parameters of the model by taking steps in the reverse direction of the model. Here we choose a hyperparameter learning rate which is denoted by alpha. It helps in deciding the step size of the gradient. 

Step 4 Repeat steps 2 and 3 iteratively to get the best parameter for the defined model 

[1234/ 4321/ 1243/ 2143]

5. Linear Regression is the Supervised Learning System - say true or false 

6. Variance is the counterpart of Bias say True or False

7. A machine learning model can neither learn the relationship between variables in the testing data nor predict or classify a new data point is called -----------[datafitting/ goodfitting/ overfitting/ underfitting]

8. The process of plotting a series of data points and drawing the best fit line to understand the

relationship between the variables is called -[datafitting/ goodfitting/ overfitting/ underfitting]

9. ---------------------- is used to convert overfitting to good fitting. [Perception/ classification/ regression/ regularization]

10. ------------------- helps to reduce the overfitting in the model as well as feature selection.

[linear regression/ ridge regression/lasso regression/ logistic regression]

2 marks

1. any two difference between Polynomial regression and Linear regression.

2. features of polynomial regression.

3. significant features of scaling and selection regression

4. Linear Regression

5. Logistic Regression

6. calculate L1 and L2 regularization

7. calculate bias and variance

15 marks

1. Gradient Descent

2. Logistic Regression with one variable and multiple variables

3.Selection of regression.

4. narrate all the advantages of Regularization over Regression.

5. Produce an application of Regularization for both linear and logistic model study