Introduction
Regression analysis is commonly used for modeling the relationship between a single dependent variable Y and one or more predictors. When we have one predictor, we call this "simple" linear regression:
E[Y] = β0 + β1X
That is, the expected value of Y is a straight-line function of X. The betas are selected by choosing the line that minimizing the squared distance between each Y value and the line of best fit. The betas are chose such that they minimize this expression:
∑i (yi – (β0 + β1X))2
An instructive graphic I found on the Internet
Source: http://www.unc.edu/~nielsen/soci709/m1/m1005.gif
When we have more than one predictor, we call it multiple linear regression:
Y = β0 + β1X1+ β2X2+ β2X3+… + βkXk
The fitted values (i.e., the predicted values) are defined as those values of Y that are generated if we plug our X values into our fitted model.
The residuals are the fitted values minus the actual observed values of Y.
Here is an example of a linear regression with two predictors and one outcome:
Instead of the "line of best fit," there is a "plane of best fit."
Source: James et al. Introduction to Statistical Learning (Springer 2013)
There are four assumptions associated with a linear regression model:
- Linearity: The relationship between X and the mean of Y is linear.
- hom*oscedasticity: The variance of residual is the same for any value of X.
- Independence: Observations are independent of each other.
- Normality: For any fixed value of X, Y is normally distributed.
We will review how to assess these assumptions later in the module.
Let's start with simple regression. In R, models are typically fitted by calling a model-fitting function, in our case lm(), with a "formula" object describing the model and a "data.frame" object containing the variables used in the formula. A typical call may look like
> myfunction <- lm(formula, data, …)
and it will return a fitted model object, here stored as myfunction. This fitted model can then be subsequently printed, summarized, or visualized; moreover, the fitted values and residuals can be extracted, and we can make predictions on new data (values of X) computed using functions such as summary(), residuals(),predict(), etc. Next, we will look at how to fit a simple linear regression.
return to top | previous page | next page
FAQs
Definition. Simple linear regression aims to find a linear relationship to describe the correlation between an independent and possibly dependent variable. The regression line can be used to predict or estimate missing values, this is known as interpolation.
How do you explain linear regression in simple terms? ›
Linear regression is a data analysis technique that predicts the value of unknown data by using another related and known data value. It mathematically models the unknown or dependent variable and the known or independent variable as a linear equation.
What is the difference between simple and multiple linear regression? ›
Simple linear regression has only one x and one y variable. Multiple linear regression has one y and two or more x variables.
What is a simple linear regression real life example? ›
A simple linear regression real life example could mean you finding a relationship between the revenue and temperature, with a sample size for revenue as the dependent variable. In case of multiple variable regression, you can find the relationship between temperature, pricing and number of workers to the revenue.
Is simple linear regression the same as correlation? ›
A correlation analysis provides information on the strength and direction of the linear relationship between two variables, while a simple linear regression analysis estimates parameters in a linear equation that can be used to predict values of one variable based on the other.
How to explain regression in layman terms? ›
Regression — as fancy as it sounds can be thought of as “relationship” between any two things. For example, imagine you stay on the ground and the temperature is 70°F. You start climbing a hill and as you climb, you realize that you are feeling colder and the temperature is dropping.
How to interpret simple linear regression? ›
It is interpreted as the proportion of observed y variation that can be explained by the simple linear regression model (attributed to an approximate linear relationship between y and x). The higher the value of r2, the more successful is the simple linear regression model in explaining y variation.
What is the difference between simple linear regression and linear regression? ›
Linear regression shows the linear relationship between the independent(predictor) variable i.e. X-axis and the dependent (output) variable i.e. Y-axis, called linear regression. If there is a single input variable X (independent variable), such linear regression is simple linear regression.
Should I use linear regression or multiple regression? ›
MLR is a statistical tool used to predict the outcome of a variable based on two or more explanatory variables. If just one variable affects the dependent variable, a simple linear regression model is sufficient. If, on the other hand, more than one thing affects that variable, MLR is needed.
When to use linear regression? ›
You can use linear regression when you want to predict a continuous dependent variable from a scale of values. Use logistic regression when you expect a binary outcome (for example, yes or no). Here are examples of linear regression: Predicting the height of an adult based on the mother's and father's height.
Simple linear regression aims to find a linear relationship to describe the correlation between an independent and possibly dependent variable. The regression line can be used to predict or estimate missing values, this is known as interpolation.
What is a good example of linear regression? ›
We could use the equation to predict weight if we knew an individual's height. In this example, if an individual was 70 inches tall, we would predict his weight to be: Weight = 80 + 2 x (70) = 220 lbs. In this simple linear regression, we are examining the impact of one independent variable on the outcome.
What is the situation where you would use simple linear regression? ›
Simple linear regression is used to estimate the relationship between two quantitative variables. You can use simple linear regression when you want to know: How strong the relationship is between two variables (e.g., the relationship between rainfall and soil erosion).
What is the main problem with using single regression line? ›
Answer: The main problem with using single regression line is it is limited to Single/Linear Relationships. linear regression only models relationships between dependent and independent variables that are linear. It assumes there is a straight-line relationship between them which is incorrect sometimes.
What is another name for simple linear regression? ›
Linear regression is the most common form of this technique. Also called simple regression or ordinary least squares (OLS), linear regression establishes the linear relationship between two variables.
Why use regression instead of correlation? ›
Correlation is almost always used when you measure both variables. It rarely is appropriate when one variable is something you experimentally manipulate. Linear regression is usually used when X is a variably you manipulate (time, concentration, etc.)
