r² and R², are they the same? (2024)

In some industries, people deal with test results in which r² or R² is something they will pay special attention. For example, a backend engineer feedback to you that one batch of product failed a particular test, and he suspects something was wrong with front end process. Most likely you will zoom in to a bunch of data and try to establish if there is linkage between the front end data, say Para_F and back end failure in Para_E. e.g. when Para_F increases in value, Para_E also increases, the significance is measured/indicated by r² .

Definition of Person Correlation Coefficient: The above context is an application of Pearson Correlation Coefficient which measure the relation among two variables. i.e. does the one variable change significantly in response to the change of the other variable. Mathematically it is calculated by the formula: ρ=cov(x,y)/sd(x)*sd(y). In R programing, you can obtain it by cor(x1,x2).

Application: Practically, people often use r or r² when they want to tell how one variable is correlated to the other. r can be ranged from -1 to 1, and a value "0" or close to "0" simply means there is no correlation among two parameters. Example: Height of twins are strongly correlated, r² very close to 1, so does r which means they are positively correlated. The rainfall and the yield of cotton is also strongly correlated, r² very close to 1, but r is close to -1 which means they are negatively correlated.

Now, let's visit R²

Definition of R²: R² is called Coefficient of Determination. Mathematically it is the ratio of SSR and SST. SSR=regression sum of squares=sum(yihat-ybar)^2, and SST=Total sum of squares=sum(yihat-ybar)^2. You often see them in anova table. By now, you may think R² and r² are different, wait a minute. Let's move on to examples as formula is always very dry.

Application: When we want to study the effect of package design, taste, ingredient, price of a bread to the sales, regression analysis is normally used [Example of Bread sales]. In this example, there are 4 factors, and one response. Again we use R programing. After you fit a model to the data, you will find R-squared (See 2nd last line) in the summary.

Take note, I used one variable only, i.e. x1.

> summary(fit)Call:lm(formula = y ~ x1)Residuals: Min 1Q Median 3Q Max -19.403 -6.121 -0.311 4.228 27.452 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 68.0454 9.4622 7.191 7.86e-07 ***x1 1.8359 0.1464 12.539 1.23e-10 ***---Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1Residual standard error: 12.19 on 19 degrees of freedomMultiple R-squared: 0.8922,Adjusted R-squared: 0.8865 F-statistic: 157.2 on 1 and 19 DF, p-value: 1.229e-10 

The reason why one is used is to explain to you the difference between r² and R². The context of one variable and one response is called simple linear regression model(LRM). Only in simple LRM, r² and R² are the same. See below output:

> cor(y,x1)[1] 0.9445543 

By square rooting R-squared: 0.8922 from LRM, you will get 0.945543 which is the same as the output of cor(y, x1)! Indeed, it can be proved mathematically r²=R².

Now, you might raise the question: When are they different?

Ans: In multi-variate regression analysis, they are different. But wait, you said they were the same, and now they are different. Confused? Let me use the Bread Sales example.

First, we want to study what are the factors that can contribute to sales positively. We list: package design, taste, ingredient, price, shelf life, distance the store to neighborhood. We plot the relation visually or use cor(x1,x2). It is reasonable to believe design, taste, ingredient, price have positive effect on the sales. The Pearson Correlation Coefficient supports our belief, with r for the 4 factors more than 0.5. And the other two - shelf life, distance seems to have weak correlation with a r value 0.45 and 0.06.

Second step, the marketing manager wants to find out how significant they contribute to sales. We now fit a linear model with all 6 factors. And you will look at the R-squared in the summary. As there are more than one variables. This R-squared tells us how much variation is contributed by the 6 variables together in relation to total variation [recall R-squared=SSR/SST ], it should be a value above 0 and below 1. It can not be negative when we fit a proper model. Under this scenario, IT IS DIFFERENT FROM PEARSON CORRELATION COEFFICIENT. Further analysis of p value suggests that only 4 factors - design, taste, ingredient, price affect sales significantly. So we reduce the variables from 6 to 4 and fit again. A new regression function will be produced with better fit. Consequently the R-squared will change. The value also explains the variation from 4 factors in relation to total variation. Although they both are called R-squared, we cannot compare them to tell which model is better as the SST and SSR both changed. In short, R² or R-squared tells variation from variables in relation to total variations. When the same number of variables is used for the response, it can be used to suggest which set of variables are better to construct a regression function. Otherwise, we should rely on other indices. Like AIC or BIC which is beyond the scope of this article.

Lastly, we use the data from Malaysia market to validate the model which is based on the data in Singapore. In the result, you may get a negative R². It means the model is not valid in Malaysia market. This type of test is called cross-validation to tell if our model is really correct using new data. Often than not, we will encounter negative R².

Recap:

1. r² is used when we begin with data to find any two among all variables are correlated or not. R² is used at subsequent step in regression to indicate how the model able to fit the data and explain the variation by fitted variables in relation to total variation.
2. r² and R² are the same in simple linear regression model.
3. When fitting using 2 and above variables, we cannot simply equate them. We need to introduce multiple correlation coefficient. It is much more complicated.
4. R² can be used to tell which regression model is better only if the number of variables used is the same, 1, or 2, or above.
5. R² can be negative, this is encountered during cross-validation when you feed new data to the model.