Residual Interpretation

[Yendall]

Core:

What would you say about this residual plot? my answer was: "The residual plot shows no clear linear pattern and confirms that the regression line is suitable to explain y from x"

Apparently, the parabola-ish tread indicates a systematic pattern, therefore it doesn't show linearity.

At what point does a residual plot parabolic trend become non-random? To me that looks fairly random.

[oneialex]

Pretty sure the pattern is in the curve of the data. From left to right the data lowers towards the middle of the plot, and then rises towards the end of the plot. If it looks like the data is correlated in any way like that, there is no linearity.

[Yendall]

Pretty sure the pattern is in the curve of the data. From left to right the data lowers towards the middle of the plot, and then rises towards the end of the plot. If it looks like the data is correlated in any way like that, there is no linearity.

Thanks! I kind of answered my own question when I started the thread haha :/

[djsandals]

What exam is that from?

[TrueTears]

It is almost impossible to simply examine a graph and determine whether there exists a pattern, although not part of the further course I will illustrate some methods which actually tests for serial correlation (http://en.wikipedia.org/wiki/Autocorrelation#Regression_analysis)

One way is to use the DW statistic: http://en.wikipedia.org/wiki/Durbin%E2%80%93Watson_statistic which is always given in any regression output, it tests for AR(1) serial correlation that is, serial correlation with itself of the first order (you can think of this like something that has a correlation with itself, so the next observation is somehow correlated to the previous observation of the same event): http://en.wikipedia.org/wiki/Autoregressive_model. If the DW statistic is close to 2, then most likely there is no serial correlation (and hence any patterns) in the residuals, however if its close to 0, then there probably is.

Next is to conduct a more formal test called the Breusch-Godfrey test http://en.wikipedia.org/wiki/Breusch%E2%80%93Godfrey_test, this method is far more flexible as it tests for serial correlation up to the qth order, by conducting simple hypothesis tests one can test for any order serial correlation. Any statistical package can automatically conduct this test.

All in all, any answer could be correct if you were to just look at the residual plot and come up with some sensible answer, however the truth can never be answered graphically, you must conduct the hypothesis test mathematically.

[Yendall]

How do you determine which values and the lower and upper critical values? Also, where does the test data come from? is it just random?

[TrueTears]

Good question about the critical values. There are many possible ways, if we know something about the population data set, ie, say in the rare case where we know the population follows a normal distribution, then clearly the critical values are just going to be from the standard normal. Obviously we have to pick a significance level, convention is to use either 1% or 5%. But in most cases we don't know anything about the distribution about the population, afterall, the whole purpose of statistical inference is to find out information about the population. So we can utilize the famous theorems, Law of Large Numbers http://en.wikipedia.org/wiki/Law_of_large_numbers and Central Limit Theorem http://en.wikipedia.org/wiki/Central_limit_theorem, these powerful theorems allows us to derive the distribution of the population under what we call the Guass-Markov Assumptions (or sometimes the Classical Linear Model Assumptions) http://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem.

Depending on what we are testing, so in this case, the residuals, through some derivations based on the assumptions we come to the conclusion that the critical values come from the t-distribution (if we are testing for 1 restriction, eg DW statistic) or the F-distribution (for multiple restrictions, eg, BG test), we can also use the chi-squared distribution.

The data is what you have been provided with, you would hope that it is random, this is one of the Gauss-Markov assumptions, if the data is not random then it is biased, if it is biased then statistical inference will be incorrect, so the foundation of all statistical analysis is that you have good data to begin with!

[Yendall]

I think the Law of Large Numbers seems suitable in this situation. We are given the data set in a previous question:

Average Sleep Time:


BMI:

So from here we could find the test data by using the Law, yes?

Average Sleep Test Data:

BMI Test Data:

Or does the test data need to be a residual value?

[TrueTears]

You can't do much with the LLN here or with the CLT.

Thanks for the data set, I have run the test on Eviews 7, this the output I get after running a simple linear regression of BMI on SLEEP



Looking at the DW statistic of 1.048, we can't say much, as it is right between 0 and 2, so then I ran the BG test, this is the output:



Looking at the p-value of the F statistic on the top right corner, we can conclude that at the 1% and 5% significance there is no evidence of serial correlation of the first order in the residuals.

[Yendall]

That's a really awesome way of looking at this! Thank you