P Values & Random Sampling

[Garves13]

Okay a quick question that I'm finding confusing before the exam tomorrow.

IF the p value is > 0.05, but the researcher has sampled the population in an appropriate manner (random sampling etc.), what conclusions can be made?

I think that because the results are statistically insignificant no conclusion can be made, and therefore no generalisations can be made to the population of interest (regardless on whether the sample is representative)

However a friend has a different perspective .... and thinks that just because they are statistically insignificant, they cannot be generalised...
EG:if you were testing a drug that you thought would cure a disease and the results were not statistically significant but you'd sampled perfectly, then you can still generalise your findings because that IS the finding: that the drug didn't work!

Could someone pleeeeeeease clear this up?

[psychlaw]

These are two different questions

In the 05 exam, there was two different questions, Q8 which said "What statistical conclusion can Rhonda infer from these results?"
To which the answer is: Rhonda cannot conclude anything about the results as they are statistically insignificant due to the p value being greater then 0.05

But the next question, Q9, said "Based on her sampling method, what conclusion can Rhonda reach about the underlying population from which the sample was drawn? Explain." - well obviously a conclusion can't be made because the p value was over 0.05, but presume it was below 0.05, the answer would be as follows

: She cannont conclude anything about the underlying population, as the participants (year 12 vce female psych students) are not entirely representative of the entire population, therefore no conclusion can me made.




With your instance you can't conclude anything because the p-value was over 0.05, thats the end of it

[melaniej]

With reference to you example, it doesn't mean that the drug didn't work, it meant that for some reason there was more than a 5% probability that the results were due to chance.
For your scenario you would say:
Although the results indicate that the drug cured the disease, no conclusion can be drawn from the results, as they are not statistically significant, meaning there is more than a 5% probability that the results were due to chance, and not to the effects of the independent variable.