On Thu, 17 Oct 2019 13:11:43 -0700 (PDT), Cosine <
asecant@gmail.com>
wrote:
Hi:
Comparing the means of two populations is a common task in statistical analysis. Independent samples and matched pair design require different approaches for conducting analysis.
Aside from the above formal methods of analysis, a popular approach employed by a naive investigator is to calculate the means of the samples of the two populations and then calculate the difference of the two means to see if the value is "large"
enough to support the claim that the two populations indeed have some differences.
1) What are the negative consequences of using the naive approach, instead of using the formal approach?
Putting in a statisitical tests makes use of the error term,
and thus includes a second /dimension/ to the statement.
That's been conventional for a long time. But knowing that
a result is "big" is important, too.
I remember learning about 50 years ago from an author:
the New England Journal of Medicine directed the authors
to delete all statistical tests for their results of a sample survey
of 25,000 people. Anything worth noting /would/ be
"significant", so, tests were extraneous.
2) What are the negative consequences of using a formal but wrong approach, for example, exchange the for independent samples and that for matched pair design?
The statistical test is wrong to whatever extent the error term
is wrong. That can be in either direction.
Consider var(a) +var(b) -2ab cov(a,b) , which is the SS error
for the deriving the two t-tests.
A positive covariance reduces the error; a negative covariance
increases the error.
Pairing usually has large r, so that pairing reduces the error.
However, I ran into a question online (year ago) where the
comparison was between (time on the Left) and (time on Right)
where the two add to a fixed constant plus-or-minus some
small amount. That made a strong negative r for L vs. R.
Because they were paired, the paired test was the /correct/ test,
even though it would "show" less difference than the unpaired test.
If I'm doing homework for you, your prof. will probably notice,
eventually.
--
Rich Ulrich
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)