• #### Q difference between hypo test and a type of reasoning

From Cosine@21:1/5 to All on Sat Mar 28 00:02:05 2020
Hi:

Would anyone clarify the difference between the reasoning we used for hypothesis testing and another one which will be described below?

For the hypothesis test, we first make the null hypothesis, H0, and then make the alternative hypothesis, Ha, which complements and contradicts to the H0.
Then we draw samples from the population and make testing on the samples. We compare the p-value with the alpha-value assigned before sampling. If the p-value is less then the alpha-value, we reject the H0 and accept the Ha as true.

However, there is another way of conducting reasoning. We have two hypotheses, H0 and Ha, which contradict each other and together add as all that could happen. For example, a product being tested could be good or bad, but no other possibility.

To prove the H0, we need to collect samples for testing to produce evidence supporting it. Nevertheless, even if the evidence found is not strong enough to prove the H0, we still must find other evidence supporting the Ha. If the evidence found is not
strong enough to support the Ha, either, we likewise cannot claim that Ha is true.

The latter type of reasoning process is different from what we used for the hypothesis, and it seems to be stricter than the one for the hypothesis test. How do we explain the result inferred by using this later method by using statistical theory?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)
• From David Jones@21:1/5 to Cosine on Sat Mar 28 11:03:44 2020
Cosine wrote:

Hi:

Would anyone clarify the difference between the reasoning we used
for hypothesis testing and another one which will be described below?

For the hypothesis test, we first make the null hypothesis, H0, and
then make the alternative hypothesis, Ha, which complements and
contradicts to the H0. Then we draw samples from the population
and make testing on the samples. We compare the p-value with the
alpha-value assigned before sampling. If the p-value is less then the alpha-value, we reject the H0 and accept the Ha as true.

However, there is another way of conducting reasoning. We have two hypotheses, H0 and Ha, which contradict each other and together add
as all that could happen. For example, a product being tested could
be good or bad, but no other possibility.

To prove the H0, we need to collect samples for testing to produce
evidence supporting it. Nevertheless, even if the evidence found is
not strong enough to prove the H0, we still must find other evidence supporting the Ha. If the evidence found is not strong enough to
support the Ha, either, we likewise cannot claim that Ha is true.

The latter type of reasoning process is different from what we used
for the hypothesis, and it seems to be stricter than the one for the hypothesis test. How do we explain the result inferred by using this
later method by using statistical theory?

(a) You have missed out an important step in your summary ... finding a
good test statistic. In "standard theory" there is standard test
statistic that has been derived to have good/optimal prooerties for the null/alternative being considered .... but notionally you can choose
to use any test statistic you like provided that it measures what you
want it to measure. In your postulated situation you just need to find
a test statistic that measures the apparent extent of evidence for the
actual decision you are trying to make.

(b) You may find it helpful to look at publications on the subject
"tests of alternative families of hypotheses". There one has two
separate families of statistical distributions and is trying to use
observed data to look at the evidence for or against either, but on
equal basis, wthout intially favouring either.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)
• From duncan smith@21:1/5 to Cosine on Sat Mar 28 15:56:52 2020
On 28/03/2020 07:02, Cosine wrote:
Hi:

Would anyone clarify the difference between the reasoning we used for hypothesis testing and another one which will be described below?

For the hypothesis test, we first make the null hypothesis, H0, and then make the alternative hypothesis, Ha, which complements and contradicts to the H0.
Then we draw samples from the population and make testing on the samples. We compare the p-value with the alpha-value assigned before sampling. If the p-value is less then the alpha-value, we reject the H0 and accept the Ha as true.

However, there is another way of conducting reasoning. We have two hypotheses, H0 and Ha, which contradict each other and together add as all that could happen. For example, a product being tested could be good or bad, but no other possibility.

To prove the H0, we need to collect samples for testing to produce evidence supporting it. Nevertheless, even if the evidence found is not strong enough to prove the H0, we still must find other evidence supporting the Ha. If the evidence found is
not strong enough to support the Ha, either, we likewise cannot claim that Ha is true.

The latter type of reasoning process is different from what we used for the hypothesis, and it seems to be stricter than the one for the hypothesis test. How do we explain the result inferred by using this later method by using statistical theory?

You might want to check out marginal likelihood.

Duncan

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)
• From Rich Ulrich@21:1/5 to dajhawkxx@gmail.com on Sat Mar 28 13:53:31 2020
On Sat, 28 Mar 2020 11:03:44 +0000 (UTC), "David Jones"
<dajhawkxx@gmail.com> wrote:

Cosine wrote:

Hi:

Would anyone clarify the difference between the reasoning we used
for hypothesis testing and another one which will be described below?

For the hypothesis test, we first make the null hypothesis, H0, and
then make the alternative hypothesis, Ha, which complements and
contradicts to the H0. Then we draw samples from the population
and make testing on the samples. We compare the p-value with the
alpha-value assigned before sampling. If the p-value is less then the
alpha-value, we reject the H0 and accept the Ha as true.

However, there is another way of conducting reasoning. We have two
as all that could happen. For example, a product being tested could
be good or bad, but no other possibility.

To prove the H0, we need to collect samples for testing to produce
evidence supporting it. Nevertheless, even if the evidence found is
not strong enough to prove the H0, we still must find other evidence
supporting the Ha. If the evidence found is not strong enough to
support the Ha, either, we likewise cannot claim that Ha is true.

The latter type of reasoning process is different from what we used
for the hypothesis, and it seems to be stricter than the one for the
hypothesis test. How do we explain the result inferred by using this
later method by using statistical theory?

(a) You have missed out an important step in your summary ... finding a
good test statistic. In "standard theory" there is standard test
statistic that has been derived to have good/optimal prooerties for the >null/alternative being considered .... but notionally you can choose
to use any test statistic you like provided that it measures what you
want it to measure. In your postulated situation you just need to find
a test statistic that measures the apparent extent of evidence for the
actual decision you are trying to make.

(b) You may find it helpful to look at publications on the subject
"tests of alternative families of hypotheses". There one has two
separate families of statistical distributions and is trying to use
observed data to look at the evidence for or against either, but on
equal basis, wthout intially favouring either.

I like that comment. It is broader and more formal than my
own response. I'm going to focus on the final question,
"How do we explain the result ... by using statistical theory?"

I think the question mis-states the ordinary test conclusion,
when conducting an asymmetrical test (small alpha).

We typically /accept/ H0, and say that there is not enough
evidence otherwise. We do not say that we "prove" the null.
We may "reject" the null and accept the alternative. Nor do
we say that we "proved" the alternative (if we are careful),
unless we have asked the question very particularly.

Example from the Q.
For an item to be Good and Not Bad, we want to have a
narrow confidence interval around the item's score, which
includes the gold-standard for Good. Quality assurance
techniques can be stiffer than the usual, simple test, when
the measured scores themselves have error.

Further.
I've seen computer output (SPSS) for two-group discriminant
function when you ask for classification information for each
case. If I recall correctly -- It assigns a case to the "likelier"
group; shows the ratio of the likeliihoods that led to that
assignment; and shows the normal likelihood for belonging
to the group that was assigned. That likelihood /might/ be
very small -- the new item might lie well beyond the normal
range of either group, or it could lie halfway between two
groups when each has a very narrow range.

For close selection, the likelihoods might be very similar
for the two. Do you want to use the selection, when the
criterion (alpha) is 50% ?

Herman Rubin used to post here, regularly emphasizing
"decision theory". He tended to dislike tests, because he
saw them abused. I say, Tests are useful, but you do have
to pay proper attention to what you are doing.

--
Rich Ulrich

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)
• From David Jones@21:1/5 to Rich Ulrich on Sun Mar 29 16:05:49 2020
Rich Ulrich wrote:

On Sat, 28 Mar 2020 11:03:44 +0000 (UTC), "David Jones"
<dajhawkxx@gmail.com> wrote:

Cosine wrote:

Hi:

Would anyone clarify the difference between the reasoning we used
for hypothesis testing and another one which will be described
below? >>
For the hypothesis test, we first make the null hypothesis, H0,
and >> then make the alternative hypothesis, Ha, which complements and
contradicts to the H0. Then we draw samples from the population
and make testing on the samples. We compare the p-value with the
alpha-value assigned before sampling. If the p-value is less then
the >> alpha-value, we reject the H0 and accept the Ha as true.

However, there is another way of conducting reasoning. We have
two >> hypotheses, H0 and Ha, which contradict each other and
together add >> as all that could happen. For example, a product
being tested could >> be good or bad, but no other possibility.

To prove the H0, we need to collect samples for testing to
produce >> evidence supporting it. Nevertheless, even if the evidence
found is >> not strong enough to prove the H0, we still must find
other evidence >> supporting the Ha. If the evidence found is not
strong enough to >> support the Ha, either, we likewise cannot claim
that Ha is true. >>
The latter type of reasoning process is different from what we
used >> for the hypothesis, and it seems to be stricter than the one
for the >> hypothesis test. How do we explain the result inferred by
using this >> later method by using statistical theory?

(a) You have missed out an important step in your summary ...
finding a good test statistic. In "standard theory" there is
standard test statistic that has been derived to have good/optimal prooerties for the null/alternative being considered .... but
notionally you can choose to use any test statistic you like
provided that it measures what you want it to measure. In your
postulated situation you just need to find a test statistic that
measures the apparent extent of evidence for the actual decision
you are trying to make.

(b) You may find it helpful to look at publications on the subject
"tests of alternative families of hypotheses". There one has two
separate families of statistical distributions and is trying to use observed data to look at the evidence for or against either, but on
equal basis, wthout intially favouring either.

I like that comment. It is broader and more formal than my
own response. I'm going to focus on the final question,
"How do we explain the result ... by using statistical theory?"

I think the question mis-states the ordinary test conclusion,
when conducting an asymmetrical test (small alpha).

We typically accept H0, and say that there is not enough
evidence otherwise. We do not say that we "prove" the null.
We may "reject" the null and accept the alternative. Nor do
we say that we "proved" the alternative (if we are careful),
unless we have asked the question very particularly.

Example from the Q.
For an item to be Good and Not Bad, we want to have a
narrow confidence interval around the item's score, which
includes the gold-standard for Good. Quality assurance
techniques can be stiffer than the usual, simple test, when
the measured scores themselves have error.

Further.
I've seen computer output (SPSS) for two-group discriminant
function when you ask for classification information for each
case. If I recall correctly -- It assigns a case to the "likelier"
group; shows the ratio of the likeliihoods that led to that
assignment; and shows the normal likelihood for belonging
to the group that was assigned. That likelihood might be
very small -- the new item might lie well beyond the normal
range of either group, or it could lie halfway between two
groups when each has a very narrow range.

For close selection, the likelihoods might be very similar
for the two. Do you want to use the selection, when the
criterion (alpha) is 50% ?

Herman Rubin used to post here, regularly emphasizing
"decision theory". He tended to dislike tests, because he
saw them abused. I say, Tests are useful, but you do have
to pay proper attention to what you are doing.

In may follow-up post I included a reference that does a comparison
with something labelled as a Bayesian approach, but I haven't looked
enough to see if it actually is a Bayesian-decision-theory type of
thing.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)
• From David Jones@21:1/5 to David Jones on Sun Mar 29 16:00:41 2020
David Jones wrote:

Cosine wrote:

Hi:

Would anyone clarify the difference between the reasoning we used
for hypothesis testing and another one which will be described
below?

For the hypothesis test, we first make the null hypothesis, H0,
and then make the alternative hypothesis, Ha, which complements and contradicts to the H0. Then we draw samples from the population
and make testing on the samples. We compare the p-value with the alpha-value assigned before sampling. If the p-value is less then
the alpha-value, we reject the H0 and accept the Ha as true.

However, there is another way of conducting reasoning. We have two hypotheses, H0 and Ha, which contradict each other and together add
as all that could happen. For example, a product being tested could
be good or bad, but no other possibility.

To prove the H0, we need to collect samples for testing to produce evidence supporting it. Nevertheless, even if the evidence found is
not strong enough to prove the H0, we still must find other evidence supporting the Ha. If the evidence found is not strong enough to
support the Ha, either, we likewise cannot claim that Ha is true.

The latter type of reasoning process is different from what we
used for the hypothesis, and it seems to be stricter than the one
for the hypothesis test. How do we explain the result inferred by
using this later method by using statistical theory?

(a) You have missed out an important step in your summary ... finding
a good test statistic. In "standard theory" there is standard test
statistic that has been derived to have good/optimal prooerties for
the null/alternative being considered .... but notionally you can
choose to use any test statistic you like provided that it measures
what you want it to measure. In your postulated situation you just
need to find a test statistic that measures the apparent extent of
evidence for the actual decision you are trying to make.

(b) You may find it helpful to look at publications on the subject
"tests of alternative families of hypotheses". There one has two
separate families of statistical distributions and is trying to use
observed data to look at the evidence for or against either, but on
equal basis, wthout intially favouring either.

The last should have been "Tests of Separate Families of Hypotheses"

A few references are :

(1) D.R.Cox (1961)
https://projecteuclid.org/euclid.bsmsp/1200512162 (open access)

(2) D.R.Cox (1962)
https://www.jstor.org/stable/2984232?seq=1 (free online access)

(3) Basilio De Braganca Pereira (2005) https://onlinelibrary.wiley.com/doi/full/10.1002/0470011815.b2a09049
(revew of progress, not open access)

(4) D.R.Cox (2013)
A return to an old paper: Tests of separate families of hypotheses https://www.semanticscholar.org/paper/A-return-to-an-old-paper%3A-‘Tests-of-separate-of-Cox/8b1408d65191ba46c83e9b1d2a77d4066832a15f
(open access)

(4) Assane et al. (2018)
https://arxiv.org/pdf/1706.07685.pdf (open access)

The last is fairly recent and includes a comparison with a Bayesian
approach. But overall there are many other paers.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)