• Q difference between hypo test and a type of reasoning

    From Cosine@21:1/5 to All on Sat Mar 28 00:02:05 2020
    Hi:

    Would anyone clarify the difference between the reasoning we used for hypothesis testing and another one which will be described below?

    For the hypothesis test, we first make the null hypothesis, H0, and then make the alternative hypothesis, Ha, which complements and contradicts to the H0.
    Then we draw samples from the population and make testing on the samples. We compare the p-value with the alpha-value assigned before sampling. If the p-value is less then the alpha-value, we reject the H0 and accept the Ha as true.

    However, there is another way of conducting reasoning. We have two hypotheses, H0 and Ha, which contradict each other and together add as all that could happen. For example, a product being tested could be good or bad, but no other possibility.

    To prove the H0, we need to collect samples for testing to produce evidence supporting it. Nevertheless, even if the evidence found is not strong enough to prove the H0, we still must find other evidence supporting the Ha. If the evidence found is not
    strong enough to support the Ha, either, we likewise cannot claim that Ha is true.

    The latter type of reasoning process is different from what we used for the hypothesis, and it seems to be stricter than the one for the hypothesis test. How do we explain the result inferred by using this later method by using statistical theory?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From David Jones@21:1/5 to Cosine on Sat Mar 28 11:03:44 2020
    Cosine wrote:

    Hi:

    Would anyone clarify the difference between the reasoning we used
    for hypothesis testing and another one which will be described below?

    For the hypothesis test, we first make the null hypothesis, H0, and
    then make the alternative hypothesis, Ha, which complements and
    contradicts to the H0. Then we draw samples from the population
    and make testing on the samples. We compare the p-value with the
    alpha-value assigned before sampling. If the p-value is less then the alpha-value, we reject the H0 and accept the Ha as true.

    However, there is another way of conducting reasoning. We have two hypotheses, H0 and Ha, which contradict each other and together add
    as all that could happen. For example, a product being tested could
    be good or bad, but no other possibility.

    To prove the H0, we need to collect samples for testing to produce
    evidence supporting it. Nevertheless, even if the evidence found is
    not strong enough to prove the H0, we still must find other evidence supporting the Ha. If the evidence found is not strong enough to
    support the Ha, either, we likewise cannot claim that Ha is true.

    The latter type of reasoning process is different from what we used
    for the hypothesis, and it seems to be stricter than the one for the hypothesis test. How do we explain the result inferred by using this
    later method by using statistical theory?

    (a) You have missed out an important step in your summary ... finding a
    good test statistic. In "standard theory" there is standard test
    statistic that has been derived to have good/optimal prooerties for the null/alternative being considered .... but notionally you can choose
    to use any test statistic you like provided that it measures what you
    want it to measure. In your postulated situation you just need to find
    a test statistic that measures the apparent extent of evidence for the
    actual decision you are trying to make.

    (b) You may find it helpful to look at publications on the subject
    "tests of alternative families of hypotheses". There one has two
    separate families of statistical distributions and is trying to use
    observed data to look at the evidence for or against either, but on
    equal basis, wthout intially favouring either.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From duncan smith@21:1/5 to Cosine on Sat Mar 28 15:56:52 2020
    On 28/03/2020 07:02, Cosine wrote:
    Hi:

    Would anyone clarify the difference between the reasoning we used for hypothesis testing and another one which will be described below?

    For the hypothesis test, we first make the null hypothesis, H0, and then make the alternative hypothesis, Ha, which complements and contradicts to the H0.
    Then we draw samples from the population and make testing on the samples. We compare the p-value with the alpha-value assigned before sampling. If the p-value is less then the alpha-value, we reject the H0 and accept the Ha as true.

    However, there is another way of conducting reasoning. We have two hypotheses, H0 and Ha, which contradict each other and together add as all that could happen. For example, a product being tested could be good or bad, but no other possibility.

    To prove the H0, we need to collect samples for testing to produce evidence supporting it. Nevertheless, even if the evidence found is not strong enough to prove the H0, we still must find other evidence supporting the Ha. If the evidence found is
    not strong enough to support the Ha, either, we likewise cannot claim that Ha is true.

    The latter type of reasoning process is different from what we used for the hypothesis, and it seems to be stricter than the one for the hypothesis test. How do we explain the result inferred by using this later method by using statistical theory?


    You might want to check out marginal likelihood.

    Duncan

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich Ulrich@21:1/5 to dajhawkxx@gmail.com on Sat Mar 28 13:53:31 2020
    On Sat, 28 Mar 2020 11:03:44 +0000 (UTC), "David Jones"
    <dajhawkxx@gmail.com> wrote:

    Cosine wrote:

    Hi:

    Would anyone clarify the difference between the reasoning we used
    for hypothesis testing and another one which will be described below?

    For the hypothesis test, we first make the null hypothesis, H0, and
    then make the alternative hypothesis, Ha, which complements and
    contradicts to the H0. Then we draw samples from the population
    and make testing on the samples. We compare the p-value with the
    alpha-value assigned before sampling. If the p-value is less then the
    alpha-value, we reject the H0 and accept the Ha as true.

    However, there is another way of conducting reasoning. We have two
    hypotheses, H0 and Ha, which contradict each other and together add
    as all that could happen. For example, a product being tested could
    be good or bad, but no other possibility.

    To prove the H0, we need to collect samples for testing to produce
    evidence supporting it. Nevertheless, even if the evidence found is
    not strong enough to prove the H0, we still must find other evidence
    supporting the Ha. If the evidence found is not strong enough to
    support the Ha, either, we likewise cannot claim that Ha is true.

    The latter type of reasoning process is different from what we used
    for the hypothesis, and it seems to be stricter than the one for the
    hypothesis test. How do we explain the result inferred by using this
    later method by using statistical theory?

    (a) You have missed out an important step in your summary ... finding a
    good test statistic. In "standard theory" there is standard test
    statistic that has been derived to have good/optimal prooerties for the >null/alternative being considered .... but notionally you can choose
    to use any test statistic you like provided that it measures what you
    want it to measure. In your postulated situation you just need to find
    a test statistic that measures the apparent extent of evidence for the
    actual decision you are trying to make.

    (b) You may find it helpful to look at publications on the subject
    "tests of alternative families of hypotheses". There one has two
    separate families of statistical distributions and is trying to use
    observed data to look at the evidence for or against either, but on
    equal basis, wthout intially favouring either.

    I like that comment. It is broader and more formal than my
    own response. I'm going to focus on the final question,
    "How do we explain the result ... by using statistical theory?"

    I think the question mis-states the ordinary test conclusion,
    when conducting an asymmetrical test (small alpha).

    We typically /accept/ H0, and say that there is not enough
    evidence otherwise. We do not say that we "prove" the null.
    We may "reject" the null and accept the alternative. Nor do
    we say that we "proved" the alternative (if we are careful),
    unless we have asked the question very particularly.

    Example from the Q.
    For an item to be Good and Not Bad, we want to have a
    narrow confidence interval around the item's score, which
    includes the gold-standard for Good. Quality assurance
    techniques can be stiffer than the usual, simple test, when
    the measured scores themselves have error.

    Further.
    I've seen computer output (SPSS) for two-group discriminant
    function when you ask for classification information for each
    case. If I recall correctly -- It assigns a case to the "likelier"
    group; shows the ratio of the likeliihoods that led to that
    assignment; and shows the normal likelihood for belonging
    to the group that was assigned. That likelihood /might/ be
    very small -- the new item might lie well beyond the normal
    range of either group, or it could lie halfway between two
    groups when each has a very narrow range.

    For close selection, the likelihoods might be very similar
    for the two. Do you want to use the selection, when the
    criterion (alpha) is 50% ?

    Herman Rubin used to post here, regularly emphasizing
    "decision theory". He tended to dislike tests, because he
    saw them abused. I say, Tests are useful, but you do have
    to pay proper attention to what you are doing.

    --
    Rich Ulrich

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From David Jones@21:1/5 to Rich Ulrich on Sun Mar 29 16:05:49 2020
    Rich Ulrich wrote:

    On Sat, 28 Mar 2020 11:03:44 +0000 (UTC), "David Jones"
    <dajhawkxx@gmail.com> wrote:

    Cosine wrote:

    Hi:

    Would anyone clarify the difference between the reasoning we used
    for hypothesis testing and another one which will be described
    below? >>
    For the hypothesis test, we first make the null hypothesis, H0,
    and >> then make the alternative hypothesis, Ha, which complements and
    contradicts to the H0. Then we draw samples from the population
    and make testing on the samples. We compare the p-value with the
    alpha-value assigned before sampling. If the p-value is less then
    the >> alpha-value, we reject the H0 and accept the Ha as true.

    However, there is another way of conducting reasoning. We have
    two >> hypotheses, H0 and Ha, which contradict each other and
    together add >> as all that could happen. For example, a product
    being tested could >> be good or bad, but no other possibility.

    To prove the H0, we need to collect samples for testing to
    produce >> evidence supporting it. Nevertheless, even if the evidence
    found is >> not strong enough to prove the H0, we still must find
    other evidence >> supporting the Ha. If the evidence found is not
    strong enough to >> support the Ha, either, we likewise cannot claim
    that Ha is true. >>
    The latter type of reasoning process is different from what we
    used >> for the hypothesis, and it seems to be stricter than the one
    for the >> hypothesis test. How do we explain the result inferred by
    using this >> later method by using statistical theory?

    (a) You have missed out an important step in your summary ...
    finding a good test statistic. In "standard theory" there is
    standard test statistic that has been derived to have good/optimal prooerties for the null/alternative being considered .... but
    notionally you can choose to use any test statistic you like
    provided that it measures what you want it to measure. In your
    postulated situation you just need to find a test statistic that
    measures the apparent extent of evidence for the actual decision
    you are trying to make.

    (b) You may find it helpful to look at publications on the subject
    "tests of alternative families of hypotheses". There one has two
    separate families of statistical distributions and is trying to use observed data to look at the evidence for or against either, but on
    equal basis, wthout intially favouring either.

    I like that comment. It is broader and more formal than my
    own response. I'm going to focus on the final question,
    "How do we explain the result ... by using statistical theory?"

    I think the question mis-states the ordinary test conclusion,
    when conducting an asymmetrical test (small alpha).

    We typically accept H0, and say that there is not enough
    evidence otherwise. We do not say that we "prove" the null.
    We may "reject" the null and accept the alternative. Nor do
    we say that we "proved" the alternative (if we are careful),
    unless we have asked the question very particularly.

    Example from the Q.
    For an item to be Good and Not Bad, we want to have a
    narrow confidence interval around the item's score, which
    includes the gold-standard for Good. Quality assurance
    techniques can be stiffer than the usual, simple test, when
    the measured scores themselves have error.

    Further.
    I've seen computer output (SPSS) for two-group discriminant
    function when you ask for classification information for each
    case. If I recall correctly -- It assigns a case to the "likelier"
    group; shows the ratio of the likeliihoods that led to that
    assignment; and shows the normal likelihood for belonging
    to the group that was assigned. That likelihood might be
    very small -- the new item might lie well beyond the normal
    range of either group, or it could lie halfway between two
    groups when each has a very narrow range.

    For close selection, the likelihoods might be very similar
    for the two. Do you want to use the selection, when the
    criterion (alpha) is 50% ?

    Herman Rubin used to post here, regularly emphasizing
    "decision theory". He tended to dislike tests, because he
    saw them abused. I say, Tests are useful, but you do have
    to pay proper attention to what you are doing.

    In may follow-up post I included a reference that does a comparison
    with something labelled as a Bayesian approach, but I haven't looked
    enough to see if it actually is a Bayesian-decision-theory type of
    thing.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From David Jones@21:1/5 to David Jones on Sun Mar 29 16:00:41 2020
    David Jones wrote:

    Cosine wrote:

    Hi:

    Would anyone clarify the difference between the reasoning we used
    for hypothesis testing and another one which will be described
    below?

    For the hypothesis test, we first make the null hypothesis, H0,
    and then make the alternative hypothesis, Ha, which complements and contradicts to the H0. Then we draw samples from the population
    and make testing on the samples. We compare the p-value with the alpha-value assigned before sampling. If the p-value is less then
    the alpha-value, we reject the H0 and accept the Ha as true.

    However, there is another way of conducting reasoning. We have two hypotheses, H0 and Ha, which contradict each other and together add
    as all that could happen. For example, a product being tested could
    be good or bad, but no other possibility.

    To prove the H0, we need to collect samples for testing to produce evidence supporting it. Nevertheless, even if the evidence found is
    not strong enough to prove the H0, we still must find other evidence supporting the Ha. If the evidence found is not strong enough to
    support the Ha, either, we likewise cannot claim that Ha is true.

    The latter type of reasoning process is different from what we
    used for the hypothesis, and it seems to be stricter than the one
    for the hypothesis test. How do we explain the result inferred by
    using this later method by using statistical theory?

    (a) You have missed out an important step in your summary ... finding
    a good test statistic. In "standard theory" there is standard test
    statistic that has been derived to have good/optimal prooerties for
    the null/alternative being considered .... but notionally you can
    choose to use any test statistic you like provided that it measures
    what you want it to measure. In your postulated situation you just
    need to find a test statistic that measures the apparent extent of
    evidence for the actual decision you are trying to make.

    (b) You may find it helpful to look at publications on the subject
    "tests of alternative families of hypotheses". There one has two
    separate families of statistical distributions and is trying to use
    observed data to look at the evidence for or against either, but on
    equal basis, wthout intially favouring either.


    The last should have been "Tests of Separate Families of Hypotheses"

    A few references are :

    (1) D.R.Cox (1961)
    https://projecteuclid.org/euclid.bsmsp/1200512162 (open access)

    (2) D.R.Cox (1962)
    https://www.jstor.org/stable/2984232?seq=1 (free online access)


    (3) Basilio De Braganca Pereira (2005) https://onlinelibrary.wiley.com/doi/full/10.1002/0470011815.b2a09049
    (revew of progress, not open access)

    (4) D.R.Cox (2013)
    A return to an old paper: Tests of separate families of hypotheses https://www.semanticscholar.org/paper/A-return-to-an-old-paper%3A-‘Tests-of-separate-of-Cox/8b1408d65191ba46c83e9b1d2a77d4066832a15f
    (open access)

    (4) Assane et al. (2018)
    https://arxiv.org/pdf/1706.07685.pdf (open access)

    The last is fairly recent and includes a comparison with a Bayesian
    approach. But overall there are many other paers.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)