• Q test of significance of the trend of the data

    From Cosine@21:1/5 to All on Fri Jan 6 03:14:44 2023
    Hi:

    Sometimes we would like to demonstrate that the data has some particular type of trend, e.g., monotone increase or decrease. How do we demonstrate that this trend has statistical significance?

    Thank you,

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From David Jones@21:1/5 to Cosine on Fri Jan 6 14:17:07 2023
    Cosine wrote:

    Hi:

    Sometimes we would like to demonstrate that the data has some
    particular type of trend, e.g., monotone increase or decrease. How do
    we demonstrate that this trend has statistical significance?

    Thank you,

    As a first step, you need to think about the context of whatever data
    you have. In particular, you need to consider whether there is temporal correlation/dependence present in addtition to whatever trend you might postulate. Also, you should think about whether a possible "trend"
    should also include a possible change in variability instead-of or in addition-to a change in location.

    In the simplest case, you can just take a model-free approach whereby
    you return to first-principles. Specifically: (a) find a numerical
    measure of how much trend there is; (b) find the null distribution of
    your numerical measure by evaluating the same numerical measure for
    random permutations of the original data. It will be clear how the
    assumptions needed for the validity of this relate to my first
    paragraph.

    In the more general case, you could resort to the usual thing of
    building a full probabilistic model and testing via maximum likelihood.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich Ulrich@21:1/5 to dajhawk18xx@@nowhere.com on Fri Jan 6 17:11:09 2023
    On Fri, 6 Jan 2023 14:17:07 -0000 (UTC), "David Jones" <dajhawk18xx@@nowhere.com> wrote:

    Cosine wrote:

    Hi:

    Sometimes we would like to demonstrate that the data has some
    particular type of trend, e.g., monotone increase or decrease. How do
    we demonstrate that this trend has statistical significance?

    Thank you,

    As a first step, you need to think about the context of whatever data
    you have. In particular, you need to consider whether there is temporal >correlation/dependence present in addtition to whatever trend you might >postulate.

    Since we don't know at all what you are doing, and since the many
    variations of "time series" offer many pitfalls, I would have stopped
    right there. In order to ask:
    What are you measuring?

    We likely can give you advice relevant to a particular problem
    (and its pitfalls). "Statistically significant" says, "not by
    chance". It does not always imply "interesting"; stupid,
    obvious confounding relationships are more common for time-series
    than for other data.

    If your trends are across something one-dimensional other than time,
    the problems are apt to be fewer.

    Also, you should think about whether a possible "trend"
    should also include a possible change in variability instead-of or in >addition-to a change in location.

    In the simplest case, you can just take a model-free approach whereby
    you return to first-principles. Specifically: (a) find a numerical
    measure of how much trend there is; (b) find the null distribution of
    your numerical measure by evaluating the same numerical measure for
    random permutations of the original data. It will be clear how the >assumptions needed for the validity of this relate to my first
    paragraph.

    In the more general case, you could resort to the usual thing of
    building a full probabilistic model and testing via maximum likelihood.

    --
    Rich Ulrich

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Cosine@21:1/5 to All on Sat Jan 7 17:48:05 2023
    Say we have five groups of subjects, and each receives different concentrations of medicine, from low to high.

    At the endpoint, we measure the diameters of the lesion of each subject and calculate the mean diameter of each group.

    We expect a monotone decrease trend of the mean diameters of the groups. But how do we demonstrate the significance?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From David Jones@21:1/5 to Cosine on Sun Jan 8 04:36:55 2023
    Cosine wrote:

    Say we have five groups of subjects, and each receives different concentrations of medicine, from low to high.

    At the endpoint, we measure the diameters of the lesion of each
    subject and calculate the mean diameter of each group.

    We expect a monotone decrease trend of the mean diameters of the
    groups. But how do we demonstrate the significance?

    As part of the first step in significance testing, you need to have a
    null hypothesis as well as an alternative hypothesis. There are two
    obvious but distinct possibilities for one aspect of what might be
    going on: in one the null hypothesis has an unspecified but varying
    pattern, to be compared to a monotone pattern, while in the other the
    null hypothesis has constant value, to be compared with a monotone
    pattern.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich Ulrich@21:1/5 to dajhawkxx@nowherel.com on Sun Jan 8 01:36:34 2023
    On Sun, 8 Jan 2023 04:36:55 -0000 (UTC), "David Jones"
    <dajhawkxx@nowherel.com> wrote:

    Cosine wrote:

    Say we have five groups of subjects, and each receives different
    concentrations of medicine, from low to high.

    At the endpoint, we measure the diameters of the lesion of each
    subject and calculate the mean diameter of each group.

    We expect a monotone decrease trend of the mean diameters of the
    groups. But how do we demonstrate the significance?

    As part of the first step in significance testing, you need to have a
    null hypothesis as well as an alternative hypothesis.

    Or - you can have a situation where you want to provide a
    precise assessment, where basic "significance" is assumed, and
    readily established by any test.

    Having 5 concentrations, without a Zero comparison, implies
    that the questions (hypotheses) concern whether the lowest
    dose (concentration) has much effect, or if there is continued
    gain from increasing dose by each step.

    A overall test:
    Assuming that the doses here are judged (by the PI) to be
    (in the relevant sense) equal intervals, a simple correlation
    will show that increasing dose /matters/. This will be HIGHLY
    significant, you hope.

    (Also, the outcome should probably take into account the size
    of the original lesion. Log of the Pre/Post ratio might be natural,
    if lesions don't decrease to 0.)

    If I had data like these, I would want to plot the Pre vs. Post
    for the 5 doses, and figure out from the picture what there is
    to describe. A strong linear trend of efficicay across dose (log concentration) with tiny contributions from the nonlinear ANOVA
    components would be the outcome most convenient to describe.


    There are two
    obvious but distinct possibilities for one aspect of what might be
    going on: in one the null hypothesis has an unspecified but varying
    pattern, to be compared to a monotone pattern, while in the other the
    null hypothesis has constant value, to be compared with a monotone
    pattern.

    --
    Rich Ulrich

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From David Jones@21:1/5 to Rich Ulrich on Sun Jan 8 10:48:46 2023
    Rich Ulrich wrote:

    On Sun, 8 Jan 2023 04:36:55 -0000 (UTC), "David Jones" <dajhawkxx@nowherel.com> wrote:

    Cosine wrote:

    Say we have five groups of subjects, and each receives different
    concentrations of medicine, from low to high.

    At the endpoint, we measure the diameters of the lesion of each
    subject and calculate the mean diameter of each group.

    We expect a monotone decrease trend of the mean diameters of the
    groups. But how do we demonstrate the significance?

    As part of the first step in significance testing, you need to have
    a null hypothesis as well as an alternative hypothesis.

    Or - you can have a situation where you want to provide a
    precise assessment, where basic "significance" is assumed, and
    readily established by any test.

    Having 5 concentrations, without a Zero comparison, implies
    that the questions (hypotheses) concern whether the lowest
    dose (concentration) has much effect, or if there is continued
    gain from increasing dose by each step.

    A overall test:
    Assuming that the doses here are judged (by the PI) to be
    (in the relevant sense) equal intervals, a simple correlation
    will show that increasing dose matters. This will be HIGHLY
    significant, you hope.

    (Also, the outcome should probably take into account the size
    of the original lesion. Log of the Pre/Post ratio might be natural,
    if lesions don't decrease to 0.)

    If I had data like these, I would want to plot the Pre vs. Post
    for the 5 doses, and figure out from the picture what there is
    to describe. A strong linear trend of efficicay across dose (log concentration) with tiny contributions from the nonlinear ANOVA
    components would be the outcome most convenient to describe.


    There are two
    obvious but distinct possibilities for one aspect of what might be
    going on: in one the null hypothesis has an unspecified but varying pattern, to be compared to a monotone pattern, while in the other
    the null hypothesis has constant value, to be compared with a
    monotone pattern.

    The OP has been very unclear, so there seems also to be at least one
    other possibility, where the null hypothesis is that there is a
    monotone pattern, with the alternative hypothesis (that which one is
    looking evidence might be happening) is that there is a change in
    direction of the pattern as the dosage increases (but possibly just one
    turning point).

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich Ulrich@21:1/5 to dajhawkxx@nowherel.com on Tue Jan 10 00:45:41 2023
    On Sun, 8 Jan 2023 10:48:46 -0000 (UTC), "David Jones"
    <dajhawkxx@nowherel.com> wrote:

    Rich Ulrich wrote:

    On Sun, 8 Jan 2023 04:36:55 -0000 (UTC), "David Jones"
    <dajhawkxx@nowherel.com> wrote:

    Cosine wrote:

    Say we have five groups of subjects, and each receives different
    concentrations of medicine, from low to high.

    At the endpoint, we measure the diameters of the lesion of each
    subject and calculate the mean diameter of each group.

    We expect a monotone decrease trend of the mean diameters of the
    groups. But how do we demonstrate the significance?

    As part of the first step in significance testing, you need to have
    a null hypothesis as well as an alternative hypothesis.

    Or - you can have a situation where you want to provide a
    precise assessment, where basic "significance" is assumed, and
    readily established by any test.

    Having 5 concentrations, without a Zero comparison, implies
    that the questions (hypotheses) concern whether the lowest
    dose (concentration) has much effect, or if there is continued
    gain from increasing dose by each step.

    A overall test:
    Assuming that the doses here are judged (by the PI) to be
    (in the relevant sense) equal intervals, a simple correlation
    will show that increasing dose matters. This will be HIGHLY
    significant, you hope.

    (Also, the outcome should probably take into account the size
    of the original lesion. Log of the Pre/Post ratio might be natural,
    if lesions don't decrease to 0.)

    If I had data like these, I would want to plot the Pre vs. Post
    for the 5 doses, and figure out from the picture what there is
    to describe. A strong linear trend of efficicay across dose (log
    concentration) with tiny contributions from the nonlinear ANOVA
    components would be the outcome most convenient to describe.


    There are two
    obvious but distinct possibilities for one aspect of what might be
    going on: in one the null hypothesis has an unspecified but varying
    pattern, to be compared to a monotone pattern, while in the other
    the null hypothesis has constant value, to be compared with a
    monotone pattern.

    The OP has been very unclear, so there seems also to be at least one
    other possibility, where the null hypothesis is that there is a
    monotone pattern, with the alternative hypothesis (that which one is
    looking evidence might be happening) is that there is a change in
    direction of the pattern as the dosage increases (but possibly just one >turning point).


    Concerning alternative hypotheses: The OP's example might have been
    made up, but I've seen real instances where the PI did not consider,
    What do I REALLY want to show? Will my numbers be able to show it?

    Oh -'randomization' is necessary if one wants the easier conclusions
    of a 'randomized trial' (compared to observational reports). If size
    of lesion varies a lot, it could be worth stratifying the
    randomization.

    "Monotonic increase in response" is not as interesting as the actual
    degree of improvement. Or: Has someone argued that 'high' will be
    bad?

    If there is special concern about the end-points, it could be
    worthwhile to use larger Ns at the ends. (Is a no-dose condition non-informative? or well-known as having No-change?)

    Also, 'statistical power' is the reason that two-group studies are
    by far the most common. Trying to reach a firm conclusion about
    whether every two groups (dose) differ, out of five groups, when
    the dose-differences are small ... would require a larger N than
    anyone ordinarily justifies.

    --
    Rich Ulrich

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)