A schoolboy question:
Thinking about the COVID issue, I haven't seen any data
on false positive/negative rates on the tests.
But first, a definition of 5% false pos.:
i) Given 100 positive results, it means 5 are erroneous?
or,
ii) Given 100 subjects non-infected, it means 5 will return positive?
You could translate these to the predictions for an individual.
Odd, that I never thought about this before -
And why is it, in medical testing generally, they focus
on the false positive rate, but discount false negatives?
On Tue, 18 Aug 2020 15:44:34 -0700 (PDT), RichD
<r_delaney2001@yahoo.com> wrote:
A schoolboy question:
Thinking about the COVID issue, I haven't seen any data
on false positive/negative rates on the tests.
Test + Test -
True + A B
True - C D
A = True Positive
B = False Negative
C = False Positive
D = True Negative
But first, a definition of 5% false pos.:
i) Given 100 positive results, it means 5 are erroneous?
C / (C+A) ? no
or,
ii) Given 100 subjects non-infected, it means 5 will return
positive?
C / (C+D) ? yes
A quick Google gives the definition,
"The false positive rate is the proportion of all negatives that still
yield positive test outcomes"
"Sensitivity" and (negative) "specificty" are the terms used
more often in epidemiology papers, to comprise the errors
in both directions, Together, they define the reliability of
the test. Technical papers will always discuss BOTH since
most of the tests have a quantitative cutoff that can be
adjusted, whereby you increase either at the expense of
the other.
The cutoff that is used will sometimes take into acount the
base-rate of the disease, since a change in base rate will
change the actual number and rate of errors in a particular
sample.
You could translate these to the predictions for an individual.
Odd, that I never thought about this before -
And why is it, in medical testing generally, they focus
on the false positive rate, but discount false negatives?
Your phrase - "in medical testing generally" - does not describe
the reality of medical testing. It may describe what you have
been reading, lately, in newspapers about covid-19.
Below are links to some sources of articles that include some
on sensitivity and specificity concerns. This content was originally
posted here by David Jones on June 1, 2020, under Subj:
"Analyzing Covid Data"
<<
I have not tried to follow any of the above. But, anyone with a
statistical interest in this epidemic should probably know about the following list of articles associtaed with the Significance magazine:
https://www.significancemagazine.com/business/647
The list features the UK rather heavily and relates to the RSS
Covid-19
Task Force, which is outlined here:
and:
https://rss.org.uk/policy-campaigns/policy/covid-19-task-force/
There is some overlap in these lists.
A schoolboy question:
Thinking about the COVID issue, I haven't seen any data
on false positive/negative rates on the tests.
But first, a definition of 5% false pos.:
i) Given 100 positive results, it means 5 are erroneous?
or,
ii) Given 100 subjects non-infected, it means 5 will return positive?
You could translate these to the predictions for an individual.
Odd, that I never thought about this before -
And why is it, in medical testing generally, they focus
on the false positive rate, but discount false negatives?
--
Rich
i) Given 100 positive results, it means 5 are erroneous?
or,
ii) Given 100 subjects non-infected, it means 5 will return positive?
For a quick look at some actual numbers regarding test accuracy, see >https://www.finddx.org/covid-19/sarscov2-eval-molecular/molecular-eval-results/
... this also shows the large number of different tests available and >indicates how well the test accuracies are known
On Wed, 19 Aug 2020 06:43:31 +0000 (UTC), "David Jones" <dajhawkxx@nowherel.com> wrote:
For a quick look at some actual numbers regarding test accuracy, see
... this also shows the large number of different tests available
and indicates how well the test accuracies are known
Thanks -
A few comments -
That says it is updated on July 3, so there are more tests
available by now, I'm sure.
Those are high values for both sensitivity and specificity,
many of them measured for their samples at 100%.
First, I am surprised and impressed. The numbers I
saw (probably in June) for a couple of tests were lower.
Those high numbers give me more reason to trust that
recent re-analysis of older data (mostly from April) where
they concluded that the "actual number" of cases ranged
from twice to 13 times (varying by place) the number of
"confirmed diagnoses".
Second, those were for "clean" samples. Collection in
the field introduces errors and inaccuracy. For instance,
the "deep nasal swab" is uncomfortably deep, and can
be done wrong, failing to get a good sample (ergo, lower
sensitivity).
One of the developments I have seen hyped in the last
week or so is the potential for a cheap saliva-test that may
be administered at home, with quick results. The lack of
ideal sensitivity is offset by cheap-and-quick. I think I read
that the US government is paying about $100 per test, though
I don't know how much of that goes to the labs.
Rich Ulrich wrote:
On Wed, 19 Aug 2020 06:43:31 +0000 (UTC), "David Jones"
<dajhawkxx@nowherel.com> wrote:
For a quick look at some actual numbers regarding test accuracy, see
https://www.finddx.org/covid-19/sarscov2-eval-molecular/molecular-eval-results/
... this also shows the large number of different tests available
and indicates how well the test accuracies are known
Thanks -
A few comments -
That says it is updated on July 3, so there are more tests
available by now, I'm sure.
Those are high values for both sensitivity and specificity,
many of them measured for their samples at 100%.
First, I am surprised and impressed. The numbers I
saw (probably in June) for a couple of tests were lower.
Those high numbers give me more reason to trust that
recent re-analysis of older data (mostly from April) where
they concluded that the "actual number" of cases ranged
from twice to 13 times (varying by place) the number of
"confirmed diagnoses".
Second, those were for "clean" samples. Collection in
the field introduces errors and inaccuracy. For instance,
the "deep nasal swab" is uncomfortably deep, and can
be done wrong, failing to get a good sample (ergo, lower
sensitivity).
One of the developments I have seen hyped in the last
week or so is the potential for a cheap saliva-test that may
be administered at home, with quick results. The lack of
ideal sensitivity is offset by cheap-and-quick. I think I read
that the US government is paying about $100 per test, though
I don't know how much of that goes to the labs.
Yes, this earlier article >https://www.significancemagazine.com/science/667-comparing-and-assessing-covid-19-tests
quotes some rather lower success rates for tests.
The high succces rates quoted seem to relate to this statemen:t
"On 19 February 2020, FIND launched an expression of interest (EOI) for
test developers of in vitro diagnostics (IVDs) that detect SARS-CoV-2
nucleic acid (molecular tests). The EOI closed on 9 March 2020. Over
200 submissions were received and applications were reviewed ..." Out
of which the 21 "best" were selected for independent evaluation and
inclusion in the tables: >https://www.finddx.org/covid-19/sarscov2-eval-molecular/
... so the tables only include tests available early on.
A wider view might be found on this page: >https://finddx.shinyapps.io/COVID19DxData/
which eventually provides a graphical scatter plot of Sensitivity
against Specificity, subject to various options ... but it does take
some time for the graph to appear, so some patience is needed. I don't
really unbderstand what is going on here, but you might want to move
the top button from "Antibody" to "Molecular".
One of the developments I have seen hyped in the last
week or so is the potential for a cheap saliva-test that may
be administered at home, with quick results. The lack of
ideal sensitivity is offset by cheap-and-quick. I think I read
that the US government is paying about $100 per test, though
I don't know how much of that goes to the labs.
On Wed, 19 Aug 2020 06:43:31 +0000 (UTC), "David Jones" <dajhawkxx@nowherel.com> wrote:
For a quick look at some actual numbers regarding test accuracy, see
... this also shows the large number of different tests available
and indicates how well the test accuracies are known
Thanks -
A few comments -
That says it is updated on July 3, so there are more tests
available by now, I'm sure.
Those are high values for both sensitivity and specificity,
many of them measured for their samples at 100%.
First, I am surprised and impressed. The numbers I
saw (probably in June) for a couple of tests were lower.
Those high numbers give me more reason to trust that
recent re-analysis of older data (mostly from April) where
they concluded that the "actual number" of cases ranged
from twice to 13 times (varying by place) the number of
"confirmed diagnoses".
Second, those were for "clean" samples. Collection in
the field introduces errors and inaccuracy. For instance,
the "deep nasal swab" is uncomfortably deep, and can
be done wrong, failing to get a good sample (ergo, lower
sensitivity).
One of the developments I have seen hyped in the last
week or so is the potential for a cheap saliva-test that may
be administered at home, with quick results. The lack of
ideal sensitivity is offset by cheap-and-quick. I think I read
that the US government is paying about $100 per test, though
I don't know how much of that goes to the labs.
Rich Ulrich wrote:
On Wed, 19 Aug 2020 06:43:31 +0000 (UTC), "David Jones"
<dajhawkxx@nowherel.com> wrote:
For a quick look at some actual numbers regarding test accuracy, see
https://www.finddx.org/covid-19/sarscov2-eval-molecular/molecular-eval-results/
... this also shows the large number of different tests available
and indicates how well the test accuracies are known
Thanks -
A few comments -
That says it is updated on July 3, so there are more tests
available by now, I'm sure.
Those are high values for both sensitivity and specificity,
many of them measured for their samples at 100%.
First, I am surprised and impressed. The numbers I
saw (probably in June) for a couple of tests were lower.
Those high numbers give me more reason to trust that
recent re-analysis of older data (mostly from April) where
they concluded that the "actual number" of cases ranged
from twice to 13 times (varying by place) the number of
"confirmed diagnoses".
Second, those were for "clean" samples. Collection in
the field introduces errors and inaccuracy. For instance,
the "deep nasal swab" is uncomfortably deep, and can
be done wrong, failing to get a good sample (ergo, lower
sensitivity).
One of the developments I have seen hyped in the last
week or so is the potential for a cheap saliva-test that may
be administered at home, with quick results. The lack of
ideal sensitivity is offset by cheap-and-quick. I think I read
that the US government is paying about $100 per test, though
I don't know how much of that goes to the labs.
THis letter from the RSS to a newspaper (September 11) >https://rss.org.uk/RSS/media/File-library/News/Press%20release/Letter-to-the-Times-on-government-moonshot-testing-plans-110920.pdf
suggests that the tests currently being used for those who might have
the infection are rather poor for certain intended uses.
On Tue, 15 Sep 2020 22:52:34 +0000 (UTC), "David Jones" <dajhawk18xx@@nowhere.com> wrote:
Rich Ulrich wrote:
see >> >On Wed, 19 Aug 2020 06:43:31 +0000 (UTC), "David Jones"
<dajhawkxx@nowherel.com> wrote:
For a quick look at some actual numbers regarding test accuracy,
... this also shows the large number of different tests available
and indicates how well the test accuracies are known
Thanks -
A few comments -
That says it is updated on July 3, so there are more tests
available by now, I'm sure.
Those are high values for both sensitivity and specificity,
many of them measured for their samples at 100%.
First, I am surprised and impressed. The numbers I
saw (probably in June) for a couple of tests were lower.
Those high numbers give me more reason to trust that
recent re-analysis of older data (mostly from April) where
they concluded that the "actual number" of cases ranged
from twice to 13 times (varying by place) the number of
"confirmed diagnoses".
Second, those were for "clean" samples. Collection in
the field introduces errors and inaccuracy. For instance,
the "deep nasal swab" is uncomfortably deep, and can
be done wrong, failing to get a good sample (ergo, lower
sensitivity).
One of the developments I have seen hyped in the last
week or so is the potential for a cheap saliva-test that may
be administered at home, with quick results. The lack of
ideal sensitivity is offset by cheap-and-quick. I think I read
that the US government is paying about $100 per test, though
I don't know how much of that goes to the labs.
THis letter from the RSS to a newspaper (September 11)
suggests that the tests currently being used for those who might
have the infection are rather poor for certain intended uses.
Thanks. I suspect that the same holds for most testing in the US,
but it is worth "discussion" - as the author says - about what
everyone is talking about as "the current tests" and the alternatives.
Do positive test resuts (somewhere) get re-tested with more precision?
I believe they were doing that at the NFL training camps, because
I did hear of a couple of false positives from their testing -- they
could not have decided that if they did not do the retest.
article -
<< Tests cause harm when they miss or wrongly diagnose cases.
Our current tests have 1 and 2% false positive rates – which, when
millions are being tested every day, risks causing personal and
economic harm to tens of thousands of people. This problem is
exacerbated if the new tests, as is likely, are less accurate than the
ones used currently.
If mass-testing can give people confidence that they are
disease-free, tests need to detect nearly all cases. Our current tests
miss around a fifth of those with the disease – if the new tests are
even less sensitive, they may not be accurate enough for the safe
running of events but could be useful for complementing social
distancing measures. >>
Rich Ulrich wrote:
One of the developments I have seen hyped in the last
week or so is the potential for a cheap saliva-test that may
be administered at home, with quick results. The lack of
ideal sensitivity is offset by cheap-and-quick. I think I read
that the US government is paying about $100 per test, though
I don't know how much of that goes to the labs.
This article ...
https://news.sky.com/story/coronavirus-what-are-the-different-types-of-covid-19-tests-12068081
outlines some of the different tests that might be available. Once
again, it is UK-centric.
Sysop: | Keyop |
---|---|
Location: | Huddersfield, West Yorkshire, UK |
Users: | 285 |
Nodes: | 16 (2 / 14) |
Uptime: | 63:37:02 |
Calls: | 6,488 |
Calls today: | 1 |
Files: | 12,096 |
Messages: | 5,274,684 |