1. Forum
  2. Usenet
  3. SCI.STAT.MATH

  • Q use of multiple metrics

    From Cosine@21:1/5 to All on Thu Nov 25 12:36:29 2021
    Hi:

    We would use more than one metrics to test the significance of a study. The most often used ones are sensitivity (SE) and specificity (SP). However, this pair would be affected by the disease prevalence. In contrast, the positive/negative predictive
    values (P/NPVs) are not affected by the prevalence.

    The first question now is: if we choose to use the P/NPVs for testing the significance, do we also test the significance for the SE and SP? Or do we only use one of the two pairs?

    The second question is that the value of the area under the receiver operating characteristic (AU-ROC) curve is a summary measure of the sensitivity and specificity. Does it mean that if we use the value of the AU-ROC for testing the significance, we do
    not test for the SE and SP?

    The third question is that is there a replacement of the value of the AU-ROC if we are concerned about the influence of the prevalence?

    Thank you,

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich Ulrich@21:1/5 to All on Fri Nov 26 13:55:54 2021
    On Thu, 25 Nov 2021 12:36:29 -0800 (PST), Cosine <[email protected]>
    wrote:

    Hi:

    We would use more than one metrics to test the significance of a
    study. The most often used ones are sensitivity (SE) and specificity
    (SP). However, this pair would be affected by the disease prevalence.
    In contrast, the positive/negative predictive values (P/NPVs) are not affected by the prevalence.

    I would not say it that way. The PPV is /based on/ the prevalence.
    It assumes a single value for the prevalence. The rarer the condition,
    the more likely the Positive is False.

    If you know the prevalence pretty well, you should use it for
    your description. "Testing the significance" uses the same set
    of numbers, same 2x2 table (I presume) for SE and SP, so you
    don't expect more power from one than for the other. It should
    be the same test.

    If you are interested in tests across the whole ROC curve, you
    test the curve. If you are interested in some specific prevalence,
    you test at that value.



    The first question now is: if we choose to use the P/NPVs for testing the significance, do we also test the significance for the SE and SP? Or do we only use one of the two pairs?

    The second question is that the value of the area under the receiver operating characteristic (AU-ROC) curve is a summary measure of the sensitivity and specificity. Does it mean that if we use the value of the AU-ROC for testing the significance, we
    do not test for the SE and SP?

    The third question is that is there a replacement of the value of the AU-ROC if we are concerned about the influence of the prevalence?

    --
    Rich Ulrich

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Bruce Weaver@21:1/5 to Rich Ulrich on Sun Nov 28 07:43:04 2021
    On Friday, November 26, 2021 at 1:56:03 PM UTC-5, Rich Ulrich wrote:
    On Thu, 25 Nov 2021 12:36:29 -0800 (PST), Cosine <[email protected]>
    wrote:
    Hi:

    We would use more than one metrics to test the significance of a
    study. The most often used ones are sensitivity (SE) and specificity
    (SP). However, this pair would be affected by the disease prevalence.
    In contrast, the positive/negative predictive values (P/NPVs) are not affected by the prevalence.
    I would not say it that way. The PPV is /based on/ the prevalence.
    It assumes a single value for the prevalence. The rarer the condition,
    the more likely the Positive is False.

    If you know the prevalence pretty well, you should use it for
    your description. "Testing the significance" uses the same set
    of numbers, same 2x2 table (I presume) for SE and SP, so you
    don't expect more power from one than for the other. It should
    be the same test.

    If you are interested in tests across the whole ROC curve, you
    test the curve. If you are interested in some specific prevalence,
    you test at that value.

    The equations in this BMJ Stats Note (Altman & Bland, 1994) show how prevalence is related to PPV and NPV:

    https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2540558/pdf/bmj00448-0038a.pdf

    Note as well that nowadays, some authors are using the following terms for the predictive values:

    Predictive value of a positive test (PV+)
    Predictive value of a negative test (PV-)

    I like this terminology and notation better than PPV/NPV, because it makes it clear that it is the *test result*, not the predictive value, that is either positive or negative.

    HTH.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)