The importance of the spectrum effect
Much of clinical practice relies on tests measuring one or more characteristics of an individual to determine whether that individual is at risk of developing a condition of interest or does or does not have a particular disease. Many scientists and clinicians are engaged in developing and/or choosing whether to use such tests and so careful evaluation of their performance is important, with implications across healthcare and patient safety.
In an article published in the British Medical Journal, Dr Juliet Usher-Smith, Mr Stephen Sharp and Professor Simon Griffin from the Primary Care Unit and MRC Epidemiology Unit (Institute of Public Health, University of Cambridge) discuss the importance of considering the spectrum effect when evaluating tests. They highlight how the spectrum effect – which describes the variation between settings in performance of tests used to predict, screen for, and diagnose disease – results in the most commonly used measures of the performance of tests (sensitivity, specificity, and likelihood ratios) being influenced by both the prevalence of the condition or disease in the sample in which they are assessed and the characteristics of the sample.
Using simulations they illustrate, for example, why tests developed in populations with a higher prevalence of disease (e.g. in hospital settings) will typically have a lower sensitivity and higher specificity when applied in populations with lower disease prevalence (e.g. in primary care). They warn that calculations of positive and negative predictive value will only partly adjust for these differences in disease prevalence so the performance of such tests should be interpreted with caution and consideration of the chosen study design and population.
Ideally new tests should be developed and evaluated using data from the populations in which they are intended to be used. Where this is not possible, calculations of positive and negative predictive value will only partly adjust for these differences in disease prevalence so the performance of such tests should be interpreted with caution and consideration of the chosen study design and population”.
– Dr Juliet Usher-Smith
Read the full journal article
J A Usher-Smith, S J Sharp, S J Griffin, “The spectrum effect in tests for risk prediction, screening, and diagnosis,” BMJ 2016;353:i3139