The change of 1-2 units or greater in PCS or MCS scores has shown to be clinically and socially relevant (Kazis et al 2004, Kazis 2006).
At the population level small changes in PCS or MCS may have relevant policy implications. CMS explored the relationship between changes in health status and health expenditures, utilization of services, and experiences of care using data from the Medicare Health Outcomes Survey, the Managed Care CHAPS Enrollee Survey and the MCBS Cost and Use Data. Controlling for age, gender, race, education, marital status, region of residence, smoking status, Medicaid dual eligibility and the presence or absence of hypertension, myocardial infarction, angina pectoris/CAD, non-skin cancer, diabetes, and emphysema/asthma/COPD their results suggest:
A 1 point increase in PCS is associated with:
A 1 point increase in MCS is associated with:
6% lower total health care expenditures (adjusted cost ratio=0.94, p<0.001)
7% lower total health care expenditures (adjusted cost ratio=0.93, p<0.001)
Selim et al (2006) explored the probability of being alive with the same or better (than would be expected by chance) PCS or MCS at 2 years and mortality, while adjusting for case-mix at the VHA and Medicare Advantage Program. They did not find significant differences in the probability of being alive with the same or better PCS except for the South region (VHA 65.8% vs. MAP 62.5%, P = .0014).
Kazis, Lewis E., Donald R. Miller, Jack A. Clark, Katherine M. Skinner, Austin Lee, Xinhua S. Ren, Avron Spiro, 3rd, William H. Rogers, and John E. Ware, Jr. “Improving the Response Choices on the Veterans Sf-36 Health Survey Role Functioning Scales: Results from the Veterans Health Study.” The Journal of ambulatory care management 27, no. 3 (2004): 263-80. http://www.ncbi.nlm.nih.gov/pubmed/15287216
Kazis, Lewis E., Donald R. Miller, Katherine M. Skinner, Austin Lee, Xinhua S. Ren, Jack A. Clark, William H. Rogers, Avron Spiro Iii, Alfredo Selim, Mark Linzer, Susan M. C. Payne, Dorcas Mansell, and B. Graeme Fincke. “Applications of Methodologies of the Veterans Health Study in the Va Healthcare System: Conclusions and Summary.” The Journal of ambulatory care management 29, no. 2 (2006): 182-8. http://www.ncbi.nlm.nih.gov/pubmed/16552327
Selim, A. J., L. E. Kazis, W. Rogers, S. X. Qian, J. A. Rothendler, A. Spiro, 3rd, X. S. Ren, D. Miller, B. J. Selim, and B. G. Fincke. “Change in Health Status and Mortality as Indicators of Outcomes: Comparison between the Medicare Advantage Program and the Veterans Health Administration.” Qual Life Res 16, no. 7 (2007): 1179-91. http://www.ncbi.nlm.nih.gov/pubmed/16565637
Clinical and Individual level data:
Depending upon the design and purpose of the study, the definition of clinically relevant may vary. Several individual repeated measures (at least three) are required to understand the clinical significance of an observed change. As figure 8 illustrates, few repeated measures may result in a misleading trend (more noise, less signal) of PCS (dotted line) due to regression towards the mean. This threat to internal validity arises when a variable is extreme on its first measurement but it will tend to be closer to the true average on its second measurement. Paradoxically, if it is extreme on its second measurement, it will tend to have been closer to the average on its first. On the other hand, more than 3 measurements are more likely to capture true trends towards improvement or worsening (less noise, more signal). The increasing use of mHealth may facilitate obtaining several measurements across time. Go to section When to use VR to read more about mHealth.
Figure 8: Repeated measures and regression towards the mean
At the individual level, 6.5 units in PCS and 7.9 units in MCS are required to consider a change clinically relevant (Ware et al. 1996).
There are several techniques to quantify the responsiveness of an instrument towards a clinically meaningful change (Lydick et al 1993, Guyatt 2002). They may be estimated using effect sizes, supplemented by more traditional anchor-based methods of benchmarking (i.e. cross-sectional or longitudinal approaches) (Samsa, 1999). Anchor-based methodology measures a patient change score against a clinically relevant or outside change (the anchor or independent standard) (Lydick et al 1993). Effect size is the mean change of the individual divided by the variability of the whole group or the subset of stable subjects. It is used to translate “the before and after changes” in a “one group” situation into a standard unit of measurement that will provide a clearer understanding of health status results (Kazis et al. 1989). An effect size of 0.2 is considered small, 0.5 is moderate, while 0.8 or greater is considered large (Cohen et al 1988).
Donta, Sam T., Charles C. Engel, Jr., Joseph F. Collins, Joel B. Baseman, Lisa L. Dever, et al. “Benefits and Harms of Doxycycline Treatment for Gulf War Veterans’ Illnesses: A Randomized, Double-Blind, Placebo-Controlled Trial.” Annals of internal medicine 141, no. 2 (2004): 85-94. http://www.ncbi.nlm.nih.gov/pubmed/15262663
Donta, Sam T., Daniel J. Clauw, Charles C. Engel, Jr., Peter Guarino, Peter Peduzzi et al. “Cognitive Behavioral Therapy and Aerobic Exercise for Gulf War Veterans’ Illnesses: A Randomized Controlled Trial.” JAMA : the journal of the American Medical Association 289, no. 11 (2003): 1396-404. http://www.ncbi.nlm.nih.gov/pubmed/12636462
Guarino, P., P. Peduzzi, S. T. Donta, C. C. Engel, D. J. Clauw, D. A. Williams, J. S. Skinner, A. Barkhuizen, L. E. Kazis, and J. R. Feussner. “A Multicenter Two by Two Factorial Trial of Cognitive Behavioral Therapy and Aerobic Exercise for Gulf War Veterans’ Illnesses: Design of a Veterans Affairs Cooperative Study (Csp #470).” Control Clin Trials 22, no. 3 (2001): 310-32. http://www.ncbi.nlm.nih.gov/pubmed/11384792
Pyne, J. M., G. Sullivan, R. Kaplan, and D. K. Williams. “Comparing the Sensitivity of Generic Effectiveness Measures with Symptom Improvement in Persons with Schizophrenia.” Med Care 41, no. 2 (2003): 208-17. http://www.ncbi.nlm.nih.gov/pubmed/12555049
References
Cohen, J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Erlbaum, 1988.
Guyatt, G. H., D. Osoba, A. W. Wu, K. W. Wyrwich, and G. R. Norman. “Methods to Explain the Clinical Significance of Health Status Measures.” Mayo Clin Proc 77, no. 4 (2002): 371-83. http://www.ncbi.nlm.nih.gov/pubmed/11936935
Lydick, E., and R. S. Epstein. “Interpretation of Quality of Life Changes.” Quality of Life Research 2, no. 3 (1993): 221-26. http://www.jstor.org/stable/4034505
Kazis, L. E., J. J. Anderson, and R. F. Meenan. “Effect sizes for interpreting changes in health status.” Medical Care 27, no. 3 Suppl (1989): S178–189. http://www.ncbi.nlm.nih.gov/pubmed/2646488
Kazis LE, Selim AJ, Rogers W. et al. Functional Status in Older Adults: Intervention Strategies for Impacting Patient Outcomes – Technical Report Literature Review (Supplement for MAO Guide on Opportunities for Improving Medicare HOS Results through Practices in Quality Preventive Health Care for the Elderly). Technical Report on the Medicare Health Outcomes Survey Website: http://www.hosonline.org/globalassets/hos-online/faqs/functional_status_in_older_adults_2011.pdf
Samsa, Dr Greg, David Edelman, Margaret L. Rothman, G. Rhys Williams, Joseph Lipscomb, and David Matchar. “Determining Clinically Important Differences in Health Status Measures.” PharmacoEconomics 15, no. 2 (1999): 141–55. http://www.ncbi.nlm.nih.gov/pubmed/10351188
Ware, J. E., Jr., M. S. Bayliss, W. H. Rogers, M. Kosinski, and A. R. Tarlov. “Differences in 4-Year Health Outcomes for Elderly and Poor, Chronically Ill Patients Treated in HMO and Fee-for-Service Systems. Results from the Medical Outcomes Study.” Jama 276, no. 13 (1996): 1039-47. http://www.ncbi.nlm.nih.gov/pubmed/8847764