A Systematic Review of Statistical Methods Used to Test for Reliability of Medical Instruments Measuring Continuous Variables

  Objective(s): Reliability measures precision or the extent to which test results can be replicated. This is the first ever systematic review to identify statistical methods used to measure reliability of equipment measuring continuous variables. This studyalso aims to highlight the inappropriate statistical method used in the reliability analysis and its implication in the medical practice.   Materials and Methods: In ۲۰۱۰, five electronic databases were searched between ۲۰۰۷ and ۲۰۰۹ to look for reliability studies. A total of ۵,۷۹۵ titles were initially identified. Only ۲۸۲ titles were potentially related, and finally ۴۲ fitted the inclusion criteria. Results: The Intra-class Correlation Coefficient (ICC) is the most popular method with ۲۵ (۶۰%) studies having used this method followed by the comparing means (۸ or ۱۹%). Out of ۲۵ studies using the ICC, only ۷ (۲۸%) reported the confidence intervals and types of ICC used. Most studies (۷۱%) also tested the agreement of instruments. Conclusion: This study finds that the Intra-class Correlation Coefficient is the most popular method used to assess the reliability of medical instruments measuring continuous outcomes. There are also inappropriate applications and interpretations of statistical methods in some studies. It is important for medical researchers to be aware of this issue, and be able to correctly perform analysis in reliability studies.