Femke Claessen - Haaglanden Medisch Centrum, Leidschendam, Orthotrauma Research Center Amsterdam, Amsterdam, the Netherlands
Ruben Zwiers - Orthotrauma Research Center Amsterdam, Amsterdam, the Netherlands
Thierry Guitton - Department of Plastic, Reconstructive, Hand en Burn Surgery, Martini Hospital, Burn Center Groningen, the Netherlands
Job Doornberg - Flinders Medical Center, Royal Adelaide Hospital, Adelaide, Australia

In orthopedic surgery, there is an increasing number of papers about online studies on the reliability of classification systems. Useful classification systems need to be reliable and valid. Measurement of validity can be variable and is prone to observer bias. These online collaboratives derive adequate power to study reliability by having a large group of trained surgeons review a small number of cases instead of the “classic” reliability studies in which a small number of observers evaluate many cases. Large online interobserver studies have advantages (i.e., less than ۱۵ minutes to complete the survey, the ability to randomize, and the ability to study factors associated with reliability, accuracy, or decision-making). This ‘handbook’ paper gives an overview of current methods for online reliability studies. We discuss the study design, sample size calculation, statistical analyses of results, pitfalls, and limitations of the study design.Level of evidence: V

interobserver studies, Methods, Reliability