Internal validity. n.
Internal validity is concerned with the ability to draw causal inferences by the extent to which a study minimizes confounding heterogeneity and systematic error – for example, involving patient selection and measurement.
External validity. n.
External validity is concerned with the generalizability or real-world applicability of findings from a study and determines whether the results and inferences from the study can be applied to the target population and settings
Pre-Post studies. n.
A Pre-Post study compares changes in outcomes following an intervention and then seeks to attribute those changes to the intervention. They are intuitive to conduct. The problem is that, without reference to a comparison group, they cannot answer whether the changes would have occurred anyway.
Observational comparative effectiveness studies. n.
Observational studies seek to draw inferences about the possible effect of an intervention as adopted in real-world practice. Observational studies can be quicker to conduct, more practical, and more cost efficient than clinical trials. External validity is usually strong but studies suffer from confounding due to non-random allocation. In addition, observational studies can rely on data collected for non-research purposes – for example, billing data (administrative claims) or clinical data (electronic health records) – which can also limit internal validity.
Explanatory clinical trial. n.
An explanatory clinical trial is a specialized randomized experiment in a specialized population under optimal conditions. – often referred to as randomized controlled trials (RCTs). In general, explanatory trials seek to maximize internal validity, often at the expense of external validity.
Pragmatic clinical trials. n.
Pragmatic (or practical) clinical trials are randomized trials that are concerned with producing answers to questions faced by decision makers. Pragmatic trials seek to increase the external validity of the findings while maintaining strong internal validity. Tunis and colleagues defined them as studies that (1) select clinically relevant alternative interventions to compare; (2) include a diverse population of study participants, (3) recruit participants from heterogeneous practice settings, and (4) collect data on a broad range of health outcomes.
Comparison Intervention
Comparison Intervention
Practitioner Expertise (Experimental)
Practitioner Expertise (Experimental)
Intervention Flexibility
Intervention Flexibility
Follow-up Intensity
Follow-up Intensity
Practitioner Expertise (comparison)
Practitioner Expertise (comparison)
Eligibility Criteria
Eligibility Criteria
Primary Analysis
Primary Analysis
Participant Compliance
Participant Compliance
Practitioner Adherence
Practitioner Adherence
Glasgow RE, Magid DJ, Beck A, Ritzwoller D, Estabrooks PA. Practical clinical trials for translating research to practice: design and measurement recommendations. Med Care2005;43(6):551-557
Glasgow RE, Gaglio B, Bennett G, Jerome GJ, Yeh HC, Sarwer DB, Appel L, Colditz G, Wadden TA, Wells B. Applying the PRECIS criteria to describe three effectiveness trials of weight loss in obese patients with comorbid conditions. Health Serv Res. 2012 Jun;47(3 Pt 1):1051-67.
Rothwell PM. External validity of randomised controlled trials: "to whom do the results of this trial apply?" Lancet. Jan 1-7 2005;365(9453):82-93.
Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG,
Tunis S, Bergel E, Harvey I,Magid DJ, Chalkidou K. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009 May;62(5):464-75. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA 2003;290(12):1624-1632.
Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Oxman AD, Moher D; CONSORT group; Pragmatic Trials in Healthcare (Practihc) group. (2008). Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ;337:a2390.
Reach
Efficacy
Adoption
Implementation
Maintenance