The” PRECIS wheel“ figure has proven to be a very convenient summary of study design features. After a little experience, a user can quickly understand the overall extent to which and the dimensions along which a study is pragmatic vs. explanatory from glancing at the size and shape of the figure that results from connecting individual PRECIS scores.
Source: Rothwell PM. (2005). External validity of randomized controlled trials: To whom do the results of this trial apply? Lancet, 365, 82-93.
The University of Dundee Health Informatics Centre, 2015: https://crs.dundee.ac.uk/precis/
PRECIS breaks down all or none thinking about a trial in terms of being pragmatic or explanatory into series of dimensions or domains, as no trial is completely explanatory or completely pragmatic. PRECIS also includes a summary ‘wheel’ or spoke and hub diagram to visually summarize how pragmatic vs. explanatory a study is across dimensions. As shown in the Figure below, this figure plots each dimension or domain on a 5 point scale with 1 representing a very explanatory trial being very close to the center of the diagram, and 5 representing a very pragmatic score on that dimension, being at the outer edge of the figure.
Originally developed to help teams design studies, the original PRECIS criteria and tool proved useful for this purpose as well as for reporting on studies and for evaluating the published literature (Loudon et al., 2013). In the same way that the CONSORT flow diagram is now required by most journals, we recommend that the PRECIS figure be submitted with reports of pragmatic trials.
The PRECIS system has recently been revised to reflect experience and lessons learned into the PRECIS-2 toolkit. Instead of the original 10 dimensions, there are now 9 domains on which studies are rated using the 1-5 point rating system.
Key changes have been to rate all dimensions relative to usual care; to remove ratings of the comparison condition (now one rates only the intervention, or if two or more interventions, the most intensive intervention); and to include recruitment and setting dimensions related to external validity.
PRECIS-2 domains have been used to help us organize most of this workbook. The Table below summarizes the PRECIS-2 domains discussed in each chapter and the specific criteria used to rate each domain are covered in the relevant section of the workbook. More information on PRECIS-2 is available from the international group that developed it at the site below.
1. Very Explanatory
2 . Rather Explanatory
3. Equally Pragmatic/Explanatory
4. Rather Pragmatic
5. Very Pragmatic
Eligibility
Eligibility
Intervention Flexibility - Adherence
Intervention Flexibility - Adherence
Recruitment
Recruitment
Follow-up
Follow-up
Outcome
Outcome
Organization Intervention
Organization Intervention
How different are the resources, provider expertise and the organisation of care delivery in the intervention arm of the trial and those available in usual care? For example, score 5 for a very pragmatic choice that uses identical organisation to usual care; score 1 for a very explanatory approach if the trial increases staff levels, gives additional training, require more than usual experience or certification and increase resources.
Analysis
Analysis
Intervention Flexibility - Delivery
Intervention Flexibility - Delivery
Glasgow RE, Magid DJ, Beck A, Ritzwoller D, Estabrooks PA. Practical clinical trials for translating research to practice: design and measurement recommendations. Med Care2005;43(6):551-557
Glasgow RE, Gaglio B, Bennett G, Jerome GJ, Yeh HC, Sarwer DB, Appel L, Colditz G, Wadden TA, Wells B. Applying the PRECIS criteria to describe three effectiveness trials of weight loss in obese patients with comorbid conditions. Health Serv Res. 2012 Jun;47(3 Pt 1):1051-67.
Rothwell PM. External validity of randomised controlled trials: "to whom do the results of this trial apply?" Lancet. Jan 1-7 2005;365(9453):82-93.
Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG,
Tunis S, Bergel E, Harvey I,Magid DJ, Chalkidou K. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009 May;62(5):464-75. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA 2003;290(12):1624-1632.
Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Oxman AD, Moher D; CONSORT group; Pragmatic Trials in Healthcare (Practihc) group. (2008). Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ;337:a2390.
The study has four phases. Phase 1 involves brainstorming and a two-round Delphi survey of authors who cited PRECIS. In Phase 2, the Delphi results will then be discussed and alternative versions of PRECIS-2 developed and user-tested by experienced trialists. Phase 3 will evaluate the validity and reliability of the most promising PRECIS-2 candidate using a sample of 15 to 20 trials rated by 15 international trialists. We will assess inter-rater reliability, and raters’ subjective global ratings of pragmatism compared to PRECIS-2 to assess convergent and face validity. Phase 4, to determine if pragmatic trials sacrifice internal validity in order to achieve applicability, will compare the internal validity and effect estimates of matched explanatory and pragmatic trials of the same intervention, condition and participants. Effect sizes for the trials will then be compared in a meta-regression. The Cochrane Risk of Bias scores will be compared with the PRECIS-2 scores of pragmatism.
In making treatment decisions, doctors and patients must take into account relevant randomised controlled trials (RCTs) and systematic reviews. Relevance depends on external validity (or generalisability)--ie, whether the results can be reasonably applied to a definable group of patients in a particular clinical setting in routine practice. There is concern among clinicians that external validity is often poor, particularly for some pharmaceutical industry trials, a perception that has led to underuse of treatments that are effective. Yet researchers, funding agencies, ethics committees, the pharmaceutical industry, medical journals, and governmental regulators alike all neglect external validity, leaving clinicians to make judgments. However, reporting of the determinants of external validity in trial publications and systematic reviews is usually inadequate. This review discusses those determinants, presents a checklist for clinicians, and makes recommendations for greater consideration of external validity in the design and reporting of RCTs.
OBJECTIVE:
To propose a tool to assist trialists in making design decisions that are consistent with their trial's stated purpose.
STUDY DESIGN AND SETTING:
Randomized trials have been broadly categorized as either having a pragmatic or explanatory attitude. Pragmatic trials seek to answer the question, "Does this intervention work under usual conditions?," whereas explanatory trials are focused on the question, "Can this intervention work under ideal conditions?" Design decisions make a trial more (or less) pragmatic or explanatory, but no tool currently exists to help researchers make the best decisions possible in accordance with their trial's primary goal. During the course of two international meetings, participants with experience in clinical care, research commissioning, health care financing, trial methodology, and reporting defined and refined aspects of trial design that distinguish pragmatic attitudes from explanatory.
RESULTS:
We have developed a tool (called PRECIS) with 10 key domains and which identifies criteria to help researchers determine how pragmatic or explanatory their trial is. The assessment is summarized graphically.
The CONSORT statement is intended to improve reporting of randomised controlled trials and focuses on minimising the risk of bias (internal validity). The applicability of a trial’s results (generalisability or external validity) is also important, particularly for pragmatic trials. A pragmatic trial (a term first used in 1967 by Schwartz and Lellouch) can be broadly defined as a randomised controlled trial whose purpose is to inform decisions about practice. This extension of the CONSORT statement is intended to improve the reporting of such trials and focuses on applicability. At two, two-day meetings held in Toronto in 2005 and 2008, we reviewed the CONSORT statement and its extensions, the literature on pragmatic trials and applicability, and our experiences in conducting pragmatic trials.