Section I: What Are Pragmatic Trials?
What Purpose Does Pragmatic Research Serve?
4 Reasons to Consider Pragmatic Trials
-
Pragmatic Trials allow us to evaluate interventions under typical conditions, with typical patients vs. under ideal conditions with a homogeneous set of selected patients (Explanatory Trials).
-
Traditional trials usually take a long time to translate into practice; it takes an average of 17 years for just 14% of research to translate into practice.
-
The conditions under which traditional trials are conducted are often not seen by practitioners as relevant to their practice or patient populations.
-
Although there are over 18,000 randomized controlled trials (RCT's) published each year, the vast majority of systematic reviews conclude that there is not enough evidence to inform clinical decisions.
Learning Objectives:
Describe the purpose and defining characteristics of pragmatic research.
What Are Pragmatic Trials?
Overview:
There are several differences between pragmatic trials and traditional efficacy or explanatory trials. It is not that one type of trial is always better than the other, but rather that they answer different questions. Some of the key differences are that in pragmatic studies, there is more emphasis on external validity of findings. Thus, stakeholders are involved throughout the study planning, implementation and reporting phases. Measures in pragmatic trials focus on more practical issues such as the reach and costs of an intervention and tend to use existing measures such as data available from electronic health records more than do explanatory trials. In contrast, explanatory trials tend to focus more on investigator-defined outcomes to answer questions about theoretical mechanisms. To advance science and application, we need answers to both.
PRAGMATIC RESEARCH BY OTHER NAMES:
You may already be familiar with Pragmatic Trials, they have also been known by some of the names defined below
- Large Simple Trials
- Phase IIb-IV Trials
- Community-Based Research
- Practical Trials
- Naturalistic Clinical Trials
Large Simple Trials
"Large Simple Trials (LSTs), usually randomized, are characterized by large sample sizes, broad entry criteria consistent with the intended target population, streamlined data collection, objectively measured endpoints (e.g., death, hospitalization), and follow-up that mimics normal clinical practice. LSTs often rely on recruitment of patients and investigators in the community setting and minimize patient visits and on-site monitoring. LSTs are ideally suited to answer many important clinical questions."
Phase IIIb:
Clinical trials conducted after regulatory submission of an NDA or other dossier, but prior to the medicine's approval and launch. These trials may supplement earlier trials, complete earlier trials, or may be directed toward new types of trials (e.g., quality of life, marketing) or Phase IV evaluations. This is the period between submission and approval of a regulatory dossier for marketing authorization.
Phase IV:
Studies or trials conducted after a medicine is marketed to provide additional details about the medicine's efficacy or safety profile. Different formulations, dosages, durations of treatment, medicine interactions, and other medicine comparisons may be evaluated. New age groups, races, and other types of patients can be studied.
Community-Based Research
A collaborative and participatory approach to research in which community organizations (schools, residents and civic leaders) partner with academic researchers (faculty, academic staff and students) to produce knowledge that solves real world problems. Community Based Research seeks community engagement at each phase of the research project. Community Based Research does not specify one model of research over another; models are selected based on what is most appropriate to the problems to be studied and solved.
Practical Trials
Clinical trials for which the hypothesis and study design are developed specifically to answer the questions faced by decision makers are called pragmatic or practical clinical trials (PCTs). The characteristic features of PCTs are that they (1) select clinically relevant alternative interventions to compare, (2) include a diverse population of study participants, (3) recruit participants from heterogeneous practice settings, and (4) collect data on a broad range of health outcomes.
Naturalistic clinical trials
Prospective “noninterventional” observational studies of phenomena, or, more broadly, retrospective analyses of existing data and studies.
Comparing Pragmatic Trials and Traditional Efficacy Trials
Pragmatic Trials address questions important to stakeholders. There are often multiple stakeholders for a given issue, including patients, providers, and administrative decision makers. Pragmatic trials include both multiple, heterogeneous settings and diverse populations that together represent typical settings and typical patients that would receive a given intervention. They also include multiple outcomes that are important to stakeholders, including decision makers. The final important characteristic of pragmatic trials is that they include comparisons of different real world alternatives, rather than placebo or no treatment conditions.
Study outcomes include the intervention reach (percent of patients offered and completing the My Own Health Report (MOHR) assessment), effectiveness (patients reporting being asked about topics, setting change goals, and receiving assistance in early versus delayed intervention practices), contextual factors influencing outcomes, and intervention costs.
Pragmatic Study
VS.
Traditional Efficacy Trial
Stakeholder Involvement
Pragmatic Studies
Engaged in all study phases including study design, conducting the study, collecting data, interpreting results, disseminating findings.
Traditional Studies
Limited engagement, often in response to investigator ideas of study subjects.
Research Design
Pragmatic Studies
Includes internal and external validity, design fidelity and local adaptation, real life settings and populations, contextual assessments.
Traditional Studies
Focus on limiting threats to internal validity, typically uses randomized controlled trial, participants and settings typically homogenous.
Outcomes
Pragmatic Studies
Reach, effectiveness, adoption, implementation, comparative effectiveness, sustainability.
Traditional Studies
Efficacy, mechanism identification, component analysis.
Measures
Pragmatic Studies
Brief, valid, actionable with rapid clinical utility, feasible in real world and low-resource settings.
Traditional Studies
Validated measures that minimize bias, focus on internal consistency and theory rather than clinical relevance.
Costs
Pragmatic Studies
Assessments include intervention costs and replication costs in relation to outcomes.
Traditional Studies
Often not collected or reported.
Data Source
Pragmatic Studies
May include existing data (electronic health records, administrative data) and brief patient reports.
Traditional Studies
Data generation and collection part of clinical trial.
Analyses
Pragmatic Studies
Process and outcome analyses relevant to stakeholders and from different perspectives.
Traditional Studies
Specified a priori and typically restricted to investigator hypotheses.
Availability of Findings
Pragmatic Studies
Rapid learning and implementation.
Traditional Studies
Delay between trial completion and availability of results.
Required
- Important to stakeholders
- Burden is low to moderate
- Sensitive to change
- Actionable
Additional
- Broadly applicable, has norms to interpret
- Low probability of harm
- Addresses public health goal(s)
- Related to theory or model
- “Maps” to “gold standard” metric or measure
Resources
The Importance of Pragmatic Clinical Trials
Russell Glasgow, PhD
Key Takeaways : Pragmatic Trials
Click To Reveal Flipcard Answer
What do Pragmatic Trials allow us to do?
Evaluate interventions under typical conditions, with typical patients vs. Under ideal conditions with a homogeneous set of selected patients (Explanatory Trials).
Some Key Aspects of Pragmatic Trials
-
Questions from and important to stakeholders;
-
Multiple heterogeneous settings
-
Diverse populations;
-
Comparison conditions are real world alternatives;
-
Multiple outcomes important to stakeholders and policy makers.
Other names for Pragmatic Trials
-
Large Simple Trials
-
Phase IIIb-IV Trials
-
Community-Based Research
-
Practical Trials
-
Naturalistic Clinical Trials
Key Reasons to Consider Pragmatic Trials
-
Traditional trials usually take a long time to translate into practice: it takes an average of 17 years for just 14% of research to translate into practice.
-
The conditions under which traditional trials are conducted are often not seen by practitioners as relevant to their practice or patient populations.
-
there is not enough evidence to inform clinical decisions.
Explanatory Clinical Trials
A specialized, randomized experiment in a specialized population under optimal conditions.
Which type of Trial is Better?
It is not that one type of trial is always better than the other, but rather that they answer different questions- about results under optimal (explanatory) versus typical (pragmatic) conditions. To advance science and application, we need answers to both.
References
The authors assess the growth in clinical research studies coupled with a slow rate of adoption, often taking an average of 17 years for new evidence-based findings to reach clinical practice. Some issues discussed include the need for a more efficient information infrastructure to better connect front-line professionals with the research community, difficulties in translating research into practice, and an inadequate system to help health care clinicians evaluate the strength of new study findings.
RATIONALE:
There is a pressing need for practical clinical trials (PCTs) that are more relevant to clinicians and decision-makers, but many are unaware of these trials. Furthermore, such trials can be challenging to conduct and to report.
OBJECTIVE:
The objective of this study was to build on the seminal paper by Tunis et al (Practical clinical trials. Increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003;290:1624-1632.) and to provide recommendations and examples of how practical clinical trials can be conducted and the results reported to enhance external validity without sacrificing internal validity.
KEY ISSUES:
We discuss evaluating practical intervention options, alternative research designs, representativeness of samples participating at both the patient and the setting/clinician level, and the need for multiple outcomes to address clinical and policy implications.
CONCLUSIONS:
We provide a set of specific recommendations for issues to be reported in PCTs to increase their relevance to clinicians and policymakers, and to help reduce the gap between research and practice.
Decision makers in health care are increasingly interested in using high quality scientific evidence to support clinical and health policy choices; however, the quality of available scientific evidence is often found to be inadequate. Reliable evidence is essential to improve health care quality and to support efficient use of limited resources. The widespread gaps in evidence based knowledge suggest that systematic flaws exist in the production of scientific evidence, in part because there is no consistent effort to conduct clinical trials designed to meet the needs of decision makers. Clinical trials for which the hypothesis and study design are developed specifically to answer the questions faced by decision makers are called pragmatic or practical clinical trials (PCTs). The characteristic features of PCTs are that they (1) select clinically relevant alternative interventions to compare, (2) include a diverse population of study participants, (3) recruit participants from heterogeneous practice settings, and (4) collect data on a broad range of health outcomes. The supply of PCTs is limited primarily because the major funders of clinical research, the National Institutes of Health and the medical products industry, do not focus on supporting such trials. Increasing the supply of PCTs will depend on the development of a mechanism to establish priorities for these studies, significant expansion of an infrastructure to conduct clinical research within the health care delivery system, more reliance on high-quality evidence by health care decision makers, and a substantial increase in public and private funding for these studies. For these changes to occur, clinical and health policy decisionmakers will need to become more involved in all aspects of clinical research, including priority setting, infrastructure development, and funding.
Background
The CONSORT statement is intended to improve reporting of randomised controlled trials and focuses on minimising the risk of bias (internal validity). The applicability of a trial’s results (generalisability or external validity) is also important, particularly for pragmatic trials. A pragmatic trial (a term first used in 1967 by Schwartz and Lellouch) can be broadly defined as a randomised controlled trial whose purpose is to inform decisions about practice. This extension of the CONSORT statement is intended to improve the reporting of such trials and focuses on applicability.
Methods
At two, two-day meetings held in Toronto in 2005 and 2008, we reviewed the CONSORT statement and its extensions, the literature on pragmatic trials and applicability, and our experiences in conducting pragmatic trials.
Recommendations
We recommend extending eight CONSORT checklist items for reporting of pragmatic trials: the background, participants, interventions, outcomes, sample size, blinding, participant flow, and generalisability of the findings. These extensions are presented, along with illustrative examples of reporting, and an explanation of each extension. Adherence to these reporting criteria will make it easier for decision makers to judge how applicable the results of randomised controlled trials are to their own conditions. Empirical studies are needed to ascertain the usefulness and comprehensiveness of these CONSORT checklist item extensions. In the meantime we recommend that those who support, conduct, and report pragmatic trials should use this extension of the CONSORT statement to facilitate the use of trial results in decisions about health care.