Home

Dissemination and Implementation Study Designs

Learning Objectives:

To Demonstrate designs to achieve effectiveness of dissemination and implementation 

What is the Research Question?

(Draft text below)
Where is your research question in the research continuum? Although the diagram is somewhat distinct and linear it has overlapping concepts. There are very different purposes of efficacy, effectiveness, and D&I research as well as different purposes for phases within the D&I stage, which may require different research designs, yet designing for D&I needs to be thought of in the early stages of efficacy and effectiveness studies (as denoted by the dotted lines). 
Overarching efficacy and effectiveness questions are the basis for determining the level of evidence.
  1. Having established the desired level of efficacy/effectiveness the focus of the questions turn to the implementation and dissemination studies.
    a.Exploration phase focuses on identifying or expanding the communities of interest.
    b.Adoption/Preparation phase identifying factors and/or strategies to increase adoption.
    c.Implementation phase includes strategies for improving program fidelity in the field.
    d.Sustainability involves strategies to maintain or extend the use in communities and organizations.
    e.We are adding Demonstration/Dissemination which involve specific strategies to actively implement the program in usual care.

What is the Right Design?

Based on the level of evidence and the aims of the study, there are a number of different phases of assessing the D&I process, and D&I study designs which would appropriately suit your needs. In order to decide which is the "right" design, please review some of the various D&I designs indicated below. 

The stages of research and phases of dissemination and implementation

Chase is currently re-creating this figure for the platform.

PREINTERVENTION

/PRELIMINARY

Also known as a pilot study or preliminary study

STEPPED

WEDGE

Involving random and sequential crossover of clustered groups

PRAGMATIC TRIALS/CER

Evalutations under "real-world" conditions

DEMONSTRATION

/DISSEMINATION

SUBTEXT ON DEMO/DISSEM

RCT

Randomized Controlled Trial

  1. Efficacy
  2. Effectiveness
  3. Cluster Randomized

HYBRIDIZED

Mixed Methods studies combining Qualitative and Quantitative data

PRECIS-2

An Evaluation criteria: Pragmatic vs. Explanatory

Pre-intervention Studies 

"In developing a new implementation strategy, it is often very sensible to conduct a small-scale study to examine feasibility and assess sources of variation to determin the size one would need to have a fully powered study . . . "
The Consolidated Framework For Implementation Research (CFIR) offers an overarching typology to promote implementation theory development and verification about what works where and why across multiple contexts.
We will indicate where pre-intervention studies lie on the figure in the overview here. . . 

RCT - Randomized Controlled Trial 

FRAMEWORK

3

Randomized controlled trials

RCT's aim to determine whether a cause-effect relation exists between treatment and outcome, and are also used in assessing the cost effectiveness of a treatment. They are characterized by study group randomization and "double blind" design features. 

In D&I research 

Studies that involve multiple levels of random assignment or allocation are often useful in increasing the precision of inferences. 

wording on efficacy
Wording on Effectiveness
Wording on Cluster-Randomized

Stepped-Wedge Study Design

The stepped wedge cluster randomized trial

This is a research study design that is increasingly being used in the evaluation of service delivery type interventions. The design involves random and sequential crossover of clusters from control to intervention until all clusters are exposed.

More clusters are exposed to the intervention towards the end of the study than in its early stages, so sample size calculations and analysis must make allowance for both the clustered nature of the design and the confounding effect of time.

 

Should we keep the above? Should Chase Re-build it? Alternative suggestion?

Hybrid Study Designs

Mixed Method Designs

"Hybrid (mixed method) designs focus on collecting, analyzing and merging both quantitative and qualitative data into one or more studies.

The Central Premise of these designs is that the use of quantitative and qualitative approaches in combination provides a better understanding of research issues than either approach alone. "

The Consolidated Framework For Implementation Research (CFIR) offers an overarching typology to promote implementation theory development and verification about what works where and why across multiple contexts.
We will indicate "where" hybrid design lies on the figure in the overview here. . . 

Pragmatic Trials

FRAMEWORK

3

Considerations for Pragmatic Trials

  • Pragmatic Trials allow us to evaluate interventions under typical conditions, with typical patients vs. under ideal conditions with a homogeneous set of selected patients (Explanatory Trials).

  • Traditional trials usually take a long time to translate into practice; it takes an average of 17 years for just 14% of research to translate into practice. 

  • The conditions under which traditional trials are conducted are often not seen by practitioners as relevant to their practice or patient populations. 

  • Although there are over 18,000 randomized controlled trials (RCT's) published each year, the vast majority of systematic reviews conclude that there is not enough evidence to inform clinical decisions. 

See the e-book on Pragmatic Trials, developed by the same team that created this e-book.

PRECIS Evaluation Criteria

Is yours a pragmatic study, or an explanatory study?

No study is completely pragmatic, nor is it completely explanatory. Rather, each study is somewhere along the continuum, having characteristics and degrees of each study type. 

 

The Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) was developed by Thorpe and colleagues to measure where a given study might fall. It can be applied to illustrate the degree to which a trial is pragmatic or explanatory. It used ten domains plotted on a “spoke-and-wheel” diagram. Since then, it has been revised to feature 9 domains related to pragmatic trials. 

 

To Learn more about PRECIS-2 and use our interactive PRECIS-2 Wheel;

See our e-book on Pragmatic Trials

Dissemination/Demonstration 

FRAMEWORK

3

Dissemination/Demonstration Studies

Need some text for this - References, etc.?

Examples in practice: D&I Study Designs

- To be dissolved into other sections

Case Example : Mixed Methods Multiple Case Study

Context:

Improving quality in children’s mental health and social service settings.

 

Objective:

This study is designed to inform efforts to develop more effective implementation strategies by fully describing the implementation experiences of a sample of community-based organizations that provide mental health services to youth in one Midwestern city.

Evaluation Design:

A mixed methods multiple case study of seven children’s social service organizations in one Midwestern city in the United States (the control group of a larger randomized controlled trial). Qualitative data included semi-structured interviews with organizational leaders and a review of documents (e.g., implementation and quality improvement plans, program manuals, etc.) to understand implementation decision-making and specific implementation strategies that are used to implement new programs and practices. Focus groups with clinicians explore their perceptions of a range of implementation strategies.

This qualitative work informs the development of a Web-based survey that will assess the perceived effectiveness, relative importance, acceptability, feasibility, and appropriateness of implementation strategies from the perspective of both clinicians and organizational leaders.

The Organizational Social Context measure will be used to assess organizational culture and climate.
NEXT →
Source:
Powell BJ, Proctor EK, Glisson CA, Kohl PL, Raghavan R, Brownson RC, Stoner BP, Carpenter CR, Palinkas LA. A mixed methods multiple case study of implementation as usual in children’s social service organizations: study protocol. Implement Sci. 2013; 8: 92.

Case Example : Pragmatic implementation trial in primary care

Context:

Understanding patient-centered health behavior and psychosocial issues in primary care.

 

Objective:

Our goal is to design a scientifically rigorous and valid pragmatic trial to test whether primary care practices can systematically implement the collection of patient-reported information and provide patients needed advice, goal setting, and counseling in response.

Evaluation Design:

A cluster randomized delayed intervention trial, of the My Own Health Report (MOHR) study. Nine pairs of diverse primary care practices are randomized to early or delayed intervention four months later. The intervention consists of fielding the MOHR assessment and subsequent provision of needed counseling and support for patients presenting for wellness or chronic care.

Stakeholder groups are engaged throughout the study design to account for local resources and characteristics.

Study outcomes include the intervention reach (percent of patients offered and completing the MOHR assessment), effectiveness (patients reporting being asked about topics, setting change goals, and receiving assistance in early versus delayed intervention practices), contextual factors influencing outcomes, and intervention costs.
NEXT →
Source:
Krist AH, Glenn BA, Glasgow RE, Balasubramanian BA, Chambers DA, Fernandez ME, Heurtin-Roberts S,Kessler R, Ory MG, Phillips SM, Ritzwoller DP, Roby DH, Rodriguez HP, Sabo RT, Sheinfeld Gorin SN,Stange KC; MOHR Study Group. Designing a valid randomized pragmatic primary care implementation trial: the my own health report (MOHR) project. Implement Sci. 2013;8:73.

Case Example : Implementation in a VA setting

Context:

Heart failure is the primary reason for discharge from the VA medical service. Furthermore, the readmission rate is high.

The Hospital to Home (H2H) Excellence in Transitions Initiative is a new national campaign to reduce preventable readmissions for patients recently hospitalized with a cardiovascular condition (www.H2Hquality.org).

 

Objective:

  1. To determine if VA facility enrollment in H2H results in improved care for VA patients with heart failure.
  2. To determine barriers and facilitators to a) enrolling facilities in H2H, and for those facilities enrolled, b) adopting the H2H interventions.
  3. To evaluate the use of the VA Heart Failure Network to aid in implementing the H2H initiative in a randomized trial. (HF Network

Implementation Design:

A 122 VA facilities with <100 discharges were randomized into intervention and control groups. From Month 1 through month 6 the implementation of the VA H2H initiative was facilitated for all the intervention facilities.

All the intervention facilities were asked to participate by (1) enrolling their facility at the H2H website as commitment to, and (2) initiating projects based on the VA H2H initiative. In Month 6 surveys were sent to both the intervention and control facilities to assess participation in the VA H2H initiative. From Month 7 to Month 12 the VA H2H initiative is being facilitated at all remaining 61 control facilities.
NEXT →
Source: Beck A, Bergman DA, Rahm AK, Dearing JW, Glasgow RE. Using Implementation and Dissemination Concepts to Spread 21st-century Well-Child Care at a Health Maintenance Organization Perm J. 2009 Summer; 13(3): 10–18.

Case Example : Implementation Design In HMO

Context:

Implementing and disseminating an evidence-based model of well-child care (WCC) that includes developmental and preventive services recommended by the American Academy of Pediatrics. 

Implementation Design:

Twenty-first Century WCC is a parent-centered, team-based, primary care model that combines online pre-visit assessments—completed by parents and caregivers—with vaccinations and anticipatory guidance.
Nurses, nurse practitioners, developmental specialists, and pediatricians all play roles in the WCC model.
Patient and clinician interaction, health records, and resources are all facilitated through a Web-based diagnostic, management, tracking, and resource information tool.
Unlike innovations that are embedded only in technical systems, validated models of team-based health care have multiple components that must be made compatible with complex sociotechnical systems. Interpersonal communication, work, coordination, and judgment are key processes that affect implementation quality.  Implementation can involve tailoring to a particular site and customizing either the model or the organizational context to accommodate it.
BEGINNING →
Source: Beck A, Bergman DA, Rahm AK, Dearing JW, Glasgow RE. Using Implementation and Dissemination Concepts to Spread 21st-century Well-Child Care at a Health Maintenance Organization Perm J. 2009 Summer; 13(3): 10–18.

Resources

​​​Boot Camp Translation for Patient-centered Outcomes

Don Nease, MD
 Slides Video

Lessons learned in engaging Practice-Based Research Networks (PBRNs) and their patients Community and Stakeholder Engagement.

Digital Media in Healthcare: Reaching Patients Anytime, Anywhere, Any Platform​​

Patrick Duparcq PhD
 Slides AudioFiles

A brief overview of the “State of Digital Engagement” including current technology trends and how future technologies could be evaluated from a management perspective.

Text Messaging in Healthcare

Lisa Schilling, MD, MSPH
 Slides Video

​An overview of SMS technology basics, regulations, intervention development, tools, and the use of text messaging in healthcare.

Message Tailoring

Amanda Dempsey, MD, PhD, MPH
Slides Video

An overview of targeting specific audiences with tailoring and personalization from lessons learned at University of Michigan Child Health Evaluation and Research Unit.

AHRQ Effective Healthcare Program

Tools and resources to help consumers, clinicians, policymakers, and others make informed health care decisions.
Click Here For Source

CDC Tools for Community Action

Evidence-based recommendations and interventions and policies that improve health and prevent disease in communities.
Click Here For Source

AHRQ Healthcare Innovations Exchange

Innovations and tools to improve quality and reduce disparities.
Click Here For Source

Key Takeaways

Click To Reveal Flipcard Answer

Before Addressing an Issue, one should know “the root of the problem.” What does this mean?

Know “what” is occurring and “why” before addressing an issue!

What are the four considerations for D&I planning?

Evidence, Audience, Engagement and Translation

Evidence: Questions to ask

  • Is the evidence for intervening compelling?
  • Has efficacy been demonstrated?
  • Has effectiveness been demonstrated?

Audience: Questions to ask

  • Who is the primary audience? Do you understand their attitudes/beliefs/current practices?
  • What is the setting?
  • Are there other stakeholders affected? Where do they stand on it? (for, against, neutral)

Engagement: Questions to ask

  • What kind of partnerships should be developed?
  • What communication norms do your audience prefer?
  • Be prepared and receptive to a re-framing of intervention/study questions and D&I strategies; it is iterative!
  • Remember stakeholder involvement enhances dissemination of the findings.

Translation: Questions to ask

  • Are there relevant implementation toolkits that can be used or adapted?
  • Remember: Multilevel approaches are better than single-prong strategies for successful translation.

What 4 elements are in the Consolidated Framework for Implementation Research?

  • Characteristics of intervention
  • Inner/outer settings
  • Characteristics of individuals
  • Characteristics of the process

Checklist To Get Started

  • What am I disseminating and implementing? Is it worthy of broad-scale adoption?

  • Who are my target audience(s) for behavior change? Do I understand their knowledge, attitudes, beliefs, and behaviors about what I want to disseminate and implement?

  • Who are other stakeholders affected by the implementation? What are their opinions – are they in favor, against, or neutral? How might this influence adoption among my target audience? How should I address this?

  • How does my target audience view what I want to disseminate and implement? Can I frame what I want to disseminate and implement so the value-added benefit is clear and persuasive?

  • What are potential barriers to adoption that I should address in the design of my translation plan? What are potential facilitators that I can leverage?

Read Chapter 4