Finding and selecting options to address a problem

This type of question aims to create a list of potentially available interventions to address a given problem. This type of question commonly includes a judgement of what options are potentially suitable for a specific contextual reality.

Some examples of this type of question are:

• What alternatives are available to treat patients with multi-drug resistant tuberculosis?
• What population-wide interventions are available to reduce people’s sugar consumption?
• What system arrangements could be changed to reduce the surgical backlog created during the acute phase of the COVID-19 pandemic?

Methodological approaches to address question A1

• Review to find options that have been used by other studies (e.g., scoping review)
• Jurisdictional scan (comparative analysis) to understand what options have been implemented by other jurisdictions
• Cross-sectional study (survey, point-in-time or snapshot study or analysis) of people’s opinions
• Ecological study (population-based study, including spatial analysis)

This type of question aims to describe what does entail to develop a given option, including its mechanism of action (or causal pathway, if applicable), and how and why it should work to address a problem.

An important distinction should be made with an option that has been already implemented, and someone may want to know why it has not had the results that should have had. This issue is addressed in stage 4 (Monitoring implementation and evaluating impact).

Some examples of this type of question are:

• How does metformin work to reduce blood glucose levels?
• How increasing the price of sugar-sweetened beverages could reduce the prevalence of obesity in a given country?
• How would the school payment mechanisms benefit teachers?

Methodological approaches to address question A2

• Randomized-controlled study (randomized experiment or randomized trial) measuring intermediate outcomes
• Review to identify existing frameworks (conceptual analysis) that explain how an intervention might work
• Interrupted time-series analysis (including joint-point regression) measuring intermediate outcomes

This type of question aims to assess whether a given option is feasible to be implemented in a given context or setting, which could be split into different dimensions (e.g., operationally feasible, legally feasible, etc.).

Some examples of this type of question are:

• How feasible is it to implement curfews to reduce COVID-19 transmission?
• What is the level of feasibility of reducing the number of hours in of training for teachers?
• How feasible is it for a patient to access an emergency room within 7 hours of having a stroke?

Methodological approaches to address question B1

• Delphi studies (to get consensus from experts)
• Jurisdictional scan (comparative analysis) to understand the feasibility of the option elsewhere
• Discrete choice experiment (stated preferences)
• Modelling to predict whether the option will be feasible (e.g., system dynamics, ARIMA models, etc.)

This type of question aims to assess the benefits and the early-occurring harms of an option to address a problem or its causes, by measuring outcomes, the effect size, and its variability.

Some examples of this type of question are:

• What is the clinical efficacy of using remdesivir to treat COVID-19 patients?
• What are the benefits of mask mandates to reduce COVID-19 transmission?
• What are the benefits of the remote monitoring of chronic patients?

Methodological approaches to address question B2

• Randomized-controlled study (randomized experiment or randomized trial)
• Controlled before-and-after study of aggregated data (including difference-in-differences study and non-equivalent control group designs)
• Interrupted time-series analysis (including joint-point regression)
• Retrospective cohort study of individual-level data (retrospective or historical longitudinal, or panel study)

This type of question aims to identify late-occurring harms and assess their probability of occurrence (i.e., risk). In case the benefits of an option are also available at the same time, this type of question might also entail assessing whether the benefits outweigh the harms (i.e., net benefit).

Some examples of this type of question are:

• What are the late-occurring harms of using HEPA filters in a classroom? 
• Do the benefits of using convalescent plasma outweigh the late-occurring harms in treating COVID-19 patients?
• Does building homeless shelters creates unintended effects, such as affecting the housing market?

Methodological approaches to address question B3

• Retrospective cohort study of individual-level data (retrospective or historical longitudinal, or panel study), including databases of adverse event reporting (e.g., pharmacovigilance)
• Randomized-controlled study (randomized experiment or randomized trial)
• Prospective cohort study of individual-level data (prospective longitudinal or panel study)
• Case-control study (case-comparison study)

This type of question aims to assess the potential costs and resource use that implementing a given option will create depending on the specific context and setting in which the option is planned to be implemented. Resources could be monetary (i.e., costs), but could also be human resources, technology, etc.

Some examples of this type of question are:

• How much would a rare disease treatment cost for a family?
• What are the costs of procuring masks for all hospitals?
• What is the budget impact of implementing universal health insurance in a given country?

Methodological approaches to address question B4

• Modelling to estimate the cost of an option
• Jurisdictional scan (comparative analysis) to understand the costs in other jurisdictions
• Cross-sectional study (survey, point-in-time or snapshot study or analysis) of data collected for this purpose (i.e., primary data)
  

This type of question aims to assess how efficient an investment of resources is, by comparing the costs of an option against the balance between benefits and harms. In this context, we already have credible evidence of the effectiveness of the option.

Some examples of this type of question are:

• What is the incremental cost-effectiveness ratio of covering trastuzumab to treat advanced-stage breast cancer?
• How economically efficient is it to invest resources in preventing diabetes?
• How economically efficient is it to procure laptops for all students to improve their learning outcomes?

Methodological approaches to address question B5

• Economic evaluations (cost-effectiveness, cost-utility, cost-benefit analyses)
• Jurisdictional scan (comparative analysis) to understand whether the option was efficient in other jurisdictions 
• Delphi studies (to get consensus from experts)

This type of question aims to understand the equity, ethical and human rights implications of a given option. On one side, implementing an option might have a differential impact (measured as any of the other outcomes included in this goal) in some population groups. On the other side, some options could have ethical and human rights implications when implemented that might arise when finding and selecting options.

Some examples of this type of question are:

• What are the equity implications of lockdowns to prevent COVID-19 transmission?
• How human rights could be affected by implementing identity controls to prevent crime?
• What are the ethical implications of using plasma transfusion in a patient against the use of this technology due to religious beliefs?

Methodological approaches to address question B6

• Delphi studies (to get consensus from experts)
• Cross-sectional study (survey, point-in-time or snapshot study or analysis) of people’s experiences (not asking about hypothetical scenarios)
• Qualitative deductive (from general to particular i.e., testing theory) methods to describe/critically analyze a phenomenon (e.g., qualitative case studies)t
• Qualitative deductive (from general to particular i.e., testing theory) methods to describe a phenomenon (e.g., qualitative description, narrative approaches)

This type of question aims to measure the level of acceptability that a given option would have in a concrete setting, and to what extent a given group is willing to receive an intervention.

Some examples of this type of question are:

• "What are the knowledge, attitudes and preferences of Aboriginal adolescents about sexual and reproductive health services?
• What data is there (if any) on the cultural acceptability of these (or other) approaches?

Methodological approaches to address question B7

• Discrete choice experiment (stated preferences)
• Qualitative deductive (from general to particular i.e., testing theory) methods to describe/critically analyze a phenomenon (e.g., qualitative case studies)
• Qualitative deductive (from general to particular i.e., testing theory) methods to describe a phenomenon (e.g., qualitative description, narrative approaches)t
• Cross-sectional study (survey, point-in-time or snapshot study or analysis) of people’s experiences (not asking about hypothetical scenarios)

This type of question aims to evaluate whether adjusting some variables (e.g., the deliverer, the intensity of the intervention, etc.) could modify the expected impact (measured as any of the questions provided in Stage 2, Goal B) of an option.

Some examples of this type of question are:

• How does the effect of corticosteroids for treating severe COVID-19 changes with its dosage?
• How the COVID-19 transmission risk varies when changing the specific distance required to promote social distancing?
• How does a company profits change when their employees are paid by performance using process vs result indicators?

Methodological approaches to address question C1

No consensus has yet been reached on what methodological approaches are most suitable to address this question. Learn more about our approach to reach consensus.

This type of question aims to explore what setting or socioecological contexts, and/or in what population groups the intervention would produce most impact (measured as any of the questions provided in Stage 2, Goal B, which includes in what population the intervention would achieve most equitable results).

Some examples of this type of question are:

• For what type of breast cancer patients, radiotherapy would be most effective?
• What population group would be the greatest benefit from a reform to police forces?
• In what settings the promotion of nurse practitioners to undertake some physician-tasks would produce the highest net benefit?"

Methodological approaches to address question C2

• Case-control study (case-comparison study)
• Prospective cohort study of individual-level data (prospective longitudinal or panel study)
• Controlled before-and-after study of aggregated data (including difference-in-differences study and non-equivalent control group designs)
• Randomized-controlled study (randomized experiment or randomized trial) using subgroup comparisons.

This type of question aims to find the right combination of interventions that would maximize the expected impacts (using any or a combination of the impacts described in Stage 2 Goal B).

Some examples of this type of question are:

• What are the best drug combinations to treat a patient with stage IV breast cancer?
• What combination of public health policies is most effective to reduce infant mortality?
• What interventions need to be combined to reduce the homeless population in a given city?

Methodological approaches to address question D1

• Evidence synthesis of studies evaluating the impact of single interventions to analyze the combined effect of packages. 
• Randomized-controlled study (randomized experiment or randomized trial) to compare packages of interventions in different arms
• Delphi study (to get consensus from experts)

This type of question aims to create a ranking of interventions based on their expected impact (measured as any of the outcomes included in Stage 2, Goal B).

Some examples of this type of question are:

• What are the five most effective treatment courses for patients with depression?
• What are the five most effective strategies to reduce childhood obesity?
• What are the five most effective strategies to reduce emergency room waiting times?

Methodological approaches to address question D2

• Economic evaluations (cost-effectiveness, cost-utility, cost-benefit analyses) to create a ranked list of options
• Ranking type Delphi study (to get consensus from experts)
• Multi-criteria (objective) decision analysis to create a ranked list of options
• Discrete choice experiment (stated preferences)