Background and Purpose
Modified Delphi methods are increasingly used to develop healthcare pathways with input from people with lived experience (PWLE) and clinicians/others. However, guidance on consensus analysis in this context remains limited. We examined consensus outcomes across different scoring methods and criteria when participants were treated as a single combined group (Objective 1) versus two distinct groups (Objective 2).
Methods
We conducted a secondary analysis of Round 1 data from a project involving PWLE (N=8) and clinicians/others (N=51). To assess agreement on 68 Delphi statements, we applied three methods for scoring percentage agreement that differed in how the middle response on a three-point Likert scale (“approve”, “not sure either way”, “do not approve”) was treated. Method 1 excluded the middle response, methods 2 and 3 grouped the middle response with “do not approve”, and “approve”, respectively. We compared consensus rates (% of items reaching consensus) using percentage agreement cutoffs of =70%, =80%, and =90% of participants.
Results
Consensus results varied by participants grouping, treatment of middle response categories, and cutoff criteria. Results from the combined group of PWLE and clinicians/others provided a simplified overview consensus outcome. Treating the participants into as separate groups provide nuanced results.
Conclusion
The analysis of data can change the results from which to draw conclusions and inform practice. Investigators should consider the alignment of each approach with the goals of their Delphi study.
Marlo Salum, Kara Schick-Makaroff, Richard Sawatzky
Disease Category: Methodological & diagnostic
Disease Name: N/A
Age Range: 0 - 120
Sex: Either
Nature of Intervention: Any
- Administrative experts
- Clinical experts
- Consumers (patients)
- Researchers
- COS methods research
- Consensus meeting
- Delphi process