An online factorial survey experiment using clinical vignettes demonstrated high test-retest reliability and preliminary validity for assessing how patient demographics and clinical factors influence ED clinicians' decisions to pursue pulmonary embolism diagnostic testing.
Visual analog scale scores for perceived PE risk demonstrated high test-retest reliability in the physician sample.
Pilot testing at a large national conference of ED clinicians showed high response and completion rates with short completion time.
Pilot participants reported higher perceived PE risk in vignettes with intermediate-risk rather than low-risk revised Geneva scores, providing early support for criterion validity.
The factorial survey experiment was designed with a 2×2×2 factorial design incorporating patient gender, race, and a case-specific clinical factor across 7 vignettes.
Cognitive debriefing interviews supported the face validity and response process validity of the factorial survey instrument.
The study identified a gap in empirical literature on how clinicians decide whether to suspect PE, despite existing evidence-based diagnostic testing guidelines.
This research describes the design and early testing of an online survey tool aimed at understanding how emergency physicians decide whether to investigate a patient for pulmonary embolism (PE), a potentially life-threatening blood clot in the lungs. The tool uses written patient scenarios (vignettes) where details like patient gender, race, and specific clinical findings are systematically varied, allowing researchers to measure whether these factors — including potentially biasing ones like race — influence physician decision-making. Doctors were not told the study was examining these specific variables, which helps reduce the chance they would change their behavior because they were being watched. Early testing showed the tool works well: physicians gave consistent answers when completing the survey twice (test-retest reliability of r=0.85), and they correctly identified higher-risk scenarios as more concerning for PE, suggesting the tool measures what it is intended to measure. Among 61 emergency physicians from 18 states, participation was high (68% response rate, 95% completion rate), and the survey took about 8 minutes to complete, suggesting it is practical for large-scale use. Interviews with participants also confirmed that the vignettes seemed realistic and that doctors were interpreting the questions as intended. This research matters because, while there are established guidelines for how to test for PE once a physician suspects it, very little is known about what leads a physician to suspect PE in the first place — and whether patient characteristics like race or gender inappropriately influence that suspicion. This validated tool could be used in future large-scale studies to detect potential disparities in PE evaluation across different patient groups, which could ultimately help improve equitable care for patients with cardiovascular symptoms in emergency settings.
Maughan B, Jarman A, Westafer L, Holmes J, Kline J, Casey S, et al.. (2026). Designing a web-based factorial survey experiment on diagnostic evaluation of cardiovascular symptoms in emergency care.. Research and practice in thrombosis and haemostasis. https://doi.org/10.1016/j.rpth.2026.106791