Focusing on microbehavior intervals yields a scalable, interpretable, and annotation-free framework for detecting psychological distress from nonverbal signals, achieving a macro F1-score of 0.75 and macro AUC of 0.80 in classifying four symptom profiles among health care workers.
Key Findings
Results
The deep learning classifier achieved robust performance in predicting four psychological distress symptom classes among health care workers using microbehavior interval features.
Macro F1-score of 0.75 on held-out data
Macro area under the receiver operating characteristic curve (AUC) of 0.80 on held-out data
Four symptom classes classified: 'moderate-severe burnout,' 'subthreshold-provisional PTSD,' 'burnout+PTSD,' and 'resilient'
Model was trained on features derived from microbehavior intervals extracted from semistructured interview recordings
Results
An average of 19.65 microbehavior intervals per interview were detected, each lasting an average of 1.31 seconds.
Mean microbehavior intervals per interview: 19.65 (SD 6.01)
Mean duration of each interval: 1.31 seconds (SD 1.10)
Intervals were isolated using an unsupervised anomaly detection model (MOMENT: a Family of Open Time-Series Foundation Models) without requiring manual labels
Analysis was conducted on 258 interview recordings from 151 HCWs
Results
Excluding gaze or arousal-valence signals caused the largest performance declines in the ablation analysis.
Ablation study systematically removed one behavioral data stream at a time
Gaze exclusion and arousal-valence signal exclusion produced the greatest drops in recall and F1-score
Other behavioral streams analyzed included facial expressions, head movement, body posture, and hand gestures
Results indicate gaze and arousal-valence are the most critical modalities for distress classification
Results
Explainability analysis revealed distinct temporal patterns across symptom classes, with irregularity and variability in microbehaviors emerging as key predictors.
Temporal irregularity and variability in microbehaviors were identified as the primary predictive features
Distinct patterns were observed across the four symptom classes: 'moderate-severe burnout,' 'subthreshold-provisional PTSD,' 'burnout+PTSD,' and 'resilient'
Explainability analysis was conducted to characterize which features drove model predictions
Findings support the interpretability of the annotation-free framework
Methods
The study analyzed interview recordings from 151 health care workers responding to five work-related, emotionally charged questions delivered via an online video platform.
258 interview recordings analyzed from 151 HCWs
Interviews were semistructured and included 5 emotion-eliciting, work-related questions
Recordings were conducted via Webex (online video platform)
Participants completed the Maslach Burnout Inventory General Survey 9-item (MBI-GS) for burnout and the PTSD Checklist for DSM-5 (PCL-5) for PTSD
Computer vision models generated time-series data of facial expressions, head movement, gaze, body posture, and hand gestures
Background
Health care workers face heightened risk for burnout and PTSD, but assessing psychological distress in this population is challenged by stigma, underreporting, and limitations of self-report tools.
HCWs face 'sustained psychological demands' placing them at heightened risk for burnout and PTSD
Stigma and underreporting were identified as barriers to conventional assessment
Self-report tools were noted to have inherent limitations
Nonverbal behaviors such as facial expressions and gaze were identified as holding 'diagnostic promise' but most approaches 'overlook the fine-grained, temporal fluctuations in these signals'
Conclusions
The multimodal microbehavior interval framework was described as scalable and annotation-free, offering a potential complement to conventional psychometric assessments.
The MOMENT unsupervised anomaly detection model isolated microbehavior intervals 'without requiring manual labels'
The approach moves 'from whole-video features to fine-grained multimodal temporal modeling'
Authors characterize it as enabling 'an objective, robust, and explainable assessment of psychological distress'
Described as 'a promising complement to conventional psychometric assessments'
What This Means
This research suggests that brief, involuntary changes in a person's facial expressions, eye movements, head movements, body posture, and hand gestures — called 'microbehavior intervals' — can be automatically detected from video-recorded interviews and used to identify different types of psychological distress in health care workers. The study recorded 258 interviews with 151 health care workers answering emotionally charged questions about their work, then used artificial intelligence to find these short behavioral fluctuations (averaging about 1.3 seconds each) without any human labeling. A deep learning model trained on features from these intervals was able to classify workers into four groups — those with burnout, those with PTSD symptoms, those with both, and those who were resilient — with meaningful accuracy (AUC of 0.80 and F1-score of 0.75).
The research also found that where someone looks (gaze) and signals related to emotional arousal and valence were the most important types of information for making accurate predictions. When these data streams were removed, the model's performance dropped more than when other signals were removed. The AI model was also designed to be explainable, meaning researchers could identify which specific behavioral patterns were linked to each distress category — with irregularity and variability in microbehaviors being the most telling signs.
This research suggests that automated video analysis of subtle, fleeting nonverbal behaviors could offer a way to assess mental health that does not rely solely on self-reported questionnaires, which health care workers may be reluctant to complete honestly due to stigma. Such a tool could potentially be used alongside standard psychological assessments to provide a more objective picture of distress, particularly in populations where underreporting is common. The system's ability to work without manual annotation makes it more practical to scale in real-world settings.
Gershov S, Hilberdink C, Zhao Y, Birnbaum S, Mueller V, Wall S, et al.. (2026). Detection of Microbehavior Intervals for Predicting Mental Health: Clinically Relevant and Advanced Multimodal Temporal Analysis.. Journal of medical Internet research. https://doi.org/10.2196/87049