Apparent correlations between psychiatric traits and metacognitive biases (decision confidence) in large-scale online studies may be largely due to surface-level questionnaire-filling behaviors: systematic rating biases and inattentive responding.
Key Findings
Results
Systematic rating biases in self-report scales can generalize to confidence ratings, producing spurious correlations between psychiatric traits and decision confidence.
A bias to report positive or negative values in self-report scales may generalize to confidence ratings, creating artifactual associations.
The previously identified transdiagnostic dimension of 'anxiety-depression' was found to map onto systematic rating biases in questionnaire-filling behavior.
This mechanism can produce what the authors call 'a mirage of apparent metacognitive alterations in mental health.'
Results
Inattentive responding produces systematic overconfidence, generating spurious positive correlations between decision confidence and endorsement of rare symptoms.
Inattentive responders show systematic overconfidence, which correlates with elevated endorsement of rare psychiatric symptoms.
The previously identified transdiagnostic dimension of 'compulsivity and intrusive thought,' shown in prior work to correlate with decision confidence, maps onto inattentive responding behavior.
Because rare symptoms are endorsed more often by inattentive participants who are also overconfident, a spurious positive correlation between confidence and compulsivity-related traits emerges.
Results
Two previously identified transdiagnostic psychiatric dimensions—'anxiety-depression' and 'compulsivity and intrusive thought'—map neatly onto rating bias and inattentive responding, respectively.
Both transdiagnostic dimensions had been previously reported in large-scale online studies as correlating with decision confidence.
Analysis of existing and new large-scale datasets revealed these dimensions correspond to surface-level questionnaire-filling behaviors rather than distinct psychiatric constructs.
The authors analyzed both existing datasets and collected new large-scale data to test these patterns.
Results
In a preregistered experiment, decision confidence and self-reported obsessive-compulsive tendencies were correlated with independent measures of inattentive and biased responding.
The experiment was preregistered, adding confirmatory weight to the findings.
Independent measures of inattentive responding and rating bias were used to assess their relationship with both decision confidence and obsessive-compulsive tendencies.
Obsessive-compulsive tendencies specifically correlated with inattentive responding as measured independently.
Decision confidence also showed correlation with independent measures of these questionnaire-filling behaviors.
Discussion
Both self-report psychiatric measures and confidence ratings are substantially influenced by inattentive and biased responding when these factors are not accounted for.
The influence of inattentive and biased responding was found to be 'substantial' across both types of measures.
Failure to control for these factors can produce spurious metacognitive findings in mental health research.
The authors call for 'concrete precautionary measures' to control for these biases in future research.
The findings apply to large-scale online studies with healthy adults, which are a common methodology in transdiagnostic psychiatric research.
What This Means
This research suggests that widely reported links between mental health traits and how confident people are in their decisions may be statistical artifacts rather than genuine psychological phenomena. When people fill out online surveys, some tend to use only one end of rating scales (always answering high or always answering low), and this same tendency may carry over into confidence ratings during decision-making tasks—creating a false appearance that certain psychiatric traits are linked to overconfidence or underconfidence. Separately, people who are not paying careful attention while completing surveys tend to randomly endorse items, which means they are more likely to accidentally endorse rare symptom questions (inflating scores on things like obsessive-compulsive tendencies) while also appearing overconfident on decision tasks.
The study examined both previously collected large-scale datasets and new data, and conducted a preregistered experiment to test these ideas. The researchers found that two major transdiagnostic psychiatric dimensions that had been previously linked to decision confidence in the scientific literature—one related to anxiety and depression, and one related to compulsivity and intrusive thoughts—closely matched patterns expected from rating bias and inattentive survey completion, respectively. This suggests that what appeared to be meaningful differences in metacognition (how people monitor their own thinking) across psychiatric traits may largely reflect how people behave when filling out questionnaires.
This matters because a growing body of research has used online surveys combined with decision-making tasks to make claims about how mental health conditions affect self-awareness and confidence. This research suggests those conclusions need to be revisited with appropriate controls for response style and attention. The authors recommend that future studies include independent checks for inattentive responding and rating biases before drawing conclusions about metacognitive differences in mental health populations.
Sarna N, Dar R, Mazor M. (2026). Biased and inattentive responding contribute to apparent metacognitive biases in mental health.. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.2520832123