Real-world use of a purpose-built mental health conversational AI tool (Ebb) showed strong engagement across 393,969 members, with product evolution to version 2.0 leading to increased retention and higher positive conversation ratings, while members emphasized the need for data safety, clinical guidelines, and the tool as a supplement rather than replacement for human care.
Key Findings
Results
Overall attitudes toward AI among Headspace members remained neutral despite majority use of general AI tools.
Cross-sectional survey included n=482 Headspace members assessing demographics, AI use, and the Artificial Intelligence Attitude Scale-4 (AIAS-4)
AIAS-4 mean score was 5.7 (SD 2.2, range 1-10), indicating neutral overall attitudes toward AI
The majority of survey respondents used and would continue to use general AI tools despite neutral attitudes
Results
CAI tool 2.0 showed substantially higher 7-day retention compared to CAI tool 1.0.
CAI tool 2.0: 77,894/153,249 (50.8%) completed 2 sessions within 7 days
CAI tool 1.0: 68,701/240,720 (28.5%) completed 2 sessions within 7 days
CAI tool 2.0 featured enhanced LLM conversational prompts, comprehensive memory, woven content recommendations, and more robust safety detection compared to 1.0
Results
CAI tool 2.0 received higher positive conversation ratings than CAI tool 1.0.
CAI tool 2.0: 37,819/40,449 (93.5%) positive conversation ratings
CAI tool 1.0: 94,308/104,323 (90.4%) positive conversation ratings
Results
Retained CAI tool 2.0 users showed greater session engagement compared to all CAI tool 2.0 users and CAI tool 1.0 users.
Retained CAI tool 2.0 users averaged 6.1 sessions per user
All CAI tool 2.0 users averaged 2.9 sessions per user
CAI tool 1.0 users averaged 2.4 sessions per user
Results
Real-world CAI tool use showed strong engagement across a large user base of 393,969 Headspace members.
Engagement descriptive analysis assessed session and message counts, retention, and conversation themes
Total sample of 393,969 Headspace members was analyzed for real-world engagement
This was a descriptive analysis of real-world use rather than a controlled trial
Results
Survey respondents viewed the CAI tool primarily as a guide to navigate mental health resources and provide in-the-moment support.
Members saw the CAI tool as a guide to navigate to mental health resources and Headspace content
Members emphasized the need for data safety and ethics transparency
Members indicated the CAI tool should be a resource in addition to human-delivered mental health care, not a replacement
Members emphasized the need for clinical guidelines structure
Results
Diary study participants envisioned using the CAI tool during specific emotional states and daily routine moments.
Diary study included n=15 Headspace members exploring the CAI tool's role within members' mental health journey
Members imagined using the CAI tool when feeling stress or anxiety
Anticipated use times included morning routines, commutes, or while winding down at night
Conclusions
The study identified three minimum viable safeguards necessary for purpose-built mental health AI products.
Safeguard 1: Transparent labeling of intended use, benefits, and limitations
Safeguard 2: Safety by design principles to monitor for overuse, detect risk, and flag needs for escalation
Safeguard 3: Child and adolescent safeguards
The authors contrasted purpose-built tools against general AI tools which lack safety guardrails, evidence-based practices, and medical regulation compliance
Background
The CAI tool (Ebb) was purpose-built using motivational interviewing techniques with clinically backed safety mechanisms.
Ebb was built by clinical psychologists and research experts
The tool uses motivational interviewing techniques for subclinical guidance
It incorporates clinically backed safety mechanisms
The tool is designed for subclinical guidance rather than clinical treatment
Callahan C, Tanner L, Coe C, Davis M, Glover J, Bernstein E, et al.. (2026). Real-World Use of a Mental Health AI Companion: Multiple Methods Study.. JMIR formative research. https://doi.org/10.2196/86904