Nicole Drapeau Gillen argues that AI chatbot safety measures currently being developed focus too narrowly on general populations and fail to address risks for individuals with serious mental illness such as psychosis. She contends that chatbot design features, including endless patience and validation without “beneficial friction,” pose a particular danger to people with active psychosis by potentially reinforcing delusions. Gillen states that current safety interventions, including crisis resource referrals and self-harm detection, were not designed with this vulnerable population in mind, representing a clinical gap in AI mental health safeguards.
