AI therapy chatbots can’t replace therapists—but they can cut paperwork, train clinicians, and flag risk. Assistive uses help people; autonomy invites harm.
getty
Therapy faces a supply problem. Nearly half of U.S. adults with mental illness do not receive treatment due to various factors like cost, coverage, and capacity. The gap in meeting this demand, along with the potential for scalability, makes AI therapists appealing to venture capital firms and patients. However, caution is advised due to safety risks highlighted in recent research from Stanford.
As a healthcare advisor, I have witnessed the consequences of AI providing confident answers without proper medical context. A published case documented a patient suffering from bromide poisoning and psychosis after following advice from a chatbot, emphasizing the potential risks involved.
Recent actions by states like Illinois, Nevada, and Utah to restrict AI for mental-health therapy without licensed professionals underscore the perceived risks and over-hype in the market.
Key Findings from Stanford Study
- Stigma in Outputs: Chatbots demonstrated higher levels of stigma towards certain conditions, indicating a lack of equitable care.
- Safety Concerns: Some bots failed to recognize suicidal ideation, leading to harmful outcomes.
- Model Behavior: Larger models did not necessarily exhibit improved behavior, with newer LLMs showing similar problematic patterns.
Effective Uses of AI
A well-implemented AI should support, not replace, human therapists. It can streamline administrative tasks, assist in training clinicians, and provide low-risk self-support options for users. It is crucial to ensure clear disclaimers and immediate human intervention when necessary.
Guidelines for Employers and Investors
When considering AI therapy tools, prioritize assistive functions over autonomous decision-making. Clearly define the clinical scope of the tool, require human oversight for any risk-related features, and emphasize safety measures to avoid potential harm. Market the tool accurately to align with regulatory standards.
The Role of AI in Therapy
AI can enhance efficiency by handling administrative tasks, allowing therapists to focus on building relationships with patients. While AI can simulate support, it cannot replace the emotional connection and transformation that occurs in human interactions. Thoughtful integration of AI can amplify the care provided by clinicians.
