Why the Risks of Using AI Are Particularly High in Mental Health
What AI can do well is generate language, identify patterns, and process large amounts of information very quickly. AI can make simple, surface-level statements that resemble empathy, but it cannot engage in complex emotional attunement. It cannot perform tasks requiring clinical judgment, nor can it engage in real-time crisis management. AI is often incapable of recognizing when someone is in crisis; it cannot recognize when to call for professional human help, and at times has provided dangerous advice.
Other concerns about AI use in mental health concern the variability in quality. Some tools are carefully developed based on evidence-based practices and are transparent about their limitations. The best tools are often meant to be used as a supplement to therapy. It may be challenging for users to judge the quality of a tool or know when they should seek the help of a therapist rather than a self-guided app.
Due to the fast-growing nature of AI technology and products entering the public sphere, the laws that govern people’s private health information have not caught up with the output of AI tools, which gives rise to serious privacy and data use concerns. People may share deeply personal information with AI systems without knowing how that data is stored, analysed, or potentially sold.
One of the most concerning risks of AI tools in mental health is the potential for inappropriate or misleading responses. AI tools can reinforce harmful beliefs, delusional thoughts, and encourage dangerous behaviours simply because they are designed to continue and affirm conversations rather than challenge them. They cannot read body language or listen to tone of voice. They are not emotionally connected to people and do not notice the subtle nuances of human behaviour. In other words, they can miss a lot of what happens between therapist and client in the therapy room. This lack of relational attunement and skill can become dangerous when someone is in or on the verge of a mental health crisis.
A Lived Experience Perspective
Carol, a client who experimented with an AI mental health chatbot described her experience to her therapist this way:
“At first, AI gave me better advice about how to approach a problem I was having than any human I had talked to about that problem. However, when I started sharing my feelings in more depth, all of the “empathic” responses felt similar and robotic. Also, I started noticing that AI affirmed and supported me no matter what I said. Nothing was ever my “fault” or my responsibility. After a while, that felt really fake and unrealistic.”
This led to a deeper conversation in therapy that helped Carol reflect on the frictionless experience that AI gave her, and to understand more about how even the ‘messiness’ of human interaction can lead to deeper, more connected relationships in her life.
AI can sound supportive. However, it is not clinically informed like a therapist. It is not ethically accountable. It has no feelings and does not genuinely care for the person in pain sitting at the computer.
Setting Realistic Expectations
AI can be a useful tool, but it cannot replace the human elements that make therapy effective: emotional attunement, clinical judgment, ethical responsibility, and genuine relationship.
Practical Tips for Users and Therapy Clients
- Be cautious about sharing personal data. Assume anything entered into AI is not private unless there is clear indication that it is.
- Don’t confuse validation with care. AI can mirror thoughts and feelings without helping you evaluate or challenge them.
- Use AI for support tasks, such as journaling or noticing patterns, rather than emotional processing.
- Avoid relying on AI in crises. Seek human help when safety is at risk.
- Look for transparency and evidence. Use tools that clearly state what they can and cannot do based on evidence-based practices.
Practical Tips for Therapists
Let’s be real. Many clients walking into a therapy office are using or interacting with AI in some way in their lives, and that number is just going to increase. It is important that therapists embrace this reality and prepare for conversations.
- Be prepared to explain to clients the difference between therapy and using a chatbot, and how to use each to their strengths. The therapy process is about exploring and understanding the complexity and grey areas in life, emotions, and relationships. Chatbot strengths include assistance with tasks, getting advice by scanning large amounts of information, creating a plan of action, and practical answers.
- Ask clients about AI use. While many people are using these tools they might not automatically tell you. Including a question about if and how a client uses AI in your intake or informed consent can be helpful.
- Offer balanced psychoeducation. Normalize curiosity while naming limits and risks of AI tools. Explain the difference between therapy and the use of AI tools
- Be informed of tools Ensure privacy and professional standards are met for any AI tool you use in your practice.
- Stay updated. The world of AI is rapidly changing day to day.
For therapists practiing EMDR therapy in particular, it’s important to explain to clients that you are practicing psychotherapy, and psychotherapy is a regulated healthcare service in most parts of the world. AI tools and machines are not authorized to provide psychotherapy and therefore cannot replace a licensed clinician in delivering EMDR therapy or any other evidence-based treatment.
Where Do We Go from Here?
AI tools are rapidly becoming part of everyday mental health care, but their adoption must be done mindfully and responsibly. Humans are complex and vulnerable, which must be taken into account when utilizing technology for mental health care. Mental health professionals are uniquely positioned to help define ethical boundaries and appropriate roles for AI, while wider public education is essential to ensure these tools are used with caution and care.
Find out more about the work of EMDRIA here.
