From AI scribes to therapy chatbots, here’s how artificial intelligence is being used in mental health care—and how states are beginning to regulate it
As behavioral health systems face workforce shortages, rising demand for care, and provider burnout, interest in and use of artificial intelligence (AI) tools have grown rapidly.
Providers are leveraging AI tools to support clinical documentation, administrative workflows, triage, and care coordination. People are also turning to AI for emotional support and therapy-like conversations—both through general-purpose large language models (LLMs) and a rapidly growing market of therapy-specific chatbots and companions.
But there are growing concerns around safety, privacy, effectiveness, bias, emotional dependency, and regulation, particularly around AI-powered mental health chatbots.
Below, we explore how AI is currently being used in mental health care, the potential benefits and risks of those applications, the emerging regulatory landscape, and the policy questions states and health systems are beginning to confront.
Three main AI use cases in mental health
Administrative support and AI scribes
One of the most common uses of AI in behavioral health is reducing administrative burden for providers. AI tools can transcribe sessions, generate draft notes and intake summaries, assist with billing workflows, and organize electronic health records. At the Steinberg Institute’s Vision 2030 Solution Symposium, panelists highlighted tools like Eleos’ AI scribe, which can generate transcripts and notes from sessions to help providers spend less time on documentation and more time focused on patients.
Research increasingly suggests this may be one of the most promising current uses of AI in health care. One qualitative study interviewed mental health care professionals on using AI chatbots and found that AI’s most realistic near-term use in mental health is for tasks that reduce administrative burden, cognitive burden, and burnout. The American Psychological Association (APA) and other experts have emphasized that AI may be most useful when augmenting clinician workflows and reducing administrative burden, rather than replacing therapeutic relationships or clinical judgment.
Intake assessments, triage, and provider matching
AI-supported tools are also being used to help determine what level of care a person may need, connect individuals with providers based on specialty or preferences, and expand access to virtual behavioral health care.
Research is limited, but recent studies have demonstrated that using AI in mental health assessments can reduce wait times, dropout rates, provider efficiency, and recovery rates. According to the Meadows Mental Health Policy Institute, AI systems may help identify suicide risk, support intake assessments, monitor outcomes, flag potential care gaps, and assist clinicians with treatment planning.
But concerns remain around inaccurate recommendations, lack of transparency on methodology, and algorithmic bias.
AI & Mental Health Use Case Breakdown |
||
| Use Case | Potential Benefits | Potential Concerns |
| Administrative support and AI scribes | Reduce administrative burden, improve efficiency, lower clinician burnout, allow providers to spend more time with patients | Privacy concerns, inaccuracies in notes, overreliance on automated documentation |
| Triage and care navigation | Improve care coordination and connect individuals to appropriate services more quickly | Inaccurate recommendations, bias, lack of transparency |
| Provider matching | Improve provider fit and expand access to specialized care | Algorithmic bias, privacy concerns, inaccurate matching |
| Telehealth and virtual care | Expand access, especially in rural or underserved areas | Digital divide, limited broadband and device access |
| Digital mental health apps (CBT activities, self-guided interventions, etc.) | Lower barriers to support and increase accessibility | Limited evidence base and inconsistent oversight |
| AI therapy chatbots and AI companions | Immediate and 24/7 support, reduced costs, expanded accessibility | Harmful guidance, emotional dependency, privacy risks, lack of regulation |
AI therapy chatbots and AI companions
The most controversial use of AI in behavioral health is the rise of conversational AI systems marketed as emotional support, companionship, or therapy.
The conversational AI mental health market has expanded significantly in recent years as demand for therapy chatbots grew and advances in generative AI lowered the barrier to building chatbot-based products.
Source: The Hemingway Report, 2026
Supporters argue these tools may expand access to support, reduce costs, and provide immediate and 24/7 availability. Some studies have increasingly suggested that these tools have promise when they are co‐designed by experts, undergo rigorous evaluation, and are based in implementation science.
But many researchers, clinicians, and professional organizations warn the technology is moving faster than the evidence and safeguards needed to support vulnerable individuals. The APA’s recent advisory highlights that AI wellness apps and chatbots may provide support in some contexts, but should not be considered substitutes for licensed mental health professionals or crisis care. Specifically, experts have raised concerns that AI chatbots:
- May provide inaccurate or harmful guidance
- Cannot understand nonverbal cues or full human context
- May reinforce harmful beliefs or emotional dependency
- Operate without consistent oversight or regulation
- Raise major privacy and data concerns
Research from Stanford University and Brown University highlights the risks of AI chatbots. Stanford research found that chatbots can contribute to stigma and provide harmful responses, whereas Brown’s research found that AI chatbots violated ethical standards of practice.
Licensed clinicians have clinical judgment, ethical responsibility, and can understand trauma, risk, and human context in nuanced ways – something that AI systems cannot yet reliably replicate. Even as AI systems become more sophisticated, behavioral health care will always depend on uniquely human qualities and the ability to understand complex lived experiences.
The regulatory landscape on AI and mental health
Generally speaking, the growth in the conversational AI mental health market has significantly outpaced federal regulatory oversight.
This is by design. The current federal administration favors reducing regulatory barriers to encourage rapid innovation and adoption of AI. A December 2025 Executive Order even discouraged states from enacting their own AI regulations.
Experts and professional organizations call for stronger oversight due to AI mental health tools lacking clear evidence standards, transparency around how systems generate responses, and sufficient protections for vulnerable users.
State regulatory actions
Currently, states are moving faster than the federal government to establish guardrails around transparency, privacy, youth protections, disclosure requirements, and emotional dependency concerns. Although the majority of state legislation focuses on general health care and artificial intelligence, several states have already enacted or advanced laws regulating conversational AI systems and AI companion chatbots:
- California: In 2025, California enacted Senate Bill 243, one of the first laws in the nation specifically regulating “companion chatbots.” The law requires certain AI chatbot operators to provide disclosures when users are interacting with AI systems, implement suicide and self-harm safety protocols, and establish additional safeguards for minors.
- New York: Similar to California, New York enacted a law requiring chatbot operators to disclose when users are interacting with AI systems and implement protocols related to suicide prevention and crisis response.
- Illinois: Illinois recently passed a law banning licensed therapists from using AI to make treatment decisions or communicate with clients. They can still use AI for administrative tasks. Under this law, companies are not allowed to offer AI-powered therapy services or advertise chatbots as therapy tools without involving a licensed professional.
- Oregon: In March 2026, Oregon passed legislation focused on protecting minors interacting with AI companion systems, including disclosure requirements and safeguards to reduce harmful emotional dependency and manipulative chatbot interactions.
- Washington: Washington’s legislation requires disclosures for AI-generated interactions and establishes consumer protections related to emotionally manipulative chatbot behavior, particularly for youth users.
Much of this legislation has been driven by growing concern around emotional dependency on AI companions, safety, and data privacy and use. The California Health Care Foundation recently published a comprehensive report on policies introduced or enacted across the country that relate to general health care and AI. Still, AI advancements outpace legislation, making it challenging to see how effective these laws will be in providing sufficient guardrails.
Experts and professional organizations call for stronger oversight due to AI mental health tools lacking clear evidence standards, transparency around how systems generate responses, and sufficient protections for vulnerable users.
Our TakeAI is already embedded in behavioral health care—at both the consumer and provider level. Use will likely continue to expand as providers and systems look for ways to address workforce shortages, administrative burden, and rising demand for care. Research shows that artificial intelligence can be an important workforce augmentation tool to reduce administrative burden, improve coordination, and support workforce capacity. As AI technology and LLMs continue to improve, there is a future where AI could assist in providing care—but only with appropriate guardrails, and never as a replacement for clinical care. The CEO Alliance for Mental Health’s Guiding Principles for AI in Behavioral Health outline effective principles for using AI while ensuring that care remains centered on human connection and community-based services. Ultimately, AI is a tool that can enhance and support behavioral health care—but should not replace the essential human element of care. As California implements major behavioral health initiatives like BHSA, CalAIM, and CYBHI, ensuring the safe, evidence-based integration of AI into behavioral health systems will be imperative to maintaining ethical and safe mental health treatment. |