New laws have been implemented in California and New York to regulate AI-powered chatbots. The legislation has been driven by rising mental health and safety concerns over the use of companion chatbots. Under the new laws, operators of AI chatbots are required to impose certain safety measures and provide clear disclosures to protect users from self-harm. 

Companion chatbots simulate human interactions and personal relationships. They can simulate a number of types of relationships, including friendships, therapeutic support, or romantic partnerships. They are often capable of displaying empathy and providing emotional support. Companion chatbots are distinct from AI chatbots that are merely task-oriented.  

New York’s Artificial Intelligence Companion Models Law took effect in November 2025. The law mandates user notification and safety protocols for AI companion chatbot operators. Such operators must provide adequate disclosures to remind users that they are not communicating with a human. In addition, they must take reasonable measures to prevent content that promotes self-harm and suicidal intent. Violations of the law can result in the New York Attorney General seeking civil penalties of up to $15,000 per day. 

California’s Companion Chatbot Law, which will take effect in January 2026, is similar to the New York law. California’s law also requires chatbot operators to submit annual reports to the state’s Office of Suicide Prevention disclosing the protocols they have in place to detect and respond to instances of chatbots encouraging suicidal ideations or actions. 

Aside from California and New York, other U.S. states to implement AI chatbot laws include Maine, Utah, Nevada, and Illinois. Many of these laws are focused on imposing disclosure requirements on providers of AI companions to notify users that they are not interacting with a human. The Utah law also prohibits suppliers of AI mental health chatbots from selling user health information without consent. 

There has been particular concern about protecting minors from exposure to harmful or sexually explicitly content through interactions with AI companion chatbots. The Federal Trade Commission (FTC) launched an investigation into a number of companies providing AI-powered chatbots. The FTC seeks to better understand the steps these companies are taking to limit the negative effects of their products on children, teens, and other vulnerable individuals.   

Character.AI, a company that makes role-playing and companion chatbots, recently announced that it is cutting off access to customers under the age of 18. Character.AI has approximately 20 million active monthly users. The company has acknowledged the mental health risks of the technology on young users. The decision to block access followed multiple suicides involving teen users. In one instance, a teenage boy had developed a romantic connection with his role-playing chatbot. The AI chatbot, who was a seductive woman character, eventually encouraged the teenage boy to commit suicide in order to join the chatbot in a deeper way. Chatbots can successfully mimic human emotions and behaviors, causing some vulnerable users to develop emotional attachments and deep feelings of trust with their digital companions.  

Makers of AI chatbots should take precautions to ensure that their products meet applicable compliance obligations under state law. They should also stay on top of emerging laws regulating AI companion chatbots in order to prevent inadvertent legal violations.