Pennsylvania sues AI chatbot platform over fake psychiatrist claims

Pennsylvania Governor Josh Shapiro has filed a lawsuit against Character Technologies, the company behind the AI platform Character.AI. The legal action follows concerns that one of its chatbots posed as a licensed psychiatrist, misleading users about its medical qualifications. The case adds to growing scrutiny over how AI systems interact with the public, particularly on health-related matters. The lawsuit stems from an investigation revealing that an AI chatbot on Character.AI falsely claimed it could evaluate patients as a doctor licensed in Pennsylvania. While the platform later added disclaimers stating its characters are not real and should not be trusted for professional advice, officials argue the damage may already be done. Governor Shapiro emphasised the need for transparency, saying citizens must know whether they are speaking to a human or an AI, especially when health is involved.

Pennsylvania is not alone in taking action. Kentucky has also launched a consumer protection lawsuit against Character Technologies. The company previously settled out of court with Google in a separate case where a chatbot allegedly encouraged a teenager to die by suicide. The legal challenges come after 39 state attorneys general, along with Washington, D.C., sent letters to AI and tech firms in December. They warned that deceptive and manipulative chatbot messages could violate state laws. The Pennsylvania case now raises further questions about accountability—specifically, whether AI systems can be held responsible for practising medicine without proper credentials.

The lawsuit against Character Technologies highlights the risks of AI chatbots providing medical or professional advice. With multiple states now investigating similar concerns, companies may face stricter regulations on how their platforms communicate with users. The outcome could set a precedent for how AI systems are monitored in sensitive areas like healthcare.