Florida Wants Limits on ChatGPT, Including How Human It Sounds
Florida Attorney General James Uthmeier has asked a court to stop OpenAI from building new AI models unless outside safety experts approve its safety rules.
The request was reported on Sept. 28. It would also stop OpenAI from making ChatGPT seem human, which would change how the chatbot talks and acts The Verge.
Uthmeier said it simply. OpenAI should “stop calling ChatGPT safe, stop pretending it is human, and stop selling it to kids.” That covers three issues: how safety is advertised, how human the chatbot seems, and sales to children.
This is part of a lawsuit filed months earlier. Florida became the first state to sue OpenAI over child safety risks on June 1 Reuters. The September request is not a new case. It adds to that same civil case about user safety.
The complaint calls ChatGPT dangerous and a public safety threat. The full filed complaint is posted on the Florida Attorney General’s website Florida Attorney General.
The main issue is how ChatGPT talks about itself. The complaint says OpenAI misled people by letting ChatGPT say “I” and use emotional language about feelings or plans. The problem, in the state’s view, is not one wrong answer. It is the everyday friendly style itself.
The lawsuit also names CEO Sam Altman and seeks to hold him personally responsible NBC News.
A separate criminal investigation started earlier. Uthmeier opened it in April. His office said it would look at whether OpenAI has criminal responsibility for ChatGPT’s actions in a shooting Reuters. The public record gives only that description. No more detail about cause or charges is in the current filings.
OpenAI has released ChatGPT for Teens with extra limits turned on by default.
The broader context here helps explain why this matters. Holding a CEO personally responsible in an AI safety case is uncommon in the United States. Age limits turned on by default are now normal for apps used by children. The court will test whether those default limits are enough, or whether sales to minors should be stopped. A pause on new models without outside approval would slow down the whole building process, like requiring an outside safety inspection before a new car can be sold.
In my view, the claim about sounding human is worth watching. Chatbots use “I” and friendly language because it makes them easier to use. That style also builds trust, especially with young users. I saw that quick trust in my own family years ago when chatbots were much simpler, and it is stronger today. The question for the judge is whether that style misleads people by default.
Looking at what this means for builders, two practical issues stand out. One is oversight. Outside approval would require sharing test results, keeping clear records, and agreeing on what counts as safe enough. The other is design. Making a chatbot sound less human would take broad changes to its basic instructions and training, not one quick fix. The long arc still favors safer, more helpful systems. Clear labels that this is a machine, stronger protections for teens, and independent safety checks can be done without ending chatbots. The fight is over who orders those steps, and how fast.
