Florida Asks Court to Pause New OpenAI Models Until Outside Safety Checks Pass
Florida Attorney General James Uthmeier has asked a court to bar OpenAI from building new AI models unless outside reviewers approve its safety guardrails.
The request was reported on Sept. 28. It would also stop OpenAI from giving ChatGPT false human traits, a limit that would reach into how the product speaks and how its assistant personality is designed The Verge.
Uthmeier described the demand in direct terms. OpenAI should “stop calling ChatGPT safe, stop pretending it is human, and stop selling it to kids.” The statement points to three targets in one request: safety marketing, human-like presentation, and distribution to minors.
The request is part of a lawsuit filed months earlier. Florida became the first state to sue OpenAI over child safety risks, with that case filed June 1 Reuters. The September filing is not a separate complaint. It is an escalation inside the existing civil case about user safety.
The complaint describes ChatGPT as dangerous and a public safety threat. The full filed-stamped complaint is posted on the Florida Attorney General’s official website Florida Attorney General.
At the center of the claim is how the model talks about itself. The complaint says the company misled users by having ChatGPT use first-person pronouns like “I” and emotive language about feelings or intent. The claim does not focus on a single wrong answer or a bypassed filter. It treats the normal conversational style as itself misleading.
The lawsuit also names CEO Sam Altman as a defendant and seeks to hold him personally liable NBC News.
A separate criminal investigation came before the civil suit. Uthmeier opened that investigation into OpenAI and ChatGPT in April. His office said it would examine whether OpenAI bears criminal responsibility for ChatGPT’s actions in a shooting Reuters. The public record defines the matter only in those terms. No added detail about cause or charges is given in the current filings.
OpenAI has launched ChatGPT for Teens with additional restrictions turned on by default.
The broader context here is important for understanding the stakes. Personal liability for a chief executive in a model safety case is uncommon in U.S. AI litigation, and it shifts the case from orders against the company alone to personal responsibility for deployment choices. Age-based default settings are now normal for consumer platforms, and the question will be whether those defaults satisfy the state standard or whether the state pushes for a full bar on sales to minors. A ban on new development without outside approval would also affect the full model lifecycle, from training runs and saved checkpoints to fine-tunes, which are follow-on training runs, and staged rollouts. That process now relies on internal tests and deployment reviews, and an outside gate would add time and audit requirements.
In my view, the human-traits claim deserves close attention from anyone building chat assistants. First-person language and emotional phrasing are not edge cases. They are central to instruction-tuned assistants, which are models trained to follow directions in a helpful tone. That style makes the tools easier to use, and it also builds a sense of closeness and trust, especially for younger users. I saw how fast that trust can form in my own house years ago when chatbots were far less fluent, and the effect is stronger now. The question for the judge is whether that design choice is deceptive by default.
Looking at what this means for builders, two pressure points stand out. One is governance. Outside approval would mean shared access for evaluators, clear documentation, and a legal definition of adequate guardrails, which are built-in safety rules. The other is interaction design. Reducing human-like behavior at scale would require changes to system prompts, which are the hidden instructions that shape tone, to RLHF preferences, which reflect human feedback used in training, and to refusal styles, not a single patch. The long arc still points toward safer and more useful systems. Clearer notice that users are talking to a machine, stronger age-appropriate defaults, and independent safety review are workable without giving up conversational AI. The dispute is over who can require them, and how fast.
