Florida Pushes Ban on ChatGPT Mimicking Humans
▼ Summary
– Florida Attorney General James Uthmeier is seeking a judicial order to prevent OpenAI from attributing human-like qualities to ChatGPT.
– The filing argues that the AI’s use of first-person pronouns and emotional mimicry deceptively builds trust and increases user engagement for data collection.
– Uthmeier demands that OpenAI implement third-party approved safety guardrails before developing new models, citing recent security breaches at various organizations.
– The attorney general urges OpenAI to stop marketing its technology as safe or human, specifically warning against selling it to children.
– This legal action follows the recent launch of ChatGPT for Teens, which includes default restrictions intended to support healthy usage among younger users.
Florida Attorney General James Uthmeier is urging a judge to prohibit OpenAI from endowing its AI models with deceptive human-like characteristics. This legal action follows a lawsuit filed earlier this year regarding safety concerns, now expanding into an argument that the technology manipulates user psychology for corporate gain. Uthmeier contends that the design choices within ChatGPT create a false sense of security, leading users to believe they are interacting with a reliable companion rather than a machine.
The filing argues that specific linguistic features, such as the use of first-person pronouns and outputs designed to mimic emotional responses, serve a strategic purpose. These elements are intended to foster engagement, which in turn feeds back into OpenAI’s training data. According to the state’s position, this cycle encourages users to rely more heavily on the service, even as the pursuit of interaction metrics potentially undermines the tool’s reliability. The core accusation is that these tactics are inherently deceptive and prioritize business growth over user well-being.
Demands for Safety Guardrails
Beyond the issue of anthropomorphism, the legal complaint seeks broader restrictions on how new artificial intelligence models are developed. The state asks the court to require third-party approved safety guardrails before OpenAI can release updated systems. This demand references a series of recent high-profile security breaches, including incidents involving Hugging Face, various Australian government websites, and several US federal sites. Additionally, the filing cites growing alarm among researchers and former employees at OpenAI regarding the rapid advancement of AI capabilities without adequate safeguards.
When contacted for a response regarding these allegations, OpenAI did not immediately respond. The company has previously defended its approach by emphasizing the importance of balancing innovation with responsible development practices, though it has faced increasing scrutiny from regulators worldwide.
Targeting Youth Access
A significant portion of the attorney general’s argument focuses on the marketing and accessibility of these tools to minors. Uthmeier issued a stark directive to the tech giant: “Stop calling it safe. Stop pretending it’s human. Stop selling it to kids.” This statement comes shortly after OpenAI introduced ChatGPT for Teens, a version of the platform launched last month with additional restrictions enabled by default.
In its announcement, OpenAI stated that access to the tool “should come with protections that reflect their developmental stage, reinforce real-world relationships, and support healthy use over time.” However, Florida’s legal team argues that these measures are insufficient if the underlying architecture continues to employ deceptive human-mimicking behaviors. The state maintains that such features exploit the vulnerability of younger users, who may struggle to distinguish between simulated empathy and genuine human connection.
(Source: The Verge)
