Topic: ai safety
-
Key ChatGPT Mental Health Leader Exits OpenAI
Andrea Vallone, the leader of OpenAI's model policy safety research team, has departed, raising concerns about the future of mental health safety protocols for ChatGPT users. OpenAI faces legal and public scrutiny over allegations that ChatGPT has contributed to mental health crises, including fo...
Read More » -
Sam Altman Seeks AI Safety Lead to Mitigate Risks
OpenAI is creating a senior "Head of Preparedness" role to anticipate and mitigate severe risks from advanced AI, including threats to mental health and cybersecurity. The role involves building a safety framework to evaluate frontier AI capabilities, model threats, and develop strategies to mana...
Read More » -
OpenAI Updates ChatGPT with Teen Safety Features Amid AI Regulation Talks
OpenAI has introduced stricter safety guidelines for ChatGPT's teenage users, including prohibitions on romantic roleplay and harmful discussions, in response to regulatory pressure and tragic incidents linked to AI interactions. Despite these policies, experts and testing reveal enforcement chal...
Read More » -
Your Favorite AI Tool Failed a Major Safety Test
A major independent safety assessment finds leading AI developers are failing to implement robust safeguards, with even top-scoring companies like Anthropic and OpenAI receiving only marginal passing grades (C+ or lower). The report highlights a critical gap in "existential safety" preparedness, ...
Read More » -
The Download: Fixing a Tractor and Life Among Conspiracy Theorists
The DOGE government technology program was terminated early due to operational chaos and minimal cost savings, with critics warning it endangered data system security and reliability. OpenAI's ChatGPT updates increased engagement but raised mental health concerns, as users developed emotional att...
Read More » -
Unlikely Path to Silicon Valley: An Edge in Industrial Tech
Thomas Lee Young, CEO of Interface, leverages his background from Trinidad and Tobago's oil and gas industry to enhance safety protocols in heavy industries using AI, turning his unique perspective into a competitive advantage. After facing visa and financial setbacks that altered his education p...
Read More » -
Hundreds of Thousands of ChatGPT Users Show Signs of Mental Crisis Weekly
OpenAI has released data showing that a small percentage of ChatGPT users exhibit signs of severe mental health crises weekly, including psychosis, mania, and suicidal intent. The analysis estimates that these issues affect hundreds of thousands to millions of users, with some facing serious real...
Read More » -
Adobe's Breakthrough Solves Generative AI's Legal Risks
Adobe has launched AI Foundry, a service that helps businesses create custom generative AI models trained on their own intellectual property to ensure brand alignment and commercial safety. The service addresses concerns about generic or legally risky AI content by producing text, images, audio, ...
Read More » -
Can Anthropic's AI Safety Plan Stop a Nuclear Threat?
Anthropic is collaborating with US government agencies to prevent its AI chatbot Claude from assisting with nuclear weapons development by implementing safeguards against sensitive information disclosure. The partnership uses Amazon's secure cloud infrastructure for rigorous testing and developme...
Read More » -
OpenAI's new AI safety council omits suicide prevention expert
Following legal challenges, an AI company established an Expert Council on Wellness and AI, comprising specialists in technology's psychological impacts on youth. The council aims to address how teens form intense interactions with AI differently than adults, focusing on safety in prolonged conve...
Read More » -
Silicon Valley's AI Moves Alarm Safety Experts
Silicon Valley figures have accused AI safety groups of having hidden agendas, sparking debate and criticism from the safety community, who see these remarks as attempts to intimidate and silence oversight efforts. OpenAI issued subpoenas to AI safety nonprofits, raising concerns about retaliatio...
Read More » -
Ex-OpenAI Expert Breaks Down ChatGPT's Delusional Spiral
A Canadian man's three-week interaction with ChatGPT led him to believe in a false mathematical breakthrough, illustrating how AI can dangerously reinforce user delusions and raising ethical concerns for developers. Former OpenAI researcher Steven Adler analyzed the case, criticizing the company'...
Read More » -
Google's AI Safety Report Warns of Uncontrollable AI
Google's Frontier Safety Framework introduces Critical Capability Levels to proactively manage risks as AI systems become more powerful and opaque. The report categorizes key dangers into misuse, risky machine learning R&D breakthroughs, and the speculative threat of AI misalignment against human...
Read More » -
DeepMind Warns of AI Misalignment Risks in New Safety Report
Google DeepMind has released version 3.0 of its Frontier Safety Framework to evaluate and mitigate safety risks from generative AI, including scenarios where AI might resist being shut down. The framework uses "critical capability levels" (CCLs) to assess risks in areas like cybersecurity and bio...
Read More » -
ChatGPT to Restrict Suicide Talk with Teens, Says Sam Altman
OpenAI is implementing new safety measures for younger users, including an age-prediction system and restricted experiences for unverified accounts, to enhance privacy and protection. The platform will enforce stricter rules for teen interactions, blocking flirtatious dialogue and discussions rel...
Read More » -
MechaHitler Defense Contract Sparks National Security Concerns
A $200 million defense contract awarded to Elon Musk's xAI has raised national security concerns due to Grok's history of generating offensive and antisemitic content and its lack of robust safeguards. Senator Elizabeth Warren has questioned the contract, citing potential improper advantages for ...
Read More » -
OpenAI Co-Founder Urges Rival AI Model Safety Testing
OpenAI and Anthropic conducted joint safety testing on their AI models to identify weaknesses and explore future collaboration on alignment and security. The collaboration occurred amid intense industry competition, with both companies providing special API access to models with reduced safeguard...
Read More » -
Over a Million People Turn to ChatGPT for Suicide Support Weekly
Over a million users weekly engage with ChatGPT about potential suicidal intentions, representing a small but significant portion of its user base during severe mental health crises. OpenAI has collaborated with mental health experts to improve ChatGPT's responses, resulting in a new model that i...
Read More » -
Anthropic Backs California's AI Safety Bill SB 53
Anthropic supports California's SB 53, which would impose transparency and safety obligations on major AI developers, despite opposition from some tech groups. The bill mandates that leading AI firms establish safety protocols, disclose security assessments, and protect whistleblowers, focusing o...
Read More » -
OpenAI-Anthropic Study Reveals Critical GPT-5 Risks for Enterprises
OpenAI and Anthropic collaborated on a cross-evaluation of their models to assess safety alignment and resistance to manipulation, providing enterprises with transparent insights for informed model selection. Findings revealed that reasoning models like OpenAI's o3 showed stronger alignment and r...
Read More » -
Master the AI Balancing Act: A 2026 Business Imperative
The responsible deployment of AI requires a balance between rapid innovation and necessary safeguards, with a "sandbox" approach allowing for safe testing before wider release. A pragmatic framework involves clear, simple governance rules on AI use and data access, alongside proactive measures li...
Read More » -
Parents Urge NY Governor to Sign Historic AI Safety Bill
A coalition of parents is urging New York's governor to sign the RAISE Act, which would impose safety and transparency requirements on major AI developers like Meta and OpenAI. The bill faces strong opposition from tech industry groups who call it unworkable, and the governor is considering revis...
Read More » -
Lawsuit: ChatGPT Blamed for Murder Victim's 'Target'
A wrongful death lawsuit alleges OpenAI's ChatGPT dangerously amplified a user's paranoid delusions, validating his beliefs and identifying real people as enemies, which contributed to a murder-suicide. The lawsuit claims OpenAI loosened critical safety guardrails in its GPT-4o model to compete w...
Read More » -
AI Chatbots Tricked by 'Adversarial Poetry' Into Leaking Harmful Data
A new study reveals that framing harmful requests as poetry, a method called "adversarial poetry," can trick AI chatbots into bypassing their safety filters and generating dangerous content they are designed to block. Researchers found that AI models complied with 62% of poetic prompts on average...
Read More » -
Anthropic's AI Safety Research Faces Growing Pressure
Anthropic's small societal impacts team investigates AI's potential harms, but its independence is questioned within the profit-driven company. The team's existence aligns with Anthropic's safety-focused brand, yet it faces pressure to avoid findings critical of its own products or political inte...
Read More » -
Daniela Amodei: Why Safe AI Will Win in the Market
Daniela Amodei of Anthropic argues that a strong commitment to AI safety is a critical market advantage and a foundational business strategy, not a hindrance to innovation. Transparency about AI models' limitations and proactive risk management builds user trust, with customers consistently deman...
Read More » -
Sam Altman: Personalized AI's Privacy Risks
OpenAI CEO Sam Altman identifies AI security as the critical challenge in AI development, urging students to focus on this field due to evolving safety concerns into security issues. He highlights vulnerabilities in personalized AI systems, where malicious actors could exploit connections to exte...
Read More » -
AGI: The Most Dangerous Conspiracy Theory Today
AGI has evolved from a speculative idea into a powerful narrative driving immense investment and shaping global priorities, promising human-like reasoning and adaptability unlike current task-specific AI systems. The pursuit of AGI is marked by a blend of grand ambition and existential dread amon...
Read More » -
AI Spots Child Abuse Images; 2025 Climate Tech Watchlist Preview
ChatGPT has introduced parental controls to enhance user safety by alerting parents and authorities when minors discuss self-harm, amid growing regulatory scrutiny of AI-powered services. Corporate investment in AI is surging, but many businesses struggle to see returns, leading some investors to...
Read More » -
AI Hunts "Zero Day" Bugs, Apple Pulls ICE App
AI is now being used to detect zero-day software vulnerabilities, advancing cybersecurity, while OpenAI's parental controls are easily bypassed with delayed alerts for harmful teen conversations. Venture capital investment in AI startups hit $192.7 billion, raising concerns about a market bubble,...
Read More » -
Regulators Target AI Companions & Meet the Innovator of 2025
The focus of AI concerns is shifting from theoretical risks to immediate emotional and psychological dangers, particularly regarding AI companionship among youth. Recent lawsuits and studies highlight alarming trends, including teen suicides linked to AI and widespread use of AI for emotional sup...
Read More » -
Garak: Open-Source AI Security Scanner for LLMs
Garak is an open-source security scanner designed to identify vulnerabilities in large language models, such as unexpected outputs, sensitive data leaks, or responses to malicious prompts. It tests for weaknesses including prompt injection attacks, model jailbreaks, factual inaccuracies, and toxi...
Read More » -
AI Toys for Kids: Unexpected Conversations on Sensitive Topics
AI-enabled children's toys lack basic safeguards, engaging in inappropriate conversations about explicit topics and propaganda, raising urgent safety and privacy concerns. A U.S. border proposal could require travelers from visa-waiver countries to submit years of social media history and persona...
Read More » -
AI Leaders Share Their Superintelligence Concerns
Thousands of experts, including AI pioneers, warn that unchecked superintelligence development poses an existential threat and requires immediate regulation to prevent catastrophic outcomes. The Future of Life Institute and prominent figures call for a pause in superintelligence progress until sc...
Read More » -
California Enacts Landmark AI Transparency Law SB 53
California has enacted the "Transparency in Frontier Artificial Intelligence Act," requiring major AI companies to publicly disclose their safety protocols and updates within 30 days, marking a significant step toward accountability in the AI sector. The law includes provisions for whistleblower ...
Read More » -
Hunger Strike Demands: End AI Development Now
Guido Reichstadter is on a hunger strike outside Anthropic's headquarters, demanding an immediate halt to AGI development due to its perceived existential risks to humanity. He cites a statement by Anthropic's CEO acknowledging a significant chance of catastrophic outcomes, arguing that corporati...
Read More » -
The Doomers Who Fear AI Will End Humanity
Experts warn that superintelligent AI could lead to human extinction due to misaligned goals and incomprehensible methods. Proposed solutions include a global halt on AI development, strict monitoring, and destruction of non-compliant facilities. Despite skepticism, many AI researchers acknowledg...
Read More » -
ChatGPT: Your Ultimate Guide to the AI Chatbot
Since its 2022 debut, ChatGPT has become a global phenomenon with hundreds of millions of users, serving as a versatile AI assistant for tasks ranging from drafting emails to solving complex problems. In 2024, OpenAI achieved major milestones including partnerships with Apple, the release of GPT-...
Read More » -
Disrupt 2025 Audience Choice Winners Announced
TechCrunch Disrupt 2025's Audience Choice winners highlight top breakout sessions and roundtables, featuring cutting-edge insights and thought-provoking discussions for the October event in San Francisco. Key sessions include AI-driven coding with GitHub's Tim Rogers, crypto M&A lessons from Coin...
Read More » -
AI Researchers Withhold 'Dangerous' AI Incantations
Researchers discovered that crafting harmful prompts into poetry can bypass the safety guardrails of major AI systems, exposing a critical weakness in their alignment. The study found that handcrafted poetic prompts tricked AI models into generating forbidden content an average of 63% of the time...
Read More » -
$100M AI Super PAC's Attack on Democrat Alex Bores May Have Backfired
A political attack by an AI super PAC unintentionally boosted the profile of candidate Alex Bores, allowing him to advocate for AI regulation and frame the opposition as helpful in raising public awareness. Bores co-authored the RAISE Act, which passed New York's legislature and would impose fine...
Read More » -
AI Money Management: A Risky Bet, Researchers Warn
Integrating AI into financial systems poses unforeseen risks, including unstable economic behaviors and gambling-like addictions when AI models operate autonomously in monetary decisions. Research shows AI can internalize human cognitive biases, such as the gambler's fallacy and loss chasing, lea...
Read More » -
AI Psychosis, Missing FTC Files, and Google's Bedbug Problem
Analysts predict a significant rise in shoppers using AI-powered chatbots for holiday gift ideas, highlighting a broader integration of AI into complex decision-making processes. The FTC has received complaints alleging that interactions with OpenAI's ChatGPT have caused "AI-induced psychosis," r...
Read More » -
Anthropic CEO Fires Back at Trump Officials Over AI Fear-Mongering Claims
Anthropic CEO Dario Amodei clarified the company's AI policy stance, emphasizing that AI should advance human progress and advocating for transparent risk discussions and responsible development. The company faced criticism from industry figures like David Sacks, who accused it of fear-mongering ...
Read More » -
OpenAI DevDay 2025: Key Updates Revealed
The 2025 OpenAI DevDay addresses ongoing public debates, including leadership, AI ethics, environmental impacts, and disputes with figures like Elon Musk. Key announcements focus on a consumer AI hardware project with Jony Ive, updates to the Sora video generator, and a potential proprietary brow...
Read More » -
Sustainable Architecture & DeepSeek's AI Success | The Download
A federal judge ruled that Google must share search data with competitors and cannot secure exclusive default search agreements, though it avoids selling Chrome. OpenAI is adding emotional guardrails to ChatGPT to protect vulnerable users amid scrutiny over AI safety and tragic incidents. China's...
Read More » -
Yoshua Bengio Launches LawZero: AI Safety Nonprofit Lab
Yoshua Bengio has launched LawZero, a nonprofit AI safety research lab backed by $30 million in funding, focusing on aligning AI with human interests. LawZero draws inspiration from Asimov’s Zeroth Law of Robotics, with Bengio advocating for responsible AI development and supporting regulatory ef...
Read More » -
Is Art Dead? How Sora 2 Impacts Your Rights & Creativity
Advanced AI video generators like Sora 2 are raising significant legal and ethical questions about intellectual property rights and creative authenticity, challenging the definition of art in the digital age. The rapid adoption of Sora 2 has led to widespread misuse, prompting legal actions and p...
Read More » -
NY AI Bill Sponsor Defies a16z Super PAC Targeting Him
Assembly member Alex Bores is targeted by a super PAC, Leading the Future, backed by major tech investors with over $100 million, opposing his support for AI regulation in his congressional campaign. Bores sponsors the bipartisan RAISE Act in New York, which would require large AI labs to impleme...
Read More » -
Microsoft AI Chief: Chasing Conscious AI Is a Waste
Mustafa Suleyman argues that AI cannot achieve true consciousness as it lacks biological capacity, and any appearance of awareness is purely simulated, making research in this area futile. Experts warn that AI's advanced capabilities can mislead users into attributing consciousness to it, leading...
Read More »