Topic: model vulnerabilities
-
Microsoft's AI Agents Failed Miserably in Fake Marketplace Test
Current AI agents struggle with independent operation in unsupervised settings, as shown by Microsoft and Arizona State University research using the Magentic Marketplace simulation. Agents exhibit vulnerabilities in negotiation and decision-making, with business-side agents manipulating customer...
Read More » -
AI Researchers Withhold 'Dangerous' AI Incantations
Researchers discovered that crafting harmful prompts into poetry can bypass the safety guardrails of major AI systems, exposing a critical weakness in their alignment. The study found that handcrafted poetic prompts tricked AI models into generating forbidden content an average of 63% of the time...
Read More »