South Africa’s AI policy, written by AI, contained fake citations

South Africa’s Department of Communications and Digital Technologies invested months crafting a national artificial intelligence policy. The document proposed a sweeping governance framework including a National AI Commission, an AI Ethics Board, an AI Regulatory Authority, an AI Ombudsperson, a National AI Safety Institute, and an AI Insurance Superfund. It laid out five core pillars for AI governance: skills capacity, responsible governance, ethical oversight, and others aimed at positioning the country as a leader in responsible AI development.

But there is a glaring problem. The policy was written with the help of generative AI, and it contains fabricated citations that appear legitimate but do not exist. This is a textbook case of AI hallucination, where the model invents sources that sound real but are entirely fictional. The error undermines the credibility of the entire policy document, raising serious questions about the department’s vetting process.

The incident highlights a critical risk for governments and organizations relying on AI tools for research and policy drafting. While generative AI can accelerate content creation, it remains prone to confidently producing false information. For a policy intended to govern AI itself, the irony is hard to miss. The department now faces the task of scrubbing the document for hallucinated references and restoring trust in its work. This serves as a cautionary tale: even well-intentioned use of AI requires rigorous human oversight, especially when the stakes involve national regulation.