Gov. Newsom Proposes AI Kill Switch for California

California Governor Gavin Newsom has issued a new executive order directing a specialized working group to evaluate and propose enhancements to the state’s artificial intelligence safety regulations. The governor has mandated that this panel deliver its findings within two months, with a specific focus on whether technology firms should be required to implement a “kill switch” for advanced AI systems. This directive comes nearly two years after Newsom vetoed legislation that would have made such shutdown capabilities mandatory for developers of frontier models.

The previous legislative effort, known as SB 1047, was designed to ensure that companies could “promptly enact a full shutdown” of their most powerful AI systems, effectively halting both operation and further training if necessary. Newsom rejected this bill on September 29, 2024, characterizing it as the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act. He argued that the proposal flawed its regulatory approach by targeting models based on their computational size and financial cost rather than assessing the actual risks associated with their intended applications.

In his latest executive action, Newsom emphasized urgency in addressing these technological challenges. “We’re not waiting to act,” he stated. While his office has not yet disclosed the composition of the newly formed working group, the order also explores the role of independent third parties in drafting safety protocols for leading AI corporations. This move signals a shift toward external oversight mechanisms, contrasting with previous attempts at direct statutory mandates.

International approaches to AI governance offer a different perspective on these requirements. Under the European Union’s AI Act, which gained relevant enforcement powers on August 2, regulators possess the authority to restrict, withdraw, or recall general-purpose models from the EU market. This power rests solely with the regulatory body, meaning compliance does not depend on individual companies implementing specific technical features like a kill switch. Additionally, the AI Act mandates that humans overseeing high-risk systems must have the ability to interrupt operations via a stop button, though this rule applies to deployed systems rather than the underlying models themselves. These obligations are currently pending implementation.

Both California and Europe are still navigating the logistics of oversight infrastructure. California’s SB 813 allows until January 2028 for the state to certify organizations capable of testing frontier models. Similarly, Europe appointed a 60-member scientific advisory panel on June 1. Neither jurisdiction has resolved questions regarding who bears the financial burden for the computing resources required for these safety tests. In the only existing precedent, the company being tested covered the costs incurred by the tester.

Newsom’s stance on AI regulation reflects a complex history with the industry. As governor, he initially supported the sector’s rapid expansion due to its economic benefits and recently championed a partnership to integrate Anthropic’s Claude model into state government agencies. Since both Anthropic and OpenAI are headquartered in California, they will likely be primary subjects of whatever recommendations emerge from the governor’s new working group.