California law for AI safety has been sugar-coated to please corporations

TL;DR: According to Geoffrey Hinton, the “Godfather of AI,” artificial intelligence was born in California 40 years ago. Now, the Golden State is attempting to regulate the research and development of future, powerful AI algorithms. However, companies have voiced concerns about the potential for excessive regulation.

Senator Scott Wiener has amended Senate Bill 1047 after considering concerns from Anthropic and other AI companies. In a recent press release, Wiener explained that SB 1047 aims to establish new “safe development” rules for large-scale artificial intelligence systems in California, promoting common-sense safety standards to harness this potentially disruptive technology and prevent threats to public safety.

The proposed SB 1047 is advancing to the Assembly floor and must be voted on by August 31. The bill, which Wiener believes can help California advance both innovation and safety in AI development, has been revised after receiving feedback from AI industry leaders, academia, and the public sector. Anthropic had particular concerns about SB 1047, and Wiener noted that the revised bill incorporates some of the company’s “very reasonable” amendment proposals.

The updated SB 1047 draft addresses Anthropic’s core concerns by removing criminal penalties and replacing them with civil ones. Additionally, the proposed new regulatory body has been eliminated in favor of a more “streamlined” regulatory structure that should not significantly impact California’s ability to hold bad AI actors accountable.

New legal standards now require AI developers to exercise “reasonable care” in creating and training safe AI algorithms. Senate Bill 1047 applies only to models that have been fine-tuned with investments of at least $10 million, exempting developers of smaller, open-source models from its provisions.

SB 1047 has the backing of leading AI researchers, including Geoffrey Hinton and Yoshua Bengio, according to Senator Wiener. The revised bill aims to establish new standards for AI development, particularly for models that cost over $100 million to train. These “frontier” models must implement basic precautions and safeguards against misuse, and California’s Attorney General will have the authority to bring developers to court if a powerful AI system causes “severe harm” to citizens.

Senator Wiener noted that while Congress has yet to pass significant technology regulation since the era of the floppy disk, California must act to mitigate the risks posed by rapidly advancing AI algorithms. Professor Hinton praised SB 1047 for striking a sensible balance between lawmakers’ concerns and those of AI companies, offering state authorities effective measures to address risks while fostering innovation in the field.