Vetoed by Governor Gavin Newsom

Vote Smart's Synopsis:

Vote to concur with House amendments and pass a bill that establishes the Frontier Model Division to regulate AI models in California.

Highlights:

• Establishes the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act to, among other things, require that a developer, before initially training a covered model, as defined, comply with various requirements, including implementing the capability to promptly enact a full shutdown, as defined, and implement a written and separate safety and security protocol, as specified (Sec. 1-8).

• Prohibits a developer from using a covered model commercially or publicly, or making a covered model or a covered model derivative available for commercial or public use, if there is an unreasonable risk that the covered model or covered model derivative can cause or enable a critical harm, as defined (Sec. 1-8).

• Requires developers, beginning January 1, 2028, annually to retain a third-party auditor to perform an independent audit of compliance with this bill's requirements (Sec. 1-8).

• Requires a developer of a covered model to submit to the Frontier Model Division, which the bill would create within the Government Operations Agency, a certification under penalty of perjury of compliance with these provisions, as specified (Sec. 1-8).

• Requires a developer of a covered model to report each artificial intelligence safety incident affecting the covered model or any covered model derivative controlled by the developer, as specified, to the Frontier Model Division (Sec. 1-8).

• Requires individuals who operate a computing cluster, as defined, to implement written policies and procedures to do certain things when a customer utilizes compute resources that are sufficient to train a covered model, including assessing whether a prospective customer intends to utilize the computing cluster to train a covered model (Sec. 1-8).

• Specifies unlawful acts under these provisions and authorizes the Attorney General or the Labor Commissioner to bring a civil action, as provided (Sec. 1-8).

• Establishes whistleblower protections, including prohibiting a developer of a covered model or a contractor or subcontractor of the developer from preventing an employee from disclosing information, or retaliating against an employee for disclosing information, to the Attorney General or Labor Commissioner if the employee has reasonable cause to believe the developer is out of compliance with certain requirements or that the covered model poses an unreasonable risk of critical harm (Sec. 1-8).

• Establishes the Board of Frontier Models within the Government Operations Agency, independent of the Department of Technology, and provides for the board’s membership (Sec. 1-8).

• Establishes the Frontier Model Division within the Government Operations Agency and under the direct supervision of the board, and would require the division to, review annual certification reports from developers received pursuant to these provisions and publicly release summarized findings based on those reports (Sec. 8).

• Requires the division to, on or before January 1, 2027, and annually after, issue regulations to update the definition of a “covered model,” as provided (Sec. ).

• Authorizes the division to assess related fees and would require a deposit of the fees into the Frontier Model Division Programs Fund (Sec. 1-8).

• Requires the Department of Technology to commission consultants, as prescribed, to create a public cloud computing cluster, to be known as CalCompute, with the primary focus of conducting research into the safe and secure deployment of large-scale artificial intelligence models and fostering equitable innovation that includes, among other things, a fully owned and hosted cloud platform (Sec. 1-8).

• Requires the state to reimburse local agencies and school districts for certain costs mandated by the state (Sec. 1-8).

Full Bill Text

Vote Smart's Synopsis:

Vote to amend and pass a bill that establishes the Frontier Model Division to regulate AI models in California.

Highlights:

• Establishes the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act to, among other things, require that a developer, before initially training a covered model, as defined, comply with various requirements, including implementing the capability to promptly enact a full shutdown, as defined, and implement a written and separate safety and security protocol, as specified (Sec. 1-8).

• Prohibits a developer from using a covered model commercially or publicly, or making a covered model or a covered model derivative available for commercial or public use, if there is an unreasonable risk that the covered model or covered model derivative can cause or enable a critical harm, as defined (Sec. 1-8).

• Requires developers, beginning January 1, 2028, annually to retain a third-party auditor to perform an independent audit of compliance with this bill's requirements (Sec. 1-8).

• Requires a developer of a covered model to submit to the Frontier Model Division, which the bill would create within the Government Operations Agency, a certification under penalty of perjury of compliance with these provisions, as specified (Sec. 1-8).

• Requires a developer of a covered model to report each artificial intelligence safety incident affecting the covered model or any covered model derivative controlled by the developer, as specified, to the Frontier Model Division (Sec. 1-8).

• Requires individuals who operate a computing cluster, as defined, to implement written policies and procedures to do certain things when a customer utilizes compute resources that are sufficient to train a covered model, including assessing whether a prospective customer intends to utilize the computing cluster to train a covered model (Sec. 1-8).

• Specifies unlawful acts under these provisions and authorizes the Attorney General or the Labor Commissioner to bring a civil action, as provided (Sec. 1-8).

• Establishes whistleblower protections, including prohibiting a developer of a covered model or a contractor or subcontractor of the developer from preventing an employee from disclosing information, or retaliating against an employee for disclosing information, to the Attorney General or Labor Commissioner if the employee has reasonable cause to believe the developer is out of compliance with certain requirements or that the covered model poses an unreasonable risk of critical harm (Sec. 1-8).

• Establishes the Board of Frontier Models within the Government Operations Agency, independent of the Department of Technology, and provides for the board’s membership (Sec. 1-8).

• Establishes the Frontier Model Division within the Government Operations Agency and under the direct supervision of the board, and would require the division to, review annual certification reports from developers received pursuant to these provisions and publicly release summarized findings based on those reports (Sec. 8).

• Requires the division to, on or before January 1, 2027, and annually after, issue regulations to update the definition of a “covered model,” as provided (Sec. ).

• Authorizes the division to assess related fees and would require a deposit of the fees into the Frontier Model Division Programs Fund (Sec. 1-8).

• Requires the Department of Technology to commission consultants, as prescribed, to create a public cloud computing cluster, to be known as CalCompute, with the primary focus of conducting research into the safe and secure deployment of large-scale artificial intelligence models and fostering equitable innovation that includes, among other things, a fully owned and hosted cloud platform (Sec. 1-8).

• Requires the state to reimburse local agencies and school districts for certain costs mandated by the state (Sec. 1-8).

Full Bill Text

Vote Smart's Synopsis:

Vote to concur with House amendments and pass a bill that establishes the Frontier Model Division to regulate AI models in California.

Highlights:

• Establishes the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act to, among other things, require that a developer, before initially training a covered model, as defined, comply with various requirements, including implementing the capability to promptly enact a full shutdown, as defined, and implement a written and separate safety and security protocol, as specified (Sec. 1-8).

• Prohibits a developer from using a covered model commercially or publicly, or making a covered model or a covered model derivative available for commercial or public use, if there is an unreasonable risk that the covered model or covered model derivative can cause or enable a critical harm, as defined (Sec. 1-8).

• Requires developers, beginning January 1, 2028, annually to retain a third-party auditor to perform an independent audit of compliance with this bill's requirements (Sec. 1-8).

• Requires a developer of a covered model to submit to the Frontier Model Division, which the bill would create within the Government Operations Agency, a certification under penalty of perjury of compliance with these provisions, as specified (Sec. 1-8).

• Requires a developer of a covered model to report each artificial intelligence safety incident affecting the covered model or any covered model derivative controlled by the developer, as specified, to the Frontier Model Division (Sec. 1-8).

• Requires individuals who operate a computing cluster, as defined, to implement written policies and procedures to do certain things when a customer utilizes compute resources that are sufficient to train a covered model, including assessing whether a prospective customer intends to utilize the computing cluster to train a covered model (Sec. 1-8).

• Specifies unlawful acts under these provisions and authorizes the Attorney General or the Labor Commissioner to bring a civil action, as provided (Sec. 1-8).

• Establishes whistleblower protections, including prohibiting a developer of a covered model or a contractor or subcontractor of the developer from preventing an employee from disclosing information, or retaliating against an employee for disclosing information, to the Attorney General or Labor Commissioner if the employee has reasonable cause to believe the developer is out of compliance with certain requirements or that the covered model poses an unreasonable risk of critical harm (Sec. 1-8).

• Establishes the Board of Frontier Models within the Government Operations Agency, independent of the Department of Technology, and provides for the board’s membership (Sec. 1-8).

• Establishes the Frontier Model Division within the Government Operations Agency and under the direct supervision of the board, and would require the division to, review annual certification reports from developers received pursuant to these provisions and publicly release summarized findings based on those reports (Sec. 8).

• Requires the division to, on or before January 1, 2027, and annually after, issue regulations to update the definition of a “covered model,” as provided (Sec. ).

• Authorizes the division to assess related fees and would require a deposit of the fees into the Frontier Model Division Programs Fund (Sec. 1-8).

• Requires the Department of Technology to commission consultants, as prescribed, to create a public cloud computing cluster, to be known as CalCompute, with the primary focus of conducting research into the safe and secure deployment of large-scale artificial intelligence models and fostering equitable innovation that includes, among other things, a fully owned and hosted cloud platform (Sec. 1-8).

• Requires the state to reimburse local agencies and school districts for certain costs mandated by the state (Sec. 1-8).

Full Bill Text