Introduction
California is at the forefront of establishing legal frameworks for artificial intelligence (AI), with several legislative measures set to take effect in the coming years. These laws address the definition, privacy implications [2] [5], and transparency requirements of AI systems, particularly focusing on generative AI and large language models (LLMs) [8]. The legislation aims to enhance consumer protection, privacy rights [4] [8], and transparency in AI applications across various sectors, including healthcare and entertainment.
Description
California has established a legal definition of artificial intelligence through AB 2885 [6], characterizing it as an engineered or machine-based system with varying levels of autonomy [2], capable of inferring outputs from inputs to influence environments [6]. This definition will take effect on January 1, 2025 [3]. An amendment to the California Consumer Privacy Act (AB 1008) clarifies that personal information may be present in AI systems [6], also effective January 1, 2025 [2] [3] [5] [6]. This amendment extends privacy rights and obligations to generative AI and large language models (LLMs) [8], introducing complexities regarding the existence of personal information within these models [8]. The legal interpretation of this issue will significantly influence the application of privacy protections [8], such as deletion and access requests [8], in generative AI systems [1] [2] [3] [5] [7] [8].
AB 3030 mandates that health facilities [7], clinics [7], and physician’s offices disclose their use of generative AI in patient communications starting January 1, 2025 [7]. This disclosure must be prominently featured at the beginning or throughout the communication [7], along with instructions for patients to reach out to a human health care provider [7]. However, communications that have been reviewed by a health care provider are exempt from this requirement [7].
Additionally, AB 2602 regulates the use of digital replicas in entertainment [6], rendering contracts unenforceable if an employer uses AI-generated replicas instead of live performances [6]. AB 1836 allows beneficiaries of deceased celebrities to seek damages for unauthorized use of AI-created replicas [6].
AB 2905 amends automatic dialing laws to require notification to call recipients when AI-generated voices are used in prerecorded messages [6]. Beginning January 1, 2026 [2] [3] [4] [5] [6], AB 2013 imposes significant compliance obligations on developers of generative AI systems [1], mandating the publication of documentation regarding the datasets used for training these systems [1]. This includes maintaining and publicly posting records detailing the sources or owners of the datasets, descriptions of data points [4], copyright or trademark protections [4], whether datasets were purchased or licensed [4], the presence of personal information [4] [6] [8], the inclusion of synthetic data [4], and any modifications made to the datasets [4]. Developers must also disclose personal information as defined by the California Consumer Privacy Act [1], necessitating a thorough evaluation of privacy issues related to the training data [1].
Generative AI models can process personal information at multiple stages [8], including during data collection for training datasets and in the model’s input and output [8]. For instance [8], querying an LLM about a public figure may yield responses that include personal information [8]. The focus of AB 2013 is on the output of generative AI models rather than the models themselves [8], indicating that models may not necessarily be defined as containing personal information [8]. If personal information is determined to exist within a model [8], simply suppressing a model’s output may not suffice for compliance with deletion requests [8]. While suppression mechanisms can prevent the generation of certain information [8], they necessitate the retention of personal information for ongoing screening [8]. An alternative approach could involve “un-learning” information [8], though this presents complexities due to the nature of AI models [8].
For generative AI systems developed or significantly modified after January 1, 2022 [1], companies may face challenges in retrospectively compiling training data documentation [1], particularly if prior records are incomplete [1]. A substantial modification is characterized by any update that materially alters the system’s functionality or performance [1], necessitating efforts to gather historical information [1]. The law also mandates transparency regarding synthetic data used in training [1], which may expose confidential internal strategies [1].
To ensure compliance with AB 2013 [1], robust data governance practices are essential [1]. Companies must implement processes for cataloging and documenting datasets regularly [1], conduct internal audits [4], and establish a reliable record-keeping system that tracks data sources and modifications [1]. Collaboration between in-house counsel and technical teams will be necessary to conduct risk assessments and privacy reviews [1], particularly to assess whether training datasets contain personal information or proprietary intellectual property [1]. Notably, the absence of exemptions for trade secret material in AB 2013 necessitates a careful balance between transparency and the protection of sensitive information [1].
The California AI Transparency Act (AB 942) obligates developers of generative AI systems with a user base of at least one million monthly users to implement tools that enable consumers to identify AI-generated content [3], along with clear disclosures regarding such content [6]. This legislation [4] [5], effective January 1, 2026 [1] [2] [3] [5] [6], imposes civil penalties of $5,000 per violation for non-compliance, with each day constituting a separate violation [5]. Covered providers must also ensure that any licensing agreements require licensees to include necessary disclosures in generated content. If a licensee alters the system to bypass these disclosures [5], the provider must revoke the license within 96 hours [5].
In addition to these measures, proposed legislation aimed at implementing impact assessments for automated decision-making tools to combat algorithmic discrimination has been stalled but is expected to be reintroduced [2]. The state’s privacy regulator is also finalizing regulations concerning the use of artificial decision-making technologies [2], which will include privacy impact assessments and cybersecurity audits [2]. Lawmakers and regulators are increasingly focused on the implications of AI in health care, with ongoing hearings and the introduction of new rules [7], urging stakeholders to stay informed about these developments to ensure compliance with the evolving legal landscape surrounding AI in health care [7]. Enhanced collaboration between regulators and technical experts is essential to foster a shared understanding of personal information in LLMs and to promote consistency in data protection regulation [8].
Conclusion
The legislative measures introduced by California represent a significant step towards regulating AI technologies, particularly in terms of privacy, transparency [1] [3] [4] [5], and consumer protection [1]. These laws will have far-reaching implications for developers, businesses, and consumers [3], necessitating careful compliance and adaptation strategies. As AI continues to evolve, ongoing collaboration between legal, technical [1] [8], and regulatory stakeholders will be crucial to address emerging challenges and ensure that AI technologies are developed and deployed responsibly.
References
[1] https://www.lexology.com/library/detail.aspx?g=04ad117a-98be-4e35-9b7e-73f3ce6e6e8d
[2] https://www.jdsupra.com/legalnews/california-privacy-and-ai-roundup-what-2126005/
[3] https://www.manatt.com/insights/newsletters/client-alert/california-privacy-and-ai-roundup-what-passed-and
[4] https://www.jdsupra.com/legalnews/california-s-new-ai-laws-focus-on-5218852/
[5] https://www.jdsupra.com/legalnews/ai-transparency-and-compliance-key-9360749/
[6] https://www.jdsupra.com/legalnews/the-privacy-and-data-security-impact-of-4063060/
[7] https://www.engage.hoganlovells.com/knowledgeservices/news/new-california-laws-impact-uses-of-ai-by-health-care-providers-insurers-and-vendors/
[8] https://fpf.org/blog/do-llms-contain-personal-information-california-ab-1008-highlights-evolving-complex-techno-legal-debate/
