Introduction
Recent legislative developments in California are set to significantly influence the regulation of artificial intelligence (AI) in healthcare, particularly in areas such as patient communications and medical necessity determinations [1] [6]. These legislative actions aim to ensure transparency, fairness [5], and the reduction of biases in patient care as AI deployment in healthcare continues to grow.
Description
Recent legislative actions in California are poised to significantly impact the regulation of artificial intelligence (AI) in health care, particularly concerning patient communications and medical necessity determinations [1] [6]. As the deployment of AI in healthcare expands, key legislation such as Assembly Bill 3030 and Senate Bill 1120 has been enacted to ensure transparency, fairness [5], and the mitigation of biases in patient care [5]. Effective January 1, 2025 [1] [5] [6], Assembly Bill 3030 mandates that health facilities [1] [2] [4] [6], clinics [1] [2] [4] [5] [6], and physician’s offices disclose their use of generative AI in patient communications [1] [6]. This includes a requirement to provide a disclaimer indicating that the communication was AI-generated, along with clear instructions for patients to contact human health care providers [6]. Notably, this requirement is waived if the AI-generated communication has been reviewed by a licensed health care provider [4]. Violations of this law will subject physicians to the jurisdiction of the Medical Board of California or the Osteopathic Medical Board of California [2].
Senate Bill 1120 [2] [4], effective in early 2025 [5], amends the Health and Safety Code and Insurance Code to establish compliance standards for health care service plans and disability insurers that utilize AI in utilization review and management [6]. This law allows AI to assist in medical necessity determinations [6], provided that a licensed physician or qualified health care professional reviews and makes the final decision [6]. Furthermore, AI’s determinations must be based on individual patient information and relevant clinical circumstances [2], rather than relying solely on group datasets, and must not replace health care provider decision-making or discriminate against patients in violation of applicable laws [2]. Health care entities are encouraged to seek guidance from the Department of Managed Health Care and the Department of Insurance for implementation [6].
In addition to these state-level regulations, Assembly Bill 2013 requires AI developers of generative AI systems available to Californians to disclose the data used for training these systems by January 1, 2026 [6]. This law applies to entities that substantially modify generative AI systems [6], with certain exclusions regarding the types of data that must be disclosed. Moreover, SB 942 requires large entities to disclose AI-generated content and provide a free AI detection tool for users starting in 2025 [5]. The California AI Transparency Act further obligates AI system providers to develop tools for detecting AI-generated content and mandates transparency regarding interactions with AI systems.
At the federal level [1] [6], while there is no comprehensive framework for AI regulation in healthcare [5], the Centers for Medicare and Medicaid Services (CMS) has outlined requirements for AI use in medical necessity determinations [6], effective January 1, 2024 [1] [5] [6]. These requirements emphasize that decisions must consider individual circumstances rather than relying solely on algorithms [6]. Additionally, the Department of Health and Human Services’ Office of the National Coordinator for Health Information Technology mandates transparency regarding training data for predictive decision support interventions under its HTI-1 Final Rule [6], effective March 11, 2024 [1] [6]. This includes requirements for developers to provide detailed information about the design [6], development [1] [3] [5] [6], training [1] [5] [6], and evaluation of these interventions [6].
As state and federal legislative efforts to regulate AI in healthcare continue to evolve [5], healthcare organizations must proactively ensure compliance with these emerging requirements [5]. Implementing governance controls and staying informed about regulatory changes will be crucial for organizations utilizing AI technologies in their operations [5]. Stakeholders are encouraged to stay informed about these developments to navigate the evolving legal landscape surrounding AI in health care effectively [6].
Conclusion
The legislative measures in California, alongside federal guidelines, underscore a growing emphasis on transparency, fairness [5], and accountability in the use of AI in healthcare. These regulations are expected to shape the future of AI deployment in the sector, ensuring that patient care remains unbiased and that AI technologies are used responsibly. Healthcare organizations must remain vigilant and adaptable to these changes to maintain compliance and uphold the integrity of patient care.
References
[1] https://www.jdsupra.com/legalnews/new-california-laws-impact-uses-of-ai-8502088/
[2] https://www.jdsupra.com/legalnews/zooming-in-on-ai-9-understanding-6034685/
[3] https://www.gunder.com/en/news-insights/insights/client-insight-artificial-intelligence-insights-the-current-regulatory-landscape-published-october-29-2024-by-katie-gardner-aaron-rubin-and-erica-davis
[4] https://www.aoshearman.com/en/insights/ao-shearman-on-tech/zooming-in-on-ai-9-understanding-californias-new-ai-legislation
[5] https://hooperlundy.com/the-ai-landscape-california-and-other-state-legislative-efforts-to-regulate-use-of-ai-in-health-care/
[6] https://www.engage.hoganlovells.com/knowledgeservices/news/new-california-laws-impact-uses-of-ai-by-health-care-providers-insurers-and-vendors/
