PATIENT DATA AND ARTIFICIAL INTELLIGENCE: BETWEEN OPTIMIZING MEDICAL CARE AND ETHICAL-LEGAL CHALLENGE

In 1970, nephrologist and economist William Benjamin Schwartz wrote in the New England Journal of Medicine that computer science would significantly transform medical practice. He predicted that computers would play a revolutionary role in medicine by the year 2000, functioning as a powerful extension of physicians' intellectual capacity.

Five decades later, Schwartz’s prediction has materialized: technological advancements are profoundly impacting medicine, from drug discovery and production to early diagnosis and disease prevention. Medical procedures have become more accurate and effective.

This development also extends to surgeries, which now incorporate innovative solutions such as robotics and Artificial Intelligence (AI)-assisted laparoscopy. These tools offer greater precision in procedures and reduce patients’ recovery time.

Beyond operating rooms, AI’s presence is intensifying in other areas of medicine. It supports clinical practice by processing large volumes of clinical data, aiding in medical decision-making, and improves hospital management by optimizing workflows and organizing care delivery.

Additionally, wearable digital health technologies have been increasingly used in health monitoring, such as smartwatches that track vital signs in real time and generate medical data that can be used preventively to anticipate changes in a patient's clinical condition. A quiet, yet profoundly transformative revolution.

AI is a revolutionary tool in the medical field. A 2009 study identified that 32% of medical errors were related to insufficient time for proper clinical evaluation, which compromised diagnoses. In this context, AI emerges as a resource capable of processing vast amounts of data and efficiently integrating clinical and historical patient information.

However, this innovation brings complex dilemmas. The pressing question is: who will be held liable for medical errors caused by AI?

The Boundary Between Innovation and Liability

The president of the International Chair in Bioethics, Rui Nunes, emphasized that the massive processing of data and the timely extraction of conclusions were unimaginable until recently. This scenario demands deep ethical and legal reflection to guide the application of AI in healthcare. In this context, he proposed the following question: Who will be held accountable for the adverse outcome of a procedure carried out based on the recommendation of artificial intelligence?

Answering this requires an interdisciplinary approach between Law, Bioethics, and Medicine. It is essential for the legal framework to keep pace with technological advancement, establishing clear parameters regarding civil liability, medical ethics, and, above all, the use and processing of patients’ personal data.

Neuroscientist Philipp Kellmeyer has also analyzed the benefits, risks, and ethical-legal challenges of using brain data obtained through neurotechnological devices. In his article Big Brain Data: On the Responsible Use of Brain Data from Clinical and Consumer-Directed Neurotechnological Devices, he raises concerns regarding privacy, security, autonomy, and discrimination in AI usage. He suggests public participation in regulatory discussions to establish an ethical model that balances technological innovation with the protection of fundamental rights.

Regulation and Data Protection: A Global Challenge

To balance innovation and fundamental rights, the European Union created the General Data Protection Regulation (GDPR) in 2016, which set rules to protect the fundamental rights of natural persons concerning the processing of personal data, while ensuring the free flow of data within the EU.

Brazil, in turn, has the Lei Geral de Proteção de Dados (LGPD) in effect since 2018. The LGPD recognizes health data as sensitive personal data and, therefore, imposes additional safeguards for its processing. Article 11 of the Law limits the legal grounds for processing such data—including physical or mental health status, medical history, diagnoses, test results, treatments, genetic and biometric information—requiring, in most cases, the data subject’s free, informed, specific, and unequivocal consent.

On the other hand, the LGPD provides for a lawful basis for data processing in the context of health protection, where consent is not required and cannot be revoked (Article 11, II, item "f"). However, this legal provision raises concerns. The Conselho Federal de Medicina - CFM (Federal Council of Medicine) maintains that consent remains mandatory for the processing of personal data in the medical field. Additionally, CFM Resolution No. 1.605/2000 provides that a physician may not disclose the content of medical records or charts without the patient's consent.

According to the International Code of Medical Ethics of the World Medical Association, AI does not replace clinical judgment, and physicians must exercise their profession with integrity, responsibility, and independence. Physicians are required to inform patients of AI use as part of their duty to disclose medical risks, complications, and outcomes, including through the Informed Consent Form.

A practical issue arises here: the article 8, §5 of the LGPD guarantees patients (data subjects) the right to revoke their consent at any time and thus immediately halt the processing of their data. However, once data has been used to train AI algorithms, its effects may be irreversible. In such cases, the physician must ensure that the data will not be reused, provide accountability to the data subject, and maintain transparency about the technical limits of consent revocation. This safeguards the right to informational self-determination and privacy.

Even though the deletion of medical records is prohibited by Law No. 13.787/2018, which mandates record retention for twenty (20) years from the last entry, the patient may request the discontinuation of data sharing with AI systems.

Meanwhile, the duty of medical confidentiality remains fully applicable in the context of digital technologies. When medical data is input into AI systems, privacy and confidentiality must be strictly protected.

Medical Supervision and the Centrality of the Human Being

Currently, Bill No. 266 of 2024 (Projeto de Lei - PL 266/24) is under review in the Brazilian Federal Senate. Its main proposal is to regulate the use of AI in healthcare. The draft bill stipulates that AI must operate under the direct supervision of a licensed physician. According to the draft bill, autonomous use of AI without a supervising physician constitutes the unauthorized practice of medicine.

As the International Code of Medical Ethics reminds us, AI does not replace clinical judgment. The physician remains responsible for exercising the profession with integrity, responsibility, and independent judgment.

Patients must not be viewed merely as a collection of analyzable data. They must be recognized as rights-bearing individuals at the center of the medical relationship. AI should not replace the ethical and clinical judgment of medical professionals, who must use it as an additional care tool—fulfilling Albert Einstein’s famous insight that “the human spirit must prevail over technology.”

Therefore, AI must be used alongside rigorous safety protocols, constant validation of data, and qualified human oversight. After all, the right to health is a human right that encompasses not only access to healthcare services but also the assurance of quality, accountability, and respect for human dignity.

About the Authors

Gustavo Borges

President of LABSUL – Laboratory of Human Rights and New Technologies

Professor of Human Rights and New Technologies in the Master’s Program inHuman Rights at UNESC, Brazil

Consultant in Law and Technology

Mariana Carlessi

Researcher at LABSUL

Master of Human Rights from UNESC, Brazil

Professor at UNESC, Brazil

Attorney-at-law in Brazil