Debating artificial intelligence in patients’ care: A double-edged sword

Arguments raised by AI advocates

The application of AI in the healthcare industry has become highly valuable and promising, and could assist in mitigating various struggles and challenges faced by healthcare systems.^[6] In the era of clinical diagnostics, a systematic review and meta-analysis of 14 studies showed a higher diagnostic performance of AI compared to healthcare providers using imaging in which reported pooled sensitivity was 87.0% (95% confidence interval [CI] : 83.0%-90.2%) for deep learning models and 86.4% (95% CI : 79.9%-91.0%) for healthcare professionals, along with a pooled specificity of 92.5% (95% CI : 85.1%-96.4%) for deep learning models and 90.5% (95% CI: 80.6%-95.7%) for healthcare professionals.^[7] In fact, you can find many examples of this in real life. A very specific example is the detection of tumors in radiology scans. Tumor cells can be quickly and safely detected by scans and imaging modalities such as computed tomography and magnetic resonance imaging, and others. Further, ML and AI have shown promise in developing the automated detection mechanism through various modalities.^[8] In the field of improving healthcare services, AI can be used to assist in scheduling appointments, offer mental health support, provide remote monitoring, and reply to patients’ enquiries.^[9,10] AI can be beneficial for the entire healthcare community, including doctors, clinicians, nurses, administrative staff, and managers, in assisting with and accomplishing various tasks, ranging from simple tasks such as discharge summary documentation to performing sophisticated surgeries.

For those concerned about overriding the individuality of healthcare providers, scientific evidence emphasizes a cultural shift to view AI as a healthcare delivery enhancer and job creator, rather than a threat.^[11] Rather, AI will allow providers, clinicians, and staff to focus on more top-of-license skill sets and activities. Based on the current statistics, by 2030, the world will have a deficit of 18 million healthcare professionals.^[12,13] We hold the view that AI amplifies and augments, rather than replaces, human intelligence.

AI-powered electronic health records (AI-powered EHR) is another feature that has the potential to optimize healthcare workflow through the utilization of innovations such as speech recognition, automated scheduling, and creation of accurate medical records. In one observational study, a group of medical residents was found to spend an average of 1.7 h only directly with patients during their shift, and 5.2 h using computers.^[14] This alarming observation was evident in several other studies and is reported as a frequent contributor to physician burnout.^[15,16] AI innovations can potentially empower healthcare workers to overcome EHR systems’ inefficiencies and provide more direct patient care with a deeper level of attention to detail.

For those with contemporary ethical concerns in the use of AI, frameworks are being introduced as one type of ethics tool, among others, such as the human-centered AI canvas.^[17] In fact, there is published evidence as a regulation for the use of AI to ensure and sustain patient safety and ethical consideration.^[18-21] The European Union took a regulatory stand by publishing the European Medical Device Regulation. They have posted the AI Act to organize AI products and services from the development phase to their application with respect to risk management, data governance, human oversight, transparency, accuracy, robustness, and cybersecurity.^[21] In 2019, documents were published by the National Institute for Health and Care Excellence in collaboration with the National Health Service on the Evidence Standards Framework for Digital Health Technologies.^[22]

Given the compelling evidence supporting the efficacy and safety of AI in patient care, the integration of AI technologies holds significant promise for enhancing healthcare outcomes. Therefore, it is essential to advocate for the adoption and utilization of AI in patient care to realize its potential benefits fully.

Arguments raised by AI opponents

Despite the potential benefits of AI in improving medical outcomes, the lack of transparency in AI algorithms poses significant challenges to their adoption and trustworthiness. AI in healthcare raises significant ethical concerns related to data privacy and security. Sensitive health data can be vulnerable to breaches, and current regulatory frameworks may not adequately address the complexities introduced by AI technologies.^[23] In addition, AI systems can perpetuate and exacerbate existing biases present in their training data, leading to discriminatory practices.^[24] This can result in unequal healthcare outcomes, particularly for minority and marginalized populations. The potential for hacking and misuse of sensitive patient information further exacerbates these concerns, emphasizing the need for data protection measures.^[25]

A Finnish court sentenced 26-year-old hacker Aleksanteri Kivimäki to 6 years and 3 months in prison for hacking tens of thousands of patient records at Vastaamo psychotherapy center and demanding ransom from patients.^[26] The 2020 data breach, which led to the filing of around 24,000 criminal complaints, revealed significant privacy violations and caused severe psychological harm, including suicides among some victims. Kivimäki, extradited from France in 2023, was found guilty of aggravated data breach, blackmail attempts, and dissemination of private information.^[26]

Integrating AI into medical practice risks diminishing the essential human elements of empathy and compassion in patient care.^[27,28] These human attributes are crucial for patient trust and effective treatment, particularly in fields such as psychiatry and pediatrics, where emotional support is vital.^[28] AI cannot exhibit genuine empathy or provide emotional support, which are essential for holistic patient care. The human touch and voice remain irreplaceable in offering hope and assurance during treatment, highlighting the limitations of AI in replicating these critical aspects of healthcare.^[29]

Furthermore, the widespread adoption of AI in healthcare could exacerbate social inequalities, as not all communities have equal access to these advanced technologies.^[30] This could widen the gap between developed and developing countries and among different socio-economic groups within countries. AI’s ability to perform tasks traditionally done by healthcare professionals also raises concerns about mechanical failures and errors. AI systems, especially in robot-assisted surgeries, can experience malfunctions, leading to severe consequences such as accidental injuries or nerve damage.^[31] Thus, while AI holds promise for enhancing healthcare, these significant ethical, practical, and social challenges must be carefully addressed.^[32]

Google developed an AI algorithm for breast cancer screening, which demonstrated promising performance but lacked transparency in its development and technical details.^[33-35] This lack of transparency raises concerns about the usefulness and safety of such AI tools, prompting calls for greater transparency in medical AI. Transparency in AI is described in terms of traceability and explainability. Traceability involves documenting the entire AI development process and monitoring its performance after deployment, while explainability refers to providing transparency for each AI prediction and decision. However, existing AI tools often lack full traceability and are described as “black box AI,” making it difficult for clinicians to understand their decision-making processes and identify errors.^[36,37]

Other issues surrounding the use of AI in healthcare are deeply intertwined with concerns about informed consent, data privacy, and cybersecurity. Informed consent, a fundamental aspect of patient autonomy and ethical healthcare practice, has become increasingly complex in the digital age, where AI algorithms and opaque decision-making processes may limit patients’ understanding and control over their health data.^[38,39]

Opaque AI algorithms and complicated consent forms hinder patient autonomy and shared decision-making, making it difficult for patients to comprehend how their data is used and to opt out of sharing it. This complexity is exemplified by the case of the Royal Free NHS Trust’s unauthorized transfer of patient records to DeepMind in 2016, which led to a breach of data protection laws and raised questions about the erosion of privacy rights in the pursuit of innovation.^[40]

Moreover, the use of AI in healthcare introduces risks of data security breaches, as demonstrated by incidents such as the Cense AI data breach in 2020, which exposed sensitive patient information to the public. Data repurposing, or “function creep,” poses another concern, as health-related data may be used for non-healthcare purposes, compromising patient privacy and autonomy.^[41-43] AI tools in healthcare are vulnerable to cyberattacks, with potentially dire consequences for patient safety. Incidents such as the cyberattack on Düsseldorf University Hospital in 2020, which resulted in a patient’s death, underscore the urgent need for robust cybersecurity measures to protect against such threats.^[44] Similarly, ransomware attacks on healthcare systems, such as the Elekta attack in 2021, can disrupt patient care and compromise sensitive medical data.^[45] Even personal medical devices controlled by AI, such as insulin pumps, are susceptible to hacking, raising concerns about the safety and integrity of AI-driven healthcare technologies.^[44,45]

Moving forward

Given the complexity of integrating AI into patient care, a strategic and subtle approach to its implementation is essential. Policymakers, healthcare providers, and researchers must collaborate to develop clear guidelines that maximize AI’s potential benefits while addressing its ethical, safety, and operational challenges. A key concern is the phenomenon of AI hallucination – inaccuracies in AI-generated results – which can have serious consequences on patient care if left unchecked.^[46] This accentuates the need for healthcare professionals to fully understand the risks posed by inaccurate information and to exercise critical judgment when applying AI in clinical contexts. Establishing a robust regulatory framework is essential, encompassing strong data protection measures, transparency, accountability, and inclusivity for all patient populations, alongside raising awareness of privacy risks.^[47] The use of open-access AI tools heightens these risks, as the vulnerability of healthcare data to breaches increases substantially.^[48] Continuous research is essential to refine AI technologies, enhance cybersecurity, and ensure their ethical deployment in healthcare. Investing in AI education for healthcare professionals is crucial for seamlessly integrating these technologies into clinical practice, enhancing their role alongside human expertise.