Accountability in AI medicine: A critical appraisal of ChatGPT in patient self-management and screening
Article information
Dear Editor,
We have read with great interest the recent publication by Yeo et al. [1], which presents an optimistic view of the potential for ChatGPT in responding to patient queries about cirrhosis and hepatocellular carcinoma (HCC). The study suggests that ChatGPT exhibits promising capabilities, which could contribute to increased awareness and enhanced management efficacy of these conditions. The authors propose that, with adequate training, language generation systems like ChatGPT could be further optimized to improve their performance in the context of patient selfmanagement.
While we commend the authors for their insightful work, we feel compelled to raise some pertinent concerns. Specifically, we question the accountability for any inaccuracies in the responses provided by ChatGPT. A previous study has indicated that, when compared to the guidelines, only 26% of ChatGPT’s responses to clinical questions were fully accurate, while 48% of the answers contained errors or were misleading [2]. This discrepancy could result in a substantial number of patients receiving misleading information, with potentially grave consequences.
Despite the significant advancements in ChatGPT’s conversational and interactive capabilities, the potential for factual inaccuracies remains [3]. It is imperative that human stakeholders take on the responsibility to ensure ChatGPT’s effectiveness and appropriate use in healthcare settings.
When medical decisions influenced by ChatGPT result in harm to individuals, the issue of accountability becomes particularly critical. It is necessary to clearly define the principles of accountability. However, the current legal frameworks in most jurisdictions offer little clarity on this issue, and it is uncertain whether traditional product liability theories apply to ChatGPT. Although medical software is clas-sified as medical devices in courts across Europe and the United States, developers may categorize ChatGPT as a service rather than a product to evade liability, complicating the process for patients seeking redress. Therefore, it must be clarified who bears the responsibility when ChatGPT is involved in medical decision-making and causes harm—the service provider, the technology supporter, or the user. This determination may require a joint assessment of the roles of the algorithmic model, training data, and user input in the tortious act, based on specific circumstances.
Moreover, it is essential to establish a risk transfer mechanism. Drawing on the insurance systems established in some countries, the socialized transfer of risks can be achieved through the purchase of commercial insurance, ensuring that victims receive timely compensation when liability is difficult to define.
Finally, ethical and legal training is indispensable. Medical professionals must be trained on the ethical and legal frameworks concerning artificial intelligence to ensure they can make responsible decisions when using ChatGPT.
We are supportive of the potential application of ChatGPT in the medical field. Nevertheless, we believe that until ethical concerns, such as accountability, are adequately addressed, clinicians should refrain from endorsing ChatGPT for patient self-management.
Notes
Authors’ contribution
Jiawen Wang: Writing - Original Draft. Tian Xia: Writing - Review & Editing.
Conflicts of Interest
The authors have no conflicts to disclose.
Abbreviations
HCC
hepatocellular carcinoma