, Hongwei Ji11,12
1Department of Endocrinology and Metabolic Hepatology, The Affiliated Hospital of Qingdao University, Qingdao, China
2Department of Gastroenterology and Hepatology, Shanghai East Hospital, Tongji University, Shanghai, China
3Department of Gastroenterology and Hepatology, The Affiliated Hospital of Qingdao University, Qingdao, China
4Shandong Provincial Key Laboratory of Metabolic Diseases and Qingdao Key Laboratory of Gout, the Affiliated Hospital of Qingdao University, Qingdao, China
5Department of Infectious Disease and Hepatology, The Affiliated Hospital of Qingdao University, Qingdao, China
6Beijing Key Laboratory of Ophthalmology and Visual Sciences, Beijing Tongren Eye Center, Beijing Institute of Ophthalmology, Beijing Tongren Hospital, Capital Medical University, Beijing, China
7Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA
8Yong Loo Lin School of Medicine, National University of Singapore, Singapore
9Singapore Eye Research Institute, Singapore National Eye Center, Singapore
10Department of Computer Science and Engineering, Shanghai JiaoTong University, Shanghai, China
11Tsinghua Medicine, Tsinghua University, Beijing, China
12Department of Internal Medicine, Beijing Tsinghua Changgung Hospital, Beijing, China
Copyright © 2024 by The Korean Association for the Study of the Liver
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Authors’ contribution
Acquisition of data: Yiwen Zhang, Hongwei Ji, Liwei Wu, Zepeng Mu. Analysis and interpretation of data: Hongwei Ji. Drafting of the manuscript: All authors. Critical revision of the manuscript for important intellectual content: All authors. Statistical analysis: Yiwen Zhang Hongwei Ji. Obtained funding: Hongwei Ji. Study supervision: Hongwei Ji.
Conflicts of Interest
The authors have no conflicts to disclose.
| GPT-3.5 | GPT-4 | Bard | Llama2 | Claude2 | |
|---|---|---|---|---|---|
| Appropriateness, n (%) | 24 (80.0) | 29 (96.7) | 27 (90.0) | 27 (90.0) | 24 (80.0) |
| 1. Risk factors | |||||
| Who is more likely to get fatty liver disease? | 1 | 3 | 3 | 3 | 3 |
| What type of diet can help better manage fatty liver disease? | 1 | 0* | 3 | 3 | 3 |
| How does alcohol consumption affect my fatty liver disease, and should I abstain from alcohol completely? | 3 | 3 | 2 | 3 | 2 |
| What type and amount of physical activity is recommended for someone with fatty liver disease? | 3 | 2 | 3 | 3 | 3 |
| I have a lean build; how did I develop fatty liver disease? | 3 | 3 | 2 | 3 | 3 |
| How does my family’s health history impact the monitoring of my fatty liver disease? | 3 | 3 | 3 | 3 | 1 |
| 2. Test and diagnosis | |||||
| What are the early signs and symptoms of fatty liver disease that I should be aware of? | 3 | 2 | 3 | 2 | 3 |
| How is fatty liver disease diagnosed? | 3 | 3 | 3 | 1 | 3 |
| Are there different types of fatty liver disease, and how do they differ? | 3 | 3 | 2 | 2 | 2 |
| Are there different stages of fatty liver disease, and how do they differ? | 1 | 3 | 1 | 1 | 1 |
| At what point is a liver biopsy recommended for individuals with fatty liver disease? | 0 | 2 | 2 | 1 | 2 |
| What is the role of imaging tests such as ultrasound, MRI, or CT scan in diagnosing fatty liver disease? | 3 | 3 | 2 | 3 | 2 |
| I have fatty liver disease and my ALT is 100 U/L; how should I interpret this? | 2 | 2 | 3 | 2 | 1 |
| I have fatty liver disease and my FIB-4 score is 1.1; how should I interpret this? | 3 | 3 | 2 | 3 | 3 |
| 3. Treatment | |||||
| How is fatty liver disease treated? | 3 | 2 | 3 | 3 | 3 |
| Are there any specific medications that are commonly prescribed for fatty liver disease? | 2 | 3 | 3 | 2 | 1 |
| How should medication be used to avoid liver damage in fatty liver disease? | 1 | 3 | 1 | 2 | 3 |
| What lifestyle interventions can aid in the treatment of fatty liver disease? | 2 | 3 | 2 | 3 | 3 |
| In severe cases, are there surgical options available for treating fatty liver disease? | 2 | 3 | 3 | 3 | 3 |
| 4. Follow up and monitoring | |||||
| How often should I be monitored if I have fatty liver disease? | 2 | 2 | 3 | 2 | 3 |
| I have fatty liver disease. What tests or procedures will be performed during follow-up appointments? | 3 | 2 | 3 | 3 | 3 |
| I have fatty liver disease. What signs or symptoms should prompt me to seek immediate medical attention? | 3 | 3 | 3 | 3 | 3 |
| 5. Comorbidities and prognosis | |||||
| What other health conditions are commonly associated with fatty liver disease? | 3 | 2 | 3 | 3 | 3 |
| What is the typical prognosis for someone with fatty liver disease? | 3 | 3 | 3 | 3 | 3 |
| Is there an increased risk of heart disease when living with fatty liver disease? | 3 | 3 | 3 | 2 | 1 |
| How does fatty liver disease affect diabetes management, and vice versa? | 3 | 3 | 3 | 3 | 3 |
| Does having fatty liver disease increases my risk for liver cancer? | 2 | 3 | 2 | 2 | 1 |
| How can I understand the stage of my fatty liver disease and the potential progression over time? | 1 | 2 | 3 | 2 | 3 |
| Can fatty liver disease be reversed? | 3 | 3 | 3 | 3 | 3 |
| Can children develop fatty liver disease, and if so, how does it affect their health as they grow up? | 2 | 3 | 1 | 3 | 2 |
The numbers represent the count of ‘appropriate’ ratings given by three reviewers. ≥2 is considered ‘appropriate.
*Response from ChatGPT-4 for this particular query: Fatty liver disease, which includes conditions such as non-alcoholic fatty liver disease (NAFLD) and non-alcoholic steatohepatitis (NASH)… 5. Hydrate: - Drink plenty of water throughout the day to stay hydrated…7. Regular Physical Activity: - Along with a healthy diet, engaging in regular physical activity can help in managing the condition…
Two graders found the advice to ‘drink plenty of water throughout the day’ to be inappropriate, noting that while hydration is important, excessive water intake can disrupt electrolyte balance, particularly in individuals with conditions like advanced liver disease that affect water regulation. One grader deemed the physical activity suggestion irrelevant to the query at hand. Additionally, the simplified examples such as NAFLD and NASH was seen as inappropriate by two graders, as the query specifically pertains to dietary modifications and such simplifications could downplay the role of alcohol and other dietary factors.
| GPT-3.5 | GPT-4 | Bard | Llama2 | Claude2 | |
|---|---|---|---|---|---|
| Appropriateness, n (%) | 24 (80.0) | 29 (96.7) | 27 (90.0) | 27 (90.0) | 24 (80.0) |
| 1. Risk factors | |||||
| Who is more likely to get fatty liver disease? | 1 | 3 | 3 | 3 | 3 |
| What type of diet can help better manage fatty liver disease? | 1 | 0* | 3 | 3 | 3 |
| How does alcohol consumption affect my fatty liver disease, and should I abstain from alcohol completely? | 3 | 3 | 2 | 3 | 2 |
| What type and amount of physical activity is recommended for someone with fatty liver disease? | 3 | 2 | 3 | 3 | 3 |
| I have a lean build; how did I develop fatty liver disease? | 3 | 3 | 2 | 3 | 3 |
| How does my family’s health history impact the monitoring of my fatty liver disease? | 3 | 3 | 3 | 3 | 1 |
| 2. Test and diagnosis | |||||
| What are the early signs and symptoms of fatty liver disease that I should be aware of? | 3 | 2 | 3 | 2 | 3 |
| How is fatty liver disease diagnosed? | 3 | 3 | 3 | 1 | 3 |
| Are there different types of fatty liver disease, and how do they differ? | 3 | 3 | 2 | 2 | 2 |
| Are there different stages of fatty liver disease, and how do they differ? | 1 | 3 | 1 | 1 | 1 |
| At what point is a liver biopsy recommended for individuals with fatty liver disease? | 0 | 2 | 2 | 1 | 2 |
| What is the role of imaging tests such as ultrasound, MRI, or CT scan in diagnosing fatty liver disease? | 3 | 3 | 2 | 3 | 2 |
| I have fatty liver disease and my ALT is 100 U/L; how should I interpret this? | 2 | 2 | 3 | 2 | 1 |
| I have fatty liver disease and my FIB-4 score is 1.1; how should I interpret this? | 3 | 3 | 2 | 3 | 3 |
| 3. Treatment | |||||
| How is fatty liver disease treated? | 3 | 2 | 3 | 3 | 3 |
| Are there any specific medications that are commonly prescribed for fatty liver disease? | 2 | 3 | 3 | 2 | 1 |
| How should medication be used to avoid liver damage in fatty liver disease? | 1 | 3 | 1 | 2 | 3 |
| What lifestyle interventions can aid in the treatment of fatty liver disease? | 2 | 3 | 2 | 3 | 3 |
| In severe cases, are there surgical options available for treating fatty liver disease? | 2 | 3 | 3 | 3 | 3 |
| 4. Follow up and monitoring | |||||
| How often should I be monitored if I have fatty liver disease? | 2 | 2 | 3 | 2 | 3 |
| I have fatty liver disease. What tests or procedures will be performed during follow-up appointments? | 3 | 2 | 3 | 3 | 3 |
| I have fatty liver disease. What signs or symptoms should prompt me to seek immediate medical attention? | 3 | 3 | 3 | 3 | 3 |
| 5. Comorbidities and prognosis | |||||
| What other health conditions are commonly associated with fatty liver disease? | 3 | 2 | 3 | 3 | 3 |
| What is the typical prognosis for someone with fatty liver disease? | 3 | 3 | 3 | 3 | 3 |
| Is there an increased risk of heart disease when living with fatty liver disease? | 3 | 3 | 3 | 2 | 1 |
| How does fatty liver disease affect diabetes management, and vice versa? | 3 | 3 | 3 | 3 | 3 |
| Does having fatty liver disease increases my risk for liver cancer? | 2 | 3 | 2 | 2 | 1 |
| How can I understand the stage of my fatty liver disease and the potential progression over time? | 1 | 2 | 3 | 2 | 3 |
| Can fatty liver disease be reversed? | 3 | 3 | 3 | 3 | 3 |
| Can children develop fatty liver disease, and if so, how does it affect their health as they grow up? | 2 | 3 | 1 | 3 | 2 |
The numbers represent the count of ‘appropriate’ ratings given by three reviewers. ≥2 is considered ‘appropriate.
Response from ChatGPT-4 for this particular query: Fatty liver disease, which includes conditions such as non-alcoholic fatty liver disease (NAFLD) and non-alcoholic steatohepatitis (NASH)… 5. Hydrate: - Drink plenty of water throughout the day to stay hydrated…7. Regular Physical Activity: - Along with a healthy diet, engaging in regular physical activity can help in managing the condition…
Two graders found the advice to ‘drink plenty of water throughout the day’ to be inappropriate, noting that while hydration is important, excessive water intake can disrupt electrolyte balance, particularly in individuals with conditions like advanced liver disease that affect water regulation. One grader deemed the physical activity suggestion irrelevant to the query at hand. Additionally, the simplified examples such as NAFLD and NASH was seen as inappropriate by two graders, as the query specifically pertains to dietary modifications and such simplifications could downplay the role of alcohol and other dietary factors.
