Skip to main navigation Skip to main content

CMH : Clinical and Molecular Hepatology

OPEN ACCESS
Search
Advanced Search
ABOUT
BROWSE ARTICLES
FOR CONTRIBUTORS

Articles

Page Path

Original Article

Artificial intelligence predicts direct-acting antivirals failure among hepatitis C virus patients: A nationwide hepatitis C virus registry program

Ming-Ying Lu1,2,3, Chung-Feng Huang2,3,4, Chao-Hung Hung5, Chi‐Ming Tai6,7, Lein-Ray Mo8, Hsing-Tao Kuo1,9, Kuo-Chih Tseng10,11, Ching-Chu Lo12, Ming-Jong Bair13,14, Szu-Jen Wang15, Jee-Fu Huang2,3, Ming-Lun Yeh2,3, Chun-Ting Chen16,17, Ming-Chang Tsai18, Chien-Wei Huang19, Pei-Lun Lee20, Tzeng-Hue Yang21, Yi-Hsiang Huang22,23, Lee-Won Chong24,25, Chien-Lin Chen26, Chi-Chieh Yang27, Sheng‐Shun Yang28, Pin-Nan Cheng29, Tsai-Yuan Hsieh16, Jui-Ting Hu30, Wen-Chih Wu31, Chien-Yu Cheng32, Guei-Ying Chen33, Guo-Xiong Zhou34, Wei-Lun Tsai35, Chien-Neng Kao36, Chih-Lang Lin37, Chia-Chi Wang38, Ta-Ya Lin39, Chih‐Lin Lin40, Wei-Wen Su41, Tzong-Hsi Lee42, Te-Sheng Chang43, Chun-Jen Liu44, Chia-Yen Dai2,3, Jia-Horng Kao44, Han-Chieh Lin22,23, Wan-Long Chuang2,3, Cheng-Yuan Peng45,46, Chun-Wei- Tsai47, Chi-Yi Chen48orcid, Ming-Lung Yu1,2,3,5orcid, TACR Study Group
Clinical and Molecular Hepatology 2024;30(1):64-79.
Published online: November 21, 2023

1School of Medicine and Doctoral Program of Clinical and Experimental Medicine, College of Medicine and Center of Excellence for Metabolic Associated Fatty Liver Disease, National Sun Yat-sen University, Kaohsiung, Taiwan

2Hepatobiliary Division, Department of Internal Medicine and Hepatitis Center, Kaohsiung Medical University Hospital, Kaohsiung Medical University, Kaohsiung, Taiwan

3Hepatitis Research Center, College of Medicine and Center for Liquid Biopsy and Cohort Research, Kaohsiung Medical University, Kaohsiung, Taiwan

4Ph.D. Program in Translational Medicine, College of Medicine, Kaohsiung Medical University, Kaohsiung, and Academia Sinica, Taipei, Taiwan

5Division of Hepatogastroenterology, Department of Internal Medicine, Kaohsiung Chang Gung Memorial Hospital, Kaohsiung, Taiwan

6Division of Gastroenterology and Hepatology, Department of Internal Medicine, E-Da Hospital, I-Shou University, Kaohsiung, Taiwan

7School of Medicine for International Students, College of Medicine, I-Shou University, Kaohsiung, Taiwan

8Division of Gastroenterology, Tainan Municipal Hospital (Managed By Show Chwan Medical Care Corporation), Tainan, Taiwan

9Division of Gastroenterology and Hepatology, Department of Internal Medicine, Chi Mei Medical Center, Yongkang District, Tainan, Taiwan

10Department of Internal Medicine, Dalin Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, Chiayi, Taiwan

11School of Medicine, Tzuchi University, Hualien, Taiwan

12Division of Gastroenterology, Department of Internal Medicine, St. Martin De Porres Hospital, Chiayi, Taiwan

13Division of Gastroenterology, Department of Internal Medicine, Taitung Mackay Memorial Hospital, Taitung, Taiwan

14Mackay Medical College, New Taipei City, Taiwan

15Division of Gastroenterology, Department of Internal Medicine, Yuan’s General Hospital, Kaohsiung, Taiwan

16Division of Gastroenterology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan

17Division of Gastroenterology, Department of Internal Medicine, Tri-Service General Hospital Penghu Branch, National Defense Medical Center, Taipei, Taiwan

18School of Medicine, Chung Shan Medical University, Department of Internal Medicine, Chung Shan Medical University Hospital, Taichung, Taiwan

19Division of Gastroenterology, Kaohsiung Armed Forces General Hospital, Kaohsiung, Taiwan

20Division of Gastroenterology and Hepatology, Department of Internal Medicine, Chi Mei Medical Center, Liouying, Tainan, Taiwan

21Lotung Poh-Ai Hospital, Yilan, Taiwan

22Division of Gastroenterology and Hepatology, Department of Medicine, Taipei Veterans General Hospital, Taipei, Taiwan

23Institute of Clinical Medicine, School of Medicine, National Yang-Ming Chiao Tung University, Taipei, Taiwan

24Division of Hepatology and Gastroenterology, Department of Internal Medicine, Shin Kong Wu Ho-Su Memorial Hospital, Taipei, Taiwan

25School of Medicine, Fu-Jen Catholic University, New Taipei City, Taiwan

26Department of Medicine, Hualien Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation and Tzu Chi University, Hualien, Taiwan

27Department of Gastroenterology, Division of Internal Medicine, Show Chwan Memorial Hospital, Changhua, Taiwan

28Division of Gastroenterology and Hepatology, Department of Internal Medicine, Taichung Veterans General Hospital, Taichung, Taiwan

29Division of Gastroenterology and Hepatology, Department of Internal Medicine, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, Tainan, Taiwan

30Liver Center, Cathay General Hospital, Taipei, Taiwan

31Wen-Chih Wu Clinic, Fengshan, Kaohsiung, Taiwan

32Division of Infectious Diseases, Department of Internal Medicine, Taoyuan General Hospital, Ministry of Health and Welfare, Taoyuan, Taiwan

33Penghu Hospital, Ministry of Health and Welfare, Penghu, Taiwan

34Zhou Guoxiong Clinic, Penghu, Taiwan

35Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan

36National Taiwan University Hospital Hsin-Chu Branch, Hsinchu, Taiwan

37Liver Research Unit, Department of Hepato-Gastroenterology and Community Medicine Research Center, Chang Gung Memorial Hospital at Keelung, College of Medicine, Chang Gung University, Keelung, Taiwan

38Taipei Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation and School of Medicine, Tzu Chi University, Taipei, Taiwan

39Cishan Hospital, Ministry of Health and Welfare, Kaohsiung, Taiwan

40Department of Gastroenterology, Renai Branch, Taipei City Hospital, Taipei, Taiwan

41Department of Gastroenterology and Hepatology, Changhua Christian Hospital, Changhua, Taiwan

42Division of Gastroenterology and Hepatology, Far Eastern Memorial Hospital, New Taipei City, Taiwan

43Division of Hepatogastroenterology, Department of Internal Medicine, Chang Gung Memorial Hospital, Chiayi, Taiwan and College of Medicine, Chang Gung University, Taoyuan, Taiwan

44Hepatitis Research Center and Department of Internal Medicine, National Taiwan University Hospital, Taipei, Taiwan

45Center for Digestive Medicine, Department of Internal Medicine, China Medical University Hospital, Taichung, Taiwan

46School of Medicine, China Medical University, Taichung, Taiwan

47Department of Computer Science and Engineering, National Sun Yat-sen University, Kaohsiung, Taiwan

48Division of Gastroenterology and Hepatology, Department of Medicine, Ditmanson Medical Foundation Chiayi Christian Hospital, Chiayi, Taiwan

Corresponding author : Ming-Lung Yu Hepatobiliary Division, Department of Internal Medicine, Kaohsiung Medical University Hospital, 100 Tzyou Road, Kaohsiung City 807, Taiwan Tel: +886-7-312-1101 ext. 7475, Fax: +886-7-312-3955, E-mail: fish6069@gmail.com
Chi-Yi Chen Department of Internal Medicine, Chiayi Christian Hospital, Chiayi, Taiwan No.539, Zhongxiao Rd., East Dist., Chiayi City 600566, Taiwan Tel: +886-5-2765041 ext. 2535, Fax: +886-5-2763713, E-mail: 5137ccy@gmail.com

Editor: Paul Kwo, Stanford University, USA

• Received: August 2, 2023   • Revised: November 2, 2023   • Accepted: November 20, 2023

Copyright © 2024 by The Korean Association for the Study of the Liver

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

  • 11,463 Views
  • 197 Download
  • 17 Web of Science
  • 15 Crossref
  • 22 Scopus
prev next

Full Article

Download PDF Download PDF
See letter "Toward hepatitis C virus elimination using artificial intelligence" in Volume 30 on page 147.
  • Background/Aims
    Despite the high efficacy of direct-acting antivirals (DAAs), approximately 1–3% of hepatitis C virus (HCV) patients fail to achieve a sustained virological response. We conducted a nationwide study to investigate risk factors associated with DAA treatment failure. Machine-learning algorithms have been applied to discriminate subjects who may fail to respond to DAA therapy.
  • Methods
    We analyzed the Taiwan HCV Registry Program database to explore predictors of DAA failure in HCV patients. Fifty-five host and virological features were assessed using multivariate logistic regression, decision tree, random forest, eXtreme Gradient Boosting (XGBoost), and artificial neural network. The primary outcome was undetectable HCV RNA at 12 weeks after the end of treatment.
  • Results
    The training (n=23,955) and validation (n=10,346) datasets had similar baseline demographics, with an overall DAA failure rate of 1.6% (n=538). Multivariate logistic regression analysis revealed that liver cirrhosis, hepatocellular carcinoma, poor DAA adherence, and higher hemoglobin A1c were significantly associated with virological failure. XGBoost outperformed the other algorithms and logistic regression models, with an area under the receiver operating characteristic curve of 1.000 in the training dataset and 0.803 in the validation dataset. The top five predictors of treatment failure were HCV RNA, body mass index, α-fetoprotein, platelets, and FIB-4 index. The accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of the XGBoost model (cutoff value=0.5) were 99.5%, 69.7%, 99.9%, 97.4%, and 99.5%, respectively, for the entire dataset.
  • Conclusions
    Machine learning algorithms effectively provide risk stratification for DAA failure and additional information on the factors associated with DAA failure.
  • Keywords: Hepatitis C virus; Antiviral agents; Artificial intelligence; Machine learning; Algorithms
• Approximately 1–3% of HCV patients experience DAA therapy failure. We conducted a nationwide study using AI to investigate the risk factors for DAA failure. The AI models outperformed the conventional logistic regression models. The AI model showed that subjects with features such as high HCV RNA levels, active hepatocellular carcinoma, or decompensated liver cirrhosis were prone to virological failure. Machine learning algorithms facilitate risk stratification in DAA failure and provide additional information on factors associated with DAA failure.
Graphical Abstract
Direct-acting antivirals (DAA) have changed the treatment landscape for patients infected with hepatitis C virus (HCV). However, despite the DAA efficacy being up to 97% across all HCV genotypes, approximately 1–3% of HCV patients fail to achieve a sustained virological response (SVR) [1,2]. Factors associated with DAA failure generally include decompensated liver cirrhosis, resistance-associated substitutions (RASs), the presence of hepatocellular carcinoma (HCC), prior treatment failures, and poor drug adherence [3,4]. As comprehensive HCV elimination programs are advocated worldwide, an increasing number of patients with HCV infection are expected to require DAA salvage therapy. Thus, all the risk factors associated with DAA failure must be considered simultaneously to reduce the retreatment burden.
Artificial intelligence (AI) emerged as a powerful tool for disease diagnosis and risk assessment in healthcare. Factors contributing to treatment failure vary among individuals, making such heterogeneous data and complex interactions difficult to evaluate through regression methods. Moreover, conventional statistical methods can only handle linear data. Alternatively, machine-learning (ML) approaches can process both linear and nonlinear information and recognize the hidden relationships between variables and outcomes in big data [5]. ML algorithms can be classified into supervised and unsupervised algorithms. Supervised ML is suitable for handling annotated data, whereas unsupervised ML can process datasets that lack class labels. Common supervised ML algorithms include decision trees (DT), random forest (RF), eXtreme Gradient Boosting (XGBoost), and artificial neural network (ANN) [6,7]. Thus, AI provides a new approach to understanding diseases by integrating multidimensional data with “automatic learning” [8]. Advances in AI have made it possible to serve as a decision-support tool and improve diagnostic quality in healthcare.
We conducted a real-world, multicenter study using the Taiwan HCV Registry (TACR) database, aiming to explore the risk factors associated with DAA failure using artificial intelligence. We applied artificial intelligence to quickly distinguish HCV patients prone to virological failure.
Subjects
The TACR Program is a nationwide HCV-registered platform implemented by the Taiwan Association for the Study of the Liver since 2020 [9]. The TACR conducted a real-world, multicenter, prospective cohort study of DAA therapy. A total of 34301 chronic hepatitis C patients >=18 years old who received DAAs with available SVR12 data were enrolled in this study. Patients with HCV who died during treatment or were lost to follow-up within 12 weeks after the completion of therapy were excluded from our study. The baseline demographics and virological characteristics before and after antiviral therapy were recorded in the TACR database. The primary outcome was the achievement of sustained virological response (SVR12), defined as undetectable HCV RNA in the serum after 12 weeks of end-of-treatment. The choice of antiviral regimens followed the international HCV treatment guidelines [10,11] and the reimbursement criteria of the Taiwan National Health Insurance Administration [2]. This study was approved by the Institutional Review Board of Kaohsiung Medical University Hospital and adhered to the Declaration of Helsinki. Written informed consent was obtained from all the participants.
Machine learning models
The subjects were randomly assigned to a 70% training dataset (n=23,955) and a 30% validation dataset (n=10,346). Fifty-five host, virological, and on-treatment features were input into the ML models (Supplementary Table 1). The algorithms included DT, RF, XGBoost, and ANN. ML analysis was performed using the rpart, randomForest, xgboost, and neural network packages of R software (https://www.r-project.org/). As some algorithms cannot handle missing data, missing data (14.3%) were imputed using the k-nearest neighbor method before generating predictions [12]. The best model was used for risk stratification of patients with DAA treatment failure. Patients were further divided into subgroups using deciles of predicted risk probability to allow for more granular management of high-risk patients.
The performances of the ML models were assessed using the area under the receiver operating characteristic curve (AUROC), accuracy of the confusion matrix, precision-recall curve, and F1-score [13]. A precision-recall curve closer to the upper-right corner indicates better performance. The F1-score is the weighted average of precision and recall and is favorable under class imbalance in the dataset. The F1-score ranged from 0 to 1; the predictive model with an F1-score closer to 1 is considered better. The Delong test was used to compare the differences in the AUC of the ROC curves [14]. The codes for the ML models are presented in the Supplementary Materials.
Statistical analyses
Student’s t-test was used to compare continuous variables. Categorical variables were evaluated using the chi-square (X2) or Fisher’s exact test. Multivariate logistic regression analysis was performed to determine independent risk factors associated with treatment failure. Data were analyzed using the Statistical Package for the Social Sciences software (SPSS, version 26; IBM Co., Armonk, NY, USA). Statistical significance was defined as a two-tailed P-value <0.05.
Baseline demographics
The baseline demographics of the study participants are presented in Table 1, with no significant differences in age, sex, body mass index (BMI), biochemical data, cirrhosis, HCV genotypes, viral load, DAA regimens, HBV coinfection, or presence of HCC between the training and validation datasets.
Logistic regression analysis of the factors associated with DAA treatment failure
The overall DAA failure rate was 1.6%. In the univariate analysis, female, fibrosis-4 index (FIB-4), cirrhosis, decompensation, presence of HCC, HCV genotypes, higher HCV viral load, protease inhibitor-based DAA regimens, treatment experience, less DAA/ribavirin adherence, and severe adverse effects significantly increased the risk of DAA treatment failure. In biochemical examinations, lower albumin, platelet, and creatinine levels significantly increased the probability of non-SVR. Elevated aspartate aminotransferase (AST), bilirubin, prothrombin time, and hemoglobin A1c (HbA1c) levels significantly increase the likelihood of virological failure. In the multivariate analysis, the presence of cirrhosis and HCC, higher HbA1c, and less DAA adherence were independent risk factors for DAA treatment failure after adjustment for the variables with P-value <0.05 in the univariate analysis (Table 2).
We developed a conventional prediction model using logistic regression as follows:
Logistic regression (LR) model=3×Liver cirrhosis (yes=1, no=0)+4×HCC (yes=1, no=0)+1×HbA1c+33×DAA adherence (<20%=5, 20–40%=4, 40–60%=3, 60–80%=2, >80%=1)
The components of the LR model were the four independent risk factors in the multivariate logistic regression analyanalysis. The coefficient for each variable was derived from the odds ratio of the multivariate logistic regression analysis. The cutoff value for discriminating DAA failure was set at 40 using Youden’s index in the ROC curve analysis.
Performance of the predictive models
In the training dataset, the AUROC was 1.000, 1.000, 0.845, 0.736, and 0.588 for the XGBoost, random forest, decision tree, artificial neural network, and logistic regression models, respectively. The accuracy, precision, and recall rates of the prediction were 100% for both XGBoost and random forest. The F1 score achieved 1.00 in both the XGBoost and random forest algorithms (Fig. 1A, B, and Table 3).
In the validation dataset, the AUROC was 0.803, 0.756, 0.644, 0.658, and 0.616 for the XGBoost, random forest, decision tree, artificial neural network, and logistic regression models, respectively (Fig. 1C, D, and Table 3). The Delong test revealed the performance of XGBoost was superior to the random forest (P=0.021), decision tree (P=4.4×10-8), artificial neural network (P=5.4×10-9), and logistic regression model (P=2.5×10-10) (Table 4). The accuracy, precision, recall, and F1-score of XGBoost are 98.3%, 98.4%, 99.9%, and 0.992, respectively.
Risk stratification based on the XGBoost algorithm
The overall HCV patients receiving DAA treatment were further stratified according to the XGBoost prediction results. XGBoost provides a risk coefficient between 0 and 1 for each case. The higher the coefficient, the higher the chance of achieving SVR. DAA efficacy was divided into ten subgroups based on risk coefficient deciles. Figure 2 shows the predicted non-SVR accuracy for each subgroup using the XGBoost algorithm. The participants were stratified into high-risk (decile 1–5), intermediate-risk (decile 6–9), and low-risk (decile 10) populations based on the risk coefficients. The DAA failure rate was 75–100% in the high-risk, 15.8–40.0% in the intermediate-risk, and 0.4% in the low-risk populations. The DAA failure rate among the top five deciles was substantially higher than that at baseline (1.6%). The accumulative non-SVR rate was 69.7% in the high-risk population. Among the 538 subjects for whom DAA treatment failed, 375 (69.7%) were successfully detected using the XGBoost model among the top five deciles (Fig. 2). When the cutoff value of the risk coefficient was set at 0.5, the accuracy, sensitivity, specificity, positive predictive value, and negative predictive value were 99.5%, 69.7%, 99.9%, 97.4%, and 99.5%, respectively (Supplementary Table 2).
Importance of predictors
The relative importance of DAA failure predictors was evaluated using the XGBoost algorithm in all cases. The x-axis represents the ratio of the number of times a variable is applied to the total number of trees. The top 12 predictors were body mass index, viral load, α-fetoprotein, bilirubin, platelets, FIB-4 index, creatinine, ALT, albumin, age, prothrombin time, and AST level (Supplementary Fig. 1).
SHAP summary plot
Shapley Additive exPlanations (SHAP) was used to separately measure the contributions to the outcome from each feature [15]. Figure 3 shows the summary of the XGBoost model explainability with SHAP in all cases. SHAP>0 indicated a higher probability of SVR, while SHAP<0 indicated a higher chance of non-SVR. A dot represents a sample, and the colors represent feature values ranging from low (yellow) to high (purple). From the color distribution of the dots, we can deduce the effect of this feature on the DAA efficacy. For example, the purple dots for HCV RNA are concentrated at SHAP<0, indicating that a higher viral load increases the probability of treatment failure, whereas the yellow dots for albumin are concentrated at SHAP<0, indicating that subjects with low albumin levels are prone to treatment failure. In brief, elevated viral load, α-fetoprotein, FIB-4 index, bilirubin, and AST levels increase the risk of DAA failure. Subjects with a lower body mass index, platelets, albumin, and younger age had a lower probability of SVR.
The detailed relationships between the features and the SVR are shown in Figure 4, revealing nonlinear relationships between the predictors and DAA efficacy. For example, HCV RNA levels <106 IU/mL increased the likelihood of SVR, and HCV RNA >2×106 IU/mL increased the risk of virological failure. Approximately, subjects aged <60 years, with serum bilirubin level >2 g/dL, albumin <3.5 g/dL, and creatinine level >15 mg/dL were prone to treatment failure. When AST ranges between 200–300 IU/L, treatment failure is more likely; in contrast, AST <100 or AST >400 IU/L implies a higher chance of achieving SVR.
The nationwide TACR study investigated the risk factors for DAA treatment failure in Taiwanese patients. Multivariate regression analysis revealed that liver cirrhosis, HCC, poor DAA adherence, and high HbA1c levels were significantly associated with virologic failure. We developed an ML-based predictive model to identify potential treatment failure populations. The performance of the XGBoost model was superior to the other algorithms and the conventional logistic regression model. The AUROC of the XGBoost algorithm is 1.000 and 0.803 for the training and validation datasets, respectively. The AI predictive model successfully detected 69.7% of the subjects who failed to achieve SVR among the top five decile subgroups, thus implying that an AI-based model can effectively strengthen the decision-making process for antiviral therapy.
The AI model showed that subjects with features of liver cirrhosis prone to decompensation (i.e., higher FIB-4 index, bilirubin, and AST levels; lower albumin and platelets) were less likely to achieve SVR. Elevated AFP levels and decreased BMI (i.e., weight loss) are hallmarks of active HCC and predispose patients to treatment failure. In addition, HCV patients with a high baseline viral load had more difficulty in clearing the virus than those with a low viral load. These AI findings are consistent with those of the multivariate regression analysis. DAA adherence and HbA1c were not incorporated into the top 12 predictors in the AI model, possibly owing to collinearity between the above factors and other variables. Comorbid diabetes in HCV-infected patients is well-known to increase the risk of HCC [16,17]. A substantial proportion of decompensated patients who do not achieve SVR may experience adverse events or death-related early discontinuation [18]. Body mass index had J- or U-shaped associations with overall and all-cancer mortality rates. Compared with healthyweight individuals, life expectancy is shorter in underweight subjects [19,20]. Liver cirrhosis and HCC were not classified as significant risk factors using the XGBoost algorithm. Liver cirrhosis vs. FIB-4 index or HCC vs. AFP levels showed a certain degree of collinearity. In contrast, the viral load was not an independent risk factor using multivariate logistic analysis but became a significant predictor using the XGBoost approach, possibly resulting from the appropriate cutoff value of HCV RNA not being embedded in the regression model. Previous studies have shown that the SVR12 rate significantly decreased in the high baseline viral load group compared to that in the low viral load group. However, the optimal cutoff values for high vs. low viral loads vary across studies [21,22]. While pan-genotypic DAAs can be safely administered in traditionally difficult-to-treat HCV populations, managing patients with active HCC, decompensated liver cirrhosis, RASs, or prior DAA failure requires special attention [3]. There are substantial amounts of nonlinear data in clinical practice that are difficult to evaluate using conventional statistical methods. The SHAP dependence plot showed the relationship between SVR and the predictors (Fig. 4). Clinicians can realize the optimal range of significant variables that contribute to SVR. The AI predictive model can assist in discriminating high-risk patients and alert clinicians to identify risk factors before initiating DAA therapy.
The advantages of ML include flexibility and scalability, making it preferable for processing nonlinear big data [5]. The HCV-TARGET study in the United States and Europe (n=6,525) applied multiple algorithms (elastic net, neural network, random forest, and gradient boosting) to predict DAA treatment failure (C-index=0.64–0.69), superior to the multivariate logistic regression model (C-index=0.51) [23]. The HCV-TARGET study revealed that the top ten predictors were albumin, liver enzymes, bilirubin, sex, HCV RNA, sodium, HCC, platelet count, and tobacco use. The TACR and HCV-TARGET studies highlighted the vital roles of HCC and cirrhosis-related risk factors in DAA treatment failure. AI approaches confirmed the viewpoints of traditional statistics and further improved predictive performance compared with conventional statistical methods.
A meta-analysis revealed that patients with active HCC had a significantly lower SVR rate (73.1%) than those with inactive HCC (92.6%) or those without HCC (93.3%) [24]. The REAL-C study enrolled propensity score-matched HCV patients and confirmed that SVR rates were reduced in patients with active HCC but not those with inactive HCC (85.5% vs. 93.7%; P=0.03) [25]. Patients with active HCC may experience more adverse effects and early DAA discontinuation [26], which partially explains the suboptimal DAA response in this population. The mechanisms underlying suboptimal antiviral efficacy in patients with active HCC remain unclear, possibly attributed to the ineffective blood delivery of DAA to target sites [27] and impairment of host immunity in HCC patients [28]. HCC Patients with curative potential should be treated aggressively before DAA administration to ensure a greater chance of viral clearance. However, the optimal timing of DAA initiation in patients with incurable HCC remains controversial. Viral eradication significantly reduces mortality in patients with HCC receiving either curative or palliative HCC therapy [29], thus suggesting that antiviral therapy should not hesitate on those subjects not eligible for curative HCC treatment [30].
Previous studies have reported that the SVR rate is lower in HCV patients with decompensated cirrhosis than in patients with compensated cirrhosis [9,31]. The probability of achieving SVR varies based on the reserved liver function [32]. Patients with higher Model for End-Stage Liver Disease (MELD) scores (>20, Child-Turcotte-Pugh class C) had lower SVR rates, more adverse effects, and a lower likelihood of liver function improvement [33-35]. For patients with a MELD score ≥20, posttransplant HCV treatment is recommended, unless the expected waitlist time is more than six months. Patients with MELD scores <15 should be treated promptly. The grey zone of a MELD score, i.e., 15–19, requires tailored therapy on a case-by-case basis [11,36]. Our study provided information to identify patients with high risk of treatment failure. Selection of proper DAA regimens with high efficacy and safety profiles and enhancing DAA adherence might help to ensure treatment efficacy [9].
XGBoost is a supervised ML algorithm under a gradient boosting framework. The ensemble method combines multiple models to produce more accurate predictions. Gradient boosting is an ensemble technique that corrects mistakes in existing models by creating new models, which are sequentially added until no further improvement can be achieved [37]. This process is called gradient boosting because it utilizes a gradient descent method to minimize loss when creating new models [38]. Moreover, XGBoost supports both regression and classification tasks. XGBoost indeed exhibits an outstanding predictive ability compared to the other algorithms in our study.
ML models are usually considered “black boxes” because their prediction process is too complex for humans to interpret. To overcome this problem, explainable AI methods have been developed based on the Shapley methods [39]. SHAP is derived from the concept of cooperative game theory, which can calculate the contribution of each feature to the prediction. SHAP provides insight into the inner workings of a “black box” by generating quantitative visualizations of the prediction process. SHAP can reflect the influence of the features in each sample and show a positive or negative impact on the outcome [15]. Through the transformation of SHAP, users can better understand how the model makes predictions. Moreover, it provides feedback on the key factors contributing to the outcome and allows for the identification of potential biases. This transparency is crucial for convincing clinicians to rely on AI-based decision support systems [40]. “Explainable AI” may help bridge the gap between the medicine and AI-predictive models.
The current study has several limitations. Some patients (e.g., hepatic decompensation, HCC) may not survive long enough to obtain SVR12 data. This population may have relatively unfavorable prognostic factors, leading to a suboptimal treatment response. Although RASs have been confirmed to be associated with viral resistance [2,41], RAS testing is not recommended in routine clinical practice [42]. Considering only 7.0% of the RASs were available in the TACR database, this predictive model may underestimate the impact of RASs on the DAA response. The performance of the validation dataset is inferior to that of the training dataset. Potential overfitting and heterogeneity in the training dataset may affect model generalizability. However, avoiding overfitting may reduce the accuracy of ML models. In such an imbalanced dataset (non-SVR rate=1.6%), we expect the accuracy of the training dataset to be at least >98.4%. Under this premise, the hyperparameter tuning of each AI model was relatively limited, making it difficult to avoid overfitting. As the generalization is suboptimal, the present AI model should be further further modified before being applied to other independent cohorts. A relatively small number of DAA failure events may limit the development of a robust model. There is no universal approach to missing data imputation—a fundamental concern in real-world clinical datasets. The input data for the AI analysis contained only clinical and virological data in the current study. A combination of genomics, proteomics, and metabolomics may improve the predictive accuracy of the validation datasets in the future.
In conclusion, this nationwide TACR study applied ML algorithms for risk stratification in DAA failure. The performance of the AI model is superior to that of the conventional logistic regression model. The XGBoost model showed that subjects with features such as higher HCV RNA levels, active HCC, or decompensated liver cirrhosis were less likely to achieve SVR. This model captured 69.7% of patients who failed to achieve SVR among the top five decile subgroups. ML algorithms facilitate risk stratification in DAA failure and provide additional information on factors associated with DAA failure.
This work was supported partially by the “ Center For Intelligent Drug Systems and Smart Bio-devices (IDS2B) “ and the “ Center of Excellence for Metabolic Associated Fatty Liver Disease, National Sun Yat-sen University, Kaohsiung” from The Featured Areas Research Center Program within the framework of the Higher Education Sprout Project by the Ministry of Education (MOE) in Taiwan, and grants from MOST 111-2314-B-037-069-MY2, MOHW111-TDU-B-221-114007, KMUH-DK(B)111002-1, KMHK-DK(C)111004, and KMHKDK(C)111006, KMU-TC111B04, KMU-TC111A04, NSTC 112-2321-B-001-006 and MOHW112-TDU-B-221-124007.

Authors’ contribution

M.Y.L analyzed the data and wrote the manuscript. C.F.H, C.H.H, C.M.T, L.R.M, H.T.K, K.C.T, C.C.L, M.J.B, S.J.W, J.F.H, M.L.Y, C.T.C, M.C.T, C.W.H, P.L.L, T.H.Y, Y.H.H, L.W.C, C.L.C, C.C.Y, S.S.Y, P.N.C, T.Y.H, J.T.H, W.C.W, C.Y.C, G.Y.C, G.X.Z, W.L.T, C.N.K, C.L.L, C.C.W, T.Y.L, C.L.L, W.W.S, T.H.L, T.S.C, C.J.L, C.Y.D, J.H.K, H.C.L, W.L.C, C.Y.P, and C.Y.C collected the clinical data. C.W.T confirmed the machine learning analysis. M.L.Y designed the study, interpreted data, and supervised the manuscript. All authors have approved the final version of the manuscript.

Conflicts of Interest

Ming-Lung Yu disclosed the following: research grant from Abbvie, Gilead, Merck, and Roche diagnostics; consultant for Abbvie, BMS, Gilead, Roche, and Roche diagnostics; and speaker for Abbvie, BMS, Eisai, Gilead, Roche, and Roche diagnostics.

Supplementary material is available at Clinical and Molecular Hepatology website (http://www.e-cmh.org).
Supplementary Table 1.
The input features in the machine learning model
cmh-2023-0287-Supplementary-Table-1.pdf
Supplementary Table 2.
Performance of XGBoost for predicting DAA treatment response in the overall cases
cmh-2023-0287-Supplementary-Table-2.pdf
Supplementary Figure 1.
Importance of predictors. “Frequency” represents the ratio of the number of times a variable is used to the number of trees. BMI, body mass index; HCV, hepatitis C virus; AFP, α-fetoprotein; PLT, platelets; FIB-4, fibrosis-4 index; ALT, alanine aminotransferase; INR, international normalized ratio; AST, aspartate aminotransferase; APRI, aminotransferase to platelet ratio index; LDL, low-density lipoprotein; DAA, direct-acting antivirals; HbA1c, hemoglobin A1c; TG, triglyceride; rGT, gamma-glutamyltransferase; HDL, high-density lipoprotein; RVR, rapid virologic response; HBsAg, hepatitis B surface antigen; HIV, human immunodeficiency virus; HTN, hypertension; CKD, chronic kidney disease; RBV, ribavirin; HCC, hepatocellular carcinoma; LC, liver cirrhosis; HLD, hyperlipidemia; DM, diabetes; SAE, serious adverse event; RAS, resistance-associated substitutions; PWID, persons who inject drugs; CAD, coronary artery disease; DLC, decompensated liver cirrhosis; CLD, chronic liver disease; CVA, cerebrovascular accident.
cmh-2023-0287-Supplementary-Fig-1.pdf
Supplementary Materials
Machine learning code
cmh-2023-0287-Supplementary-1.pdf
Figure 1.
Performance of the predictive models. Figure 1 shows the ROC curves and precision-recall curves of the eXtreme Gradient Boosting (XGBoost), random forest (RF), decision tree (DT), artificial neural network (ANN) algorithms, and logistic regression (LR) models in the training dataset (A, B) and validation dataset (C, D). A precision-recall curve closer to the upper-right corner indicates better performance.
cmh-2023-0287f1.jpg
Figure 2.
DAA treatment failure rate by decile risk subgroups assessed using the XGBoost model among overall cases. The overall HCV patients were further divided into ten subgroups using the deciles of risk coefficients obtained from the XGBoost model. Patients with risk coefficients ranging from 0 to 0.1 belong to decile 1, from 0.1 to 0.2 were decile 2, …, and so on. The bars represent the predictive non-SVR accuracy in each subgroup. The red line represents the accumulated non-SVR rate. DAA, direct-acting antivirals; XGBoost, eXtreme Gradient Boosting; HCV, hepatitis C virus; SVR, sustained virological response.
cmh-2023-0287f2.jpg
Figure 3.
SHAP summary plot. The SHAP summary plot combined the feature importance and effects on DAA efficacy in all cases. The x-axis represents the SHAP value of the feature. A SHAP value >0 represents a positive correlation with SVR, and a SHAP value <0 represents a negative correlation with SVR. The overlapping points jittered along the x-axis represent the samples; the colors represent feature values ranging from low (yellow) to high (purple). SHAP, Shapley additive explanations; DAA, direct-acting antivirals; SVR, sustained virological response; BMI, body mass index; AFP, α-fetoprotein; PLT, platelets; FIB-4, fibrosis-4 index; AST, aspartate aminotransferase; INR, international normalized ratio; APRI, aminotransferase to platelet ratio index.
cmh-2023-0287f3.jpg
Figure 4.
SHAP dependence plot. SHAP dependence plot revealed that global model interpretations depend on the given features. SHAP, Shapley additive explanations; HCV, hepatitis C virus; BMI, body mass index; AFP, α-fetoprotein; PLT, platelets; FIB-4, fibrosis-4 index; AST, aspartate aminotransferase; INR, international normalized ratio; APRI, aminotransferase to platelet ratio index.
cmh-2023-0287f4.jpg
cmh-2023-0287f5.jpg
Table 1.
Baseline demographics
Table 1.
Variable Total Training Validation P-value
Number (%) 34,301 (100) 23,955 (70) 10,346 (30)
Age 61.9±12.7 61.9±12.6 61.9±12.8 0.710
Male 17,972 (52.4) 12,562 (52.4) 5,410 (52.3) 0.799
BMI (kg/m2) 24.7±4.04 24.7±4.06 24.7±4.01 0.285
FIB-4 3.31±3.37 3.30±3.37 3.33±3.36 0.438
Cirrhosis 7,051 (20.6) 4,903 (20.5) 2,148 (20.8) 0.542
Decompensated liver cirrhosis 932 (2.7) 637 (2.7) 295 (2.9) 0.316
HCC 2,822 (8.2) 1,967 (8.2) 855 (8.3) 0.874
Liver transplant 84 (0.2) 62 (0.3) 22 (0.2) 0.427
Genotype
 1 16,638 (48.5) 11,577 (48.3) 5,061 (48.9) 0.924
 2 13,298 (38.8) 9,332 (39.0) 3,966 (38.3)
 3 543 (1.6) 381 (1.6) 162 (1.6)
 4 17 (0.0) 13 (0.1) 4 (0.0)
 5 6 (0.0) 5 (0.0) 1 (0.0)
 6 2,478 (7.2) 1,730 (7.2) 748 (7.2)
 Mixed 1,073 (3.1) 748 (3.1) 325 (3.1)
 Unclassified 247 (0.7) 168 (0.7) 79 (0.8)
HCV RNA (log IU/mL) 5.90±1.02 5.90±1.02 5.90±1.02 0.623
HBsAg (+) 2,381 (7.3) 1,654 (7.3) 727 (7.4) 0.668
AST (IU/L) 60.7±54.1 60.4±53.9 61.3±54.5 0.165
ALT (IU/L) 73.0±77.0 72.8±77.7 73.3±75.4 0.635
Albumin (g/dL) 4.18±0.43 4.18±0.43 4.17±0.43 0.755
Total bilirubin (g/dL) 0.83±0.51 0.82±0.51 0.84±0.51 0.078
Platelet (x103/μL) 181.4±72.1 181.8±72.3 180.5±71.5 0.122
Prothrombin time (INR) 1.05±0.31 1.05±0.30 1.05±0.34 0.174
HbA1c (%) 6.06±1.24 6.1±1.3 6.0±1.2 0.308
DAA regimens
 Daclatasvir/Asunaprevir 981 (2.9) 682 (2.8) 299 (2.9) 0.986
 Viekirax/Exviera 3,394 (9.9) 2,377 (9.9) 1,017 (9.8)
 Elbasvir/grazoprevir 3,933 (11.5) 2,720 (11.4) 1,213 (11.7)
 Ledipasvir/sofosbuvir 7,592 (22.1) 5,301 (22.1) 2,291 (22.1)
 Sofosbuvir 2,549 (7.4) 1,770 (7.4) 779 (7.5)
 Sofosbuvir/daclatasvir 726 (2.1) 512 (2.1) 214 (2.1)
 Glecaprevir/pibrentasvir 7,568 (22.1) 5,300 (22.1) 2,268 (21.9)
 Sofosbuvir/velpatasvir 7,415 (21.6) 5,196 (21.7) 2,219 (21.5)
 Sofosbuvir/velpatasvir/voxilaprevir 90 (0.3) 60 (0.3) 30 (0.3)
 Others 42 (0.1) 28 (0.1) 14 (0.1)
Ribavirin (+) 4,346 (12.7) 3,019 (12.6) 1,327 (12.8) 0.572
Treatment naive 29,540 (86.1) 20,616 (86.1) 8,924 (86.3) 0.663

Values are presented as number (%).

BMI, body mass index; FIB-4, fibrosis index-4; HCC, hepatocellular carcinoma; HBsAg, hepatitis B surface antigen; AST, aspartate aminotransferase; ALT, alanine aminotransferase; HbA1c, hemoglobin A1c; DAA, direct-acting antivirals.

Table 2.
Multivariate logistic regression analysis of the factors associated with DAA treatment failure
Table 2.
Variable Univariate
 Multivariate
SVR Non-SVR P-value adj. OR (95% CI) P-value
Number (%) 33,763 (98.4) 538 (1.6)
Age 61.9±12.7 61.6±12.9 0.624
Male 17,722 (52.5) 250 (46.5) 0.006 0.53 (0.26–1.11) 0.091
BMI (kg/m2) 24.7±4.0 24.9±4.2 0.381
FIB-4 3.30±3.36 3.94±3.85 1.7×10-4 0.95 (0.85–1.06) 0.331
Cirrhosis 6,870 (20.4) 181 (33.6) 3.8×10-14 2.60 (1.13–5.97) 0.025
Decompensated LC 907 (2.7) 25 (4.6) 0.006 0.92 (0.31–2.69) 0.877
HCC 2,715 (8.0) 107 (19.9) 3.4×10-23 4.43 (2.17–9.02) 4.2×10-4
Liver transplant 82 (0.2) 2 (0.4) 0.380
Genotype
 1 16,429 (48.7) 209 (38.8) 1.2×10-15 0.99 (0.74–1.33) 0.949
 2 13,057 (38.7) 241 (44.8)
 3 516 (1.5) 27 (5.0)
 4 14 (0.0) 3 (0.6)
 5 6 (0.0) 0 (0.0)
 6 2,440 (7.2) 38 (7.1)
 Mixed 1,056 (3.1) 17 (3.2)
 Unclassified 244 (0.7) 3 (0.6)
HCV RNA (log IU/mL) 5.90±1.02 6.16±0.87 2.2×10-11 1.48 (0.99–2.20) 0.056
HBsAg (+) 2345 (7.3) 36 (7.0) 0.785
Albumin (g/dL) 4.18±0.43 4.07±0.51 4.9×10-7 1.23 (0.57–2.66) 0.592
AST (IU/L) 60.6±54.1 67.4±54.2 0.004 1.01 (1.00–1.01) 0.068
ALT (IU/L) 72.9±77.1 76.0±73.6 0.351
Total bilirubin (mg/dL) 0.83±0.51 0.90±0.55 0.003 0.90 (0.56–1.43) 0.645
Platelet (x103/μL) 181.6±72.0 169.4±74.9 9.9×10-5 1.00 (0.99–1.01) 0.889
Prothrombin time (INR) 1.05±0.31 1.08±0.44 0.036 0.86 (0.18–4.14) 0.846
Cr (mg/dL) 1.17±1.54 1.05±1.30 0.028 0.21 (0.04–1.05) 0.058
HbA1c (%) 6.1±1.2 6.3±1.5 0.012 1.28 (1.05–1.56) 0.014
DAA regimens
 Daclatasvir/Asunaprevir 936 (2.8) 45 (8.4) 1.4×10-34 0.99 (0.76–1.29) 0.933
 Viekirax/Exviera 3,353 (9.9) 41 (7.6)
 Elbasvir/grazoprevir 3,888 (11.5) 45 (8.4)
 Ledipasvir/sofosbuvir 7,470 (22.1) 122 (22.7)
 Sofosbuvir 2,448 (7.3) 101 (18.8)
 Sofosbuvir/daclatasvir 714 (2.1) 12 (2.2)
 Glecaprevir/pibrentasvir 7,467 (22.1) 101 (18.8)
 Sofosbuvir/velpatasvir 7,344 (21.8) 71 (13.2)
 Sofosbuvir/velpatasvir/voxilaprevir 90 (0.3) 0 (0)
 Others 42 (0.1) 0 (0)
Treatment naive 29,113 (86.2) 427 (79.4) 4.6×10-6 0.85 (0.58–2.40) 0.654
DAA adherence
 >80% 33,656 (99.7) 492 (91.8) 2.0×10-259 33.48 (2.46–456.7) 0.008
 60-80% 44 (0.1) 10 (1.9)
 40-60% 21 (0.1) 7 (1.3)
 20-40% 19 (0.1) 11 (2.1)
 <20% 5 (0.0) 16 (3.0)
Ribavirin adherence
 >80% 3,910 (93.2) 117 (87.3) 0.004 0.75 (0.43–1.31) 0.313
 60-80% 125 (3.0) 7 (5.2)
 40-60% 77 (1.8) 6 (4.5)
 20-40% 49 (1.2) 0 (0)
 <20% 35 (0.8) 4 (3.0)
Severe adverse effects 365 (1.3) 13 (2.9) 0.003 1.10 (0.35–3.48) 0.876

Values are presented as number (%).

DAA, direct-acting antivirals; BMI, body mass index; LC, liver cirrhosis; FIB-4, fibrosis index-4; HCC, hepatocellular carcinoma; HBsAg, hepatitis B surface antigen; AST, aspartate aminotransferase; ALT, alanine aminotransferase; Cr, creatinine; HbA1c, hemoglobin A1c; adj.

OR, adjusted odds ratio; CI, confidence interval.

Table 3.
Performance of the predictive models for the response of direct-acting antivirals
Table 3.
Model AUC (95% CI) P-value Accuracy Precision Recall F1-score
Training dataset
 XGBoost 1.000 (1.000–1.000) 2.2×10-238 100% 100% 100% 1.000
 Random Forest 1.000 (1.000–1.000) 2.2×10-238 100% 100% 100% 1.000
 Decision tree 0.845 (0.825–0.865) 1.1×10-114 98.6% 98.6% 100% 0.993
 Neural network 0.736 (0.711–0.762) 8.3×10-55 98.5% 98.5% 100% 0.992
 Logistic regression 0.588 (0.558–0.619) 1.3×10-8 81.6% 98.7% 82.3% 0.898
Validation dataset
 XGBoost 0.803 (0.769–0.837) 4.9×10-42 98.3% 98.4% 99.9% 0.992
 Random Forest 0.756 (0.722–0.790) 1.6×10-30 98.4% 98.4% 100% 0.992
 Decision tree 0.644 (0.594–0.695) 9.8×10-11 98.2% 98.4% 99.8% 0.991
 Neural network 0.658 (0.616–0.700) 1.6×10-12 98.4% 98.4% 100% 0.992
 Logistic regression 0.616 (0.571–0.662) 6.2×10-7 82.1% 98.8% 82.8% 0.901

CI, confidence interval; XGBoost, eXtreme Gradient Boosting.

Table 4.
Delong test
Table 4.
P-value (z-score) Random Forest Decision tree Neural network Logistic regression
Training
 XGBoost 1.000 (0.0) <0.0001 (15.5) <0.0001 (19.9) <0.0001 (26.5)
 Random Forest <0.0001 (15.5) <0.0001 (19.9) <0.0001 (26.5)
 Decision tree <0.0001 (8.0) <0.0001 (15.0)
 Neural network <0.0001 (8.7)
Validation
 XGBoost 0.021 (2.3) 4.4×10-8 (5.5) 5.4×10-9 (5.8) 2.5×10-10 (6.3)
 Random Forest 3.9×10-5 (4.1) 6.6×10-7 (5.0) 2.5×10-8 (5.6)
 Decision tree 0.643 (–0.5) 0.354 (0.9)
 Neural network 0.102 (1.6)

XGBoost, eXtreme Gradient Boosting.

AFP

α-fetoprotein

AI

artificial intelligence

ALT

alanine aminotransferase

ANN

artificial neural network

AST

aspartate aminotransferase

AUROC

area under the receiver operator characteristic curve

BMI

body mass index

DAA

direct-acting antivirals

DT

decision tree

FIB-4

fibrosis-4 index

HbA1c

hemoglobin A1c

HBV

hepatitis B virus

HCC

hepatocellular carcinoma

HCV

hepatitis C virus

INR

international normalized ratio

KNN

k-nearest neighbor

LC

liver cirrhosis

ML

machine learning

RAS

resistance-associated substitutions

RF

random forest

SHAP

Shapley additive explanations

SVR

sustained virological response

TACR

Taiwan HCV registry program

XGBoost

eXtreme Gradient Boosting
  • 1. Hayes CN, Imamura M, Tanaka J, Chayama K. Road to elimination of HCV: Clinical challenges in HCV management. Liver Int 2022;42:1935-1944.
  • 2. Hong CM, Lin YY, Liu CJ, Lai YY, Yeh SH, Yang HC, et al. Drug resistance profile and clinical features for hepatitis C patients experiencing DAA failure in Taiwan. Viruses 2021;13:2294.
  • 3. Solitano V, Plaz Torres MC, Pugliese N, Aghemo A. Management and treatment of hepatitis C: Are there still unsolved problems and unique populations? Viruses 2021;13:1048.
  • 4. Huang CF, Yu ML. Unmet needs of chronic hepatitis C in the era of direct-acting antiviral therapy. Clin Mol Hepatol 2020;26:251-260.
  • 5. Hassabis D, Kumaran D, Summerfield C, Botvinick M. Neuroscience-inspired artificial intelligence. Neuron 2017;95:245-258.
  • 6. Su TH, Wu CH, Kao JH. Artificial intelligence in precision medicine in hepatology. J Gastroenterol Hepatol 2021;36:569-580.
  • 7. Le Berre C, Sandborn WJ, Aridhi S, Devignes MD, Fournier L, Smaïl-Tabbone M, et al. Application of artificial intelligence to gastroenterology and hepatology. Gastroenterology 2020;158:76-94.e2.
  • 8. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med 2019;25:44-56.
  • 9. Chen CY, Huang CF, Cheng PN, Tseng KC, Lo CC, Kuo HT, et al. Factors associated with treatment failure of direct-acting antivirals for chronic hepatitis C: A real-world nationwide hepatitis C virus registry programme in Taiwan. Liver Int 2021;41:1265-1277.
  • 10. Yu ML, Chen PJ, Dai CY, Hu TH, Huang CF, Huang YH, et al. 2020 Taiwan consensus statement on the management of hepatitis C: part (I) general population. J Formos Med Assoc 2020;119:1019-1040.
  • 11. Yu ML, Chen PJ, Dai CY, Hu TH, Huang CF, Huang YH, et al. 2020 Taiwan consensus statement on the management of hepatitis C: Part (II) special populations. J Formos Med Assoc 2020;119:1135-1157.
  • 12. Petrazzini BO, Naya H, Lopez-Bello F, Vazquez G, Spangenberg L. Evaluation of different approaches for missing data imputation on features associated to genomic data. BioData Min 2021;14:44.
  • 13. Dankers FJWM, Traverso A, Wee L, van Kuijk SMJ. Prediction modeling methodology. In: Kubben P, Dumontier M, Dekker A, eds. Fundamentals of clinical data science. Cham (CH): Springer, 2019:101-120.
  • 14. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837-845.
  • 15. Belle V, Papantonis I. Principles and practice of explainable machine learning. Front Big Data 2021;4:688969.
  • 16. Davila JA, Morgan RO, Shaib Y, McGlynn KA, El-Serag HB. Diabetes increases the risk of hepatocellular carcinoma in the United States: a population based case control study. Gut 2005;54:533-539.
  • 17. Tsai PC, Kuo HT, Hung CH, Tseng KC, Lai HC, Peng CY, et al. Metformin reduces hepatocellular carcinoma incidence after successful antiviral therapy in patients with diabetes and chronic hepatitis C in Taiwan. J Hepatol 2023;78:281-292.
  • 18. Charlton M, Everson GT, Flamm SL, Kumar P, Landis C, Brown RS Jr, et al. Ledipasvir and sofosbuvir plus ribavirin for treatment of HCV infection in patients with advanced liver disease. Gastroenterology 2015;149:649-659.
  • 19. Bhaskaran K, Dos-Santos-Silva I, Leon DA, Douglas IJ, Smeeth L. Association of BMI with overall and cause-specific mortality: a population-based cohort study of 3·6 million adults in the UK. Lancet Diabetes Endocrinol 2018;6:944-953.
  • 20. Global BMI Mortality Collaboration, Di Angelantonio E, Bhupathiraju ShN, Wormser D, Gao P, Kaptoge S, et al. Body-mass index and all-cause mortality: individual-participant-data meta-analysis of 239 prospective studies in four continents. Lancet 2016;388:776-786.
  • 21. Chen WM, Wei KL, Tung SY, Shen CH, Chang TS, Yen CW, et al. High viral load predicts virologic failure in chronic genotype 2 hepatitis C virus-infected patients receiving glecaprevir/pibrentasvir therapy. J Formos Med Assoc 2020;119:1593-1600.
  • 22. Salmon D, Bani-Sadr F, Gilbert C, Rosenthal E, Valantin MA, Simon A, et al. HCV viral load at baseline and at week 4 of telaprevir/boceprevir based triple therapies are associated with virological outcome in HIV/hepatitis C co-infected patients. J Clin Virol 2015;73:32-35.
  • 23. Park H, Lo-Ciganic WH, Huang J, Wu Y, Henry L, Peter J, et al. Machine learning algorithms for predicting direct-acting antiviral treatment failure in chronic hepatitis C: An HCV-TARGET analysis. Hepatology 2022;76:483-491.
  • 24. Ji F, Yeo YH, Wei MT, Ogawa E, Enomoto M, Lee DH, et al. Sustained virologic response to direct-acting antiviral therapy in patients with chronic hepatitis C and hepatocellular carcinoma: A systematic review and meta-analysis. J Hepatol 2019;71:473-485.
  • 25. Ogawa E, Toyoda H, Iio E, Jun DW, Huang CF, Enomoto M, et al. Hepatitis C virus cure rates are reduced in patients with active but not inactive hepatocellular carcinoma: A practice implication. Clin Infect Dis 2020;71:2840-2848.
  • 26. Huang CF, Yeh ML, Huang CI, Liang PC, Lin YH, Hsieh MY, et al. Equal treatment efficacy of direct-acting antivirals in patients with chronic hepatitis C and hepatocellular carcinoma? A prospective cohort study. BMJ Open 2019;9:e026703.
  • 27. Konjeti VR, John BV. Interaction between hepatocellular carcinoma and hepatitis C eradication with direct-acting antiviral therapy. Curr Treat Options Gastroenterol 2018;16:203-214.
  • 28. Sachdeva M, Chawla YK, Arora SK. Immunology of hepatocellular carcinoma. World J Hepatol 2015;7:2080-2090.
  • 29. Dang H, Yeo YH, Yasuda S, Huang CF, Iio E, Landis C, et al. Cure with interferon-free direct-acting antiviral is associated with increased survival in patients with hepatitis C virus-related hepatocellular carcinoma from both East and West. Hepatology 2020;71:1910-1922.
  • 30. Lee SW, Chen LS, Yang SS, Huang YH, Lee TY. Direct-acting antiviral therapy for hepatitis C virus in patients with BCLC stage B/C hepatocellular carcinoma. Viruses 2022;14:2316.
  • 31. Huang CF, Iio E, Jun DW, Ogawa E, Toyoda H, Hsu YC, et al. Direct-acting antivirals in East Asian hepatitis C patients: realworld experience from the REAL-C Consortium. Hepatol Int 2019;13:587-598.
  • 32. Curry MP, O’Leary JG, Bzowej N, Muir AJ, Korenblat KM, Fenkel JM, et al. Sofosbuvir and velpatasvir for HCV in patients with decompensated cirrhosis. N Engl J Med 2015;373:2618-2628.
  • 33. Modi AA, Nazario H, Trotter JF, Gautam M, Weinstein J, Mantry P, et al. Safety and efficacy of simeprevir plus sofosbuvir with or without ribavirin in patients with decompensated genotype 1 hepatitis C cirrhosis. Liver Transpl 2016;22:281-286.
  • 34. Belli LS, Duvoux C, Berenguer M, Berg T, Coilly A, Colle I, et al. ELITA consensus statements on the use of DAAs in liver transplant candidates and recipients. J Hepatol 2017;67:585-602.
  • 35. Bunchorntavakul C, Reddy RK. HCV therapy in decompensated cirrhosis before or after liver transplantation: A paradoxical quandary. Am J Gastroenterol 2018;113:449-452.
  • 36. Ekpanyapong S, Reddy KR. Hepatitis C virus therapy in advanced liver disease: Outcomes and challenges. United European Gastroenterol J 2019;7:642-650.
  • 37. Mayr A, Binder H, Gefeller O, Schmid M. The evolution of boosting algorithms. From machine learning to statistical modelling. Methods Inf Med 2014;53:419-427.
  • 38. Ilboudo WEL, Kobayashi T, Sugimoto K. Robust stochastic gradient descent with student-t distribution based first-order momentum. IEEE Trans Neural Netw Learn Syst 2022;33:1324-1337.
  • 39. Bussmann N, Giudici P, Marinelli D, Papenbrock J. Explainable AI in fintech risk management. Front Artif Intell 2020;3:26.
  • 40. Calderaro J, Seraphin TP, Luedde T, Simon TG. Artificial intelligence for the prevention and clinical management of hepatocellular carcinoma. J Hepatol 2022;76:1348-1361.
  • 41. Sarrazin C. Treatment failure with DAA therapy: Importance of resistance. J Hepatol 2021;74:1472-1482.
  • 42. Ridruejo E, Pereson MJ, Flichman DM, Di Lello FA. Hepatitis C virus treatment failure: Clinical utility for testing resistanceassociated substitutions. World J Hepatol 2021;13:1069-1078.

Download Citation

Download a citation file in RIS format that can be imported by all major citation management software, including EndNote, ProCite, RefWorks, and Reference Manager.

Format:

Include:

Artificial intelligence predicts direct-acting antivirals failure among hepatitis C virus patients: A nationwide hepatitis C virus registry program
Clin Mol Hepatol. 2024;30(1):64-79.   Published online November 21, 2023
DOI: https://doi.org/10.3350/cmh.2023.0287
Download Citation

Download a citation file in RIS format that can be imported by all major citation management software, including EndNote, ProCite, RefWorks, and Reference Manager.
Format:
Include:
Artificial intelligence predicts direct-acting antivirals failure among hepatitis C virus patients: A nationwide hepatitis C virus registry program
Clin Mol Hepatol. 2024;30(1):64-79.   Published online November 21, 2023
DOI: https://doi.org/10.3350/cmh.2023.0287
Close

Figure

  • 0
  • 1
  • 2
  • 3
  • 4

Related articles

Artificial intelligence predicts direct-acting antivirals failure among hepatitis C virus patients: A nationwide hepatitis C virus registry program
Image Image Image Image Image
Figure 1. Performance of the predictive models. Figure 1 shows the ROC curves and precision-recall curves of the eXtreme Gradient Boosting (XGBoost), random forest (RF), decision tree (DT), artificial neural network (ANN) algorithms, and logistic regression (LR) models in the training dataset (A, B) and validation dataset (C, D). A precision-recall curve closer to the upper-right corner indicates better performance.
Figure 2. DAA treatment failure rate by decile risk subgroups assessed using the XGBoost model among overall cases. The overall HCV patients were further divided into ten subgroups using the deciles of risk coefficients obtained from the XGBoost model. Patients with risk coefficients ranging from 0 to 0.1 belong to decile 1, from 0.1 to 0.2 were decile 2, …, and so on. The bars represent the predictive non-SVR accuracy in each subgroup. The red line represents the accumulated non-SVR rate. DAA, direct-acting antivirals; XGBoost, eXtreme Gradient Boosting; HCV, hepatitis C virus; SVR, sustained virological response.
Figure 3. SHAP summary plot. The SHAP summary plot combined the feature importance and effects on DAA efficacy in all cases. The x-axis represents the SHAP value of the feature. A SHAP value >0 represents a positive correlation with SVR, and a SHAP value <0 represents a negative correlation with SVR. The overlapping points jittered along the x-axis represent the samples; the colors represent feature values ranging from low (yellow) to high (purple). SHAP, Shapley additive explanations; DAA, direct-acting antivirals; SVR, sustained virological response; BMI, body mass index; AFP, α-fetoprotein; PLT, platelets; FIB-4, fibrosis-4 index; AST, aspartate aminotransferase; INR, international normalized ratio; APRI, aminotransferase to platelet ratio index.
Figure 4. SHAP dependence plot. SHAP dependence plot revealed that global model interpretations depend on the given features. SHAP, Shapley additive explanations; HCV, hepatitis C virus; BMI, body mass index; AFP, α-fetoprotein; PLT, platelets; FIB-4, fibrosis-4 index; AST, aspartate aminotransferase; INR, international normalized ratio; APRI, aminotransferase to platelet ratio index.
Graphical abstract

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Graphical abstract

Artificial intelligence predicts direct-acting antivirals failure among hepatitis C virus patients: A nationwide hepatitis C virus registry program
Variable Total Training Validation P-value
Number (%) 34,301 (100) 23,955 (70) 10,346 (30)
Age 61.9±12.7 61.9±12.6 61.9±12.8 0.710
Male 17,972 (52.4) 12,562 (52.4) 5,410 (52.3) 0.799
BMI (kg/m2) 24.7±4.04 24.7±4.06 24.7±4.01 0.285
FIB-4 3.31±3.37 3.30±3.37 3.33±3.36 0.438
Cirrhosis 7,051 (20.6) 4,903 (20.5) 2,148 (20.8) 0.542
Decompensated liver cirrhosis 932 (2.7) 637 (2.7) 295 (2.9) 0.316
HCC 2,822 (8.2) 1,967 (8.2) 855 (8.3) 0.874
Liver transplant 84 (0.2) 62 (0.3) 22 (0.2) 0.427
Genotype
 1 16,638 (48.5) 11,577 (48.3) 5,061 (48.9) 0.924
 2 13,298 (38.8) 9,332 (39.0) 3,966 (38.3)
 3 543 (1.6) 381 (1.6) 162 (1.6)
 4 17 (0.0) 13 (0.1) 4 (0.0)
 5 6 (0.0) 5 (0.0) 1 (0.0)
 6 2,478 (7.2) 1,730 (7.2) 748 (7.2)
 Mixed 1,073 (3.1) 748 (3.1) 325 (3.1)
 Unclassified 247 (0.7) 168 (0.7) 79 (0.8)
HCV RNA (log IU/mL) 5.90±1.02 5.90±1.02 5.90±1.02 0.623
HBsAg (+) 2,381 (7.3) 1,654 (7.3) 727 (7.4) 0.668
AST (IU/L) 60.7±54.1 60.4±53.9 61.3±54.5 0.165
ALT (IU/L) 73.0±77.0 72.8±77.7 73.3±75.4 0.635
Albumin (g/dL) 4.18±0.43 4.18±0.43 4.17±0.43 0.755
Total bilirubin (g/dL) 0.83±0.51 0.82±0.51 0.84±0.51 0.078
Platelet (x103/μL) 181.4±72.1 181.8±72.3 180.5±71.5 0.122
Prothrombin time (INR) 1.05±0.31 1.05±0.30 1.05±0.34 0.174
HbA1c (%) 6.06±1.24 6.1±1.3 6.0±1.2 0.308
DAA regimens
 Daclatasvir/Asunaprevir 981 (2.9) 682 (2.8) 299 (2.9) 0.986
 Viekirax/Exviera 3,394 (9.9) 2,377 (9.9) 1,017 (9.8)
 Elbasvir/grazoprevir 3,933 (11.5) 2,720 (11.4) 1,213 (11.7)
 Ledipasvir/sofosbuvir 7,592 (22.1) 5,301 (22.1) 2,291 (22.1)
 Sofosbuvir 2,549 (7.4) 1,770 (7.4) 779 (7.5)
 Sofosbuvir/daclatasvir 726 (2.1) 512 (2.1) 214 (2.1)
 Glecaprevir/pibrentasvir 7,568 (22.1) 5,300 (22.1) 2,268 (21.9)
 Sofosbuvir/velpatasvir 7,415 (21.6) 5,196 (21.7) 2,219 (21.5)
 Sofosbuvir/velpatasvir/voxilaprevir 90 (0.3) 60 (0.3) 30 (0.3)
 Others 42 (0.1) 28 (0.1) 14 (0.1)
Ribavirin (+) 4,346 (12.7) 3,019 (12.6) 1,327 (12.8) 0.572
Treatment naive 29,540 (86.1) 20,616 (86.1) 8,924 (86.3) 0.663
Variable Univariate
 Multivariate
SVR Non-SVR P-value adj. OR (95% CI) P-value
Number (%) 33,763 (98.4) 538 (1.6)
Age 61.9±12.7 61.6±12.9 0.624
Male 17,722 (52.5) 250 (46.5) 0.006 0.53 (0.26–1.11) 0.091
BMI (kg/m2) 24.7±4.0 24.9±4.2 0.381
FIB-4 3.30±3.36 3.94±3.85 1.7×10-4 0.95 (0.85–1.06) 0.331
Cirrhosis 6,870 (20.4) 181 (33.6) 3.8×10-14 2.60 (1.13–5.97) 0.025
Decompensated LC 907 (2.7) 25 (4.6) 0.006 0.92 (0.31–2.69) 0.877
HCC 2,715 (8.0) 107 (19.9) 3.4×10-23 4.43 (2.17–9.02) 4.2×10-4
Liver transplant 82 (0.2) 2 (0.4) 0.380
Genotype
 1 16,429 (48.7) 209 (38.8) 1.2×10-15 0.99 (0.74–1.33) 0.949
 2 13,057 (38.7) 241 (44.8)
 3 516 (1.5) 27 (5.0)
 4 14 (0.0) 3 (0.6)
 5 6 (0.0) 0 (0.0)
 6 2,440 (7.2) 38 (7.1)
 Mixed 1,056 (3.1) 17 (3.2)
 Unclassified 244 (0.7) 3 (0.6)
HCV RNA (log IU/mL) 5.90±1.02 6.16±0.87 2.2×10-11 1.48 (0.99–2.20) 0.056
HBsAg (+) 2345 (7.3) 36 (7.0) 0.785
Albumin (g/dL) 4.18±0.43 4.07±0.51 4.9×10-7 1.23 (0.57–2.66) 0.592
AST (IU/L) 60.6±54.1 67.4±54.2 0.004 1.01 (1.00–1.01) 0.068
ALT (IU/L) 72.9±77.1 76.0±73.6 0.351
Total bilirubin (mg/dL) 0.83±0.51 0.90±0.55 0.003 0.90 (0.56–1.43) 0.645
Platelet (x103/μL) 181.6±72.0 169.4±74.9 9.9×10-5 1.00 (0.99–1.01) 0.889
Prothrombin time (INR) 1.05±0.31 1.08±0.44 0.036 0.86 (0.18–4.14) 0.846
Cr (mg/dL) 1.17±1.54 1.05±1.30 0.028 0.21 (0.04–1.05) 0.058
HbA1c (%) 6.1±1.2 6.3±1.5 0.012 1.28 (1.05–1.56) 0.014
DAA regimens
 Daclatasvir/Asunaprevir 936 (2.8) 45 (8.4) 1.4×10-34 0.99 (0.76–1.29) 0.933
 Viekirax/Exviera 3,353 (9.9) 41 (7.6)
 Elbasvir/grazoprevir 3,888 (11.5) 45 (8.4)
 Ledipasvir/sofosbuvir 7,470 (22.1) 122 (22.7)
 Sofosbuvir 2,448 (7.3) 101 (18.8)
 Sofosbuvir/daclatasvir 714 (2.1) 12 (2.2)
 Glecaprevir/pibrentasvir 7,467 (22.1) 101 (18.8)
 Sofosbuvir/velpatasvir 7,344 (21.8) 71 (13.2)
 Sofosbuvir/velpatasvir/voxilaprevir 90 (0.3) 0 (0)
 Others 42 (0.1) 0 (0)
Treatment naive 29,113 (86.2) 427 (79.4) 4.6×10-6 0.85 (0.58–2.40) 0.654
DAA adherence
 >80% 33,656 (99.7) 492 (91.8) 2.0×10-259 33.48 (2.46–456.7) 0.008
 60-80% 44 (0.1) 10 (1.9)
 40-60% 21 (0.1) 7 (1.3)
 20-40% 19 (0.1) 11 (2.1)
 <20% 5 (0.0) 16 (3.0)
Ribavirin adherence
 >80% 3,910 (93.2) 117 (87.3) 0.004 0.75 (0.43–1.31) 0.313
 60-80% 125 (3.0) 7 (5.2)
 40-60% 77 (1.8) 6 (4.5)
 20-40% 49 (1.2) 0 (0)
 <20% 35 (0.8) 4 (3.0)
Severe adverse effects 365 (1.3) 13 (2.9) 0.003 1.10 (0.35–3.48) 0.876
Model AUC (95% CI) P-value Accuracy Precision Recall F1-score
Training dataset
 XGBoost 1.000 (1.000–1.000) 2.2×10-238 100% 100% 100% 1.000
 Random Forest 1.000 (1.000–1.000) 2.2×10-238 100% 100% 100% 1.000
 Decision tree 0.845 (0.825–0.865) 1.1×10-114 98.6% 98.6% 100% 0.993
 Neural network 0.736 (0.711–0.762) 8.3×10-55 98.5% 98.5% 100% 0.992
 Logistic regression 0.588 (0.558–0.619) 1.3×10-8 81.6% 98.7% 82.3% 0.898
Validation dataset
 XGBoost 0.803 (0.769–0.837) 4.9×10-42 98.3% 98.4% 99.9% 0.992
 Random Forest 0.756 (0.722–0.790) 1.6×10-30 98.4% 98.4% 100% 0.992
 Decision tree 0.644 (0.594–0.695) 9.8×10-11 98.2% 98.4% 99.8% 0.991
 Neural network 0.658 (0.616–0.700) 1.6×10-12 98.4% 98.4% 100% 0.992
 Logistic regression 0.616 (0.571–0.662) 6.2×10-7 82.1% 98.8% 82.8% 0.901
P-value (z-score) Random Forest Decision tree Neural network Logistic regression
Training
 XGBoost 1.000 (0.0) <0.0001 (15.5) <0.0001 (19.9) <0.0001 (26.5)
 Random Forest <0.0001 (15.5) <0.0001 (19.9) <0.0001 (26.5)
 Decision tree <0.0001 (8.0) <0.0001 (15.0)
 Neural network <0.0001 (8.7)
Validation
 XGBoost 0.021 (2.3) 4.4×10-8 (5.5) 5.4×10-9 (5.8) 2.5×10-10 (6.3)
 Random Forest 3.9×10-5 (4.1) 6.6×10-7 (5.0) 2.5×10-8 (5.6)
 Decision tree 0.643 (–0.5) 0.354 (0.9)
 Neural network 0.102 (1.6)
Table 1. Baseline demographics

Values are presented as number (%).

BMI, body mass index; FIB-4, fibrosis index-4; HCC, hepatocellular carcinoma; HBsAg, hepatitis B surface antigen; AST, aspartate aminotransferase; ALT, alanine aminotransferase; HbA1c, hemoglobin A1c; DAA, direct-acting antivirals.

Table 2. Multivariate logistic regression analysis of the factors associated with DAA treatment failure

Values are presented as number (%).

DAA, direct-acting antivirals; BMI, body mass index; LC, liver cirrhosis; FIB-4, fibrosis index-4; HCC, hepatocellular carcinoma; HBsAg, hepatitis B surface antigen; AST, aspartate aminotransferase; ALT, alanine aminotransferase; Cr, creatinine; HbA1c, hemoglobin A1c; adj.

OR, adjusted odds ratio; CI, confidence interval.

Table 3. Performance of the predictive models for the response of direct-acting antivirals

CI, confidence interval; XGBoost, eXtreme Gradient Boosting.

Table 4. Delong test

XGBoost, eXtreme Gradient Boosting.

Table 1.

Table 2.

Table 3.

Table 4.

CMH : Clinical and Molecular Hepatology
Twitter

ABOUT

BROWSE ARTICLES

FOR CONTRIBUTORS

© The Korean Association for the Study of the Liver.