Critical appraisal of novel prediction models and risk calculators for post-hepatectomy liver failure and complications: practicability and generalisability in the real-world setting

Over the last few decades, the evolution of liver resection has progressed through numerous milestones in peri-operative management, operative techniques and novel technologies that have dramatically improved patient safety and outcomes (1). Consequently, such developments have enabled surgeons to embark on liver resections of lesions in technically challenging locations, whereby extended resection or bilovascular reconstruction may be required to ensure oncologic clearance. In the context of extended resections or resection of lesions from heavily diseased livers, concerns remain regarding the adequacy of the remnant future liver remnant (FLR) and liver function, placing patients at risk of the clinical phenomenon known as post-hepatectomy liver failure (PHLF). Although relatively uncommon, PHLF has a reported incidence of up to 32% in the literature and remains an important cause of post-hepatectomy morbidity and mortality (2). Presently, several definitions have been proposed to describe PHLF, the most recent of which was proposed by the International Study Group of Liver Surgery (ISGLS). In this definition, PHLF was defined as an increased international normalized ratio (INR) or hyperbilirubinemia on or after post-operative day 5, with further stratification of severity grades (A, B or C) based on the extent of clinical management (3). While definitions in PHLF assist in providing a common diagnostic framework among physicians, establishing predictors in PHLF is conceivably more helpful as it allows surgeons to have important decision-making details prior to planned liver resection.

In their recent publication, Wang et al. (4) described the development of prediction models and risk calculators aimed at predicting two different post-hepatectomy sequelae: PHLF, defined as ISGLS grade B or C PHLF and severe post-operative morbidity, defined by a comprehensive complication index (CCI) score (5) of greater than 40. These outcomes were intentionally selected to embody the full spectrum of complications that may potentially occur post-hepatectomy. In this study, statistical models were derived utilizing data pooled from a diverse, international cohort comprising patients from various eastern and western centres. The dataset was subsequently divided into two sets: a training set for which the prediction model was derived and a validation dataset for which the model was internally validated. Both models demonstrated good discrimination and calibration, with an area under the curve (AUC) of 0.80 and 0.76 for the PHLF and CCI models, respectively, when both pre- and intra-operative variables were included. This AUC was 0.78 and 0.71, respectively, when only pre-operative variables were included.

One of the strengths of this manuscript was the rigorous statistical methods used for developing the predictive models. Pre- and intra-operative variables were incorporated by adopting logistic regression models with a lasso penalty. From a statistical viewpoint, this approach is particularly commendable as it not only enhances the robustness of the models by facilitating selective variable inclusion and regularisation, but also minimizes the potential for overfitting. The evaluation of these models is thorough, utilizing both discrimination (AUC) and calibration metrics to ensure that the predicted probabilities are reliable and that the models exhibit robust discriminative capabilities. Additionally, the meticulous handling of missing data through advanced imputation techniques substantially reinforces the integrity of the models. Finally, the added benefit of a large sample size derived from a heterogeneous cohort reflects the authors’ endeavour to develop a reliable and generalisable model applicable to actual clinical practice.

Despite the strengths of this study, certain limitations may challenge the relevance of these models in clinical practice. Beyond model discrimination and calibration, one of the considerations for a risk calculator is its ease of use. The current model comprises 20 working variables, 16 if this excludes intra-operative variables. Moreover, a closer look at the variables selected for the model reveals that several essential determinants of PHLF, such as FLR or indocyanine green (ICG) retention, were not available in the model, a limitation acknowledged by the authors. In an article by Schwarz et al., pre-operative ICG clearance was associated with not just liver dysfunction in major but also minor hepatectomies (6). Similarly, Kim et al. reported the correlation between standardized FLR (sFLR) and ICG retention as a predictor of post-operative hepatic function (7). The relevance of ICG clearance in PHLF is further elaborated in numerous studies establishing the optimal cut-offs for safe resection, including Maakuchi’s criteria in which ICG retention is used to estimate liver functional reserve while providing recommendations on the limit of liver resection (8). In addition, despite incorporating an Asian cohort, the generalizability of this prediction model may be less applicable to most Asians. In most western cohorts and Japan, hepatitis C is the predominant hepatotropic risk factor for liver disease, whereas most of East and Southeast Asia is hepatitis B virus (HBV) endemic. In a systematic review of the global prevalence of chronic hepatitis B infections, countries like China and South Korea have a prevalence estimate of 5.49% and 4.36%, respectively (9), aligning with the burden of hepatocellular carcinoma (HCC) in these regions. The added impact of liver injury from peri-operative HBV activation in patients without appropriate anti-viral treatment further jeopardizes remnant liver function (10) and may contribute to PHLF. As such, omitting patients from HBV-endemic areas and certain important viral-specific variables may further compromise the model’s generalisability.

As with most studies aimed at developing prediction tools, one key concern is the practical application of the developed model in clinical environments. One of the challenges in prediction studies is the gap between statistical modelling and actual clinical utility. To truly bridge this gap, the manuscript would benefit significantly from a more detailed exposition on the practical translation of this statistical model into actionable clinical tools by giving specific examples or guidelines on their implementation in daily clinical practice and workflows. This would help clinicians understand how such models can be applied in patient assessment, surgical planning, and risk stratification. Moreover, the operationalisation of this model, especially the incorporation of intraoperative variables, is inadequately addressed, leaving questions on how the model would perform in real-time surgical care. For instance, a look at the online calculator on Evidencio (https://www.evidencio.com/home) reveals that omitting information pertaining to intraoperative factors can dramatically change the probability of PHLF. In another example, adopting an open as opposed to a laparoscopic approach increases the risk of PHLF; while performing resections with vascular resection—an intuitive surrogate for complexity, paradoxically reduces the risk of PHLF. This underscores some significant challenges when applying statistical predictions directly to patient management during operations. Moving forward, the clinical applicability of such models should be externally validated and rigorously tested to provide more precise insights on their predictive and clinical applicability before recommendation into routine clinical practice.

Acknowledgments

Funding: None.

Footnote

Provenance and Peer Review: This article was commissioned by the editorial office, HepatoBiliary Surgery and Nutrition. The article did not undergo external peer review.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://hbsn.amegroups.com/article/view/10.21037/hbsn-24-289/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Jarnagin WR, Gonen M, Fong Y, et al. Improvement in perioperative outcome after hepatic resection: analysis of 1,803 consecutive cases over the past decade. Ann Surg 2002;236:397-406; discussion 406-7. [Crossref] [PubMed]
2. Farges O, Malassagne B, Flejou JF, et al. Risk of major liver resection in patients with underlying chronic liver disease: a reappraisal. Ann Surg 1999;229:210-5. [Crossref] [PubMed]
3. Rahbari NN, Garden OJ, Padbury R, et al. Posthepatectomy liver failure: a definition and grading by the International Study Group of Liver Surgery (ISGLS). Surgery 2011;149:713-24. [Crossref] [PubMed]
4. Wang JJ, Feng J, Gomes C, et al. Development and Validation of Prediction Models and Risk Calculators for Posthepatectomy Liver Failure and Postoperative Complications Using a Diverse International Cohort of Major Hepatectomies. Ann Surg 2023;278:976-84. [Crossref] [PubMed]
5. Slankamenac K, Graf R, Barkun J, et al. The comprehensive complication index: a novel continuous scale to measure surgical morbidity. Ann Surg 2013;258:1-7. [Crossref] [PubMed]
6. Schwarz C, Plass I, Fitschek F, et al. The value of indocyanine green clearance assessment to predict postoperative liver dysfunction in patients undergoing liver resection. Sci Rep 2019;9:8421. [Crossref] [PubMed]
7. Kim HJ, Kim CY, Park EK, et al. Volumetric analysis and indocyanine green retention rate at 15 min as predictors of post-hepatectomy liver failure. HPB (Oxford) 2015;17:159-67. [Crossref] [PubMed]
8. Kobayashi Y, Kiya Y, Sugawara T, et al. Expanded Makuuchi’s criteria using estimated indocyanine green clearance rate of future liver remnant as a safety limit for maximum extent of liver resection. HPB (Oxford) 2019;21:990-7. [Crossref] [PubMed]
9. Schweitzer A, Horn J, Mikolajczyk RT, et al. Estimations of worldwide prevalence of chronic hepatitis B virus infection: a systematic review of data published between 1965 and 2013. Lancet 2015;386:1546-55. [Crossref] [PubMed]
10. Chong CC, Wong GL, Lai PB. Impact of antiviral therapy on post-hepatectomy outcome for hepatitis B-related hepatocellular carcinoma. World J Gastroenterol 2014;20:6006-12. [Crossref] [PubMed]