Using the win ratio to compare laparoscopic versus open liver resection for colorectal cancer liver metastases

Introduction

Colorectal cancer represents the third most common cancer and the second leading cause of cancer-related death in the United States (1,2). The liver is the most common site of metastatic spread with at least 25% of patients diagnosed with colorectal cancer eventually developing colorectal liver metastases (CRLMs) (3). Surgical resection is the only potentially curative treatment, and is associated with a 5-year survival of approximately 40–50% (4,5). Traditionally, open hepatectomy has been the surgical approach of choice in CRLM patients. Since its introduction in the early 1990s, the use of laparoscopic liver resection in the treatment of CRLM has become increasingly common (6-11).

Notably, a laparoscopic approach to liver resection may offer several short-term benefits over an open approach. Specifically, laparoscopic hepatectomy requires smaller incisions and may result in lower overall morbidity, reduced intraoperative blood loss, a shorter length of postoperative stay and lower overall costs versus open hepatectomy (12-14). The use of laparoscopic resection for CRLM remains debated, however, due to the lack of data on oncologic and long-term outcomes. Several observational studies have reported comparable oncologic and long-term outcomes for patients who undergo laparoscopic resection as opposed to open resection (15-18). The results of these studies are difficult to interpret, however, due to sample size, center bias, as well as intrinsic selection bias.

The “win ratio” is a composite outcome metric originally introduced in the context of cardiovascular clinical trials (19). The calculation of a win ratio involves assessing potential pairs of patients who have undergone two different treatment interventions. Patients in each pair (i.e., treatment A vs. treatment B) are subsequently compared based on the hierarchical order of the component outcomes comprising a “win”. If either patient fares better than the other patient relative to the first rank ordered component outcome then a “win” for that given treatment and a “loss” for the alternative treatment are recorded; if not, the pair is considered a “tie” and the two patients are compared based on the second component outcome, and so on (20). In this manner, the win ratio is the total number of wins for a given specific treatment divided by the total number of losses. Thus, the win ratio accounts not only for the proportion of patients who achieve each individual component outcome (i.e., no complication, readmission, etc.), but also rank orders outcomes based on relative hierarchical importance. Additionally, the win ratio can combine information on time-to-event, continuous and/or categorical outcomes relative to both short- and long-term outcomes following surgery.

While not well-studied relative to surgical procedures, the win ratio may represent a useful composite means to facilitate a more comprehensive comparison of two different therapeutic interventions such as open versus laparoscopic hepatic resection for CRLM. Therefore, the objective of the current study was to determine the win ratio associated with open versus laparoscopic hepatic resection in a large international cohort of patients diagnosed with CRLM. Additional analysis was stratified by relevant demographic and clinical characteristics to identify subgroups of patients who may have benefited more from the use of either approach. We present this article in accordance with the STROBE reporting checklist (available at https://hbsn.amegroups.com/article/view/10.21037/hbsn-22-36/rc).

Methods

Ethical statement

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). Obtaining informed consent from all individual participants was not required as the study relied on secondary data analysis of de-identified patient data.

Patient population

Patients who underwent curative-intent resection for CRLM between 2001 and 2018 were identified from an international multi-institutional database involving five major hepatobiliary centers: the Ohio State University Comprehensive Cancer Center (Columbus, OH, USA), Stanford University (Stanford, CA, USA), Cleveland Clinic Foundation (Cleveland, OH, USA), Fundeni Clinical Institute (Bucharest, Romania) and the University of Verona (Verona, Italy). Patients were excluded if they underwent palliative resection, underwent robotic resection or had extrahepatic disease at diagnosis.

Variables, definitions and outcomes

Patients were split into two separate, intention-to-treat cohorts based on whether they underwent open or laparoscopic hepatic resection for CRLM. Patients who underwent a laparoscopic hepatectomy that required conversion to an open approach were categorized into the laparoscopic hepatectomy cohort for the purpose of analysis. Other variables of interest included patient age, sex, location of the primary colorectal cancer, American Joint Committee on Cancer (AJCC) T stage, presence of associated lymph node metastases, as well as whether the disease-free interval between diagnosis of the primary and diagnosis of the metastases was longer than 12 months. The number of hepatic lesions and the size of the largest hepatic lesion were reported separately, as well as combined to categorize patients into low, medium or high tumor burden groups. Additionally, disease was characterized as unilobar or bilobar. Information on preoperative levels of carcinoembryonic antigen (CEA) and KRAS mutational status were obtained. Any tumor with a KRAS mutation identified either on the primary tumor or on hepatic metastases was considered to be KRAS mutated, as the KRAS mutation concordance rate between the two is over 90% (21). Details about the treatment patients received were also included, such as extent of hepatic resection (major or minor), concurrent ablation at the time of surgery, and receipt of chemotherapy before or after surgery. Major hepatectomy was defined as a resection of at least three Couinaud liver segments (22).

The main outcome of interest was the win ratio, a composite outcome metric used to compare two alternative treatment or management options (19,20). Unlike other composite outcomes, the win ratio not only accounts for the achievement of each of the component outcomes, but also accounts for their relative priority. Additionally, the win ratio is not restricted to a single variable type, but can include time-to-event, continuous and/or categorical outcomes (23,24). To calculate the win ratio, all patients from one treatment group (i.e., open) were paired with all patients in the other treatment group (i.e., laparoscopic); outcomes were then compared with regards to their hierarchical ordering. If a patient undergoing one specific treatment had a better outcome, it was defined as a “win”. If instead the patient undergoing the reference treatment had a better outcome, it was defined as a “loss”. Otherwise, it was considered a “tie”. The win ratio was then calculated by dividing the number of wins by the number of losses. A more thorough explanation of the win ratio approach has been provided by Redfors et al. (19).

The outcomes selected for inclusion in the win ratio were: surgical resection margin status, severity of postoperative complications, 90-day mortality, time to recurrence, and time to death. 90-day mortality was chosen as a measure of perioperative mortality as this metric has been demonstrated to capture better the perioperative risk associated with hepatectomy than 30-day mortality (25). Severity of postoperative complications was graded according to the Clavien-Dindo classification (26). In the current study, not all unmatched patient-to-patient pairs were considered. Instead, patients were matched based on age, number of hepatic lesions, size of the largest lesion, primary tumor lymph node status and receipt of neoadjuvant chemotherapy. For continuous variables, members of a pair were not allowed to differ by more than one standard deviation; for categorical variables, members of a pair were only allowed to have the same value. The matching process avoided pairwise comparisons between patients with drastically different underlying risk profiles. This particular approach to matching is akin to variable ratio matching in which one patient in the laparoscopic cohort may be matched with one patient in the open cohort whereas in another case, one patient in the laparoscopic cohort may be matched with multiple patients in the open cohort. This particular approach is commonly used in matching schemes to minimize bias relative to 1:1 matching schemes (27,28). The variables used for the matching process were selected based on their established association with outcomes following surgical resection of CRLM (29-31). The win ratio was calculated for the overall cohort, as well as for subsets of the patient population.

Statistical analysis

Descriptive statistics were presented as median (interquartile range; IQR) for continuous variables and frequency (relative frequency, %) for categorical variables. Bivariate associations between surgical approach (laparoscopic or open) and patient demographic and clinical characteristics were assessed using the Mann-Whitney U test for continuous variables and Chi-square tests for categorical variables. To account for patient differences and ensure comparisons to calculate the win ratio were only among pairs of similar patients, matching was performed such that categorical factors must have been an exact match between patients and for continuous variables, patients must have been within one standard deviation (measured on the entire cohort) of one another. To assess how the variable ratio matching approach implemented in the primary analysis may have influenced the WR or confidence interval (CI) estimates, a sensitivity analysis was performed on the overall cohort and then stratified using the more traditional, unmatched approach. All analyses were performed using SAS version 9.4. Statistical significance was assessed at α=0.05.

Results

Patient characteristics

A total of 962 patients undergoing hepatic resection for CRLM between 2001 and 2018 met inclusion criteria. Median age was 61 years (IQR, 53–68.3) and the majority of patients were male (n=583, 60.6%) (Table 1). The primary colorectal cancer most often involved the left colon (n=418, 43.5%), followed by the rectum (n=288, 29.9%) and the right colon (n=235, 24.4%). More than two-thirds of patients had T3 colorectal cancer at the time of diagnosis (n=659, 68.5%), and slightly more than one-half of patients had lymph node metastasis associated with the primary colorectal cancer (n=499, 51.9%). Of note, more than two thirds of patients had a disease-free interval <12 months (n=650, 67.6%). The median number of hepatic lesions was 2 (IQR, 1–3), while the median size of the largest hepatic lesion was 3 cm (IQR, 1.9–4.5). In turn, almost three fourths of patients were classified as having a medium tumor burden (n=696, 72.3%), while other patients either had a low (n=103, 10.7%) or a high (n=163, 16.9%) tumor burden. The median preoperative CEA level was 9.5 ng/mL (IQR, 3.6–42). KRAS mutational status could be determined in 49.8% (n=479) of patients. Specifically, 18.0% (n=173) of patients had a mutated KRAS, while 31.8% (n=306) of patients had a wild-type KRAS. Most patients received neoadjuvant chemotherapy prior to resection (n=562, 58.4%); resection consisted of a minor hepatic resection in about two thirds of cases (n=660, 68.6%). Only a small subset of patients had an ablation concomitant with resection (n=58, 6.0%). Following resection, about 6 in 10 patients received adjuvant chemotherapy (n=589, 61.2%).

The majority of patients underwent an open hepatectomy (n=832, 86.5%), while a subset of patients underwent a laparoscopic hepatectomy (n=130, 13.5%) (Table 1). Of note, patients who had a laparoscopic resection were older (63.8 years, IQR, 55.3–72.6 vs. 61 years, IQR, 52.7–68.0) and were more often female (48.5% vs. 38.0%) compared with patients who had an open resection (both P<0.05). Additionally, patients in the laparoscopic group had fewer hepatic lesions (1, IQR, 1–2 vs. 2, IQR, 1–3), as well as smaller size lesions (2.0 cm, IQR, 1.3–3.5 vs. 3.0 cm, IQR, 2.0–4.9) than patients in the open group (both P<0.001). As a result, patients undergoing laparoscopic surgery were less likely to have a high overall tumor burden (4.6% vs. 18.9%) or bilobar disease (13.1% vs. 31.9%) versus patients undergoing open surgery (both P<0.001). In turn, patients in the laparoscopic group were more likely to undergo a minor hepatectomy compared with patients in the open group (92.3% vs. 64.9%; P<0.001).

Win ratio component outcomes

Table 2 lists the component outcomes that were included in the calculation of the win ratio in addition to long-term overall survival and recurrence-free survival. Overall, 90-day survival following surgery was 96.9% (n=932), and most patients had no postoperative complications (n=643, 66.8%). Patients who developed postoperative complications most often had a grade II complication (n=153, 15.9%), followed by a grade III complication (n=87, 9.0%), a grade I complication (n=54, 5.6%) or a grade IV complication (n=25, 2.6%). In terms of oncologic outcomes, the majority of patients had negative surgical margins following resection (n=750, 78.0%). Of note, patients who underwent laparoscopic hepatectomy were less likely to experience 90-day mortality or postoperative complications versus patients who underwent open hepatectomy (0% vs. 3.6% and 16.2% vs. 35.8%, respectively; both P<0.05) (Table 2). The proportion of patients who had negative resection margins was lower among patients who underwent laparoscopic resection versus patients who underwent open resection (70.0% vs. 79.2%, P=0.008).

Win ratio

Calculation of the win ratio for the overall patient population is illustrated in Figure 1. The win ratio for the overall patient population was 1.77 (95% CI: 1.42–2.34), indicating that for every matched patient-to-patient pair the odds that the patient undergoing laparoscopic resection was the “winner” were 1.77 higher than a given patient who underwent an open resection. Calculation of the win ratio was stratified by relevant patient demographic and clinical characteristics (Table 3 and Figure 2). Specifically, the win ratio was more favorable following laparoscopic hepatectomy among female (WR: 1.98, 95% CI: 1.25–3.11) and male patients (WR: 1.76, 95% CI: 1.27–2.42), as well as older (WR: 2.01, 95% CI: 1.36–2.89) and younger patients (WR: 1.74, 95% CI: 1.23–2.49). Notably, the win ratio was also in favor of laparoscopic hepatectomy regardless of whether the patient had a low (WR: 2.94, 95% CI: 1.20–6.39), medium (WR: 1.56, 95% CI: 1.16–2.10) or high (WR: 7.25, 95% CI: 1.13–32.0) tumor burden, as well as unilobar (WR: 1.71, 95% CI: 1.25–2.31) or bilobar (WR: 4.57, 95% CI: 2.36–8.64) disease. The win ratio was more favorable among patients who received neoadjuvant chemotherapy following by laparoscopic resection (WR: 1.85, 95% CI: 1.31–2.62); in contrast, among patients who did not receive neoadjuvant therapy, the win ratio did not appear to favor either surgical approach (WR: 1.58, 95% CI: 0.99–2.46). The win ratio also favored laparoscopic resection among patients with a primary colon cancer (WR: 1.69, 95% CI: 1.26–2.23), but did not favor either approach among patients with a primary rectal cancer (WR: 1.83, 95% CI: 0.89–3.92). Results of the sensitivity analysis using the unmatched approach closely resembled the results obtained through the matched approach (see Table S1).

Figure 1 Illustration of how the win ratio was calculated. Each box contains the cumulative incidence of wins, ties and losses, as well as the associated win ratio, after each component outcome is included in the calculation.

Figure 2 Forest plot showing the win ratio for the overall patient population, as well as the win ratios for subgroups of the patient population. A logarithmic scale was used for the x-axis.

Discussion

Several advantages to a laparoscopic approach to hepatectomy for the treatment of CRLM have been suggested. For example, observational studies have reported that CRLM patients undergoing laparoscopic resection experience less intraoperative blood loss, fewer complications and a shorter in-hospital length-of stay (12,17,18,32). Results of these observational studies have been substantiated by the OSLO-COMET randomized controlled trial in which CRLM patients undergoing laparoscopic parenchyma-sparing resections experienced fewer postoperative complications and had a shorter hospital stay than patients undergoing open parenchyma-sparing resections (33,34). However, the OSLO-COMET trial was underpowered to detect even moderate differences in long-term survival between the two groups (35). As such, strong evidence supporting the noninferiority of the laparoscopic approach in terms of long-term outcomes is currently lacking. Methodological approaches to compare the two approaches using composite measures that include information on short-, long-term and oncologic outcomes may allow for a more holistic comparison of the two procedures. Composite measures have been proposed as potentially superior to individual measures to assess performance, as well as more reflect the “all-or-none” approach to outcomes from a patient-centered perspective (36). One such composite measure is the win ratio, which has the added benefit of being able to assess component outcomes hierarchically, giving the most clinically relevant outcomes precedence (19). In turn, the current study was important because we used the win ratio to compare laparoscopic versus open hepatectomy in the treatment of patients with CRLM. Unlike more conventional quality assessments that have been traditionally restricted to a single variable, win ratio provided a means to assess a composite endpoint composed of time-to-event, continuous, and/or categorical outcomes. The hierarchical structure, statistical power, and flexibility of the win ratio approach made it an attractive alternative to compare the efficacy of these different treatment approaches. Of note, among any given pair of patients with CRLM, patients who underwent laparoscopic resection had 77% increased odds of “winning” relative to the composite hierarchy of outcomes (Figure 1). In fact, the laparoscopic approach had an advantage over the open approach among patients regardless of age, sex, tumor burden or tumor location within the liver.

While CRLM patients who underwent laparoscopic versus open resection had overall increased odds of “winning”, the odds of “winning” were particularly pronounced relative to short-term outcomes (i.e., 90-day mortality and severity of postoperative complications) (WR: 4.06, 95% CI: 2.33–7.78; see Figure 1). Several previous observational studies had reported a lower rate of postoperative complications among patients undergoing laparoscopic resection than with open resection (37-39). In an analysis of 266 CRLM patients, Lewin et al. noted that patients undergoing laparoscopic rather than open liver resection had a lower incidence of both liver-specific and non-liver-specific complications (18). In the current study, patients who underwent laparoscopic resection not only had fewer perioperative complications, but the complications tended to be less severe (Table 2). Collectively these data are important as there had been initial concerns about the safety of the laparoscopic approach for hepatic resection (i.e., risk of gas embolism, ability to control catastrophic bleeding, etc.) (40). In fact, in a systemic review of 127 articles that equated to 2,804 laparoscopic liver resections, Nguyen and colleague reported that overall morbidity was only 10.5% with no intraoperative deaths (41). In the current study, only 3 out of 130 patients undergoing laparoscopic liver resection had a perioperative complication of grade III or higher, further reinforcing the idea that such initial concerns may be unfounded (Table 2).

Of note, the differential in the win ratio decreased after accounting for long-term overall and recurrence-free survival, as well as the incidence of negative surgical margins (Figure 1). Results of the current study suggested that the main advantage of a laparoscopic approach to hepatic resection in CRLM patients resided in improved perioperative—rather than long-term—outcomes. As such, surgeons who are not confident in their ability to perform a laparoscopic resection should not hesitate to perform an open resection knowing that long-term outcomes are comparable. Of note, in the current study, choice of laparoscopic versus open approach was at the surgeon-level and was based on clinical factors and experience. The incidence of positive surgical margins among patients who underwent laparoscopic hepatectomy was, however, higher among patients who underwent a laparoscopic versus open hepatectomy (Table 2). These findings differed from the data reported by Lewin et al., which noted that patients who underwent laparoscopic resection had a lower incidence of microscopically positive surgical margins compared with patients who had an open resection (8% vs. 18%) (18). Several other previous studies had noted, however, no overall differences in the incidence of positive surgical margins between the laparoscopic versus open approaches (14,42). The reasons for these disparate data are likely multifactorial. Early studies on laparoscopic liver resection largely involved only patients who had a bisectionectomy or other minor liver resections in which achieving a negative surgical margin can be more easily achieved. In contrast, in the current study, roughly 1 in 13 patients underwent a major laparoscopic hepatic resection. One of the limitations of a laparoscopic approach to a major hepatectomy may be the inability to palpate the liver, which may in turn lead to the need for repeat ultrasounds to plan the parenchymal resection plan to obtain adequate surgical margins (43). It is important to note that the win ratio decreased only slightly after including information on surgical margins (Figure 1). Additionally, surgical margin status was assessed hierarchically only after differences in overall and recurrence-free survival had already been accounted for in the algorithm. Therefore, while there was a modest difference in negative surgical margin status among patients who had laparoscopic versus open hepatectomy, this difference had a negligible impact on long-term outcomes.

Patients undergoing laparoscopic resection had higher odds of winning than patients undergoing open resection independent of sex, age, tumor burden or hepatic tumor location (Table 3 and Figure 2). These data were particularly interesting as tumor size and location are two of the main factors used to estimate the difficulty of a laparoscopic liver resection according to Ban’s difficulty score (44). Specifically, the Ban’s difficulty score takes into account the extent of liver resection, tumor location, tumor size, liver function, and tumor proximity to major vessels. In the current study, all of these factors were not included in the win ratio calculation due to lack of data relative to specific segments involved or the tumor’s proximity to vascular structures. We did note, however, that the benefit of a laparoscopic approach to CRLM resection persisted even among a high tumor burden score. The benefits of a laparoscopic approach appear were also more pronounced among patients who received preoperative chemotherapy prior to resection (Table 3 and Figure 2). A positive response to preoperative chemotherapy can often result in cytoreduction within the liver, perhaps enhancing the technical feasibility of laparoscopic hepatectomy. In addition, laparoscopic liver resection may help mitigate the risk of certain perioperative complications that have been associated with preoperative chemotherapy (45-48). In addition, the win ratio demonstrated that laparoscopic liver resection was more favorable than open liver resection among patients diagnosed with a primary colon cancer rather than a primary rectal cancer (Table 3 and Figure 2). In an analysis of 2,972 patients from the South Australian Metastatic Colorectal Cancer Registry, Price et al. noted differences in long-term outcomes among patients with metastatic right-sided versus left-sided colorectal cancer, yet no difference relative to left-sided colon cancer versus rectal cancer (49). The use of minimally invasive resections for patients with synchronous metastasis and primary colorectal cancer has been a topic of particular discussion (50,51). Recently an adapted Delphi method of a panel of liver surgeon experts, in conjunction with a systematic literature review, reported that the use of minimally invasive technique for simultaneous treatment of synchronous CRLM was similar to the documented benefits of two separate surgeries (52).

Several limitations should be considered when interpreting the results of the current study. As patients were identified using a large multi-institutional database, practice patterns were not standardized across institutions. While there may have been variations in clinical practice at different centers, the involvement of multiple institutions and patient populations markedly improved the generalizability of the findings. When calculating the win ratio, patients in a pair were matched based on age, number and size of hepatic lesions, primary tumor lymph node status, as well as receipt of neoadjuvant chemotherapy. Residual selection bias may still exist, however, due to confounders that were not accounted for in the matching process. Of note, the current study focused on the first resection of CRLM; repeat hepatectomy after recurrence was not considered. As such, receipt of repeat hepatectomy following recurrence may represent a confounding factor in the relationship between treatment modality (i.e., open versus laparoscopic resection) and long-term survival. In addition, the majority of laparoscopic procedures were minor liver resections; therefore, future studies that include a higher proportion of patients who have undergone minimally invasive major liver resection will be needed (53).

Conclusions

In conclusion, using a novel statistical approach called the win ratio, we assessed the relative benefit of laparoscopic versus open liver resection among CRLM patients. The win ratio provided a single measure that accounted for both short- and long-term outcomes, as well as the relative hierarchical priority of the outcomes. Notably, patients undergoing laparoscopic liver resection had 77% increased odds of “winning” versus patients who underwent open liver resection. Importantly, patients undergoing laparoscopic liver resection had increased odds of “winning” among patients independent of age, sex, tumor burden or location. The data strongly suggest that laparoscopic liver resection, on balance, should be strongly considered as a preferred approach to resection in CRLM patients.

Acknowledgments

Funding: None.

Footnote

Reporting Checklist: The authors have completed the STROBE reporting checklist. Available at https://hbsn.amegroups.com/article/view/10.21037/hbsn-22-36/rc

Data Sharing Statement: Available at https://hbsn.amegroups.com/article/view/10.21037/hbsn-22-36/dss

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://hbsn.amegroups.com/article/view/10.21037/hbsn-22-36/coif). TMP serves as the unpaid Deputy Editor-in-Chief of Hepatobiliary Surgery and Nutrition. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). Obtaining informed consent from all individual participants was not required as the study relied on secondary data analysis of de-identified patient data.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Siegel RL, Miller KD, Goding Sauer A, et al. Colorectal cancer statistics, 2020. CA Cancer J Clin 2020;70:145-64. [Crossref] [PubMed]
2. American Cancer Society. Key Statistics for Colorectal Cancer 2021 [updated Jan, 12 2021. Available online: https://www.cancer.org/cancer/colon-rectal-cancer/about/key-statistics.html
3. Martin J, Petrillo A, Smyth EC, et al. Colorectal liver metastases: Current management and future perspectives. World J Clin Oncol 2020;11:761-808. [Crossref] [PubMed]
4. de Jong MC, Pulitano C, Ribero D, et al. Rates and patterns of recurrence following curative intent surgery for colorectal liver metastasis: an international multi-institutional analysis of 1669 patients. Ann Surg 2009;250:440-8. [Crossref] [PubMed]
5. Choti MA, Sitzmann JV, Tiburi MF, et al. Trends in long-term survival following liver resection for hepatic colorectal metastases. Ann Surg 2002;235:759-66. [Crossref] [PubMed]
6. Azagra JS, Goergen M, Gilbart E, et al. Laparoscopic anatomical (hepatic) left lateral segmentectomy-technical aspects. Surg Endosc 1996;10:758-61. [Crossref] [PubMed]
7. Hüscher CG, Lirici MM, Chiodini S, et al. Current position of advanced laparoscopic surgery of the liver. J R Coll Surg Edinb 1997;42:219-25. [PubMed]
8. Cherqui D, Husson E, Hammoud R, et al. Laparoscopic liver resections: a feasibility study in 30 patients. Ann Surg 2000;232:753-62. [Crossref] [PubMed]
9. O'Rourke N, Shaw I, Nathanson L, et al. Laparoscopic resection of hepatic colorectal metastases. HPB (Oxford) 2004;6:230-5. [Crossref] [PubMed]
10. Nguyen KT, Laurent A, Dagher I, et al. Minimally invasive liver resection for metastatic colorectal cancer: a multi-institutional, international report of safety, feasibility, and early outcomes. Ann Surg 2009;250:842-8. [Crossref] [PubMed]
11. Pilgrim CH, To H, Usatoff V, et al. Laparoscopic hepatectomy is a safe procedure for cancer patients. HPB (Oxford) 2009;11:247-51. [Crossref] [PubMed]
12. Cannon RM, Scoggins CR, Callender GG, et al. Financial comparison of laparoscopic versus open hepatic resection using deviation-based cost modeling. Ann Surg Oncol 2013;20:2887-92. [Crossref] [PubMed]
13. Schiffman SC, Kim KH, Tsung A, et al. Laparoscopic versus open liver resection for metastatic colorectal cancer: a metaanalysis of 610 patients. Surgery 2015;157:211-22. [Crossref] [PubMed]
14. Jackson NR, Hauch A, Hu T, et al. The safety and efficacy of approaches to liver resection: a meta-analysis. JSLS 2015;19:e2014.00186.
15. Castaing D, Vibert E, Ricca L, et al. Oncologic results of laparoscopic versus open hepatectomy for colorectal liver metastases in two specialized centers. Ann Surg 2009;250:849-55. [Crossref] [PubMed]
16. Montalti R, Berardi G, Laurent S, et al. Laparoscopic liver resection compared to open approach in patients with colorectal liver metastases improves further resectability: Oncological outcomes of a case-control matched-pairs analysis. Eur J Surg Oncol 2014;40:536-44. [Crossref] [PubMed]
17. Cipriani F, Rawashdeh M, Stanton L, et al. Propensity score-based analysis of outcomes of laparoscopic versus open liver resection for colorectal metastases. Br J Surg 2016;103:1504-12. [Crossref] [PubMed]
18. Lewin JW, O'Rourke NA, Chiow AKH, et al. Long-term survival in laparoscopic vs open resection for colorectal liver metastases: inverse probability of treatment weighting using propensity scores. HPB (Oxford) 2016;18:183-91. [Crossref] [PubMed]
19. Redfors B, Gregson J, Crowley A, et al. The win ratio approach for composite endpoints: practical guidance based on previous experience. Eur Heart J 2020;41:4391-9. [Crossref] [PubMed]
20. Pocock SJ, Ariti CA, Collier TJ, et al. The win ratio: a new approach to the analysis of composite endpoints in clinical trials based on clinical priorities. Eur Heart J 2012;33:176-82. [Crossref] [PubMed]
21. Cejas P, López-Gómez M, Aguayo C, et al. KRAS mutations in primary colorectal cancer tumors and related metastases: a potential role in prediction of lung metastasis. PLoS One 2009;4:e8199. [Crossref] [PubMed]
22. Strasberg SM. Nomenclature of hepatic anatomy and resections: a review of the Brisbane 2000 system. J Hepatobiliary Pancreat Surg 2005;12:351-5. [Crossref] [PubMed]
23. Finkelstein DM, Schoenfeld DA. Graphing the Win Ratio and its components over time. Stat Med 2019;38:53-61. [Crossref] [PubMed]
24. Oakes D. On the win-ratio statistic in clinical trials with multiple types of event. Biometrika 2016;103:742-5. [Crossref]
25. Mayo SC, Shore AD, Nathan H, et al. Refining the definition of perioperative mortality following hepatectomy using death within 90 days as the standard criterion. HPB (Oxford) 2011;13:473-82. [Crossref] [PubMed]
26. Dindo D, Demartines N, Clavien PA. Classification of surgical complications: a new proposal with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg 2004;240:205-13. [Crossref] [PubMed]
27. Stuart EA. Matching methods for causal inference: A review and a look forward. Stat Sci 2010;25:1-21. [Crossref] [PubMed]
28. Ming K, Rosenbaum PR. A note on optimal matching with variable controls using the assignment algorithm. J Comput Graph Stat 2001;10:455-63. [Crossref]
29. Fong Y, Fortner J, Sun RL, et al. Clinical score for predicting recurrence after hepatic resection for metastatic colorectal cancer: analysis of 1001 consecutive cases. Ann Surg 1999;230:309-18; discussion 318-21. [Crossref] [PubMed]
30. Margonis GA, Sasaki K, Gholami S, et al. Genetic And Morphological Evaluation (GAME) score for patients with colorectal liver metastases. Br J Surg 2018;105:1210-20. [Crossref] [PubMed]
31. Guo M, Jin N, Pawlik T, et al. Neoadjuvant chemotherapy for colorectal liver metastases: A contemporary review of the literature. World J Gastrointest Oncol 2021;13:1043-61. [Crossref] [PubMed]
32. Untereiner X, Cagniet A, Memeo R, et al. Laparoscopic hepatectomy versus open hepatectomy for colorectal cancer liver metastases: comparative study with propensity score matching. Hepatobiliary Surg Nutr 2016;5:290-9. [Crossref] [PubMed]
33. Fretland ÅA, Dagenborg VJ, Bjørnelv GMW, et al. Laparoscopic Versus Open Resection for Colorectal Liver Metastases: The OSLO-COMET Randomized Controlled Trial. Ann Surg 2018;267:199-207. [Crossref] [PubMed]
34. Chan AKC, Jamdar S, Sheen AJ, et al. The OSLO-COMET Randomized Controlled Trial of Laparoscopic Versus Open Resection for Colorectal Liver Metastases. Ann Surg 2018;268:e69. [Crossref] [PubMed]
35. Aghayan DL, Kazaryan AM, Dagenborg VJ, et al. Long-Term Oncologic Outcomes After Laparoscopic Versus Open Resection for Colorectal Liver Metastases: A Randomized Trial. Ann Intern Med 2021;174:175-82. [Crossref] [PubMed]
36. Shwartz M, Restuccia JD, Rosen AK. Composite Measures of Health Care Provider Performance: A Description of Approaches. Milbank Q 2015;93:788-825. [Crossref] [PubMed]
37. Slakey DP, Simms E, Drew B, et al. Complications of liver resection: laparoscopic versus open procedures. JSLS 2013;17:46-55. [Crossref] [PubMed]
38. Hu MG, Ou-yang CG, Zhao GD, et al. Outcomes of open versus laparoscopic procedure for synchronous radical resection of liver metastatic colorectal cancer: a comparative study. Surg Laparosc Endosc Percutan Tech 2012;22:364-9. [Crossref] [PubMed]
39. Zhou Y, Xiao Y, Wu L, et al. Laparoscopic liver resection as a safe and efficacious alternative to open resection for colorectal liver metastasis: a meta-analysis. BMC Surg 2013;13:44. [Crossref] [PubMed]
40. Gigot JF, Glineur D, Santiago Azagra J, et al. Laparoscopic liver resection for malignant liver tumors: preliminary results of a multicenter European study. Ann Surg 2002;236:90-7. [Crossref] [PubMed]
41. Nguyen KT, Geller DA. Laparoscopic liver resection--current update. Surg Clin North Am 2010;90:749-60. [Crossref] [PubMed]
42. Inoue Y, Hayashi M, Tanaka R, et al. Short-term results of laparoscopic versus open liver resection for liver metastasis from colorectal cancer: a comparative study. Am Surg 2013;79:495-501. [Crossref] [PubMed]
43. Robles-Campos R, Lopez-Lopez V, Brusadin R, et al. Open versus minimally invasive liver surgery for colorectal liver metastases (LapOpHuva): a prospective randomized controlled trial. Surg Endosc 2019;33:3926-36. [Crossref] [PubMed]
44. Ban D, Tanabe M, Ito H, et al. A novel difficulty scoring system for laparoscopic liver resection. J Hepatobiliary Pancreat Sci 2014;21:745-53. [Crossref] [PubMed]
45. Karoui M, Penna C, Amin-Hashem M, et al. Influence of preoperative chemotherapy on the risk of major hepatectomy for colorectal liver metastases. Ann Surg 2006;243:1-7. [Crossref] [PubMed]
46. Bilchik AJ, Poston G, Curley SA, et al. Neoadjuvant chemotherapy for metastatic colon cancer: a cautionary note. J Clin Oncol 2005;23:9073-8. [Crossref] [PubMed]
47. Fernandez FG, Ritter J, Goodwin JW, et al. Effect of steatohepatitis associated with irinotecan or oxaliplatin pretreatment on resectability of hepatic colorectal metastases. J Am Coll Surg 2005;200:845-53. [Crossref] [PubMed]
48. Wolf PS, Park JO, Bao F, et al. Preoperative chemotherapy and the risk of hepatotoxicity and morbidity after liver resection for metastatic colorectal cancer: a single institution experience. J Am Coll Surg 2013;216:41-9. [Crossref] [PubMed]
49. Price TJ, Beeke C, Ullah S, et al. Does the primary site of colorectal cancer impact outcomes for patients with metastatic disease? Cancer 2015;121:830-5. [Crossref] [PubMed]
50. Ejaz A, Semenov E, Spolverato G, et al. Synchronous primary colorectal and liver metastasis: impact of operative approach on clinical outcomes and hospital charges. HPB (Oxford) 2014;16:1117-26. [Crossref] [PubMed]
51. Mayo SC, Pulitano C, Marques H, et al. Surgical management of patients with synchronous colorectal liver metastasis: a multicenter international analysis. J Am Coll Surg 2013;216:707-16; discussion 716-8. [Crossref] [PubMed]
52. Rocca A, Cipriani F, Belli G, et al. The Italian Consensus on minimally invasive simultaneous resections for synchronous liver metastasis and primary colorectal cancer: A Delphi methodology. Updates Surg 2021;73:1247-65. [Crossref] [PubMed]
53. Wakabayashi G, Cherqui D, Geller DA, et al. Recommendations for laparoscopic liver resection: a report from the second international consensus conference held in Morioka. Ann Surg 2015;261:619-29. [PubMed]