Development and validation of a novel nomogram for predicting systemic inflammatory response syndrome’s occurrence in patients undertaking flexible ureteroscopy

Introduction

With the rapid development of intracorporeal lithotripsy technology and the continuous improvement of supporting equipment, ureteroscopic surgery has increasingly highlighted its minimally invasive and efficient characteristics, becoming one of the main methods for treating upper urinary calculi (1,2).

Uro-sepsis is caused by infection that occurs in the urinary tract and has an incidence between 0.1–4.3% after ureteroscopy, which may be fatal in some cases (3,4). Systemic inflammatory response syndrome (SIRS) is an early hyper-inflammatory phase of sepsis characterized by fever and hyper-metabolism, which can lead to multiple organ dysfunction and septic shock (5,6). While some prediction models or prognostic factors have been reported for SIRS and sepsis after flexible ureteroscopy (fURS), a standard has not been reached (7-10).

This study analyzed the preoperative data of SIRS in patients after fURS to find the prognostic factors of SIRS and achieve the purpose of early detection, early prevention, and early treatment. Our study aimed to identify SIRS patients postoperatively by developing and validating a feasible prognostic model based on a retrospective cohort analysis. We present the following article in accordance with the TRIPOD reporting checklist (available at https://tau.amegroups.com/article/view/10.21037/tau-22-34/rc).

Methods

Source of data and participants

All procedures performed in this study involving human participants were in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Ethics Committee of Dongguan Kanghua Hospital (Dongguan, China) (No. 2021020). Individual consent for this retrospective analysis was waived. A total of 311 patients diagnosed with upper urinary stones who underwent fURS at Dongguan Kanghua Hospital (Dongguan, China) from January 2016 to December 2020 were included and the population was randomly divided into two cohort: a primary cohort (n=219) and validation cohort (n=92), with no missing data in this study. All patients underwent a preoperative CT scan, X-ray scan, blood test, regular urine test, and urine culture, and the variables of each test are shown in Table 1. Stone burden was calculated in mm² according to the formula Σ(0.785 × length(max) × width(max)) (11), and surgery time was recorded during the procedure. Patients who were suffering SIRS or severe infection before surgery were excluded.

fURS procedure

The operations were performed by a qualified and well-experienced urologist, and all surgeries were performed after stenting (Cook Medical, Bloomington, USA), which stayed in the ureter for two weeks (or longer) to dilate the operating tract. All patients were given antibiotics 30 minutes just before surgery, and general anesthesia in the lithotomy position was applied. A Karl Storz Flex-X^c digital flexible ureteroscope (Karl Storz, Tuttlingen, Germany) and 65 w Holmium laser (Miracle Laser, Wuhan, China) were used for lithotripsy. The 12/14 Fr ureteral access sheath (UAS) (Cook Medical, Bloomington, USA) was placed just beneath the pelvic-ureter junction (PUJ) with guide-wire guidance to maintain low pressure during surgery, while the irrigate pressure was set at 50 mL/min. A 4.7 Fr double J stent (Cook Medical, Bloomington, USA) was placed after lithotripsy and removed between two or four weeks later in all cases.

Outcome

The presence of SIRS was identified according to the criteria of the International Sepsis Definitions Conference in 2001 (two or more items as follows) (12): (I) Temperature >38 °C or <36 °C; (II) heart rate >90 beats per minute; (III) respiratory rate >20 breaths per minute or PaCO₂ (arterial carbon dioxide tension) <32 mmHg; (IV) WBC count >12,000 or <4,000 cells/µL.

Statistical analysis

The primary statistical analysis was performed to identify risk factors using STATA software (version 16.0), and the results were compared using the Chi-squared test or Fisher's exact test. Mann-Whitney U test and t-test were used to compare non-parametric and continuous variables, and the subsequent analysis was performed in R software (version 4.1.2), as described in the results section. LASSO regression analysis was used to reduce data dimension and to select significant prognostic factors. The nomogram was built with the results of multivariable logistic regression analysis using the selected factors from LASSO regression analysis, and validation of the model’s performance was quantified with calibration, discrimination, and clinical usefulness.

The accuracy of calibration and discrimination was assessed by Spiegelhalter Z-test, unreliability test, and the receiver operating characteristic (ROC) curve. The nomogram’s clinical usefulness was evaluated by DCA through calculating net benefits at different threshold probabilities. P<0.05 with two-tailed were considered significant statistically.

Results

Population and clinical characteristics

We enrolled 311 patients diagnosed with upper urinary stones treated with fURS into the study, and all procedures were performed successfully. Table 1 shows the clinical demographics of patients and reveals their median age was 47.0 (range, 38 to 56), and there were 101 males and 210 females. There were 28 (9.0%) patients with diabetes and 42 (13.8%) were hypertensive. Most stones (236, 75.9%) were located in the kidney and combined position (kidney and ureter), while 107 (34.4%) patients had stones located in the inferior calyx. Bilateral procedures were performed in 51 cases, and the median surgery time of all cases was 65 (range, 50 to 95) minutes. The total occurrence of SIRS postoperatively was 28 (9.0%). Blood and urine samples were collected and recorded preoperatively, and 37 (11.9%) patients showed a positive urine culture.

The population was randomly divided into a primary cohort (n=219) and validation cohort (n=92), for the purpose of statistical analysis. Table 2 shows the comparison between two cohorts, with the SIRS rate 9.1% [20] and 8.7% [8], respectively. Gender, diabetes, stone burden, neutrophil (N), lymphocyte (L), neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR), urine-WBC, nitrite (Nit), urine culture, and surgery time were significantly correlated with the SIRS group in the primary cohort.

Nomogram development

To select characters, the LASSO regression technique [“glmnet” package (13) in R software] was used in the primary cohort. The LASSO regression analysis selected variables based on non-zero coefficients from the whole of 26 variables collected from patients. The lambda.min was 0.0149 and selected 11 variables, while the lambda.1se was 0.0796 and selected three variables (Figure 1). In this study, the prognostic factors were chosen by the lambda.1se criteria, and the three predictors included NLR, PLR, and urine culture. A multivariable logistic regression model was built with the three factors, and the model was developed as a nomogram as presented in Figure 2 using “rms” package (14) in R software. The risk score of SIRS was calculated by summarizing the scores of all selected factors.

Figure 1 LASSO regression analysis for factors selection. (A) Selection of the tuning parameter (λ) by using 10-fold cross-validation via minimum and 1-SE criteria in the LASSO regression analysis. (B) Non-zero coefficients selection by using the 10-fold cross-validation, while the optimal λ (1-SE criteria, the right line) selected three non-zero coefficients (NLR, PLR, and urine culture). NLR, neutrophil-to-lymphocyte ratio; PLR, platelet-to-lymphocyte ratio.

Figure 2 Presentation of the prognostic nomogram model. The nomogram was constructed with three factors (NLR, PLR, and urine culture) selected by LASSO regression analysis in the primary cohort. NLR, neutrophil-to-lymphocyte ratio; PLR, platelet-to-lymphocyte ratio.

Validation of the nomogram

To validate the performance of the nomogram in predicting the probability of SIRS after fURS, we drew the calibration and discrimination curves by using “rms” (14) and “pROC” (15) packages in R software (Figure 3). The Brier scores were 0.064 and 0.034, with P values of 0.671 and 0.749, respectively, in the primary and validation cohort, indicating good accuracy of calibration. The ROC curves presented the AUC values of 0.897 (95% CI: 0.837–0.957) and 0.976 (95% CI: 0.947–1.000), respectively, revealing excellent discrimination of the nomogram.

Figure 3 Calibration and discrimination curves of the prognostic nomogram model. Calibration curves of the primary cohort (A) and the validation cohort (B). The Y-axis scale represents the actual value probability of SIRS and the X-axis scale represents the predicted value generated using the nomogram model. The dotted grey line represents the nomogram's prediction performance while the solid black line represents an ideal model. Discrimination curve: ROC curves showed AUC values of 0.897 and 0.976, respectively, in the primary cohort (C) and validation cohort (D). ROC, receiver operating characteristic; AUC, area under ROC curve; SIRS, systemic inflammatory response syndrome.

Clinical usefulness assessment

The DCA curve presented in Figure 4 was constructed using the “rmda” package (16) in R software. DCA showed a significant range threshold probability, indicating the nomogram had an excellent net benefit to identify SIRS patients after fURS, and that patients would benefit significantly by using it. Additionally, the predicted probability of SIRS’s occurrence was very close to the actual rate as the risk threshold increased by higher than 60% in clinical impact curve analysis.

Figure 4 The DCA curves (A) and the clinical impact curve (B) of the nomogram model. (A) The Y-axis scale represents net benefit. The red and green lines represent the performance of the nomogram model in the primary and validation cohort, respectively. (B) By applying the nomogram model in predicting probabilities for 1,000 patients, the red line represents the number of high-risk patients divided according to the nomogram model with different probabilities and the dotted blue curve is the actual number of patients with SIRS in different probabilities. DCA, decision curve analysis; SIRS, systemic inflammatory response syndrome.

Discussion

In this study, we found there was no significant difference in WBC count, stone location, stone density, hydronephrosis and surgery time between SIRS and non-SIRS groups. Interestingly, although diabetes, gender, and stone burden were considered significant, the LASSO regression model excluded these characters by the tuning parameters through the cohort analysis. Therefore, it is difficult to predict a SIRS case using regular blood tests or imaging findings. We further calculated blood inflammation markers such as NLR, PLR, and LMR (17), and urine inflammation markers to explore potential correlation with the occurrence of SIRS’s. After LASSO regression analysis was performed, NLR, PLR, and urine culture were selected as predictors for building the nomogram.

Several studies have demonstrated the value of NLR and PLR in the diagnosis, treatment, and prognosis assessment of many diseases, such as malignant tumours, cardiovascular diseases, sepsis, and even coronavirus (COVID-19) disease (18-21). As the body’s inflammation-immune-related markers, NLR and PLR play essential roles in the occurrence and development of inflammation, and in the early stage of sepsis many mature neutrophils are rapidly activated and the number increases rapidly. The lymphocyte count reflects the patient’s immune status, and usually with the progression of inflammation, the patient’s immune system function declines and many T cells are inhibited or even apoptosed. Simultaneously, the release of various anti-inflammatory cytokines into the blood will lead to immunosuppression and the decrease of T lymphocyte proliferation activity. Platelets (PLT) participate in the amplification process of sepsis inflammation, induce neutrophils to secrete many inflammatory factors, and aggravate the inflammatory response (22). A meta-analysis (23) involving 14 studies and comprising 11,564 cases, demonstrated that the initial NLR value might be a promising candidate as a prognostic biomarker of sepsis. Similarly, in this study, the preoperative NLR and PLR levels may reflect the reserve capacity of the inflammation-immune response in patients, which means, as the NLR and PLR increase, the inflammation-immune capacity may decline simultaneously. Thus, these measures can predict the occurrence of postoperative SIRS, but advanced studies are needed to confirm this.

A positive urine culture indicates the presence of pathogenic bacteria in the urine or stones (7). At the same time, lithotripsy is accompanied by the perfusion of saline, bacteria, and endotoxins in the urine or stones may flow into the blood leading to postoperative SIRS and sepsis, resulting in increased surgery times. Therefore, those with positive urine culture before surgery should be carefully monitored, and sensitive antibiotics for anti-infection treatment based on the results of drug sensitivity or clinical experience should be considered (7,9,24). Similarly, our study revealed that urine culture was one of the main predictors for SIRS.

It is more important to prevent sepsis or SIRS than deal with it without preparation. To this end, we developed a prognostic nomogram based on preoperative predictors. The Brier scores were <0.25 (0.064 and 0.034) and P>0.05 (0.671 and 0.749), indicating good calibration accuracy (25,26), and the validation results showed excellent AUC (0.897 and 0.976), respectively, in the primary and validation cohort, representing good discrimination (27). Moreover, the DCA curve was drawn and showed a significant range threshold probability, suggesting excellent clinical usefulness and indicating patients would obtain a considerable net benefit by using the nomogram. Finally, the clinical impact curve showed patients with a risk score which increases higher than 60% will benefit from the nomogram model. Accordingly, we recommend a reconsideration of performing RIRS surgery while the risk score is higher than 60%, or taking adjuvant treatment before or after the surgery, to prevent SIRS.

There are several limitations to this study. Firstly, this was a retrospective analysis of a single medical institution, and selection bias was prone to occur, which is likely to cause some deviations in the results. It needs to be further confirmed by multi-center clinical trials. Secondly, the LASSO method we used in selecting predictors is a data- based algorithm, although it showed excellent performance in this study, we may take some clinical variables in to construct a more reasonable and accurate nomogram based on practice experience while multi-center data added in.

In summary, we established a novel and feasible predictive nomogram with NLR, PLR, and urine culture to predict the occurrence of SIRS after fURS and validated it in internal and external cohorts. Although the nomogram was mainly constructed with preoperative laboratory findings, it showed excellent performance in discriminatory, calibration, and clinical usefulness. We believe that patients will obtain promising benefits from the use of this nomogram, and clinicians should incorporate it into surgery decision making and pretreatment preoperatively. In future research more specific variables during surgery and the outcomes of pretreating these high-risk patients should be observed.

Acknowledgments

Funding: The study was funded by the Dongguan Social Development Science and Technology Project (No. 20211800903162).

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://tau.amegroups.com/article/view/10.21037/tau-22-34/rc

Data Sharing Statement: Available at https://tau.amegroups.com/article/view/10.21037/tau-22-34/dss

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://tau.amegroups.com/article/view/10.21037/tau-22-34/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. All procedures performed in this study involving human participants were in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Ethics Committee of Dongguan Kanghua Hospital (Dongguan, China) (No. 2021020). Individual consent for this retrospective analysis was waived.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Scotland KB, Kroczak T, Pace KT, et al. Stone technology: intracorporeal lithotripters. World J Urol 2017;35:1347-51. [Crossref] [PubMed]
2. Doizi S, Traxer O. Flexible ureteroscopy: technique, tips and tricks. Urolithiasis 2018;46:47-58. [Crossref] [PubMed]
3. De Coninck V, Keller EX, Somani B, et al. Complications of ureteroscopy: a complete overview. World J Urol 2020;38:2147-66. [Crossref] [PubMed]
4. Cindolo L, Castellan P, Scoffone CM, et al. Mortality and flexible ureteroscopy: analysis of six cases. World J Urol 2016;34:305-10. [Crossref] [PubMed]
5. Hotchkiss RS, Monneret G, Payen D. Sepsis-induced immunosuppression: from cellular dysfunctions to immunotherapy. Nat Rev Immunol 2013;13:862-74. [Crossref] [PubMed]
6. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315:801-10. [Crossref] [PubMed]
7. Fan S, Gong B, Hao Z, et al. Risk factors of infectious complications following flexible ureteroscope with a holmium laser: a retrospective study. Int J Clin Exp Med 2015;8:11252-9. [PubMed]
8. Li T, Sun XZ, Lai DH, et al. Fever and systemic inflammatory response syndrome after retrograde intrarenal surgery: Risk factors and predictive model. Kaohsiung J Med Sci 2018;34:400-8. [Crossref] [PubMed]
9. Ozgor F, Sahan M, Cubuk A, et al. Factors affecting infectious complications following flexible ureterorenoscopy. Urolithiasis 2019;47:481-6. [Crossref] [PubMed]
10. Lu J, Xun Y, Yu X, et al. Albumin-globulin ratio: a novel predictor of sepsis after flexible ureteroscopy in patients with solitary proximal ureteral stones. Transl Androl Urol 2020;9:1980-9. [Crossref] [PubMed]
11. Tiselius HG, Andersson A. Stone burden in an average Swedish population of stone formers requiring active stone removal: how can the stone size be estimated in the clinical routine? Eur Urol 2003;43:275-81. [Crossref] [PubMed]
12. Levy MM, Fink MP, Marshall JC, et al. 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Crit Care Med 2003;31:1250-6. [Crossref] [PubMed]
13. Friedman J, Hastie T, Tibshirani R. Regularization Paths for Generalized Linear Models via Coordinate Descent. J Stat Softw 2010;33:1-22. [Crossref] [PubMed]
14. Harrell FE Jr. rms: Regression Modeling Strategies. R package version 6.2-0. 2021. Available online: https://CRAN.R-project.org/package=rms
15. Robin X, Turck N, Hainard A, et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics 2011;12:77. [Crossref] [PubMed]
16. Marshall B. rmda: Risk Model Decision Analysis. R package version 1.6. 2018. Available online: https://CRAN.R-project.org/package=rmda
17. Wang J, Zhang F, Jiang F, et al. Distribution and reference interval establishment of neutral-to-lymphocyte ratio (NLR), lymphocyte-to-monocyte ratio (LMR), and platelet-to-lymphocyte ratio (PLR) in Chinese healthy adults. J Clin Lab Anal 2021;35:e23935. [Crossref] [PubMed]
18. Kumarasamy C, Sabarimurugan S, Madurantakam RM, et al. Prognostic significance of blood inflammatory biomarkers NLR, PLR, and LMR in cancer-A protocol for systematic review and meta-analysis. Medicine (Baltimore) 2019;98:e14834. [Crossref] [PubMed]
19. Russell CD, Parajuli A, Gale HJ, et al. The utility of peripheral blood leucocyte ratios as biomarkers in infectious diseases: A systematic review and meta-analysis. J Infect 2019;78:339-48. [Crossref] [PubMed]
20. Köse N, Yıldırım T, Akın F, et al. Prognostic role of NLR, PLR, and LMR in patients with pulmonary embolism. Bosn J Basic Med Sci 2020;20:248-53. [PubMed]
21. Erdogan A, Can FE, Gönüllü H. Evaluation of the prognostic role of NLR, LMR, PLR, and LCR ratio in COVID-19 patients. J Med Virol 2021;93:5555-9. [Crossref] [PubMed]
22. Cecconi M, Evans L, Levy M, et al. Sepsis and septic shock. Lancet 2018;392:75-87. [Crossref] [PubMed]
23. Huang Z, Fu Z, Huang W, et al. Prognostic value of neutrophil-to-lymphocyte ratio in sepsis: A meta-analysis. Am J Emerg Med 2020;38:641-7. [Crossref] [PubMed]
24. Kumar A, Roberts D, Wood KE, et al. Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med 2006;34:1589-96. [Crossref] [PubMed]
25. Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010;21:128-38. [Crossref] [PubMed]
26. Huang Y, Li W, Macheret F, et al. A tutorial on calibration measurements and calibration models for clinical prediction models. J Am Med Inform Assoc 2020;27:621-33. [Crossref] [PubMed]
27. Royston P, Altman DG. External validation of a Cox prognostic model: principles and methods. BMC Med Res Methodol 2013;13:33. [Crossref] [PubMed]