- Research article
- Open Access
Clinical prediction score for superficial surgical site infection after appendectomy in adults with complicated appendicitis
World Journal of Emergency Surgery volume 13, Article number: 23 (2018)
Superficial surgical site infection (SSI) is common after appendectomy. This study aims to determine a clinical prediction score for SSI after appendectomy in complicated appendicitis.
Data from randomized controlled trial of delayed versus primary wound closures in complicated appendicitis was used. Nineteen patient- and operative-related predictors were selected in the logit model. Clinical prediction score was then constructed using coefficients of significant predictors. Risk stratification was done by receiver operating characteristic (ROC) curve analysis. Bootstrap technique was used to internal validate the score.
Among 607 patients, the SSI incidence was 8.7% (95% CI 6.4, 11.2). Four predictors were significantly associated with SSI, i.e., presence of diabetes, incisional length > 7 cm, fecal contamination, and operative time > 75 min with the odds ratio of 2.6 (95% CI 1.2, 5.9), 2.8 (1.5, 5.4), 3.6 (1.9, 6.8), and 3.4 (1.8, 6.5), respectively. Clinical prediction score ranged from 0 to 4.5 with its discrimination concordance (C) statistic of 0.74 (95% CI 0.66, 0.81). Risk stratification classified patients into very low, low, moderate, and high risk groups for SSI when none, one, two, and more than two risk factors were presented with positive likelihood ratio of 1.00, 1.45, 3.32, and 9.28, respectively. A bootstrap demonstrated well calibration and thus good internal validation.
Diabetes, incisional length, fecal contamination, and operative time could be used to predict SSI with acceptable discrimination. This clinical risk prediction should be useful in prediction of SSI. However, external validation should be performed.
Superficial surgical site infection (SSI) is common after appendectomy especially in complicated appendicitis (i.e., gangrenous and ruptured) with incidence of 9 to 53% [1, 2]. It increases pain, length of stay, and costs . Risk factors associated with SSI are classified into patient-related, operative-related, and microbe-related factors. It can also be considered according to time of operation as preoperative, intraoperative, and postoperative [4,5,6].
Accurate prediction of SSI is helpful for management, set up surveillance protocol, and risk stratification for further clinical trial . Risk prediction of SSI after appendectomy had been developed by the National Healthcare Safety Network . The results demonstrated that emergency procedure, gender, hospital bed size > 500, and contaminated wound classification were significant with concordance statistic of 0.70. However, it is difficult to apply into clinical practice due to lack of risk stratification. Other prediction scores of SSI in other contaminated abdominal surgery were available with acceptable discriminative performances [4, 9], but they included heterogeneous groups of patients (i.e., pancreatic, hepatobiliary, and colorectal surgery), which could not be applied to complicated appendicitis [4, 10, 11]. Therefore, this study was conducted to identify risk factors and created clinical prediction score with risk stratification for SSI in complicated appendicitis.
This study was a part of a randomized controlled trial (RCT) comparing SSI between delayed and primary wound closure in complicated appendicitis . Patients were recruited from two university hospitals (i.e., Ramathibodi and Thammasat University Hospitals) and four regional hospitals (i.e., Lampang Hospital from the North, Chonburi hospital from the Eastern, Surin form the North-eastern, and Pathumthani hospital from the Central). Preoperative inclusion criteria were adult patients, aged ≥ 18 years, who had appendectomy via right lower quadrant abdominal incision. The intraoperative eligible criteria were complicated appendicitis (i.e., gangrenous and rupture). Definition of gangrenous appendicitis was inflamed appendix with necrotic wall (dark, grayish color); ruptured appendicitis was defined as presence of a hole in the appendix, intraoperative rupture, or presence of frank pus. Exclusion criteria were obesity (body mass index (BMI) ≥ 40 kg/m2), autoimmune disease, end-stage renal/liver disease, or human immunodeficiency virus. Preoperative and postoperative intravenous antibiotics covering gram-negative and facultative/anaerobic bacilli had been regularly prescribed in all patients until their body temperature (BT) < 37.8 °C for 24 to 48 h, then switched to oral antibiotics for 7 to 10 days as specified in the RCT protocol . The study was reported according to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement . The original study was approved by the Ethics committee of Ramathibodi Hospital and the collaborating hospitals, in addition to Chonburi hospital for the current study.
Outcome and predictors
SSI was defined following the Center of Disease Control (CDC) criteria as infection within 30 days that involved skin and subcutaneous tissue, with any of the following: purulent drainage, positive culture of organism isolated from fluid or tissue, or any of the following symptoms/signs: pain or tenderness, localized swelling, redness/heat, or the wound was opened by physician without a positive culture . SSI was assessed before discharging home, at 1-week and 1-month follow-up.
Predictors included in the analysis were patient-related and operative-related factors. Patient-related factors were age, gender, BMI, active smoking, American Society of Anesthesiologists’ (ASA) classification, diabetes, hypertension, duration of symptom before admission, presence of fever at admission (BT ≥ 37.8 °C), white blood cell count, and anemia (hemoglobin ≤ 10 g/dL). Operative-related factors included wound length, subcutaneous fat thickness, type of appendicitis (gangrenous or ruptured), wound contamination with exudative fluid, frank pus, fecal contamination (presence of fecolith or wound contamination with fecal material), presence of phlegmon, operative time, and use of closed suction drain.
As for a rule of thumb of simulation study, at least 10–20 of interested events (i.e., SSI) were required per one predictor in the final model to prevent model optimism . Our data contained 48 SSIs ; thus, the final model should include not more than five predictors.
Baseline characteristics of the patients were described. Predictors were compared between SSI and non-SSI using chi-square (or exact test) and Student’s t test (or Mann-Whitney U test) for categorical and continuous data, respectively. Variables with p value less than 0.10 would be included in multivariable analysis. For adjusting purpose, type of wound closure (i.e., delayed primary or primary closure) was included in the multivariable analysis. Forward stepwise logistic regression was applied to identify predictors that significantly associated with SSI (p value < 0.05) and thus should be kept in a final parsimonious model.
The coefficients of significant predictors in the parsimonious model were then used to generate risk scores. Then, area under the curve (or concordance (C) statistics) along with its 95% confidence interval (CI) was estimated to quantify discrimination performance. Calibration of the model was assessed using Hosmer-Lemeshow goodness of fit test. Sensitivity, specificity, and positive likelihood ratio (LR) were also estimated. Cutoff point for risk stratification was calibrated according to the LR performance. Posttest probability was finally estimated according to the stratified risk groups.
An internal validation of the risk prediction score was assessed using a bootstrap with 1000 replications . For calibration, the Somer’D coefficient which is an estimate of the correlation between the observed and predicted values of SSI was estimated for all bootstrap data (called Dboot) and derived (original) data (called Dorg). A bias was then estimated by subtracting the Dorg with the mean Dboot, lower value reflected less bias and thus better calibration. In addition, the original C statistic was compared to an average C statistic from the bootstraps for discrimination performance. All analysis was performed using STATA version 15.0.
A total of 607 patients were enrolled from November 2012 to February 2016. Mean age was 45 (standard deviation (SD) 18) years, and 53% of patients were male. Four hundred and nine (76%) had ruptured whereas 148 (24%) had gangrenous appendicitis. A total of 52 patients had SSIs with incidence of 8.7% (95% CI 6.4, 11.0). One patient developed organ space SSI with auto-drainage through incision and was successfully treated non-operatively. Some variables were missing leaving 543 cases with 48 SSIs for the main analysis.
Model development and validation
Univariable analysis was performed to assess association between each risk factor and SSI, see Table 1. For ease of application, continuous variables were categorized into two groups based on 75 percentiles except operative time that was categorized according to National Nosocomial Infections Surveillance (NNIS) . The cut points were 25.7 kg/m2, 48 h, 7 cm, 4.5 cm, and 75 min for BMI, duration symptoms, incisional length, subcutaneous fat thickness, and operative time, respectively. Among 19 predictors, eight had p < 0.10 including BMI, diabetes, duration of symptoms, appendicitis classification, incisional length, subcutaneous fat thickness, fecal contamination, and length of operative time. Stepwise logistic regression identified four predictors in the parsimonious model, i.e., presence of diabetes, incisional length, fecal contamination, and operative time > 75 min, see Table 2. The risk prediction equation was written as
The C statistics of this model was 0.74 (95% CI 0.66, 0.81) indicating acceptable discrimination of SSI from non-SSI. Hosmer-Lemeshow goodness of fit test was applied and demonstrated good calibration of the model (chi-square = 4.42, p value = 0.491), see Additional File 1: Table S1. Coefficients of significant predictors were used to calculate prediction score. Diabetes and incision length were weighted 1; operative time and fecal contamination were weighted 1.2 and 1.3, respectively. Adding them up resulted in risk score ranged from 0 to 4.5, see Table 2. Cutoff points were identified based on positive LR of each distinct score from ROC curve analysis. The score was classified as 0, 1–1.3, 2–2.5, and > 2.5 for very low, low, moderate, and high risk groups, respectively. Sensitivity, specificity, positive LR, and posttest probabilities were demonstrated in Table 3. Fagan’s nomogram was constructed, and posttest probabilities were calculated based on incidence of SSI in this study which was 8.7% (95% CI 6.4, 11.0), see Fig. 1 . Patients had probabilities of SSI of 12.2 (95% CI 9.0, 15.5), 24.0 (95% CI 18.5, 29.5), and 46.9 (95% CI 38.8, 53.9) for low, moderate, and high risk groups, respectively.
A bootstrap with 1000 replications was performed to assess internal validation, which yielded estimated Dorg and Dboot coefficients of 0.476 and 0.500 (95% CI 0.496, 0.506) for the derivative and bootstrap models, respectively. The average bias was only − 0.025 (95% CI − 0.030, − 0.019), suggesting good calibration. The bootstrap C statistics was 0.750 (95% CI 0.748, 0.753), with a bias of − 0.012 (95% CI − 0.015, − 0.010).
Our study demonstrated that presence of diabetes, incisional length > 7 cm, fecal contamination, and operative time > 75 min were significantly associated with superficial SSI after open appendectomy via right lower quadrant abdominal incision in complicated appendicitis. A risk prediction score was then constructed with good performances, i.e., well calibrated and good discrimination with C statistic of 0.74 (95% CI 0.66, 0.81). The score was classified into four groups: very low, low, moderate, and high risk with positive LR of 1.453, 3.315, and 9.281 for the latter three groups, respectively.
All predictors included in the model were consistently demonstrated to be associated with SSI in other studies (i.e., diabetes [17, 18], incisional length , fecal contamination [18, 19], and operative time [9, 20]). Both preoperative and intraoperative factors play roles in SSI. Higher BMI [21, 22] and subcutaneous fat thickness  had been shown to be associated with SSI in other studies but they were not and thus excluded from the model during a selection process in this study.
About 11.8% of total patients whose data were missing were dropped out during development of risk prediction model. Characteristics of patients between complete and incomplete data were explored, distributions of BMI, diabetes, hypertension, and operative time were quite different, but only BMI was significant, see Additional file 2: Table S2. Multiple imputation by chain equations with 10 imputations was attempted to fill in missing data. The final risk prediction model was constructed, and their coefficients were not much different comparing with using only complete data, see Additional file 3: Table S3. Bias from using only complete data should be less likely.
Laparoscopic appendectomy has been associated with less incidence of SSI than open surgery and has replaced open appendectomy in some countries . However, it required much experienced with high cost. Therefore, it is still less performed in developing country than developed country, and open appendectomy is still the standard of care .
Applying our model is simplified by counting number of four risk factors (i.e., diabetes, operative time > 75 min, fecal contamination, and incision length > 7 cm). According to the risk stratification scores, patient with 0, 1, 2, and > 2 risk factors will be respectively classified into very low, low, moderate, and high risk groups with positive LR for low, moderate, and high scores of 1.4, 3.3, and 9.2, in which positive LR of 5 to 10 could moderately shift pre- to posttest probability as for EBM working group recommendation . To calculate posttest probabilities of SSI, Fagan’s nomogram could be used or to estimate posttest probability of SSI occurrence according to risk prediction score . For instance, if patient has history of diabetes, operative time > 75 mins, and fecal contamination, she/he will be classified as high risk group (three factors); posttest probability of SSI for this patient is about 46%. If patient does not have history of diabetes, but has fecal contamination and incision length > 7 cm, this patient will be classified as moderate risk and thus her/his posttest probability is about 20–30%.
The results from previous randomized controlled trial  demonstrated that delayed primary wound closure had no benefit over primary wound closure in normal risk patients and primary wound closure should be done with risk of SSI of 8.7% (95% CI 6.4, 11.2). We further suggested closing the wound primarily in the low risk group with risk of SSI of 12.3 (95% CI 9.0, 15.5). However, in moderate and high risk groups, primarily closing the wound should be done with caution. Other wound interventions such as wound edge protectors , subcutaneous wound drainage , or daily wound probing  should be considered to apply.
Early detection and treatment of SSI is also important to reduce impact of SSI to a patient, which includes pain, isolation, insecurity, and costs . Knowing risk of SSI can help physician educate, inform, and arrange appropriate infection surveillance protocol. Patients should be advised about symptoms and signs of SSI before discharging home. Patient information sheets or self-assessment questionnaires may be useful .
The model was derived with standard methodological analysis including 4 markedly significant variables. Risk stratification was also easy to apply by just counting occurring variables at the end of operation. The internal validation also demonstrated good discrimination and calibration. The final prediction model contained 4 variables that should not result in model optimism with 48 events of SSI as for recommendation by TRIPOD . An internal validation was performed using a bootstrapping technique, which was more appropriated than splitting data when a number of event of interest was small . However, external validation and impact of applying the model to outcomes should be done in the future .
The risk prediction score of SSI has been developed containing diabetes, operative time, fecal contamination, and incisional length. The score internally performed well with good calibration and discrimination. This can be ease of use in clinical practice. However, the score should also be further externally validated.
American Society of Anesthesiologists
Body mass index
Center of Disease Control
National Nosocomial Infections Surveillance
Receiver operating characteristic
Superficial surgical site infection
Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis
Siribumrungwong B, Srikuea K, Thakkinstian A. Comparison of superficial surgical site infection between delayed primary and primary wound closures in ruptured appendicitis. Asian J Surg. 2014;37:120–4.
Siribumrungwong B, Noorit P, Wilasrusmee C, Thakkinstian A. A systematic review and meta-analysis of randomised controlled trials of delayed primary wound closure in contaminated abdominal wounds. World J Emerg Surg. 2014;9:49.
Sullivan E, Gupta A, Cook CH. Cost and consequences of surgical site infections: a call to arms. Surg Infect. 2017;18:451–4.
Ejaz A, Schmidt C, Johnston FM, Frank SM, Pawlik TM. Risk factors and prediction model for inpatient surgical site infection after major abdominal surgery. J Surg Res. 2017;217:153–9.
Pedroso-Fernandez Y, Aguirre-Jaime A, Ramos MJ, Hernandez M, Cuervo M, Bravo A, Carrillo A. Prediction of surgical site infection after colorectal surgery. Am J Infect Control. 2016;44:450–4.
Alavi K, Sturrock PR, Sweeney WB, Maykel JA, Cervera-Servin JA, Tseng J, Cook EF. A simple risk score for predicting surgical site infections in inflammatory bowel disease. Dis Colon rectum. 2010;53:1480–6.
Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): the TRIPOD Statement. Br J Surg. 2015;102:148–58.
Mu Y, Edwards JR, Horan TC, Berrios-Torres SI, Fridkin SK. Improving risk-adjusted measures of surgical site infection for the national healthcare safety network. Infect Control Hosp Epidemiol. 2011;32:970–86.
de Oliveira AC, Ciosak SI, Ferraz EM, Grinbaum RS. Surgical site infection in patients submitted to digestive surgery: risk prediction and the NNIS risk index. Am J Infect Control. 2006;34:201–7.
Neumayer L, Hosokawa P, Itani K, El-Tamer M, Henderson WG, Khuri SF. Multivariable predictors of postoperative surgical site infection after general and vascular surgery: results from the patient safety in surgery study. J Am Coll Surg. 2007;204:1178–87.
van Walraven C, Musselman R. The Surgical Site Infection Risk Score (SSIRS): a model to predict the risk of surgical site infections. PLoS One. 2013;8:e67167.
Siribumrungwong B, Chantip A, Noorit P, Wilasrusmee C, Ungpinitpong W, Chotiya P, Leerapan B, Woratanarat P, McEvoy M, Attia J, Thakkinstian A. Comparison of superficial surgical site infection between delayed primary versus primary wound closure in complicated appendicitis: a randomized controlled trial. Ann Surg. 2018; 267(4):631–37.
Horan TC, Andrus M, Dudeck MA. CDC/NHSN surveillance definition of health care-associated infection and criteria for specific types of infections in the acute care setting. Am J Infect Control. 2008;36:309–32.
Courvoisier DS, Combescure C, Agoritsas T, Gayet-Ageron A, Perneger TV. Performance of logistic regression modeling: beyond the number of events per variable, the role of data structure. J Clin Epidemiol. 2011;64:993–1000.
Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–87.
System NNIS. National Nosocomial Infections Surveillance (NNIS) System Report, data summary from January 1992 through June 2004, issued October 2004. Am J Infect Control. 2004;32:470–85.
Martin ET, Kaye KS, Knott C, Nguyen H, Santarossa M, Evans R, Bertran E, Jaber L. Diabetes and risk of surgical site infection: a systematic review and meta-analysis. Infect Control Hosp Epidemiol. 2016;37:88–99.
Reichman DE, Greenberg JA. Reducing surgical site infections: a review. Rev Obstet Gynecol. 2009;2:212–21.
Itatsu K, Sugawara G, Kaneoka Y, Kato T, Takeuchi E, Kanai M, Hasegawa H, Arai T, Yokoyama Y, Nagino M. Risk factors for incisional surgical site infections in elective surgery for colorectal cancer: focus on intraoperative meticulous wound management. Surg Today. 2014;44:1242–52.
Imai E, Ueda M, Kanao K, Miyaki K, Kubota T, Kitajima M. Surgical site infection surveillance after open gastrectomy and risk factors for surgical site infection. J Infect Chemother. 2005;11:141–5.
Waisbren E, Rosen H, Bader AM, Lipsitz SR, Rogers SO Jr, Eriksson E. Percent body fat and prediction of surgical site infection. J Am Coll Surg. 2010;210:381–9.
Tserenpuntsag B, Haley V, Van Antwerpen C, Doughty D, Gase KA, Hazamy PA, Tsivitis M. Surgical site infection risk factors identified for patients undergoing colon procedures, New York State 2009–2010. Infect Control Hosp Epidemiol. 2014;35:1006–12.
Fujii T, Tsutsumi S, Matsumoto A, Fukasawa T, Tabe Y, Yajima R, Asao T, Kuwano H. Thickness of subcutaneous fat as a strong risk factor for wound infections in elective colorectal surgery: impact of prediction using preoperative CT. Dig Surg. 2010;27:331–5.
Jaschinski T, Mosch C, Eikermann M, Neugebauer EA. Laparoscopic versus open appendectomy in patients with suspected appendicitis: a systematic review of meta-analyses of randomised controlled trials. BMC Gastroenterol. 2015;15:48.
Camilleri-Brennan J, Drake T, Spence R, Bhangu A, Harrison E. Management and outcomes from appendectomy: an international, prospective, multicentre study. S Afr J Surg. 2017;55:86–7.
Jaeschke RGG, Lijmer J. Diagnostic tests. In: Users’ guides to the medical literature: a manual for evidence-based clinical practice. Chicago: United States of America; 2008.
Safari S, Baratloo A, Elfil M, Negida A. Evidence based emergency medicine; part 4: pre-test and post-test probabilities and Fagan’s nomogram. Emerg (Tehran). 2016;4:48–51.
Sajid MS, Rathore MA, Sains P, Singh KK. A systematic review of clinical effectiveness of wound edge protector devices in reducing surgical site infections in patients undergoing abdominal surgery. Updat Surg. 2017;69:21–8.
Manzoor B, Heywood N, Sharma A. Review of subcutaneous wound drainage in reducing surgical site infections after laparotomy. Surg Res Pract. 2015;2015:715803.
Towfigh S, Clarke T, Yacoub W, Pooli AH, Mason RJ, Katkhouda N, Berne TV. Significant reduction of wound infections with daily probing of contaminated wounds: a prospective randomized clinical trial. Arch Surg. 2011;146:448–52.
Andersson AE, Bergh I, Karlsson J, Nilsson K. Patients’ experiences of acquiring a deep surgical site infection: an interview study. Am J Infect Control. 2010;38:711–7.
Pham JC, Ashton MJ, Kimata C, Lin DM, Nakamoto BK. Surgical site infection: comparing surgeon versus patient self-report. J Surg Res. 2016;202:95–102.
Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, Riley RD, Hemingway H, Altman DG. Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med. 2013;10:e1001381.
This study was granted by the Health Systems Research Institute, Thailand (grant number 59–062). The funding was for conducting a randomized controlled trial of comparing SSI between delayed and primary wound closure.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
The randomized controlled trial was registered with ClinicalTrials.gov (ID NCT01659983). The study was approved by all collaborating hospitals with the committee reference number of MTU-EC-SC-4-057/55. Every patient was informed and signed informed consent before participating into the trial. For this retrospective study, the protocol was approved by the ethic committees of Chonburi and Ramathibodi hospitals with protocol numbers 03/61/S/h3 and 04-55-18, respectively.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Table S1. Hosmer-Lemeshow goodness of fit. Table describing details of Hosmer-Lemeshow goodness of fit. (DOCX 17 kb)
Table S2. Describe distribution of variables between complete and incomplete data. Baseline characteristics comparison between cases with complete and incomplete data. (DOCX 21 kb)
Table S3. Risk factors of superficial surgical site infection based on imputed data. Significant variables with their estimated coefficiencies in the final parsimonious model estimated from imputed data. (DOCX 16 kb)
About this article
Cite this article
Noorit, P., Siribumrungwong, B. & Thakkinstian, A. Clinical prediction score for superficial surgical site infection after appendectomy in adults with complicated appendicitis. World J Emerg Surg 13, 23 (2018). https://doi.org/10.1186/s13017-018-0186-1
- Surgical site infection
- Wound infection
- Risk factors
- Prediction scores