- Research article
- Open Access
Performance of new adjustments to the TRISS equation model in developed and developing countries
World Journal of Emergency Surgery volume 12, Article number: 17 (2017)
The Trauma and Injury Severity Score (TRISS) has been criticized for being based on data from the USA and Canada—high-income countries—and therefore, it may not be applicable to low-income and middle-income countries. The present study evaluated the accuracy of three adjustments to the TRISS equation model (NTRISS-like; TRISS SpO2; NTRISS-like SpO2) in a high-income and a middle-income country to compare their performance when derived and applied to different groups.
This was a retrospective study of trauma patients admitted to two institutions: a university medical center in São Paulo, Brazil (a middle-income country), and a level 1 university trauma center in San Diego, USA (a high-income country). Patients were admitted between January 1, 2006, and December 31, 2010. The subjects were 2416 patients from Brazil and 8172 patients from the USA. All equations had adjusted coefficients for São Paulo and San Diego and for blunt and penetrating trauma. Receiver operating characteristic (ROC) curves were used to evaluate performance of the models.
Regardless of the population where the equation was generated, it performed better when applied to patients in the USA (AUC from 0.911 to 0.982) compared to patients in Brazil (AUC from 0.840 to 0.852). When the severity was considered and homogenized, the performance of equations were similar to both application in the USA and Brazil.
Survival probability models showed better performance when applied in data collected in the high-income countries (HIC) regardless the country they were derived. The severity is an important factor to consider when using non-adjusted survival probability models for the local population. Adjusted models for severely traumatized patients better predict survival probability in less severely traumatized populations. Other factors besides physiological and anatomical data may impact final outcomes and should be identified in each environment if they are to be used in the development of the trauma care performance improvement process in middle-income countries.
The Trauma and Injury Severity Score (TRISS) methodology is widely accepted in assessing quality of care and promoting improvements in trauma care despite criticism for being derived from a database on trauma victims in the USA and Canada, presenting results and regression coefficients that relate to these countries’ realities . USA and Canada present mature trauma systems, and their results can serve as targets to be achieved by other countries. However, those systems do not reflect the difficulties in providing care faced in other countries due to specific geographic, economic, and sociodemographic characteristics, standards for health care systems, and local morbidity and mortality rates .
Trauma quality improvement programs are a fundamental part of trauma care systems in developed countries and have been used in some low-income and middle-income countries with good results . These programs seek to provide care to trauma patients in a planned sequence, assess compliance with established standards, and reduce variability of care in order to reduce costs while maintaining quality of care . The TRISS and its derivations are an important tool in various techniques used to monitor quality of care for trauma patients, so the better its accuracy in estimating the probability of survival, the better its performance in trauma care quality improvement programs.
The adjustment of the survival probability equation coefficients according to the local population has been indicated despite the results of studies that show similar performance. Equations based on data from high-income countries do not seem to fit for use in low-income and middle-income countries [5–11].
The present study aimed to adjust the coefficients of three variations of the TRISS—NTRISS-like, TRISS SpO2, and NTRISS-like SpO2—for patients in the institutions in São Paulo (Brazil) and San Diego (USA) and compare the discriminatory ability of these equations when applied to different groups of trauma patients. These models have been evaluated in other study in which they have shown good accuracy (about 89.5%) and performance similar to other, previously published adjustments of the TRISS [12–21]. In these three variations, the Revised Trauma Score (RTS) was replaced in order to obtain the most adequate variables, according to their availability and which reflect the patient’s physiological condition.
The objective of the present study was to evaluate the accuracy of three variations of the TRISS in two different contexts: a level 1 university trauma center in San Diego County in the state of California, USA (a high-income country), and a university medical center located in the city of São Paulo, Brazil (a middle-income country).
This was a retrospective study of trauma patients admitted to two centers: the Clinical Hospital of the College of Medicine of the University of São Paulo (HCFMUSP), a university medical center in São Paulo, Brazil (a middle-income country), and the Medical Center of the University of California at San Diego (UCSDMC), a level 1 university-based trauma center in the USA (a high-income country). The patients were admitted between January 1, 2006, and December 31, 2010. Victims of blunt or penetrating trauma aged 14 years or older were included in the study. Patients who were admitted to HCFMUSP and UCSDMC after 24 h of the traumatic event or transferred from other hospitals were excluded from the study. The traumatic events considered are listed in Chapter XX of the World Health Organization’s 10th International Classification of Diseases , excluding cases of hanging, suffocation, drowning or near drowning, poisoning, burns, and electrocution.
The data source for collecting information on patients in HCFMUSP was hospital records selected from a list containing the names and hospital record numbers of all patients hospitalized due to trauma. Patients who met the inclusion criteria were identified through the Hospital Information and Management System. Of the 3576 patients identified, 2416 were included in the study, making it a non-random sample composed of 67.6% of all cases.
At UCSDMC, the trauma patients admitted during the study period were identified in the institution’s database, which already contained all the necessary information for the research. In total, 8172 patients met the inclusion criteria, and access to information was obtained for all of them.
Thus, two matrix databases were generated: one for HCFMUSP, with 2416 patients, and one for UCSDMC, with 8172 patients. Three hundred patients were randomly selected from each of these databases, each record having all the information required to calculate the probability of survival rates, and these databases were designated the test databases. The information from other patients was grouped into derivation databases, forming two other databases: one for HCFMUSP, with 2116 patients, and one for UCSDMC, composed of 7872 patients.
Considering the disproportion between the number of patients in the HCFMUSP and the UCSDMC derivation database (2116 versus 7872), weight of 3.72 was given to each of the HCFMUSP patient in order that both database had the same weight in the coefficient derivation.
The test databases were used to evaluate the accuracy of the models, while the derivation databases were used to adjust the coefficients to the study population. The models applied were as follows: NTRISS-like (best motor response (BMR), systolic blood pressure (SBP), New Injury Severity Score (NISS), and age); TRISS SpO2 (Glasgow Coma Scale (GCS), SBP, peripheral oxygen saturation (SpO2), Injury Severity Score (ISS), and age); and NTRISS-like SpO2 (BMR, SBP, SpO2, NISS, and age). In the TRISS SpO2 and NTRISS-like SpO2, respiration rate was replaced with SpO2, considering that respiration rate is often not available in the primary assessment scenario, and observed changes in it cannot be directly related to respiratory function because of pain and psychological stress . Furthermore, obtaining respiratory rate consumes time at a stage in treatment in which there is an urgent need for other approaches. In the NTRISS-like and NTRISS-like SpO2, the GCS was replaced by the item BMR on the scale, to enable the inclusion of intubated patients in analysis. Moreover, the NISS, which corrects the limitations of the ISS, was the anatomical index included in these variations of the TRISS [19, 24].
The demographic and clinical variables of patients were submitted to descriptive analysis, comparing the HCFMUSP and UCSDMC groups. The Pearson’s chi-square test was used for categorical variables, and ANOVA was used for continuous variables. The coefficients generated for all equations analyzed in this study were derived by means of logistic regression analysis. A 5% significance level was used for all tests.
Four equations were derived for each of the TRISS variations: two from the HCFMUSP and two from UCSDMC derivation database (one for blunt and other for penetrating trauma). These equations were validated in the HCFMUSP and UCSDMC test databases. The ROC curve and predictive ability of the models in each of the applications were evaluated. ROC curve performance was compared by DeLong’s Algorithm using pROC e clinfun programs [25–27].
The patient groups from HCFMUSP and UCSDMC were compared in relation to the characteristics presented in Table 1. The results showed statistically significant differences between the two groups in terms of all variables analyzed (p < 0.001).
Coefficients were derived for each of the TRISS variations, as shown in Table 2. All of the equations generated were validated in the HCFMUSP and UCSDMC test databases, and their performance was compared, as shown in Fig. 1 and Table 3.
Figure 1 shows that there was no statistically significant difference (p > 0.05%) between the three models in each of the applications; therefore, the variations of TRISS had equivalent performance when applied to the same sample. Together with the information from Table 3, Fig. 1 shows that the performance of the equations was different when applied to patients in the origin institution and patients of the other institution: the equations derived from the HCFMUSP patients applied on the test database of the same hospital had an AUC of approximately 0.85 and in the San Diego group had an AUC between 0.911 and 0.982. Moreover, the application of equations derived in San Diego resulted in an AUC of about 0.95 when applied to the group of patients from the same institution and less than 0.85 on the group from HCFMUSP. The AUC was higher when the equations were applied at UCSDMC.
This may be because UCSDMC data included more low severity patients, which are easy population to predict outcome, than HCFMUSP. Patient characteristics are obviously different, so the predictive performance could be different. To exclude this possible bias, the models were derived and applied in patients with the same severity according to the ISS (< or ≥16). Table 4 shows that the models had similar AUC when applied to HCFMUSP and UCSDMC.
The equations derived with data from HCFMUSP and that derived with data from UCSDMC showed greater accuracy when applied to the San Diego population than to patients from São Paulo. However, deriving and applying the models (HCFMUSP and UCSDMC) in these two populations classified by severity, equations presented similar accuracy both in São Paulo and in San Diego.
The application of the equations to patients from HCFMUSP with coefficients adjusted to this population did not increase the accuracy of the models (AUC between 0.848 and 0.852), since the equations adjusted to patients from UCSDMC had similar performance in this hospital (AUC between 0.840 and 0.849). This result raises uncertainties about the importance of adjustments of survival probability rates to the local realities where they are applied. This uncertainty is also reinforced by the results of other studies that have shown equivalent or worse performance of the TRISS after these coefficient adjustments [1, 6, 28–32].
The accuracy of the models was similar to the same test population, and all the equations presented better performance when validated for patients from UCSDMC, regardless of the group in which they were derived. Considerations should be made about these observations.
The characteristics of the population in which the equations were validated have contributed to this result, since the performance was better in the UCSDMC group. The subjects in the groups differed significantly in all statistical comparisons. Most of the patients from the two institutions received prehospital care. In HCFMUSP, there was a predominance of basic life support, while in UCSDMC, there was a predominance of advanced life support (ALS). Considering the severity of HCFMUSP, it is desirable that they receive ALS prehospital care. The length of hospital stay, frequency of admission to ICU, and surgical procedures were significantly higher among those admitted to HCFMUSP. As regards severity, the São Paulo patients were more severely traumatized than those in San Diego. Mortality in HCFMUSP was significantly greater than that in UCSDMC.
UCSDMC data included more low severity patients, which are easy population to predict outcome, than HCFMUSP. As the characteristics are obviously different, so the predictive performance could be different. When the accuracy was evaluated considering the trauma severity by ISS, the results showed similar AUC for both HCFMUSP and UCSDMC equations, regardless of the group they were applied.
Evaluating the AUC of the applied equations in the HCFMUSP population, five in six presented better performance when the equations were derived and applied in patients with the same severity (0.862–0.926) than in general population (0.840–0.852). These findings were not found in the population of San Diego, who presented smaller AUC when analyzed by equations derived and applied in similar patients for severity (0.849–0.936 versus 0.911–0.982).
In addition to severity, other factors should be considered. The two patient groups were inserted into different social, political, and economic realities. Additionally, the group that was more severely traumatized was cared for in a hospital that is a reference center for trauma care in an emergency care network that is in an organizational phase (HCFMUSP). The other group of patients, who were less severely traumatized, was cared for in a level I trauma care center (UCSDMC), inserted into a consolidated system of care for trauma victims with established prevention programs.
The UCSDMC is part of a consolidated and well-established trauma system and has a low mortality rate. A well-structured care system allows greater uniformity and predictability of care provided to victims of trauma, with less chance of non-adherence to care protocols . In addition, UCSDMC works with an activation protocol for the trauma team, which prepares to receive patients prior to arrival at the hospital. Better results have been shown for individuals who are treated in accordance with this protocol , which has certain advantages: it enables better communication between the emergency and hospital care teams; it allows prior availability of resources that patients might need; it involves teams in order to evaluate patients as a whole and properly prioritize investigations; and it creates a suitable setting for trauma education and research .
In HCFMUSP, trauma patients receive care in the emergency room and are treated by the medical staff of emergency general surgery, who are responsible for treating all surgical emergencies admitted to this sector. Other health care professionals, including nurses, do not have specific training for trauma. The mortality rate in this institution was 17.5%, but mortality between 1 and 38% was found in the literature .
Studies have linked improvements in the results of care for trauma patients with regionalized trauma systems with trauma center designation [13, 16]. Well-structured trauma care systems enable better distribution of resources, respecting the “golden hour” and offering all severely traumatized patients’ complexity of treatment according to need. Patients with similar needs receive treatment according to specific protocols based on those needs.
There is an increasingly critical need for the creation of a Brazilian system of trauma care, not only to improve trauma records but also to strengthen quality improvement programs, enhance the results of trauma care, decrease mortality rates, and allow the return of these individuals to society with quality of life. The training of professionals deserves special attention in this process. Some trauma training programs of the American College of Surgeons Committee on Trauma, Society of Trauma Nurses, and National Association of Emergency Medical Technicians are available nationally.
Access to the highest level of care, injury prevention strategies, system-wide quality assurance, population-based surveillance of injury-related problems, disaster preparedness programs, financial viability, and integration with the existing health system are critical elements of an integrated system and should be addressed to an effective trauma system in Brazil.
This study presented some limitations. The sample of patients from HCFMUSP was not random, because access to the records was according to their availability. However, according to information from persons responsible for the division of medical records, the search for the records requested was casual. We used the 2005 version 2008 update for coding the injuries of the patients from HCFMUSP and the 1998 version for patients from UCSDMC. This difference occurred because in the data from HCFMUSP, the injuries were coded recently and the current AIS version was used. In San Diego, the AIS codes were established at the same time as the inclusion of records in the database, and the records were coded according to the 1998 version. In one study in Australia comparing the use of these two versions in patients with ISS >15, the ISS presented better performance when calculated with the AIS 2008; however, there was no difference in the performance of the NISS, TRISS, and NTRISS with the use of the 1998 and 2008 versions of AIS .
Survival probability models showed better performance when applied in data collected in the HIC, regardless the country they were derived—high- or middle-income country. The severity is an important factor to consider when using non-adjusted survival probability models for the local population—adjusted models for severely traumatized patients better predict survival probability in less severely traumatized populations, as adjusted survival probability models for the local population. More severely traumatized populations benefit when survival probability models were adjusted according to the trauma patients’ severity.
Kroezen F, Bijlsma TS, Liem MSL, Meeuwis JD, Leenen LPH. Base deficit-predictive modeling of outcome in trauma patients admitted to intensive care units in Dutch trauma centers. J Trauma. 2007;63:908–13.
Corfield AR, MacKay DF, Pell JP. Association between trauma and socioeconomic deprivation: a registry-based, Scotland-wide retrospective cohort study of 9,238 patients. Scand J Trauma Resusc Emerg Med. 2016;24:90.
World Health Organization; International Association for Trauma Surgery and Intensive Care. Guidelines for trauma quality improvement programmes. Geneva: World Health Organization; 2009.
Hoyt DB, Coimbra R. Trauma systems. Surg Clin North Am. 2007;87:21–35.
Domingues CA, Nogueira LS, Sttervall CHC, Sousa RMC. Performance of Trauma and Injury Severity Score (TRISS) adjustments: an integrative review. Rev Esc Enferm USP. 2015;49:138–46.
Schluter PJ, Cameron CM, Davey TM, Civil I, Orchard J, Dansey R, et al. Contemporary New Zealand coefficients for the Trauma Injury Severity Score: TRISS(NZ). N Z Med J. 2009;122:54–64.
Bouamra O, Wrotchford A, Hollis S, Vail A, Woodford M, Lecky F. Outcome prediction in trauma. Injury. 2006;37:1092–7.
Kilgo PD, Meredith JW, Osler TM. Incorporating recent advances to make the TRISS approach universally available. J Trauma. 2006;60:1002–8.
Hannan EL, Farrell LS, Cayten CG. Predicting survival of victims of motor vehicle crashes in New York state. Injury. 1997;28:607–15.
Lane PL, Doig G, Mikrogianakis A, Charyk ST, Stefanits T. An evaluation of Ontario trauma outcomes and the development of regional norms for Trauma and Injury Severity Score (TRISS) analysis. J Trauma. 1996;41:731–4.
Millham FH, LaMorte WW. Factors associated with mortality in trauma: re-evaluation of the TRISS method using the National Trauma Data Bank. J Trauma. 2004;56:1090–6.
Domingues CA. Trauma and Injury Severity Score: análise de novos ajustes no índice [Internet]. São Paulo: Escola de Enfermagem, Universidade de São Paulo; 2013 [citado 2015 mar. 10]. Avaiable in: http://www.teses.usp.br/teses/disponiveis/7/7139/tde-14102013-090011/pt-br.php
Stewart TC, Lane PL, Stefanits T. An evaluation of patient outcomes before and after trauma center designation using Trauma and Injury Severity Score analysis. J Trauma. 1995;39:1036–40.
Offner PJ, Jurkovich GJ, Gurney J, Rivara FP. Revision of TRISS for intubated patients. J Trauma. 1992;32:32–5.
Garber BG, Hebert PC, Wells G, Yelle JD. Differential performance of TRISS-like in early and late blunt trauma deaths. J Trauma. 1997;43:1–5.
Petrie D, Lane P, Stewart TC. An evaluation of patient outcomes comparing trauma team activated versus trauma team not activated using TRISS analysis. J Trauma. 1996;41:870–5.
Dahl B, Schiodt FV, Nielsen M, Kiaer T, Williams JG, Ott P. Admission level of Gc-globulin predicts outcome after multiple trauma. Injury. 1999;30:275–81.
Lane PL, Doig G, Stewart TC, Mikrogianakis A, Stefanits T. Trauma outcome analysis and the development of regional norms. Accid Anal Prev. 1997;29:53–6.
Domingues CA, Sousa RMC, Nogueira LS, Poggetti RS, Fontes B, Muñoz D. The role of the New Trauma and Injury Severity Score (NTRISS) for survival prediction. Rev Esc Enferm USP. 2011;45:1353–8.
Moini M, Rezaishiraz H, Zafarghandi MR. Characteristics and outcome of injured patients treated in urban trauma centers in Iran. J Trauma. 2000;48:503–7.
Aydin SA, Bulut M, Ozgüç H, Ercan I, Türkmen N, Eren B, et al. Should the New Injury Severity Score replace the Injury Severity Score in the Trauma and Injury Severity Score? Ulus Travma Acil Cerrahi Derg. 2008;14:308–12.
World Health Organization. International statistical classification of diseases and related health problems 10th revision. 2010.
Whitaker IY, Gennari TD, Whitaker AL. The difference between ISS and NISS in a series of trauma patients in Brazil. Annu Proc Assoc Adv Automot Med. 2003;47:301–9.
Tay SY, Sloan EP, Zun L, Zaret P. Comparison of the New Injury Severity Score and the Injury Severity Score. J Trauma. 2004;56:162–4.
van Buuren S, Groothuis-Oudshoorn K. Mice: multivariate imputation by chained equations in R. Online J Stat Softw. 2011;45(3):1–67. http://www.jstatsoft.org/v45/i03, [cited 2012 Nov 20].
DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–45.
Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. 2011;12:77.
Bergeron E, Rossignol M, Osler T, Clas D, Lavoie A. Improving the TRISS methodology by restructuring age categories and adding comorbidities. J Trauma. 2004;56:760–7.
Hannan EL, Mendeloff J, Farrell LS, Cayten CG, Murphy JG. Validation of TRISS and ASCOT using a non-MTOS trauma registry. J Trauma. 1995;38:83–8.
Voskresensky IV, Rivera-Tyler T, Dossett LA, Riordan Jr WP, Cotton BA. Use of scene vital signs improves TRISS predicted survival in intubated trauma patients. J Surg Res. 2009;154:105–11.
Osler TM, Rogers FB, Badger GJ, Healey M, Vane DW, Shackford SR. A simple mathematical modification of TRISS markedly improves calibration. J Trauma. 2002;53:630–4.
Schall LC, Potoka DA, Ford HR. A new method for estimating probability of survival in pediatric patients using revised TRISS methodology based on age-adjusted weights. J Trauma. 2002;52:235–41.
MacKenzie EJ, Rivara FP, Jurkovich GJ, Nathens AB, Frey KP, Egleston BL, et al. A national evaluation of the effect of trauma-center care on mortality. N Engl J Med. 2006;354:366–78.
Tohira H, Jacobs I, Mountain D, Gibson N, Yeo A. Systematic review of predictive performance of injury severity scoring tools. Scand J Trauma Resusc Emerg Med. 2012;20:63.
Tohira H, Jacobs I, Mountain D, Gibson N, Yeo A. Comparisons of the outcome prediction performance of injury severity scoring tools using the abbreviated injury scale 90 update 98 (AIS 98) and 2005 update 2008 (AIS 2008). Ann Adv Automot Med. 2011;55:255–65.
We thank Dr. Victor Alexandre Percinio Gianvecchio for his valuable contribution in allowing access to necropsy data in the Legal Medical Institute, and all the professionals of the Trauma Center at the University of California San Diego, in particular, Dale A. Fortlage, for providing service data.
Cristiane de Alencar Domingues has received scholarship funds from Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES).
Availability of data and materials
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
All authors have made substantial contributions to all of the following: (1) the conception and design of the study, acquisition of data, or analysis and interpretation of data; (2) drafting the article or revising it critically for important intellectual content; and (3) final approval of the version to be submitted.
The authors declare that they have no competing interests.
Consent for publication
Not applicable. A retrospective study whose data were obtained from medical records and institutional databases.
Ethics approval and consent to participate
The project received approval from the Ethics Committee of the Clinical Hospital of the College of Medicine of the University of São Paulo (Comissão de Ética para Análise de Projetos de Pesquisa – CAPPesp, protocol number 1019/09).
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Domingues, C.d.A., Coimbra, R., Poggetti, R.S. et al. Performance of new adjustments to the TRISS equation model in developed and developing countries. World J Emerg Surg 12, 17 (2017). https://doi.org/10.1186/s13017-017-0129-2
- Wounds and injuries
- Injury severity score
- Outcome assessment