A benchmarking study of two trauma centres highlighting limitations when standardising mortality for comorbidity
© Budd et al; licensee BioMed Central Ltd. 2008
Received: 12 September 2007
Accepted: 16 January 2008
Published: 16 January 2008
A continuous process of trauma centre evaluation is essential to ensure the development and progression of trauma care at regional, national and international levels. Evaluation may be by comparison between pooled datasets or by direct benchmarking between centres. This study attempts to benchmark mortality at two trauma centres standardising this for multiple case-mix factors, which includes the prevalence of individual background pre-existing diseases within the study population.
Trauma patients with an Injury Severity Score (ISS) >15 admitted to the two centres in 2001 and 2002 were included in the study with the exception of those who died in the emergency department. Patient characteristics were analysed in terms of 18 case-mix factors including Glasgow Coma Scale on arrival, Injury Severity Score and the presence or absence of 9 co-morbidity types, and patient outcome was compared based on in-hospital mortality before and after standardisation.
Crude mortality was greater at UHNS (18.2 vs 14.5%) with a non-significant odds ratio of 1.31 prior to adjusting for case-mix (P = 0.171). Adjustment for case mix using logistic regression analysis altered the odds ratio to 1.64, which was not significant (P = 0.069).
This study did not demonstrate any significant difference in the outcome of patients treated at either hospital during the study period. More importantly it has raised several important methodological issues pertinent to researchers undertaking registry based benchmarking studies. Data at the two registries was collected by personnel with differing backgrounds, in formats that were not completely compatible and was collected for patients that met different admissions criteria. The inclusion of a meaningful analysis of pre-existing disease was limited by the availability of robust data and sample size. We suggest greater communication between trauma research coordinators to ensure equivalent data collection and facilitate future benchmarking studies.
The outcome of trauma patients managed by individual centres is influenced by multiple factors including investment in the trauma service infrastructure, the facilities, technology and experience available at each centre, the type of injuries sustained and the background demographics of the patients managed. The performance of trauma centres has been evaluated by comparing outcomes with norms derived from pooled datasets, as in TRISS analysis and its modifications, and by benchmarking studies [1–3]. The logistic regression used in TRISS analysis uses few case-mix factors: age, Injury Severity Score, Glasgow Coma Score, systolic blood pressure and respiratory rate. Data collected by trauma centre registries are however generally more detailed than those submitted to pooled national databases and provide a richer set of case mix factors to allow a more discerning comparison.
A previous benchmarking study of trauma centre outcome compared the University Hospital of North Staffordshire (UHNS) with Oregon Health Sciences University (OHSU) in the United States . The crude mortality analysis demonstrated a highly significant odds-ratio of 1.64 in favour of OHSU that was rendered non-significant after adjusting for case mix, which included twenty-three factors with equivalent definitions. The presence or absence of a significant past medical history was included as a binary variable (yes/no), providing this as the only indicator of co-morbidity with no qualification of severity or type. Adjusting case-mix for this binary indicator of co-morbidity does not accurately allow discrimination between the outcomes of two centres treating patients with a differing prevalence of significant pre-existing diseases, each of which can detrimentally effect the outcome of major trauma to a differing extent, as was demonstrated in a case control study by Morris et al . In this study comparing two major trauma centres, one in the United Kingdom and one in Australia, we have attempted to account for the influence of pre-existing disease on mortality by including the prevalence of individual co-morbid factors within the case-mix adjustment.
The University Hospital of North Staffordshire, Stoke-on-Trent, United Kingdom
The University Hospital of North Staffordshire (UHNS) is a large acute hospital with over 1000 beds. It has a local catchment population of 0.5 million and accepts tertiary referrals (except major burns) from a total of 1.7 million. The trauma research department collects data on all injured patients who stay longer than 72 hours in hospital, are admitted to the intensive care unit, die or are transferred from other hospitals. Co-morbidity data are routinely collected, but in non-mandatory fields.
The Royal Melbourne Hospital, Melbourne, Australia
The Royal Melbourne Hospital (RMH) provides one of two adult major trauma services accepting tertiary referrals from the state of Victoria. It shares a total catchment population of over three million with the Alfred Hospital, Melbourne (AH). All surgical specialties (except for a burns service) are available on-site. Its trauma research department collects data for all patients who are admitted for longer than 24 hours or who die. Patient co-morbidity data are collected in a structured way, using the pre-defined conditions from the American College of Surgeons' trauma dataset .
All patients admitted to the two centres between 1st January 2001 and 31st December 2002 with an Injury Severity Score (ISS) over 15 were identified, including those transferred from other hospitals. Patients who were dead on arrival or who died in the emergency department were excluded.
Case Mix Factors
1. Date of admission
4. Mechanism of injury
5. Type of trauma (blunt/penetrating)
6. Systolic blood pressure on arrival in primary hospital
7. GCS on arrival in primary hospital
8. Whether transferred from another hospital (yes/no)
9. Injury Severity Score
Abbreviated Injury Scale (AIS) severity in the 9 body regions:
10. Head AIS
11. Face AIS
12. Neck AIS
13. Chest AIS
14. Abdomen AIS
15. Spine AIS
16. Upper limbs AIS
17. Lower limb AIS
18. External region AIS
19. Coronary artery disease (yes/no)
20. Hypertension (yes/no)
21. Non-insulin dependent diabetes mellitus (yes/no)
22. Psychiatric illness (yes/no)
23. Seizures (yes/no)
24. Stroke (yes/no)
25. Asthma (yes/no)
26. Drug abuse (yes/no)
27. Alcohol problem (yes/no)
A multiple linear logistic regression model, which allows the effects of continuous and categorical factors to be evaluated and weighted simultaneously, was used in a multi-factor analysis. Twenty-seven factors were studied, with automatic variable selection procedures (stepwise forward and backward algorithms) to suggest the best fitting logistic model. A 5% level of statistical significance was used to test statistical hypothesis and construct confidence intervals.
Number of cases
Mean number of co-morbidity factors
Mean systolic blood pressure
Mean Injury Severity Score
Mechanism of Injury
Motor vehicle crash
Fall > 2 m
Fall < 2 m
Coronary artery disease
Non-insulin dependent diabetes
Odds Ratios for Mortality Before and After Case Mix Adjustment
Degrees of Freedom
This study is the first benchmarking trauma centre comparison to attempt to account for the prevalence of individual pre-existing diseases when calculating mortality rates of severely injured patients. Pre-existing disease in trauma patients is an independent risk factor for mortality, though some studies have indicated that it is less important than the mechanism of injury, the severity and site of injuries, and patient age [7–9]. The crude mortality rate was higher for Stoke than Melbourne (18.2% vs 14.5%) but while the odds ratio increased from 1.31 to 1.64 after standardisation for case-mix, including co-morbidity, this did not reach statistical significance.
The mean number of co-morbidity factors per patient was higher at RMH (1.04 versus 0.75), which may relate to differences in patient health behaviour, medical practice or simply by the completeness of co-morbidity recording in the two trauma registries. The incidence of the nine specific co-morbidities was similar with the exception of hypertension and chronic drug abuse, both of which were higher in the RMH population. A previous study looking at the presence of pre-existing conditions and mortality in older patients has shown hypertension to have a protective effect on outcome, possibly relating to the prescription of beta-blockers .
Less than 8% of major trauma (ISS>15) was penetrating in both centres, with RMH having double the incidence of UHNS. The lower rate of injuries in the 'external' region may reflect the presence of a nearby specialist burns unit in Melbourne, reducing the likelihood of primary transfer to RMH. Similarly, the slightly higher mean age at RMH may be due to the presence of a regional paediatric Major Trauma Service operated from Melbourne Royal Children's Hospital.
Victoria State (227,620 sq. km) is more than seventy-five times larger than the North West Midlands in the UK (2,997 sq. km). It would be surprising if the contrasting geographical features of the Australian catchment area did not have an influence on outcome. The proportion of major trauma patients transferred into RMH from primary hospitals was more than twice that transferred into UHNS, however in the analysis whether the patient was transferred or not was not a significant discriminator.
This study attempts to standardise mortality taking into account pre-existing disease but has several important limitations that merit discussion. The categorisation of co-morbidity was limited to a binary system relating to the presence or absence of individual co-morbidity and information regarding disease severity was not included due to the difficulty in quantifying this retrospectively. In addition, due to the size of the study populations only the 9 most prevalent disease types (of the 46 types recorded) could be included in analysis, excluding conditions such as cancer, renal failure and hepatic disease that have previously been shown to increase significantly the risk of death in elderly trauma patients . Patient outcome is influenced by the nature and expediency of pre-hospital care, which we could not include within the case-mix standardisation because of the limited compatibility of available pre-hospital data recorded within the registries, and limited power due to population size. For inclusion of the above case-mix factors in a benchmarking study the sample size would have to be significantly larger if it were to prove a statistically significant difference in outcome. This would only be possible if individual trauma registries collected data for equivalent patients groups in a standardised format, and this was undertaken by staff with similar experience and training submitting information to the national registry. While this study has several methodological limitations it has highlighted the need for greater cooperation between trauma registry programme coordinators to ensure that data, including that relating to co-morbidity, is classified in a standardised format pertinent to future benchmarking studies necessary to drive global improvements in trauma care.
As trauma systems have improved, unexpected death rates have fallen making mortality a less discerning indicator of performance. Disability and morbidity are being heralded as better discriminators of outcome, although the validity of the current disability measures remains uncertain. As our ability to quantify outcome in this way improves, the importance of a robust analysis of co-morbidity becomes obvious.
Paul Garwood, Trauma Research Department, University Hospital North Staffordshire
- Celso B, Tepas J, Langland-Orban B, Pracht E, Papa L, Lottenberg L, Flint L: Systematic review and meta analysis comparing outcome of severely injured patients treated in trauma centers following the establishment of trauma systems. J Trauma. 2006, 60: 371-8.View ArticlePubMedGoogle Scholar
- Boyd C, Tolson M, Copes W: Evaluating trauma care: the TRISS method. Journal of Trauma. 1987, 27 (4): 370-8.View ArticlePubMedGoogle Scholar
- Hollis S, Yates DW, Woodford M, Foster P: Standardised comparison of performance indicators in trauma: a new approach to case-mix variation. J Trauma. 1995, 38: 763-766.View ArticlePubMedGoogle Scholar
- Templeton J, Oakley PA, MacKenzie G, Cook AL, Brand D, Mullins RJ, Trunkey DD: A comparison of patient characteristics and survival in two trauma centres located in different countries. Injury. 2000, 31: 493-501. 10.1016/S0020-1383(00)00029-2.View ArticlePubMedGoogle Scholar
- Morris JA, MacKenzie EJ, Edelstein SL: The effect of preexisting conditions on mortality in trauma patients. JAMA. 1990, 263 (14): 1942-6. 10.1001/jama.263.14.1942.View ArticlePubMedGoogle Scholar
- American College of Surgeons – National Trauma Data Bank. Data Submission File Format Codes. November 2005. Appendix A: Pre-Existing Comorbidity Factors.Google Scholar
- McGwin G, MacLennan PA, Fife J, Davis GG, Rue LW: Pre-existing conditions and mortality in older trauma patients. J Trauma. 2004, 56: 1291-1296.View ArticlePubMedGoogle Scholar
- Milzman D, Boulanger B, Rodriguez A: Pre-existing Disease in Trauma Patients: A Predictor of Fate Independent of Age and Injury Severity Score. The Journal of Trauma. 1992, 32 (2): 236-243.View ArticlePubMedGoogle Scholar
- Millham F, La Morte W: Factors Associated with Mortality in Trauma: Re-Evaluation of the TRISS Method using the National Trauma Data Bank. J Trauma. 2004, 56: 1090-1096.View ArticlePubMedGoogle Scholar
- Grossman M, Miller D, Scaff D, Arcona S: When is an elder old? Effect of pre-existing conditions on mortality in trauma patients. JAMA. 1990, 263: 1942-1946. 10.1001/jama.263.14.1942.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.