Iranian Journal of Medical Sciences

Document Type : Review Article

Authors

1 Emergency Medicine Research Team, School of Medicine, Tabriz University of Medical Sciences, Tabriz, Iran

2 Research Center for Evidence-Based Medicine, Iranian EBM Center: A Joanna Briggs Institute Affiliated Center, Tabriz University of Medical Sciences, Tabriz, Iran

Abstract

Background: Emergency department (ED) physicians often need to quickly assess patients and determine vital signs to prioritize them by the severity of their condition and make optimal treatment decisions. Effective triage requires optimal scoring systems to accelerate and positively influence the treatment of trauma cases. To this end, a variety of scoring systems have been developed to enable rapid assessment of ED patients. The present systematic review and meta-analysis aimed to investigate the accuracy of the rapid emergency medicine score (REMS) system in predicting the mortality rate in non-surgical ED patients.
Methods: A systematic search of articles published between 1990 and 2020 was conducted using various scientific databases (Medline, Embase, Scopus, Web of Science, ProQuest, Cochrane Library, IranDOC, Magiran, and Scientific Information Database). Both cross-sectional and cohort studies assessing the REMS system to predict mortality in ED settings were considered. Two reviewers appraised the selected articles independently using the National Institutes of Health (NIH) quality assessment tool. The random-effects model was used for meta-analysis. I2 index and Q statistic were used to examine heterogeneity between the articles.
Results: The search resulted in 1,310 hits from which, 29 articles were eventually selected. Out of these, for 25 articles, the area under the curve value of REMS ranged from 0.52 to 0.986. The predictive power of REMS for the in-hospital mortality rate was high in 19 articles (67.85%) and low in nine articles (32.15%).
Conclusion: The results showed that the REMS system is an effective tool to predict mortality in non-surgical patients presented to the ED. However, further evidence using high-quality design studies is required to substantiate our findings.

Keywords

What’s Known

Previous studies have shown that the rapid emergency medicine score (REMS) system could be a valuable predictor of long-term mortality in non-surgical emergency department (ED) patients. REMS is reported to have good prognostic potential (AUC=0.815) to predict hospital mortality in severely injured patients.

What’s New

Results of our systematic review showed that most of the included studies confirmed the REMS system as an effective tool to predict mortality in ED patients. REMS is recommended as a valuable tool to predict in-hospital mortality in non-surgical patients admitted to the ED.

Introduction

The emergency department (ED) plays a pivotal role in managing complex and acute patients. 1 Triage in ED focuses on effective patient flow management, providing appropriate care, and preventing unnecessary interventions to improve medical outcome. 2 Emergency physicians often need to quickly assess patients, determine vital signs for prioritization, and make optimal decisions. Effective triage requires optimal scoring systems to accelerate treatment and positively influence treatment outcomes.

During the past decades, a variety of scoring systems have been developed to assess patients upon admission. The core element in these systems is an objective assessment of disease severity based on deviations in various physiological variables. More recently, researchers such as Nguyen 3 and Hyzy 4 have developed new scoring systems for critically ill trauma patients. However, none of these systems are dedicated to non-surgical ED patients. 3 The Acute Physiology and Chronic Health Evaluation II (APACHE II) system has been developed based on 12 physiological variables for use in the intensive care unit (ICU). However, APACHE II cannot be applied to ED patients due to the use of biochemical parameters. 5 The Rapid Acute Physiology Score (RAPS), a shortened version of APACHE II, 6 is one of the most appropriate scoring systems used in ED. It evaluates physiological parameters such as blood pressure, respiratory rate, pulse rate, and Glasgow coma scale (GCS). RAPS is further improved by including oxygen saturation and patient age, introducing a new system known as rapid emergency medicine score (REMS). 7 The benefit of these additions is that oxygen saturation can be easily measured in the ED, and age is an independent risk factor for severe diseases and mortality. A previous study showed that REMS is a powerful predictor of patient outcomes in the ED versus other scoring systems. 8 Another study reported that REMS could be a valuable predictor of long-term mortality in non-surgical ED patients. 9 In contrast, Söyüncü and Bektaş indicated that other scoring systems are more reliable than REMS. 10 Due to the lack of comprehensive data on the prognostic value of scoring systems, we performed a systematic review of the literature and meta-analysis to investigate the accuracy of REMS in predicting the mortality rate in non-surgical ED patients.

Materials and Methods

The study was approved by the Local Ethics Committee (code: IR.TBZMED.VCR.REC.1399.003). We conducted a systematic search from 1990 to 2020 using Medline (Ovid, PubMed), Embase, Scopus, Web of Science, ProQuest, and Cochrane Library. We also searched Iranian databases such as IranDOC, Magiran, and Scientific Information Database (SID). The search strategy included a combination of MeSH terms and free-text such as REMS, rapid emergency medicine score, rapid emergency medical score, and mortality (appendix 1-3). PICO (population, interventions, comparisons, outcomes) components were respectively non-surgical patients referred to ED, rapid emergency medicine score, other scoring systems, and mortality.

All identified citations were collated and uploaded into EndNote X9 (Clarivate Analytics, USA) followed by the exclusion of duplicate citations. Then, titles and abstracts were independently screened by two reviewers. The full texts of the screened articles were retrieved and assessed in detail. Inclusion criteria were using REMS as a predictive tool for mortality, studies conducted in ED, cross-sectional and cohort studies, non-surgical patients, and articles in English or Persian. Exclusion criteria were articles published before 1990, the use of languages other than English or Persian, and studies with patients discharged from ED or admitted to ED with cardiac arrest. The assessment was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA). 11 The quality of eligible articles was determined using the National Institutes of Health (NIH) quality assessment tool for observational cohort and cross-sectional studies. 12 Disagreements between reviewers were resolved through discussion until consensus was reached.

Eligible articles were appraised independently by two reviewers for methodological quality using standard critical appraisal tools. Disagreements between reviewers were resolved through mutual discussion. Following a critical appraisal, based on the degree of study bias, articles not fulfilling the quality threshold (i.e., meeting at least two items from the checklist) were excluded. The extracted data from the selected articles were the name of first author, publication year, country, setting, type of study, sample size, age, sex, admission reasons, study period (months), length of hospital stay (days), number of deceased patients, REMS score for survivors and non-survivors, the area under the curve (AUC) value of REMS, and the predictive power of REMS.

Statistical Analysis

The data were analyzed using Comprehensive Meta-Analysis software, version 3.0 (BioStat Inc., USA). The random-effects model was used for meta-analysis. I2 index and Q statistic were used to examine heterogeneity between the articles. Subgroup analysis was conducted based on the age of patients. P values less than 0.05 were considered statistically significant.

Results

The search resulted in 1,310 hits, of which 497 duplicate articles were removed. From the remaining 813 articles, those that did not meet the inclusion criteria (n=755) were removed. The full texts of the remaining 58 were assessed for eligibility, resulting in the exclusion of a further 29 articles because of non-original type of research, different study settings, or using REMS for assessing patients for procedures other than non-surgical approaches. Subsequently, a total of 29 studies were included in our systematic review. As depicted in figure 1, the selection process was in accordance with the PRISMA checklist. Of the 29 included articles, eight were cross-sectional 13 - 20 and 21 were cohort 1 , 7 , 9 , 21 - 38 studies. A total of 550,966 patients were included in this study of which 324775 (58.95%), 226,191 (41.05%) were men and women, respectively. The mean age of the patients was 49.13 years (range: 6.2-90.8 years). The reported setting was ED and the patients were admitted because of sepsis, injuries, vibrio vulnificus infection, splenic abscess, hepatic portal venous gas; severe fever with thrombocytopenia syndrome, trauma, S. aureus bacteremia or other suspected infections, febrile; non-surgical, acute coronary syndrome, or internal diseases. The average study duration was 27.04±1.0 months (range: 5-183 months). More than 50% of the studies reported an average hospital stay of about six days. Most studies reported the number of deceased patients with an average mortality rate of 7.95% (table 1).

Figure 1. The search strategy for the systematic review is illustrated according to the PRISMA guidelines.

Author Publication year Country Setting Type of study Sample size (n) Average age Admission reasons Study period (month) Length of hospital stay (day) Number of deceased patients REMS score for survivors REMS score for non-survivors AUC of REMS Predictive power of REMS
Male Female
Alter 21 2017 USA A county-based advanced life support EMS agency Cohort 28,035 33,311 51.9 12 4.3 High
Brabrand 22 2017 Denmark Hospital Cohort 2,917 2,867 67 5 1 (median) 193 0.77 Low
Bulut 23 2014 Turkey Hospital Cohort 1,039 961 61.41±18.92 6 153 5 7 0.589 High
Cardenete-Reyes 13 2017 Spain Hospital Cross-sectional 67 37 60.25±11.06 Acute coronary syndrome 12 High
Carugati 24 2018 Tanzania Hospital Cohort Febrile 11 44 0.52 Low
Cattermole 14 2009 Hong Kong Hospital Cross-sectional 195 135 61.3±20.6 12 0.771 Low
Crowe 25 2010 USA Hospital Cohort 108 108 Severe sepsis or septic shock 13 1-45 10 11 0.62 Low
Dundar 15 2015 Turkey Hospital Cross-sectional 507 432 71 (median) Geriatric patients 12 73 1 5 0.833 High
Ghanem-Zoubi 26 2011 Israel Community based hospital Cohort 582 490 74.7±16.1 Sepsis 15 8.77 387 8.4 11.9 0.77 High
Gok 16 2018 Turkey Hospital Cross-sectional 144 106 57.60±20.82 Internal diseases, surgery, and trauma 24 0.703 Low
Goodacre 27 2006 UK Hospital Cohort 3,222 2,361 63.4 55 744 8.4 0.74 High
Ha 17 2015 Vietnam Hospital Cross-sectional 806 940 7 (median) 172 6 9 0.712 High
Hilderink 28 2015 Netherlands Hospital Cohort 296 304 64.6 Sepsis 12 75 0.78 Low
Howell 29 2007 Israel Tertiary care hospital Cohort 1,020 1,112 61 Suspected infection 10 83 5 10 0.80 High
Hung 1 2017 Taiwan Hospital Cohort 77 37 56.33±16.12 Splenic abscess 183 0.10 0.16 0.67 Low
Imhoff 30 2014 USA Level one trauma center Cohort 2,718 962 36.5 Trauma 48 7.6 191 3.4 11.8 0.91 High
Kuo 31 2013 Taiwan Hospital Cohort 96 75 63.1±12.3 Vibrio vulnificus infection 16.8±14.6 (mean±SD) 43 5.4±2.3 9.7±2.6 0.895 High
Miller 32 2017 USA Level one trauma center Cohort 263,957 165,656 Blunt and/or penetrating injuries 5.2 (mean) 3,382 2.9 17.7 0.967 High
Nakhjavan-Shahraki 18 2017 Iran Hospital Cross-sectional 1,623 525 39.50±17.27 Trauma 123 0.92 High
Nakhjavan-Shahraki 19 2017 Iran Hospital Cross-sectional 605 209 11.65±5.36 Trauma 6 26 0.986 High
Olsson 33 2003 Sweden Hospital Cohort 513 513 70±18.1 5 116 0.911 High
Olsson 9 2004 Sweden Hospital Cohort 5,663 6,087 61.9±20.7 Non-surgical disorders 12 3.2 Predictor of long-term mortality
Olsson 7 2004 Sweden Hospital Cohort 5,663 6,087 61.9±20.7 Non-surgical disorders 12 3.2 285 5.5 10.5 0.852 High
Park 34 2017 South Korea Hospital Cohort 4,298 2,607 57.42±18.51 Trauma 60 24.95 212 4.31 9.71 0.9 High
Polita 35 2014 Brazil Hospital Cohort 131 32 38±18 Trauma 5 17 4.9 0.761 Low
Seak 36 2017 Taiwan Hospital Cohort 36 30 69.23±16.64 Hepatic portal venous gas 38 6.86 14.21 0.9286 High
Sharma 37 2013 USA Tertiary care community hospital Cohort 241 56.95±17.62 S. aureus bacteremia 17 55 5.24 9.58 0.806 High
Yang 38 2017 China Hospital Cohort 62 61 59±12 Severe fever with thrombocytopenia syndrome 38 31 8.55 12.45 0.746 Low
Ala 20 2020 Iran Hospital Cross-sectional 154 146 59.21±19.86 Non-surgical disorders 30 40 High
REMS: Rapid emergency medicine score; AUC: Area under the curve
Table 1.Detailed characteristics of included articles retrieved from the data extraction form

Methodological Quality Assessment

The quality of the included articles was assessed by two reviewers independently using the NIH quality assessment tool for observational cohort and cross-sectional studies. 12 All articles were judged to be fair or good (table 2). Since most of the articles used secondary data and were retrospective studies, three questions in the Critical Appraisal Skills Programme (CASP) checklist (numbers 8, 10, and 12) were deemed not applicable and therefore omitted (table 3).

No. Author Publication year Quality rating (reviewer 1) Quality rating (reviewer 2)
1 Alter 21 2017 Good Good
2 Brabrand 22 2017 Fair Fair
3 Bulut 23 2014 Fair Fair
4 Cattermole 14 2009 Fair Fair
5 Carugati24 2018 Good Good
6 Crowe 25 2010 Good Good
7 Dundar 15 2015 Fair Fair
8 Ghanem-Zoubi 26 2011 Fair Fair
9 Gok 16 2018 Fair Fair
10 Goodacre 27 2006 Fair Fair
11 Ha 17 2015 Fair Fair
12 Hilderink 28 2015 Fair Fair
13 Howell 29 2007 Fair Fair
14 Hung1 2017 Fair Fair
15 Imhoff 30 2014 Fair Fair
16 Kuo 31 2013 Good Good
17 Miller 32 2017 Fair Fair
18 Nakhjavan-Shahraki 18 2017 Fair Fair
19 Nakhjavan-Shahraki 19 2017 Fair Fair
20 Olsson 33 2003 Fair Fair
21 Olsson 9 2004 Fair Fair
22 Olsson 7 2004 Fair Fair
23 Park 34 2017 Fair Fair
24 Polita 35 2014 Fair Fair
25 Cardenete-Reyes 13 2017 Fair Fair
26 Seak 36 2017 Fair Fair
27 Sharma 37 2013 Good Good
28 Yang 38 2017 Good Good
29 Ala 20 2020 Good Good
Table 2.The quality rating of included articles using the National Institutes of Health quality assessment tool for observational cohort and cross-sectional studies
Author 1: Objective 2: Population definition 3: Participation rate 4: Selection criteria 5: Sample size 6: Exposure assessment 7: Timeframe 9: Exposure measures 11: Outcome measures 13: Loss to follow up 14: Statistical analysis
Alter 21 Yes Yes Yes Yes No Yes Yes Yes Yes CD Yes
Brabrand 22 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Bulut 23 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Cattermole 14 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Carugati 24 Yes Yes Yes Yes No Yes Yes Yes Yes CD Yes
Crowe 25 Yes Yes Yes Yes No Yes Yes Yes Yes CD Yes
Dundar 15 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Ghanem-Zoubi 26 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Gok 16 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Goodacre 27 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Ha 17 Yes Yes CD Yes No Yes Yes Yes Yes Yes Yes
Hilderink 28 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Howell 29 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Hung 1 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Imhoff 30 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Kuo 31 Yes Yes Yes Yes No Yes Yes Yes Yes CD Yes
Miller 32 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Nakhjavan-Shahraki 18 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Nakhjavan-Shahraki 19 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Olsson, et al 33 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Olsson 9 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Olsson 7 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Park 34 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Polita 35 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Cardenete-Reyes 13 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Seak 36 Yes Yes CD Yes No Yes Yes Yes Yes CD Yes
Sharma 37 Yes Yes Yes Yes No Yes Yes Yes Yes CD Yes
Yang 38 Yes Yes Yes Yes No Yes Yes Yes Yes CD Yes
Ala 20 Yes Yes CD Yes No Yes yes No No CD Yes
CD: Could not be determined
Table 3.Methodological quality assessment of included articles using the Critical Appraisal Skills Programme (CASP) checklist

Predictive Power of REMS

Almost all articles reported the average REMS score for survivors (5.10) and non-survivors (9.88). Except for four articles, average AUC values (0.79; range: 0.52-0.986) were reported. In these articles, REMS was considered independently or in comparison with other scoring systems. The predictive power of REMS for in-hospital mortality rate was high in 19 articles (67.85%) 7 , 9 , 13 , 15 , 17 - 19 , 21 , 23 , 26 , 27 , 29 - 34 , 36 , 37 and low in nine articles (32.15%). 1 , 14 , 16 , 22 , 24 , 25 , 28 , 35 , 38 Only one study reported that REMS was a good predictor of long-term mortality (4.7 years). 9

Meta-analysis

Twenty-two articles reported the percentage of mortality by surveying 477,186 ED cases. Publication bias was assessed using the funnel plot and Egger’s regression test. The results showed that diffusion between the articles was not statistically significant (t=0.59, df=20, P=0.281). Furthermore, the funnel plot showed symmetry between the articles (figure 2). Heterogeneity between the articles was significant (Q=11,340.14, df=21, I2=99.81, P<0.001), and the percentage of mortality was 8.69% (pooled death=0.0869, 95% CI: 4.50-16.11, P<0.001). The forest plot of the result of our meta-analysis is shown in figure 3.

Figure 2. Funnel plot illustrates bias in the results of the meta-analysis.

Figure 3. Forest plot depicted the mortality rates, which is extracted from the reviewed articles.

Subgroups Analysis

Subgroup analysis was conducted based on the age of the patients and the predictive power of REMS (table 4, figures 4 and 5). The results showed that the mortality rate in patients under versus above 60 years was 8.5% and 10.44%, respectively. Moreover, studies that evaluated the predictive power of REMS reported high and low levels of mortality rates at 8.72% and 8.59%, respectively.

Subgroup Effect size and 95% interval Null hypothesis Heterogeneity
Number Studies Proportion of patient deaths Lower limit Upper limit Z-value P value Q-value df P value I2
Predictive power of REMS High 17 0.0872 0.0407 0.1771 -5.67 <0.001 10,881.68 16 <0.001 99.85
Low 5 0.0859 0.0204 0.2973 -3.08 0.002 184.62 4 <0.001 97.83
Age ≤60 8 0.0857 0.0398 0.1749 -5.69 <0.001 308.67 7 <0.001 97.73
>60 12 0.1044 0.0566 0.1847 -6.34 <0.001 1,856.35 11 <0.001 99.41
Table 4.Tabular presentation of the results of subgroups analysis

Figure 4. Forest plot depicted the reported mortality rates in patients aged below and above 60 years.

Figure 5. Forest plot indicated the reported mortality rates in terms of the high and low predictive power of REMS.

Discussion

The results of the present systematic review showed that the AUC value of REMS was 0.79. The majority of the included studies (67.85%) reported that the REMS system has a high or good predictive value for mortality. In contrast, a previous study reported the lack of sufficient evidence to conclude on the accuracy of prognostic models in patients with suspected infection admitted to the ED. 39 The results of another systematic review aimed at validating 10 different scoring systems, including REMS, reported that none of the systems could accurately predict the risk of in-hospital mortality and admission to the ICU. However, they found that REMS had an acceptable discriminatory power but poor calibration. 40

In the present study, we mainly focused on the ED setting, whereas some other review studies focused on other healthcare settings. Nonetheless, their findings on the predictive power of REMS were in line with our study. El-Sarnagawy and Hafez assessed different scoring systems, including REMS, in predicting the need for mechanical ventilation in patients with a drug overdose. They reported that REMS had a 100% positive predictive value and recommended this scoring system as an appropriate tool. 41 In contrast with our study, Yu and colleagues compared REMS with other scoring systems in terms of its predictive ability to detect clinical deterioration in non-ICU patients diagnosed with an infection. They measured each score serially to characterize how these scores changed with time. They reported that REMS had an AUC value of 0.70 and lacked adequate predictive value that other systems. 42 Ji and colleagues conducted a study in the ED and coronary care unit (CCU) of a hospital and showed that REMS did not have adequate predictive value for short-term risk of death in patients with acute myocardial infarction (AMI). After comparing REMS with Global Registry of Acute Coronary Events (GRACE) and APACHE II risk scores, they reported that the AUC value of REMS for predicting mortality in AMI patients within 30 days was 0.615. 43 In the present study, the average AUC value of REMS for non-surgical patients was 0.79, which is an acceptable predictive value.

We also compared the findings of the studies included in our systematic review with the results of other studies. One of the included articles reported that REMS was a good predictor of long-term in-hospital mortality (4.7 years). 9 Similarly, Olsson and colleagues showed that while REMS can be a predictor of long-term mortality, it cannot independently predict short-term (three-day, seven-day) mortality in non-surgical ED patients. 44 Seven studies in our systematic review were conducted in traumatic patients, five of which reported that REMS could accurately predict in-hospital mortality. 18 , 19 , 30 , 32 , 34 Lee and colleagues also reported that REMS had a good prognostic ability (AUC=0.815) to predict hospital mortality in severely injured patients. 45 Although most of the studies in our systematic review assessed patients with infectious diseases, the reported overall AUC>0.70 was in line with other studies conducted on traumatic patients. Furthermore, three studies were conducted on ED patients diagnosed with sepsis in the ED setting. Among these, one study reported the high predictive power of REMS for in-hospital mortality. 26 In line with our findings, another study reported that REMS had a good prognostic ability (AUC=0.72) to predict mortality in adult ED patients diagnosed with sepsis. 46

In the present systematic review, we selected studies that specifically focused on non-surgical patients. It is recommended that future studies include other categories of patients to further confirm the high prognostic ability of REMS to predict mortality. The main limitation of our systematic review was related to poor quality or lack of access to the full text of some of the selected articles, as well as the exclusion of studies published in languages other than English and Persian.

Conclusion

The results of the present systematic review and meta-analysis showed that the REMS system is an effective tool to predict hospital mortality in non-surgical patients admitted to ED. The use of the REMS system is recommended in ED to predict mortality and serve as a basis for developing an efficient care plan. However, further evidence using high-quality design studies is required to substantiate our findings.

Acknowledgement

The authors would like to express their gratitude for the financial support from the Vice-Chancellor for Research of Tabriz University of Medical Sciences, Tabriz, Iran.

Authors’ Contribution

A.Gh: Acquisition and analysis of data, Drafting and critical revision of the manuscript for important intellectual content; N.V: Systematic search, analysis of data, Drafting and critical revision of the manuscript for important intellectual content; S.Sh.V: Study design, Critical reviews, Drafting of the manuscript; A.A: Study concept and design, Drafting and critical revision of the manuscript for important intellectual content; M.J: Study concept and design, Critical reviews, Acquisition of Data, Drafting and critical revision of the manuscript for important intellectual content; All authors have read and approved the final manuscript and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Conflict of Interest: None declared.

References

  1. Hung SK, Ng CJ, Kuo CF, Goh ZNL, Huang LH, Li CH, et al. Comparison of the Mortality in Emergency Department Sepsis Score, Modified Early Warning Score, Rapid Emergency Medicine Score and Rapid Acute Physiology Score for predicting the outcomes of adult splenic abscess patients in the emergency department. PLoS One. 2017; 12:e0187495. Publisher Full Text | DOI | PubMed
  2. Wilson R, Goodacre SW, Klingbajl M, Kelly AM, Rainer T, Coats T, et al. Evaluation of the DAVROS (Development And Validation of Risk-adjusted Outcomes for Systems of emergency care) risk-adjustment model as a quality indicator for healthcare. Emerg Med J. 2014; 31:471-5. Publisher Full Text | DOI | PubMed
  3. Nguyen HB, Rivers EP, Havstad S, Knoblich B, Ressler JA, Muzzin AM, et al. Critical care in the emergency department a physiologic assessment and outcome evaluation. Academic Emergency Medicine. 2000; 7:1354-61. DOI
  4. Hyzy RC. ICU scoring and clinical decision making. Chest. 1995; 107:1482-3. DOI | PubMed
  5. Bhagwanjee S, Paruk F, Moodley J, Muckart DJ. Intensive care unit morbidity and mortality from eclampsia: an evaluation of the Acute Physiology and Chronic Health Evaluation II score and the Glasgow Coma Scale score. Crit Care Med. 2000; 28:120-4. DOI | PubMed
  6. Rhee KJ, Fisher CJ, Willitis NH. The Rapid Acute Physiology Score. Am J Emerg Med. 1987; 5:278-82. DOI | PubMed
  7. Olsson T, Terent A, Lind L. Rapid Emergency Medicine score: a new prognostic tool for in-hospital mortality in nonsurgical emergency department patients. J Intern Med. 2004; 255:579-87. DOI | PubMed
  8. Wei X, Ma H, Liu R, Zhao Y. Comparing the effectiveness of three scoring systems in predicting adult patient outcomes in the emergency department. Medicine (Baltimore). 2019; 98:e14289. Publisher Full Text | DOI | PubMed
  9. Olsson T, Terent A, Lind L. Rapid Emergency Medicine Score can predict long-term mortality in nonsurgical emergency department patients. Acad Emerg Med. 2004; 11:1008-13. DOI | PubMed
  10. Söyüncü S, Bektaş F. Comparison of the scoring systems for predicting mortality in intoxicated patients hospitalized to the ICU: a prospective observational study. Erciyes Medical Journal. 2011; 33:29-34.
  11. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015; 4:1. Publisher Full Text | DOI | PubMed
  12. National Health, Lung, and Blood Institutes [Internet]. Study Quality Assessment Tools. [update July 2021]. Available at: https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools.
  13. Cardenete-Reyes C, Gonzalez-Pascual JL, Tellez-Galan G, Velarde-Garcia JF, Polo-Portes CE, Palacios-Cena D. Mortality risk and complications relating to interhospital transfers of patients with acute coronary syndrome receiving primary coronary angioplasty performed by the Medical Emergency Service of Madrid 112: a group comparison study. Hong Kong Journal of Emergency Medicine. 2017; 24:177-85.
  14. Cattermole GN, Mak SK, Liow CH, Ho MF, Hung KY, Keung KM, et al. Derivation of a prognostic score for identifying critically ill patients in an emergency department resuscitation room. Resuscitation. 2009; 80:1000-5. DOI | PubMed
  15. Dundar ZD, Karamercan MA, Ergin M, Colak T, Tuncar A, Ayrancı K, et al. Rapid emergency medicine score and HOTEL score in geriatric patients admitted to the emergency department. International Journal of Gerontology. 2015; 9:87-92. DOI
  16. Gok RGY, Gok A, Bulut M. Assessing prognosis with modified early warning score, rapid emergency medicine score and worthing physiological scoring system in patients admitted to intensive care unit from emergency department. Int Emerg Nurs. 2019; 43:9-14. DOI | PubMed
  17. Ha DT, Dang TQ, Tran NV, Vo NY, Nguyen ND, Nguyen TV. Prognostic performance of the Rapid Emergency Medicine Score (REMS) and Worthing Physiological Scoring system (WPS) in emergency department. Int J Emerg Med. 2015; 8:18. Publisher Full Text | DOI | PubMed
  18. Nakhjavan-Shahraki B, Baikpour M, Yousefifard M, Nikseresht ZS, Abiri S, Mirzay Razaz J, et al. Rapid Acute Physiology Score versus Rapid Emergency Medicine Score in Trauma Outcome Prediction; a Comparative Study. Emerg (Tehran). 2017; 5:e30. Publisher Full Text | PubMed
  19. Nakhjavan-Shahraki B, Yousefifard M, Faridaalaee G, Shahsavari K, Oraii A, Hajighanbari MJ, et al. Performance of physiology scoring systems in prediction of in-hospital mortality of traumatic children: A prospective observational study. J Clin Orthop Trauma. 2017; 8:S43-S8. Publisher Full Text | DOI | PubMed
  20. Ala A, Vahdati SS, Jalali M, Parsay S. Rapid Emergency Medicine Score as a Predictive Value for 30-day Outcome of Nonsurgical Patients Referred to the Emergency Department. Indian J Crit Care Med. 2020; 24:418-22. Publisher Full Text | DOI | PubMed
  21. Alter SM, Infinger A, Swanson D, Studnek JR. Evaluating clinical care in the prehospital setting: Is Rapid Emergency Medicine Score the missing metric of EMS?. Am J Emerg Med. 2017; 35:218-21. DOI | PubMed
  22. Brabrand M, Hallas P, Hansen SN, Jensen KM, Madsen JLB, Posth S. Using scores to identify patients at risk of short term mortality at arrival to the acute medical unit: A validation study of six existing scores. Eur J Intern Med. 2017; 45:32-6. DOI | PubMed
  23. Bulut M, Cebicci H, Sigirli D, Sak A, Durmus O, Top AA, et al. The comparison of modified early warning score with rapid emergency medicine score: a prospective multicentre observational cohort study on medical and surgical patients presenting to emergency department. Emerg Med J. 2014; 31:476-81. DOI | PubMed
  24. Carugati M, Zhang HL, Kilonzo KG, Maze MJ, Maro VP, Rubach MP, et al. Predicting Mortality for Adolescent and Adult Patients with Fever in Resource-Limited Settings. Am J Trop Med Hyg. 2018; 99:1246-54. Publisher Full Text | DOI | PubMed
  25. Crowe CA, Kulstad EB, Mistry CD, Kulstad CE. Comparison of severity of illness scoring systems in the prediction of hospital mortality in severe sepsis and septic shock. J Emerg Trauma Shock. 2010; 3:342-7. Publisher Full Text | DOI | PubMed
  26. Ghanem-Zoubi NO, Vardi M, Laor A, Weber G, Bitterman H. Assessment of disease-severity scoring systems for patients with sepsis in general internal medicine departments. Crit Care. 2011; 15:R95. Publisher Full Text | DOI | PubMed
  27. Goodacre S, Turner J, Nicholl J. Prediction of mortality among emergency medical admissions. Emerg Med J. 2006; 23:372-5. Publisher Full Text | DOI | PubMed
  28. Hilderink MJ, Roest AA, Hermans M, Keulemans YC, Stehouwer CD, Stassen PM. Predictive accuracy and feasibility of risk stratification scores for 28-day mortality of patients with sepsis in an emergency department. Eur J Emerg Med. 2015; 22:331-7. DOI | PubMed
  29. Howell MD, Donnino MW, Talmor D, Clardy P, Ngo L, Shapiro NI. Performance of severity of illness scoring systems in emergency department patients with infection. Acad Emerg Med. 2007; 14:709-14. DOI | PubMed
  30. Imhoff BF, Thompson NJ, Hastings MA, Nazir N, Moncure M, Cannon CM. Rapid Emergency Medicine Score (REMS) in the trauma population: a retrospective study. BMJ Open. 2014; 4:e004738. Publisher Full Text | DOI | PubMed
  31. Kuo SH, Tsai CF, Li CR, Tsai SJ, Chao WN, Chan KS, et al. Rapid Emergency Medicine Score as a main predictor of mortality in Vibrio vulnificus-related patients. Am J Emerg Med. 2013; 31:1037-41. DOI | PubMed
  32. Miller RT, Nazir N, McDonald T, Cannon CM. The modified rapid emergency medicine score: A novel trauma triage tool to predict in-hospital mortality. Injury. 2017; 48:1870-7. DOI | PubMed
  33. Olsson T, Lind L. Comparison of the rapid emergency medicine score and APACHE II in nonsurgical emergency department patients. Acad Emerg Med. 2003; 10:1040-8. DOI | PubMed
  34. Park HO, Kim JW, Kim SH, Moon SH, Byun JH, Kim KN, et al. Usability verification of the Emergency Trauma Score (EMTRAS) and Rapid Emergency Medicine Score (REMS) in patients with trauma: A retrospective cohort study. Medicine (Baltimore). 2017; 96:e8449. Publisher Full Text | DOI | PubMed
  35. Polita JR, Gomez J, Friedman G, Ribeiro SP. Comparison of APACHE II and three abbreviated APACHE II scores for predicting outcome among emergency trauma patients. Rev Assoc Med Bras (1992). 2014; 60:381-6. DOI | PubMed
  36. Seak CJ, Yen DH, Ng CJ, Wong YC, Hsu KH, Seak JC, et al. Rapid Emergency Medicine Score: A novel prognostic tool for predicting the outcomes of adult patients with hepatic portal venous gas in the emergency department. PLoS One. 2017; 12:e0184813. Publisher Full Text | DOI | PubMed
  37. Sharma M, Szpunar S, Khatib R. Validating severity of illness scoring systems in the prediction of outcomes in Staphylococcus aureus bacteremia. Am J Med Sci. 2013; 346:87-91. DOI | PubMed
  38. Yang B, Wang X, Li Y, Wu A, Liu Q, Lu Y, et al. A Newly Established Severity Scoring System in Predicting the Prognosis of Patients with Severe Fever with Thrombocytopenia Syndrome. Tohoku J Exp Med. 2017; 242:19-25. DOI | PubMed
  39. Calle P, Cerro L, Valencia J, Jaimes F. Usefulness of severity scores in patients with suspected infection in the emergency department: a systematic review. J Emerg Med. 2012; 42:379-91. DOI | PubMed
  40. Brabrand M, Folkestad L, Clausen NG, Knudsen T, Hallas J. Risk scoring systems for adults admitted to the emergency department: a systematic review. Scand J Trauma Resusc Emerg Med. 2010; 18:8. Publisher Full Text | DOI | PubMed
  41. El-Sarnagawy GN, Hafez AS. Comparison of different scores as predictors of mechanical ventilation in drug overdose patients. Hum Exp Toxicol. 2017; 36:539-46. DOI | PubMed
  42. Yu S, Leung S, Heo M, Soto GJ, Shah RT, Gunda S, et al. Comparison of risk prediction scoring systems for ward patients: a retrospective nested case-control study. Crit Care. 2014; 18:R132. Publisher Full Text | DOI | PubMed
  43. Ji CL, Zhou HR, Peng CH, Yang XL, Zhang Q. [Comparison of value of GRACE, APACHEII and REMS for early prognosis of death in patients with acute myocardial infarction]. Zhonghua Wei Zhong Bing Ji Jiu Yi Xue. 2013; 25:660-3. DOI | PubMed
  44. Olsson T, Terent A, Lind L. Charlson Comorbidity Index can add prognostic information to Rapid Emergency Medicine Score as a predictor of long-term mortality. Eur J Emerg Med. 2005; 12:220-4. DOI | PubMed
  45. Lee SH, Park JM, Park JS, Kim KH, Shin DW, Jeon WC, et al. Utility of the Rapid Emergency Medicine Score (REMS) for predicting hospital mortality in severely injured patients. Journal of the Korean Society of Emergency Medicine. 2016; 27:199-205.
  46. Taylor RA, Pare JR, Venkatesh AK, Mowafi H, Melnick ER, Fleischman W, et al. Prediction of In-hospital Mortality in Emergency Department Patients With Sepsis: A Local Big Data-Driven, Machine Learning Approach. Acad Emerg Med. 2016; 23:269-78. Publisher Full Text | DOI | PubMed