Skip to main content
Download PDF

The performance of the Dutch Safety Management System frailty tool to predict the risk of readmission or mortality in older hospitalised cardiac patients

BMC Geriatrics volume 21, Article number: 299 (2021) Cite this article

Abstract

Background

Early identification of older cardiac patients at high risk of readmission or mortality facilitates targeted deployment of preventive interventions. In the Netherlands, the frailty tool of the Dutch Safety Management System (DSMS-tool) consists of (the risk of) delirium, falling, functional impairment, and malnutrition and is currently used in all older hospitalised patients. However, its predictive performance in older cardiac patients is unknown.

Aim

To estimate the performance of the DSMS-tool alone and combined with other predictors in predicting hospital readmission or mortality within 6 months in acutely hospitalised older cardiac patients.

Methods

An individual patient data meta-analysis was performed on 529 acutely hospitalised cardiac patients ≥70 years from four prospective cohorts. Missing values for predictor and outcome variables were multiply imputed. We explored discrimination and calibration of: (1) the DSMS-tool alone; (2) the four components of the DSMS-tool and adding easily obtainable clinical predictors; (3) the four components of the DSMS-tool and more difficult to obtain predictors. Predictors in model 2 and 3 were selected using backward selection using a threshold of p = 0.157. We used shrunk c-statistics, calibration plots, regression slopes and Hosmer-Lemeshow p-values (PHL) to describe predictive performance in terms of discrimination and calibration.

Results

The population mean age was 82 years, 52% were males and 51% were admitted for heart failure. DSMS-tool was positive in 45% for delirium, 41% for falling, 37% for functional impairments and 29% for malnutrition. The incidence of hospital readmission or mortality gradually increased from 37 to 60% with increasing DSMS scores. Overall, the DSMS-tool discriminated limited (c-statistic 0.61, 95% 0.56–0.66). The final model included the DSMS-tool, diagnosis at admission and Charlson Comorbidity Index and had a c-statistic of 0.69 (95% 0.63–0.73; PHL was 0.658).

Discussion

The DSMS-tool alone has limited capacity to accurately estimate the risk of readmission or mortality in hospitalised older cardiac patients. Adding disease-specific risk factor information to the DSMS-tool resulted in a moderately performing model. To optimise the early identification of older hospitalised cardiac patients at high risk, the combination of geriatric and disease-specific predictors should be further explored.

Peer Review reports

Background

Hospitalisation of older cardiac patients is associated with increased risk of functional loss, readmission or mortality [1,2,3]. Geriatric conditions such as malnutrition, tendency to fall and functional impairment are common in older cardiac patients and contribute to these adverse health outcomes [2, 4, 5].

Measurement of risk in older cardiac patients facilitates early initiation of targeted interventions to delay or prevent complications such as (further) functional loss, readmission or mortality in those patients susceptible to such interventions [6]. Risk stratification may help to determine in which patients guideline-recommended treatments may be deployed and for which patients harms outweigh benefits [4, 7, 8].

The Dutch Safety Management System (VeiligheidsManagementSysteem, DSMS) of the Ministry of Health, Welfare and Sport, developed the DSMS-screening tool to detect hospitalised older patients at high risk of functional loss [9]. The DSMS-tool has been in use since 2012 and all Dutch hospitals are required to screen hospitalised older patients on (their risk of) four geriatric domains; delirium, falling, functional impairment and malnutrition. Functional loss is associated with a high risk of readmission and mortality [10,11,12,13]. As the DSMS detects frail older patients at high risk of functional loss, the tool may also be capable of identifying patients at high risk of these adverse outcomes and if so, would enable timely targeted deployment of preventive interventions. Therefore, the aim of this study is to estimate the performance of the DSMS-tool alone and combined with other predictors in predicting all-cause unplanned hospital readmission or mortality within 6 months in acutely hospitalised older cardiac patients.

Methods

An individual patient data meta-analysis was performed on 529 acutely hospitalised cardiac patients ≥70 years from four prospective cohort studies: 1) The Hospital-ADL study [12] examined the development and course of geriatric conditions during and after hospitalisation; 2) the Surprise Question Cohort [14] examined to what extent a negative answer of healthcare professionals to the question “would I be surprised if this patient died in the next year?”, corresponded to mortality within the next year; 3) the Transitional Care Bridge study [15], a multi-centre randomised trial (RCT) on nurse-coordinated transitional care. Only patients of the control group were included in this study because the intervention was found to have a statistically significant effect on mortality; 4) the Cardiac Care Bridge [16], a multi-centre RCT. All patients were included in the current study because the interventions proved to be ineffective.

Patients were eligible for the current study if they 1) had been admitted with a cardiac disease, 2) had been acutely hospitalised for ≥48 h, and 3) were aged ≥70 years.

The DSMS-screening tool

Table 1 shows the content of the DSMS-tool [9]. The tool consists of single yes/no questions that assess the four geriatric conditions to identify patients at high risk of functional loss. The answers to the questions can also be added up to form the total score. Based on the number of geriatric conditions, the DSMS-score therefore ranges between 0 and 4.

Table 1 Screening tool for vulnerable elderly of the Dutch Safety Management System
Full size table

Outcome

The primary outcome was the performance of the DSMS-tool in predicting six-month all-cause unplanned readmission or mortality. Readmission data were collected from medical files in the participating hospitals and supplemented with patients’ and family members’ self-reported readmissions in other hospitals. Mortality was registered within the original cohorts and originates from medical files, the Dutch National Personal Records Database [19], or information from family members at follow-up.

Statistical analyses

Missing data

Additional file 1 shows the frequency of missing data in the four cohorts. Missing values for predictor and outcome variables were imputed 20 times using the MICE package in R-Studio (version 3.6.1), involving 19 variables, including 3 indicator variables to identify the 4 cohorts [20]. The only continuous variable with missing values, length of stay (days), was log-transformed before imputation. We used predictive mean matching throughout. The complete datasets (m = 20) were analysed separately and the results pooled using the pooled sampling variance method [21].

Descriptive statistics

Descriptive statistics are reported as means with standard deviation (SD) for normally distributed continuous variables and medians with interquartile range (IQR) for non-normally distributed data. Categorical variables are reported as frequencies and percentages. The incidence of all-cause unplanned readmission or mortality at 6 months is reported per DSMS-score. DSMS-scores 3 and 4 were merged to indicate high-risk patients due to the limited numbers with score 4.

Regression models

The prediction model for readmission or mortality within 6 months was developed and tested by using an individual patient data meta-analysis of prediction models. Both geriatric and disease-specific candidate predictors associated with readmission or mortality were selected. We explored discrimination and calibration of: 1) DSMS alone (delirium, falling, functional impairment and malnutrition); 2) clinical candidate predictors easily obtainable from medical files or by short questions: age, sex, educational level, living arrangement, polypharmacy (≥ 5 medicines), admission in the previous 6 months and cardiac diagnosis at admission, first without and then including the items of the DSMS; 3) a model based on the four components of the DSMS and more difficult to obtain candidate predictors: Charlson comorbidity index, Mini-Mental State Examination (MMSE), handgrip strength, Short Physical Performance Battery (SPPB) and Geriatric Depression Scale-15 and forcing the DSMS-items into the model. In steps 2 and 3, a backward selection procedure was performed. Predictors were retained in the model if their p-value was < 0.157, corresponding with Akaike’s information criterion [22]. No dummy variables were included for the included cohorts. Subsequently, we tested the differences in c-statistics between the models by using the D3 method for pooling Likelihood ratio statistics [23]. The DeLong method was used to obtain the standard error of each imputed dataset and subsequently, the difference in pooled c-statistic was calculated [24].

We internally validated the models using 250 bootstrap samples, which were drawn from the original dataset with missing values and missing values filled in by multiple imputation (m = 20) in every single bootstrap sample. We used shrunk c-statistics, calibration plots (Fig. 4, Additional files 2, 3 and 4), regression slopes and Hosmer-Lemeshow p-values (PHL) to describe discrimination and calibration. Regression coefficients were shrunk by a single shrinkage factor to reduce over-optimism of model performance in new populations [25]. Since two of the data sets were from randomised trials, that used frailty instruments as an inclusion criterion, we tested model calibration on the combined data of the two observational cohorts to ensure application to a more natural target population. We used the psfmi package in R-studio (version 3.6.1) for these analyses. The psfmi package is fully described elsewhere [24].

Results

Population characteristics

In total, 529 patients were included in this study (Fig. 1, Table 2). The mean age was 82 years and 52% were males. Most patients had been admitted for heart failure (51%), 38% had been admitted to the hospital in the previous 6 months and 25% of the included patients had cognitive impairment (MMSE < 24). Regarding the DSMS-score, a positive screening was observed in 45% for the risk of delirium, 41% for fall risk, 37% for functional impairment and 29% for malnutrition. The prevalence’s were 21, 31, 30 and, 19% for a DSMS-score of 0, 1, 2 and 3 or 4, respectively. Figure 2 shows the crude incidence of the composite outcome and readmission and mortality separately at 6 months follow-up. The crude incidences of the composite outcome at 6 months were 32, 41, 46 and 58% in patients with DSMS score 0, 1, 2 and 3 or 4, respectively.

Fig. 1
figure 1

Flowchart

Full size image
Table 2 Baseline characteristics
Full size table
Fig. 2
figure 2

Incidence of adverse outcomes at 6 months follow-up

Full size image

Performance of the DSMS-tool

Table 3 and Fig. 3 show the predictive performance of the three models in predicting readmission or mortality within 6 months. In model 1, including the DSMS only, malnutrition was the strongest predictor (OR 2.29, 95% CI 1.47–3.56). The model discriminated limited (c-statistic 0.61, 95% CI 0.56–0.66) and after internal validation discrimination decreased (c-statistic 0.55).

Table 3 Multivariable analyses and predictive performance for readmission or mortality at six-monthsa
Full size table
Fig. 3
figure 3

Areas under the curve and 95% confidence intervals for predictors of six-month readmission or mortality. Model 1: DSMS delirium, DSMS fall risk, DSMS functional impairment, DSMS malnutrition. Model 2a: sex, admission in the previous 6 months and cardiovascular diagnosis. Model 2b: sex, admission in the previous 6 months and cardiovascular diagnosis + model 1. Model 3: Charlson comorbidity index [26], cardiovascular diagnosis + model 1

Full size image

In model 2a (without the DSMS-items) only sex, admission in the previous 6 months and diagnosis at admission remained in the model. In model 2b, the DSMS-items were added to the predictors in 2a which slightly improved discrimination (c-statistic 0.66, 95% CI 0.61–0.71). The discrimination of model 2b was statistically significantly better than that of model 1 (p = 0.002). In the observational cohorts, the c-statistic of model 2b was 0.57 (95% CI 0.48–0.65), however, the model was well calibrated (corrected slope 0.71, PHL = 0.89) (Additional files 2 and 3).

In model 3, the admission diagnosis and Charlson comorbidity index were selected, which yielded a model c-statistic of 0.69 (95% CI 0.63–0.73), which fell to 0.66 after internal validation. Model 3 discriminated statistically significantly better than model 1 (p < 0.001) and model 2b (p = 0.007). The calibration plot of model 3 is shown in Additional file 4. In the observational cohorts, the discriminative performance was lower (c-statistic 0.58, 95% CI 0.47–0.68) but well calibrated (corrected slope 0.76, PHL = 0.66) as shown in Fig. 4.

Fig. 4
figure 4

Calibration plot of readmission or mortality within 6 months (model 3) in the two observational cohorts

Full size image

Discussion

We examined the performance of the DSMS-tool, alone and combined with other predictors, on all-cause unplanned hospital readmission or mortality within 6 months in older patients acutely hospitalised for a cardiac reason. Our results show that the DSMS-tool’s performance is limited in this population. However, in combination with the diagnosis on admission and the Charlson comorbidity index, reasonably good predictions could be made.

Originally, the DSMS-items were introduced into Dutch hospitals to assess the risk of functional loss in older patients on admission and to selectively deploy interventions to prevent functional loss early [9]. However, the predictive performance has not been studied before implementation in 2012. Heim et al. [27] studied discrimination of the DSMS-tool in predicting the occurrence of a composite outcome of death, high healthcare demand or at least one additional dependency in activities of daily living within 3 months follow-up among acutely and electively hospitalised patients ≥70 years at departments of neurology, urology, surgery and orthopaedics. On external validation in 812 patients (of which 105 only had data on healthcare demand), they found a sensitivity of 0.61 and a specificity of 0.75 (c-statistics 0.68) for the DSMS-tool reinforced by information on age (cut-off at 80 years). Using different methods (cardiac patients, all acutely admitted, six-month composite outcome of readmission or death, multiple imputation of missing values, bootstrapping and shrinkage), we found that discrimination of the DSMS-tool to predict the occurrence of six-month hospital readmission or mortality was much lower (shrunk c-statistic = 0.55). Although the contrasting c-statistics may be explained by the different outcome measures and time window, it could also be explained by differences between the study populations. For example, Heim et al. [27] included both acutely and electively hospitalised patients including a high percentage of surgical and orthopaedic patients, whereas we focussed solely on the acutely hospitalised cardiac population in which a high prevalence of geriatric conditions and comorbidities were found. In addition, more patients in our study were cognitively impaired (MMSE ≤23 21.3% versus 15.9%) [27]. Surprisingly, and despite a fairly wide range of ages in our study, age was not a strong predictor and was not selected in any of the models.

Hermans et al. [28] studied, in a retrospective analysis of routine data, the association between the DSMS-score and the occurrence of mortality or a composite of various complications after a percutaneous coronary intervention within 30 days in patients with ST-elevated myocardial infarction ≥70 years. They found an OR of 9.6 (95%CI 1.6–56.9) for a DSMS-score (≥ 1) to predict 30-day mortality. However, the authors were hindered by the low incidence of mortality (n = 11; 5%) which may have led to severe overfitting of their regression model.

Until now, only few studies have studied the performance of the DSMS-tool. These studies vary in study population, time window, outcomes and methods and are therefore difficult to compare. As a result, more research is needed to study the performance of the DSMS-tool, especially since in the Netherlands its use is compulsory in all patients ≥70 years who are hospitalised. In our analyses, we focussed on the original and routinely used, binary cut-off points within the four domains of the DSMS. For further research, it would be interesting to elucidate the performance of the most efficient subset of these 13 items within the four domains, possibly modeling these as a continuous score.

In addition, it is important to not only identify patients at risk but also act on it, that is, initiate early preventive interventions in those patients indicated by their predicted high risk. As far as we are aware, treatment thresholds, in terms of predicted risk, are seldom specified. Within the DSMS-tool, attention is payed to practical hospital-based interdisciplinary interventions in patients with one or more risk factors present [9]. However, it is known that common geriatric syndromes are often still present 3 months post-discharge [12]. The DSMS recommends transferring risk information to caregivers in primary care. However, more attention may be needed to continue interventions from hospital to home. For example, transitional care interventions contribute to continuity of care across care settings and have been shown to reduce the risk of readmission and mortality in several populations [29, 30].

We conclude that a combination of variables reflecting geriatric conditions (the DSMS-items and the Charlson comorbidity index) and a disease-related factor (diagnosis at admission), led to better predictive performance than a model of the DSMS-items alone. A recent systematic review of risk prediction models in cardiac patients showed that only few studies use geriatric predictors, such as physical performance or dementia, to estimate patients’ probabilities of experiencing an unplanned readmission (van Grootven, submitted). However, models containing geriatric predictors did not seem to predict differently than those without. This may partly be explained by the relatively low mean age in the underlying studies as most studies included patients ≤70 years. This lowers the presence of geriatric syndromes, which may hinder accurate detection of potential predictive capabilities. The SILVER-AMI study included patients ≥75 years and developed risk prediction models for 30 and 180-day readmission [2, 31]. In accordance with our results, they found that a combination of geriatric as well as disease-specific risk factors best predicted the risk of readmission.

Strengths and limitations

In this study we combined data of older cardiac patients of four studies to examine the performance of the DSMS-tool and the contribution of additional variables using rigorous statistical methods. Our study contributes to the evidence on how to identify older cardiac patients at risk of readmission or mortality.

Some limitations should however be considered. First, we examined the performance of the DSMS-tool on the risk estimation of hospital readmission or mortality in older cardiac patients. However, the tool has originally been developed to identify older patients at risk of functional loss. Since functional loss is strongly related to hospital readmission or mortality, testing the performance of the DSMS-tool on these outcomes is considered plausible [10, 11]. Second, no c-statistics for readmission and mortality as separate outcomes were reported due to the limited number of events per outcome. However, the outcome-specific preventive options to be considered after high-probability predictions of readmission or death from a model are comparable [32]. Third, while we were able to select a broad range of geriatric predictors, some important medical (disease-specific) predictors (e.g. left ventricular ejection fraction, and stage of disease (NYHA)) were not available. Information on these tests is usually not available on hospital admission (and in our four cohorts) and were therefore not included in our model which focusses on the early admission phase. However, data about the disease history and comorbidities may be available at hospital admission. For example, the presence of specific comorbidities such as renal failure, diabetes [33, 34] or chronic obstructive pulmonary disease [2, 31] are known to increase the risk of adverse outcomes and may be of additional value in future risk prediction models for older cardiac patients. Fourth, in the two intervention cohorts a selected subgroup of 87% frail older cardiac patients according to the DSMS-tool was included, compared to 44% in the two observational cohorts. We therefore performed a second internal validation process on the two observational cohorts to reflect model performance in a hospitalised older cardiac patient population representative of that encountered in clinical practice. Last, despite rigorous steps taken to assess the internal validity of our models, an additional external validation in independent datasets is recommended to examine the generalisability of our results.

Conclusion

The DSMS-tool alone has limited capacity to accurately estimate the risk of readmission or mortality in hospitalised older cardiac patients. Adding disease-specific risk factor information to the DSMS-tool resulted in a moderately performing model. To optimise the early identification of older hospitalised cardiac patients at risk, the combination of geriatric and disease-specific predictors should be further explored.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

CI:

Confidence interval

DSMS:

Dutch Safety Management System

HL:

Hosmer-Lemeshow

IQR:

Interquartile range

MMSE:

Mini-Mental State Examination

OR:

Odds ratio

RCT:

Randomised controlled trial

SD:

Standard deviation

SPPB:

Short Physical Performance Battery

References

  1. Khera R, Wang Y, Bernheim SM, Lin Z, Krumholz HM. Post-discharge acute care and outcomes following readmission reduction initiatives: national retrospective cohort study of Medicare beneficiaries in the United States. BMJ. 2020;368:l6831.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Dodson JA, Hajduk AM, Murphy TE, Geda M, Krumholz HM, Tsang S, et al. Thirty-day readmission risk model for older adults hospitalized with acute myocardial infarction. Circ Cardiovasc Qual Outcomes. 2019;12(5):e005320. https://0-doi-org.brum.beds.ac.uk/10.1161/CIRCOUTCOMES.118.005320.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Jepma P, Ter Riet G, van Rijn M, Latour CHM, Peters RJG, WJM SOR, et al. Readmission and mortality in patients >/=70 years with acute myocardial infarction or heart failure in the Netherlands: a retrospective cohort study of incidences and changes in risk factors over time. Neth Heart J. 2019;27(3):134–41. https://0-doi-org.brum.beds.ac.uk/10.1007/s12471-019-1227-4.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Gorodeski EZ, Goyal P, Hummel SL, Krishnaswami A, Goodlin SJ, Hart LL, et al. Domain management approach to heart failure in the geriatric patient: present and future. J Am Coll Cardiol. 2018;71(17):1921–36. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jacc.2018.02.059.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Bell SP, Orr NM, Dodson JA, Rich MW, Wenger NK, Blum K, et al. What to expect from the evolving field of geriatric cardiology. J Am Coll Cardiol. 2015;66(11):1286–99. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jacc.2015.07.048.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Vitale C, Jankowska E, Hill L, Piepoli M, Doehner W, Anker SD, et al. Heart Failure Association/European Society of Cardiology position paper on frailty in patients with heart failure. Eur J Heart Fail. 2019;21(11):1299–305. https://0-doi-org.brum.beds.ac.uk/10.1002/ejhf.1611.

    Article  PubMed  Google Scholar 

  7. Boyd C, Smith CD, Masoudi FA, Blaum CS, Dodson JA, Green AR, et al. Decision making for older adults with multiple chronic conditions: executive summary for the American Geriatrics Society guiding principles on the care of older adults with multimorbidity. J Am Geriatr Soc. 2019;67(4):665–73. https://0-doi-org.brum.beds.ac.uk/10.1111/jgs.15809.

    Article  PubMed  Google Scholar 

  8. Zão A, Magalhães S, Santos M. Frailty in cardiovascular disease: screening tools. Rev Port Cardiol. 2019;38(2):143–58. https://0-doi-org.brum.beds.ac.uk/10.1016/j.repc.2018.05.019.

    Article  PubMed  Google Scholar 

  9. Dutch Safety Management Program. Practical guide for frail older patients [in Dutch]. Place unknown: Dutch Safety Management Program; 2009.

  10. van Seben R, Covinsky KE, Reichardt LA, Aarden JJ, van der Schaaf M, van der Esch M, et al. Insight into the posthospital syndrome: a 3-month longitudinal follow up on geriatric syndromes and their association with functional decline, readmission, and mortality. J Gerontol A Biol Sci Med Sci. 2020;75(7):1403–10. https://0-doi-org.brum.beds.ac.uk/10.1093/gerona/glaa039.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Koch D, Kutz A, Haubitz S, Baechli C, Gregoriano C, Conca A, et al. Association of functional status and hospital-acquired functional decline with 30-day outcomes in medical inpatients: a prospective cohort study. Appl Nurs Res. 2020;54:151274. https://0-doi-org.brum.beds.ac.uk/10.1016/j.apnr.2020.151274.

    Article  PubMed  Google Scholar 

  12. van Seben R, Reichardt LA, Aarden JJ, van der Schaaf M, van der Esch M, Engelbert RHH, et al. the course of geriatric syndromes in acutely hospitalized older adults: the hospital-ADL Study. J Am Med Dir Assoc. 2019;20(2):152–158.e2.

    Article  PubMed  Google Scholar 

  13. Buurman BM, Hoogerduijn JG, van Gemert EA, de Haan RJ, Schuurmans MJ, de Rooij SE. Clinical characteristics and outcomes of hospitalized older patients with distinct risk profiles for functional decline: a prospective cohort study. PLoS One. 2012;7(1):e29621. https://0-doi-org.brum.beds.ac.uk/10.1371/journal.pone.0029621.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Flierman I, van Rijn M, Willems DL, Buurman BM. Usability of the surprise question by nurses to identify 12-month mortality in hospitalized older patients: a prospective cohort study. Int J Nurse Stud. 2020;109:103609. https://0-doi-org.brum.beds.ac.uk/10.1016/j.ijnurstu.2020.103609.

    Article  Google Scholar 

  15. Buurman B, Parlevliet J, Allore H, Blok W, van Deelen B, Moll van Charante E, et al. Comprehensive geriatric assessment and transitional care in acutely hospitalized patients - the transitional care bridge randomized clinical trial. JAMA Intern Med. 2016;176(3):302–9. https://0-doi-org.brum.beds.ac.uk/10.1001/jamainternmed.2015.8042.

    Article  PubMed  Google Scholar 

  16. Verweij L, Jepma P, Buurman BM, Latour CHM, Engelbert RHH, Ter Riet G, et al. The cardiac care bridge program: design of a randomized trial of nurse-coordinated transitional care in older hospitalized cardiac patients at high risk of readmission and mortality. BMC Health Serv Res. 2018;18(1):508. https://0-doi-org.brum.beds.ac.uk/10.1186/s12913-018-3301-9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Katz S, Ford A, Moskowitz R, Jackson B, Jaffe M. Studies of illness in the aged. The index of Adl: a standardized measure of biological and psychosocial function. JAMA. 1963;185:914–9.

    Article  CAS  PubMed  Google Scholar 

  18. Kruizenga HM, Seidell JC, de Vet HC, Wierdsma NJ, Van Bokhorst-de van der Schueren MA. Development and validation of a hospital screening tool for malnutrition: the short nutritional assessment questionnaire (SNAQ). Clin Nutr. 2005;24(1):75–82. https://0-doi-org.brum.beds.ac.uk/10.1016/j.clnu.2004.07.015.

    Article  CAS  PubMed  Google Scholar 

  19. Government of the Netherlands. What information is in the Personal Records Database? Available at: https://www.government.nl/topics/personal-data/question-and-answer/what-information-is-in-the-personal-records-database. Accessed 02 Jul 2020.

  20. Held U, Kessels A, Garcia Aymerich J, Basagaña X, Ter Riet G, Moons KG, et al. Methods for handling missing variables in risk prediction models. Am J Epidemiol. 2016;184(7):545–51. https://0-doi-org.brum.beds.ac.uk/10.1093/aje/kwv346.

    Article  PubMed  Google Scholar 

  21. Heymans M, Eekhout I. Chapter 13. Pooling methods for categorical variables. Available at: https://bookdown.org/mwheymans/bookmi/pooling-methods-for-categorical-variables.html. Accessed 04 Feb 2021.

  22. Sauerbrei W. The use of resampling methods to simplify regression models in medical statistics. J R Stat Soc Ser. 1999;48(3):313–29. https://0-doi-org.brum.beds.ac.uk/10.1111/1467-9876.00155.

    Article  Google Scholar 

  23. Meng X, Rubin D. Performing likelihood ratio tests with multiply-imputed data sets. Biometrica. 1992;29:103–11.

    Article  Google Scholar 

  24. Heymans M. Psfmi package. Available at: https://mwheymans.github.io/psfmi/index.html. Accessed 20 Jan 2021.

  25. Van Houwelingen JC, Le Cessie S. Predictive value of statistical models. Stat Med. 1990;9(11):1303–25. https://0-doi-org.brum.beds.ac.uk/10.1002/sim.4780091109.

    Article  PubMed  Google Scholar 

  26. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83. https://0-doi-org.brum.beds.ac.uk/10.1016/0021-9681(87)90171-8.

    Article  CAS  PubMed  Google Scholar 

  27. Heim N, van Fenema EM, Weverling-Rijnsburger AW, Tuijl JP, Jue P, Oleksik AM, et al. Optimal screening for increased risk for adverse outcomes in hospitalised older adults. Age Ageing. 2015;44(2):239–44. https://0-doi-org.brum.beds.ac.uk/10.1093/ageing/afu187.

    Article  PubMed  Google Scholar 

  28. Hermans MPJ, Eindhoven DC, van Winden LAM, de Grooth GJ, Blauw GJ, Muller M, et al. Frailty score for elderly patients is associated with short-term clinical outcomes in patients with ST-segment elevated myocardial infarction treated with primary percutaneous coronary intervention. Neth Heart J. 2019;27(3):127–33. https://0-doi-org.brum.beds.ac.uk/10.1007/s12471-019-1240-7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Naylor MD, Shaid EC, Carpenter D, Gass B, Levine C, Li J, et al. Components of comprehensive and effective transitional care. J Am Geriatr Soc. 2017;65(6):1119–25. https://0-doi-org.brum.beds.ac.uk/10.1111/jgs.14782.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Naylor MD, Aiken LH, Kurtzman ET, Olds DM, Hirschman KB. The care span: The importance of transitional care in achieving health reform. Health Aff. 2011;30(4):746–54 P.747.

    Article  Google Scholar 

  31. Dodson JA, Hajduk AM, Murphy TE, Geda M, Krumholz HM, Tsang S, et al. 180-day readmission risk model for older adults with acute myocardial infarction: the SILVER-AMI study. Open Heart. 2021;8(1):e001442. https://0-doi-org.brum.beds.ac.uk/10.1136/openhrt-2020-001442.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Le Berre M, Maimon G, Sourial N, Gueriton M, Vedel I. Impact of transitional care services for chronically ill older patients: a systematic evidence review. J Am Geriatr Soc. 2017;65(7):1597–608. https://0-doi-org.brum.beds.ac.uk/10.1111/jgs.14828.

    Article  PubMed  Google Scholar 

  33. Keenan PS, Normand SL, Lin Z, Drye EE, Bhat KR, Ross JS, et al. An administrative claims measure suitable for profiling hospital performance on the basis of 30-day all-cause readmission rates among patients with heart failure. Circ Cardiovasc Qual Outcomes. 2008;1(1):29–37. https://0-doi-org.brum.beds.ac.uk/10.1161/CIRCOUTCOMES.108.802686.

    Article  PubMed  Google Scholar 

  34. Krumholz HM, Lin Z, Drye EE, Desai MM, Han LF, Rapp MT, et al. An administrative claims measure suitable for profiling hospital performance based on 30-day all-cause readmission rates among patients with acute myocardial infarction. Circ Cardiovasc Qual Outcomes. 2011;4(2):243–52. https://0-doi-org.brum.beds.ac.uk/10.1161/CIRCOUTCOMES.110.957498.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank all researchers and patients of the Hospital-ADL study, the Surprise Question cohort, the Transitional Care Bridge study and the Cardiac Care Bridge study.

Funding

This work was supported by the Dutch Research Council (NWO) [023.009.036 [PJ] and 023.008.024 [LV]]. The funder had neither a role in study design, data collection and analysis, nor in the preparation or publication of the manuscript.

Author information

Authors and Affiliations

  1. Department of Cardiology, Amsterdam UMC, Amsterdam, the Netherlands

    Patricia Jepma, Lotte Verweij, Ron J. G. Peters, Wilma J. M. Scholte op Reimer & Gerben ter Riet

  2. Centre of Expertise Urban Vitality, Faculty of Health, Amsterdam University of Applied Sciences, Amsterdam, the Netherlands

    Patricia Jepma, Lotte Verweij, Arno Tijssen, Corine H. M. Latour, Bianca M. Buurman & Gerben ter Riet

  3. Department of Epidemiology and Data Science, Amsterdam UMC, Amsterdam, the Netherlands

    Martijn W. Heymans

  4. Department of Internal Medicine, section of Geriatric Medicine, Amsterdam UMC, Amsterdam, the Netherlands

    Isabelle Flierman & Bianca M. Buurman

  5. Research Group Chronic Diseases, HU University of Applied Sciences Utrecht, Utrecht, the Netherlands

    Wilma J. M. Scholte op Reimer

Authors
  1. Patricia Jepma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lotte Verweij
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Arno Tijssen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Martijn W. Heymans
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Isabelle Flierman
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Corine H. M. Latour
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ron J. G. Peters
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Wilma J. M. Scholte op Reimer
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Bianca M. Buurman
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Gerben ter Riet
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors have made substantial contributions to the conception and design of the work. PJ, LV, AT, MW and GtR were responsible for the acquisition, analysis and interpretation of the data. PJ, LV and GtR have drafted the manuscript. AT, MH, IF, CL, RP, WS and BB substantively revised the manuscript. All authors have approved the submitted version. All authors have agreed both to be personally accountable for the author’s own contributions and have ensured that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature.

Corresponding author

Correspondence to Patricia Jepma.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

None declared.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Frequency of missing data per variable in the four cohorts.

Additional file 2:

Supplemental Figure 1. Calibration plot of readmission or mortality within 6 months (model 2b) in 250 bootstrapped samples.

Additional file 3:

Supplemental Figure 2. Calibration plot of readmission or mortality within 6 months (model 2b) in the two observational cohorts.

Additional file 4:

Supplemental Figure 3. Calibration plot of readmission or mortality within 6 months (model 3), in 250 bootstrapped samples.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jepma, P., Verweij, L., Tijssen, A. et al. The performance of the Dutch Safety Management System frailty tool to predict the risk of readmission or mortality in older hospitalised cardiac patients. BMC Geriatr 21, 299 (2021). https://0-doi-org.brum.beds.ac.uk/10.1186/s12877-021-02243-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12877-021-02243-5

Keywords

BMC Geriatrics

ISSN: 1471-2318

Contact us