A Nomogram to Predict the Probability of Relapse during Hospitalization after Minor Ischemic Stroke in Chinese Patients.


 Background: Predicting the risk of recurrence during hospitalization in patients with minor ischemic stroke (MIS) is of great significance for clinical and treatment. Compared with early models and prognostic scores, nomogram is a better visualization tool for predicting clinical outcomes. It combines different factors to develop a graphical continuous scoring system, and accurately calculates the risk probability of adverse outcomes based on individual characteristics. Our goal is to develop and validate a nomogram for individualized prediction of hospitalization recurrence in patients with mild ischemic stroke in the Chinese population.Methods: Based on retrospective collection, a single center study was conducted in the first affiliated Hospital of Anhui Medical University from January 2014 to December 2019. The subjects were stroke patients with NIHSS≤5.In order to generate the nomogram, age, systolic blood pressure,previous heart disease, serum total bilirubin, ferritin and smoking were integrated into the model. The predictive accuracy of the nomogram model to predict the probability of unfavorable outcome was assessed by calculation of the area under the receiver operating characteristic curve (AUC–ROC). Calibration of the risk prediction model was assessed by the plot comparing the observed probability of unfavorable outcome against the predicted, and by using the Hosmer–Lemeshow test.Results: Age at admission (OR,0.946; 95% CI,-0.002 to 0.048), SBP (OR,0.012,95%CI,0.000 to 0.024), previous heart disease (OR,0.867,95%CI, 0.084 to 1.651), UA (OR,-0.003,95%CI,-0.006 to 0.001), serum total bilirubin (OR,-0.022,95%CI,-0.036 to -0.008), ferritin (OR,0.004,95%CI, 0.002 to 0.005), smoking (OR,0.494,95%CI,-0.115 to 1.103) are significant predictors of in-hospital recurrence in Chinese patients with minor ischemic stroke.The model shows good discrimination, the AUC-ROC value is 0.737 (95%CI:0.676-0.798), and has perfect calibration performance. Calibration was good (p=0.1457 for the Hosmer-Lemesshow test), which could predict the risk of recurrence of MIS patients during hospitalization.Conclusion: The nomogram developed and validated in this study can provide individualized, intuitive and accurate prediction of recurrence in Chinese patients with minor ischemic stroke during hospitalization.

One out of every three patients with acute ischemic cerebrovascular disease is MIS. While symptoms of MIS is minor, its prognosis is not optimistic. Studies [3,4] have shown that the risk of recurrent stroke is up to 10% in the week after a transient ischaemic attack (TIA) or minor stroke. It has been shown that 25% ~ 50% of MIS patients still leave varying degrees of disability at 90 days after onset. [5,6] In-depth understanding of factors affecting the recurrence of MIS will not only help to predict the prognosis of patients, but also provide a feasible reference for the clinical management and treatment of MIS. With the increase in the number of the elderly and the extension of life expectancy, the number of patients with disabilities caused by MIS is expected to further increase in the future. Therefore, it is of great signi cance to be able to establish a formal statistical model and provide a formal prediction of adverse outcomes after recurrence of MIS during hospitalization combined with clinical experience.
Disease prediction is an important part of medicine. Nomogram is considered as a means to improve disease prediction.it is an visualized graphic statistical tool, which is used to calculate the continuous probability of a speci c outcome of a single patient, and can provide an estimated numerical prognosis for each patient; it can combine different data to form a continuous scoring system to predict the risk of the individual. Nomogram is an important part of modern medical decision-making. it has become a risk strati cation tool used in routine clinical practice, including cancer and medical and surgical diseases. [7,8] This retrospective investigation did not require individual consent based on the institutional guidelines for waiving consent, was performed according to local ethical committee regulation, in accord with the Helsinki declaration, and was inally cleared by the Institutional Review Board as no-risk retrospective study.

Statistical Analysis
Mann-Whitney U test was used for bivariate analysis of continuous and ordered distribution variables.
Fisher's exact test was used for bivariate analysis of classi ed variables. The nomogram model was generated to predict the probability of recurrence during hospitalization in patients with mild ischemic stroke in study population.In univariate analysis, all variables with probability<0.20 entered the selection range, and the likelihood ratio test with Akaike's information criterion (AIC) [10]as the termination rule was used for a forward stepwise method, and the predictive variables with non-zero coe cient characteristics were screened out by lasso regression to compare the results.Regression coe cients and odds ratios (OR) with twosided 95% con dence intervals (CI) for each of the variables included in the models were nally calculated.
The nomogram enables discrimination of patients with favorable outcome from those with unfavorable outcome, the predictive accuracy of the nomogram model was assessed by calculation of the area under curve (AUC) of the receiver-operating characteristic. The model confusion matrix is listed and the error rate is calculated. The R package used in the study is reported in "Supplemental Information".All tests were two sided and P < 0.05 was considered statistically signi cant.
The statistical analysis was carried out using the statistical software package R version 3.6.2 (R Foundation for Statistical Computing; https://www.r-project.org/) and STATA 15.0 (StataCorp Texas, USA).

Construction and validation of a nomogram
We included the patients in the rst four years (2014-2017) as the training cohort and the patients in the last two years (2018-2019) as the validating cohort.The nomogram is constructed by using the regression coe cient obtained from the multivariable Logistic regression model and the RMS package in the R software. Nomogram is a graphical representation of complex mathematical formulas. Medical nomogram uses laboratory tests and clinical variables such as blood glucose, uric acid and patient age to graphically describe statistical prognostic models that generate the probability of clinical events (such as disease recurrence or death) for a given individual.Rapid calculation through an intuitive digital interface, coupled with higher accuracy and easier to understand prognosis, can seamlessly combine the prognosis obtained by nomogram to help clinical decision-making.
The performance of the model is evaluated by discrimination (the ability of the proposed model to distinguish patients with different outcomes) and calibration (the relative distance between the prediction and the actual results).The predictive accuracy of the nomogram model to predict the probability of unfavorable outcome was assessed by calculation of the area under the receiver operating characteristic curve (AUC-ROC). Calibration of the risk prediction model was assessed by the plot comparing the observed probability of unfavorable outcome against the predicted, and by using the Hosmer-Lemeshow test.Calibration was carried out using a calibration plot, in which the predicted probabilities were plotted against the frequency of the observed unfavorable outcome.The prediction of a well-calibrated model should be mirrored by a 45°diagonal line.

Results
As a result, the nal study population included 1234 patients whose clinical features, demographic and laboratory data were shown in Table 1.All the 27 clinical features, demographic and laboratory data were reduced to 7 potential predictors on the basis of information of patients in the training cohort ( Figure 1A and 1B). These features were nonzero coe cients in the LASSO logistic regression model. Finally, the predictive variables screened by lasso regression are highly consistent with the stepwise regression method, and the variables of seven constructing models were obtained( Table2  included into the multiple logistic regression models (P < 0.2) and Additionally traditional stroke risk factors such as Hypertension, diabetes and cardiopathy were added into the model. The nomogram was created by assigning a graphic preliminary score to each of the seven predictors with a point range from 0 to 10, which was then summed to generate a total score, nally converting it into the individual risk of recurrence of MIS patients during hospitalization (from 0% to 100%).The higher total score of nomogram, the higher possibility of adverse outcome. The AUC-ROC of the Nomogram of the training cohort is 0.737 (95%CI 0.676-0.798) (Figure 2). In the validating cohort, the AUC-ROC value is 0.706 (95%CI 0.532-0.881) (Figure 3).The confusion matrix of the model in the training cohort is shown in the gure , and the error rate is 0.06; the confusion matrix of the model in the validating cohort is shown in the gure, and the error rate is 0.015 (Table 3).The calibration curve of the nomogram model shows that there is su cient consistency between the predicted values calculated by the nomogram and the actual results. Hosmer-Lemesshow goodness-of-t test showed that the nomogram was well corrected (p=0.1457), which could predict the risk of recurrence of MIS patients during hospitalization.

Discussion
Studies have shown that [11], among MIS patients, the risk of stroke recurrence within 90 days is 10%-19% (average 15%) while the risk of recurrence of acute ischemic stroke within 90 days is 2%-7% (average 5%), which is signi cantly lower than that of MIS,MIS is a relatively urgent emergency. More importantly, although MIS attacks do not usually lead to immediate functional or cognitive decline, patients have a signi cantly increased risk of ischemic events in the future, more frequently in the days or weeks after the initial relief of symptoms [12].Because of these evidences, post-MIS disability has now became a public health care problem. Therefore, it is of great signi cance to propose a more accurate and valuable predictive model through the early prevention and identi cation of MIS, through reliable predictors.
This is the rst time that the probability of adverse events which can better predict the recurrence of Chinese MIS patients during hospitalization was proposed through the comprehensive in uence of a variety of variables (such as age and total bilirubin, previous heart disease and smoking, etc.).That is, age, SBP, previous heart disease, UA, SBIL, ferritin, and smoking were independently associated with the incidence of cerebral infarction in previous studies. Therefore, we developed and validated a nomogram containing these seven variable combinations, which can relatively accurately predict the probability of relapse during the hospitalization of MIS patients (AUC=0.706).
When we use R to generate nomogram, we nd an interesting phenomenon. When inputing seven variables into R software, we get the original version of nomogram ( Figure 4).But when we interacted between age and a previous heart disease, the regenerated nomogram ( Figure 5), results caught our eyes.
Under the condition of previous heart disease (cardiopathy=0 or cardiopathy=1), age is no longer simply signi cantly positively correlated with the recurrence of MIS, but the younger patients with heart disease, the higher incidence of adverse events. A study of atrial brillation may explain our results, which showed that cerebral infarction wasclosely related to cardiovascular disease and was the most common cause of cardiovascular death. However, all-cause mortality was the highest in young patients with atrial brillation and decreased with age. The effect of atrial brillation on mortality varies with age. The absolute mortality (death toll) is lower in young patients with atrial brillation, but the effect of atrial brillation on mortality is higher than that in elderly patients. In contrast, the absolute mortality rate of elderly patients with atrial brillation is higher, although standardized mortality ratio(SMR) is observed to be lower than that of young patients with atrial brillation, indicating that the effect of atrial brillation on mortality decreases with age. As individuals without atrial brillation get older, they will develop complications other than atrial brillation, which may increase mortality and thus weaken the impact of atrial brillation on patients. [13]There is also a warning role to remind young people at risk of high-risk diseases that they should pay more attention to prevention, actively improve lifestyle behavior and seek medical treatment in time in daily life, and preventive drug treatment if necessary. Blood pressure (BP) level is closely related to stroke mortality. We already know that early antihypertensive therapy is bene cial to patients with acute intracranial hemorrhage and ischemic stroke. [14]Tan's study also proved that systolic blood pressure can be used as an independent risk factor for MIS. [15]Some studies have shown that BP can distinguish between MIS and non-cerebrovascular events in the population, and early and accurate diagnosis of MIS is particularly important to reduce the risk of recurrence and disability. [16]We found that elevated SBP can be used as a warning indicator by physicians in the short term, as our nomogram shows, the higher the SBP, the higher the corresponding score, and the higher the total score, indicating a higher risk of the disease. This reminds doctors to further examine the neurological symptoms and imaging features of patients in order to make treatment decisions in a timely manner.
Previous studies have shown that the level of serum uric acid is an independent protective factor for cognitive impairment in patients with MIS, and the lower the level of uric acid is, the more disadvantageous it is to the cognitive function of patients. [17]Serum uric acid is a good antioxidant and will increase rapidly after acute cerebral infarction.An animal study shows that in the establishment of rat middle cerebral artery occlusion model, apoptosis and brain tissue injury will produce reactive oxygen species, and after the addition of appropriate high concentration of uric acid, the degree of brain tissue injury is reduced and the production of reactive oxygen species is reduced. However, low concentration uric acid can not improve the cell survival rate, so in a certain range, the increase of uric acid level has a certain neuroprotective effect. [18]Our study shows that there is a negative correlation between high concentration of uric acid and recurrence of MIS, which further con rms this view. The clinical studies of Zhang et al have shown that UA plays a neuroprotective role in acute ischemic stroke, and appropriate high concentration of uric acid is bene cial to the prognosis of adolescent stroke. [19]The meta-analysis of Wang supports that serum uric acid level has a protective effect on neurological prognosis after acute ischemic stroke, and high uric acid level at onset is a biomarker with better prognosis in patients with acute ischemic stroke. [20]Therefore, an appropriate high level of serum uric acid can prevent the recurrence of MIS.
It can be seen from nomogram that ferritin is closely related to the progression of the disease. Ferritin is an iron-binding plasma protein. It is also an acute phase reactant that increases after proin ammatory stimulation. The high concentration of plasma ferritin is related to clinical deterioration. This suggests that the excitotoxicity of iron free radicals may be one of the reasons for the progression of the disease.
Some studies have pointed out that the increase of ferritin concentration in plasma and cerebrospinal uid within 24 hours after the onset of ischemic stroke is related to the deterioration of early neurological function, and the increase of iron reserve may lead to stroke progression by enhancing the cytotoxic mechanism of cerebral ischemia. This study is consistent with our ndings, that is, there is a positive correlation between ferritin and recurrence of MIS. [21] Bilirubin, as an antioxidant, can always oxidize lipids and lipoproteins and play a certain role in preventing atherosclerosis. Bilirubin reduces oxidative stress by scavenging free radicals, thus improving endothelial dysfunction. The improvement of endothelial function will prevent the progression of atherosclerosis. Therefore, higher physiological levels of serum bilirubin have a protective effect on vascular events [22]. Studies have shown that higher levels of total bilirubin are associated with lower risk of asymptomatic cerebral infarction. [24] Tan et al [17]also proved that total bilirubin can be used as an independent risk factor for MIS. Indeed, as nomogram shows, it is reasonable that higher levels of bilirubin can prevent the recurrence of cerebral infarction to some extent.
The view that smoking is harmful to vascular health has been around for a long time. The impact of smoking on the disease is mainly manifested in three points: the rst is the thickening and stiffness of the blood vessel wall; the second is the slow blood ow; and the third is the increase of blood viscosity. [25]Smoke also contains a lot of nicotine, these toxic substances will cause the body to increase the secretion of adrenaline and thyroxine, can make the heartbeat faster, blood pressure rise, resulting in adverse consequences. Quitting smoking as soon as possible plays an important role in preventing the recurrence of mild stroke. Quitting smoking is not only the most economical intervention to reduce the risk of stroke, but also one of the important measures to prevent cerebrovascular disease. Therefore, smoking is also one of the important risk factors for MIS. [26] Nomogram is an important part of modern medical decision-making. The well-constructed nomogram is designed to answer a focus question, which, when properly explained and applied, can be of great value to clinicians and patients. However, they must undergo rigorous scrutiny and need to be aware of their performance and limitations before using them in clinical decisions. Only in this way, nomogram can better predict patients health outcomes. Despite these limitations, as far as we know, this is the rst attempt to construct and verify a nomogram to predict the possibility of recurrence of MIS.
We have to admit that the current research also has some limitations. First, this is an inherent weakness of any single-center exploratory study, such as collection and entry bias, and possible residual confusion, whose clinical value may be diminished by aws in its experimental design. Secondly, these methods can prevent over-interpretation of data, but can not eliminate all deviations caused by over tting that may be inherent in the selection of variables and thresholds. Third, the group of patients with limited race or geographical area has always been a concern in the development of predictive models. [27]In consequence, external veri cation, especially in completely different patient cohorts, is necessary. [28]

Conclusion
Before hospitalization of MIS patients, we use this nomogram for individual risk assessment to identify high-risk patients and provide them or their relatives with more detailed risk information to support sound decision-making.In the future, a multicenter clinical test should be conducted to verify whether the model can help patients prevent adverse events and promote their understanding.This is essential to provide a reasonable method of patient management and to help patients and their families understand the process of the disease, and clinicians and patients can also take more necessary measures in lifestyle monitoring and medical intervention. This retrospective investigation did not require individual consent based on the institutional guidelines for waiving consent, was performed according to local ethical committee regulation, in accord with the Helsinki declaration, and was inally cleared by the Institutional Review Board as no-risk retrospective study.

Consent for publication
Not applicable.

Availability of data and materials
The datasets generated and analysed during the current study are not publicly available due to protection of patients' privacy but are available from the corresponding author on reasonable request.

Funding
This research received grant from National Natural Science Foundation of China (No. 81771154).The funding bodies played no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.