A novel method for handling missing data in health care real-world study: Optimal Intact Subset Method

doi:10.21203/rs.3.rs-1761573/v1

Download PDF

Research Article

A novel method for handling missing data in health care real-world study: Optimal Intact Subset Method

https://doi.org/10.21203/rs.3.rs-1761573/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this older preprint version

Read the latest preprint version →

Background

Handling missing data is indispensable in health care real-world data processing. Deleting or imputing missing data may introduce error or lead to multicollinearity. Therefore, we tried to explore a novel missing data processing method to avoid the above issues.

Method

By exploring an optimal deleting way of columns and rows with missing data, we developed a missing data processing method which can retain most information of original datasets. Traditionally, the goal can be realized by traversing all possible deleting combinations. But the computational cost is too high to use in large datasets. Therefore, we established an Optimal Intact Subset Method (OIS.Method) by using an indicator containing missing information of both columns and rows to determine an optimal deleting order of columns. OIS.Method can ascertain the optimal deleting way and simplify computing meanwhile. In order to validate the effectiveness of OIS.Method, we compared OIS.Method with five other data-imputation methods in 700 classification datasets (simulated datasets 1) generated by computer. In order to simulate real-world datasets, we generated simulated datasets 2: introducing redundant variables in simulated datasets 1. We also compared OIS.Method with control methods on that. Finally, we validated OIS.Method in two real-world classification tasks: 1. predict the risk of hypotension during dialysis, 2. predict the risk of drug adverse reaction in elderly patients with type 2 diabetes.

Results

In simulated datasets 1, we found that OIS.Method performed well when the distribution of missing data was unbalanced among columns. In simulated datasets 2, the comprehensive performance of OIS.Method was better in all evaluating dimensions. In two real-world datasets, OIS.Method could acquire better classification performance. We used the area under ROC curve (AUC) to evaluate it: OIS.Method VS Simple Impute VS Random Forest VS Modified Random Forest, 0.8179 ± 0.0005VS0.8116 ± 0.0002VS0.8087 ± 0.0009VS0.8093 ± 0.0014 in task1, and 0.7028VS0.6963VS0.6957VS0.6699 in task2.

Conclusions

Our study provided a novel method for handing missing data in real-world study. Compared with other existing missing data processing methods, the calculation of OIS.Method is smaller, and OIS.Method can reflect the true data situation of original datasets. Moreover, OIS.Method is well-suited for real-world datasets with large sample size and multiple variables.

Missing data

real-world study

data preprocessing

missing data imputation

Missing data is inevitable in medical real-world study ^[1]. The reasons could include: ⑴Patients refused to answer sensitive questions. ⑵Patients were lost to follow up ^[2]. ⑶Doctors did not prescribe certain expensive screening examinations. ⑷Mechanical failures or mistakes of investigators ^[3].

Missing data will cause negative effects on statistical analysis: ⑴As the amount of efficient data decreases, the accuracy of statistical analysis will degrade. ⑵Some hypothesis testing based on complete data will not be suittable. ⑶Missing data will affect the interpretation of statistical analysis results, and undermine the effectiveness of conclusion^[2].

Therefore, we should first handle missing data. Currently, the commonly-used missing data processing methods mainly include: ⑴Delete samples with missing values^[4]. ⑵Imputation: ①Impute fixed values: replace missing data with mean, median, or mode^[5]. ②Impute predicted values: impute values predicted on the base of observed values, such as regression predicting^[6]. Multiple imputation is based on predicted imputing. It will impute several possible values for each missing value, and then select the optimal one as the final result^[7].

Above methods, however, have following problems: ⑴The patterns of missingness can usually be nonrandom, and the distribution of missingness depends on missing values themselves. ⑵Imputing fixed values doesn’t take the influence of other variables on the missing data into consideration. ⑶Imputing predicted values may increase the correlation between independent variables, and introduce multicollinearity. ⑷Computational resources will be wasted when lots of sparse variables were imputed. ⑸Imputing other values may introduce error, and mask the actual regularity of original datasets.

In general, real-world datasets have many redundant variables, the distribution of missingness is unbalanced and the amount of data is large. So, it is necessary to explore a new method for the real-world study. We have developed a new method which can find an optimal intact subset from original dataset (Optimal Intact Subset Method, OIS.Method). In this article, we described the detailed procedures of OIS.Method and validated OIS.Method in computer-generated datasets and real-world datasets.

Algorithm

Complete subsets can be obtained by deleting rows and columns with missing data. Different deleting ways will generate different intact subsets. We can use the optimal intact subset containing most rows and columns in the downstream statistical analysis.

1. Find the optimal intact subset by traversing method

For an original dataset, set the number of rows = n, the number of columns = f. Take dataset A in Fig. 1 with n = 8 and f = 4 for an example. Symbols of ‘V’ represented observed values, and blank cells represented missing values. If we delete X4 and corresponding rows with missing values, the subset contains four rows: n2, n5, n6, n8, three columns: X1, X2, X3 and 12 figures. Similarly, if we delete X1, X2, the intact subset contains two rows: n2, n8, two columns: X3, X4 and four figures. Deleting different column(s), the retention data of intact subset is different. (See Table 1 for further details). Through traversing all deleting methods in Table 1, we can find that deleting X4 is the best way because the subset has the most reserved figures.

2. The derivation of the total number of deleting ways

Traversing method has a fatal shortcoming: high computational cost. In this example, there were \({N=C}_{4}^{1}+{C}_{4}^{2}+{C}_{4}^{3}+{C}_{4}^{4}={2}^{4}=16\) kinds of deleting ways (Table 1).

Table 1

All deleting ways of dataset A in Fig. 1
The number of deleted columns	The combination of deleted columns	The number of reserved rows	The number of reserved columns	The number of reserved figures
0	none	2	4	8
1	X1	2	3	6
	X2	2	3	6
	X3	2	3	6
	X4	4	3	12
2	X1、X2	2	2	4
	X1、X3	2	2	4
	X1、X4	4	2	8
	X2、X3	2	2	4
	X2、X4	4	2	8
	X3、X4	5	2	10
3	X1、X2、X3	2	1	2
	X1、X2、X4	5	1	5
	X1、X3、X4	5	1	5
	X2、X3、X4	6	1	6
4	X1、X2、X3、X4	0	0	0
There were 16 kinds of deleting ways, among which deleting X4 is the best way.

Generally, for a dataset with f columns, there will be N = 2^f kinds of deleting ways. The times of traversing will increase exponentially with the increase of f value. For example, when f = 12, \(N={2}^{12}=4096\), when f = 50, \(N={2}^{50}=1.1\times {10}^{15}\). Obviously, it has far exceeded the computing power of ordinary computing devices Therefore, it’s necessary for us to explore a substitute algorithm to simplify computing.

3. The basic principle and steps of OIS.method

In order to avoid traversing all deleting ways, we used an indicator containing the missing information of both rows and columns to determine the deleting order of columns. Set the indicator as S_m.

S_m\(=\sum _{1}^{n}\frac{{a}_{i,j}}{f}\times 100+bj\times 100\) (1)

n: the number of rows, f: the number of columns, \(a\): the number of observed values, \({a}_{i,j}\):the number of observed values in the i-th row, j -th column, b_j: the number of observed values in the j -th column.

Take dataset A in Fig. 1 for an example. The detailed steps of OIS.method were as follows, and the flowchart of OIS.method was shown in Fig. 1.

Step1: Compute the value of\({a}\)

Count the number of observed values of each row and record as a.

Step2: Replace

Replace all observed values with ‘100’, and replace missing data with ‘\(\frac{{a}_{i,j}}{f}\)’.

Step 3 Compute the value of S_m

Compute the value of S_m of each column according to Formula (1). After that, Sort columns according to the S_m value from smallest to largest.

Step 4: Delete columns

For each combination of deleted columns, we should delete column(s) according to the order in the step 3.

Cell represents the amount of data of intact subset. \(Cell={n}_{subset}\times {f}_{subset}\). It has maximum value in a certain combination of deleted columns (Fig. 2).

Step 5: Choose the optimal deleting method

Cell had the maximum when deleting X4.

OIS.method can avoid traversing all deleting ways by identifying the optimal deleting order via S_m value. OIS.method has been validated in another two datasets, too. The process of validation was shown in Supplementary Table 1–2. All results were consistent with traversing method.

Model validation

1. Validation in computer-generated datasets

1.1 Series of simulated datasets 1

Step1: generate datasets with missing values

Firstly, we generated 25 categorical datasets using Python sklearn library. These datasets were full combinations of the number of rows: 11, 2000, 5000, 7500, 10 000, and the number of columns: 11, 20, 50, 80, 100. Secondly, set different patterns of missingness: missing rates included: 20%,30%,40%,50%,60%,70%,80%, distributions of missingness included: random distribution, exponential distribution, uniform distribution and lognormvariate distribution. Totally, we have generated \(25\times 7\times 4=700\) datasets with missing data.

Step2: handle missing values

We used five control methods on these datasets to handle missing data and compared against OIS.method. Therefore, we can obtain \(700\times 6=4200\) intact datasets. The individual methods in this comparison were:

⑴IterativeImputer: Missing values are estimated by modeling features with missing values as functions of other features in a cyclic manner.

⑵IterativeSVD: It’s a low-rank matrix imputation method based on singular value decomposition (SVD). The observation matrix is decomposed into three specific matrices, then, multiplying the three matrices as the imputing value of the observation matrix at the corresponding position. The method executes multiple SVD algorithms to impute observation matrix, and use it to replace observation matrix without imputed. It is performed iteratively.

⑶K-Nearest Neighbor: Missing values were imputed using K-Nearest Neighbor (KNN) imputation ^[8].

⑷Random Forest: The missing values were imputed based on random forest model ^[9].

⑸Simple Fill: Missing values were imputed with the mean or median of each column.

Step3: Build and evaluate machine learning models

Five machine learning models were established using 4200 intact datasets:

⑴Logistic Regression: It can perform classification task by establishing a regression formula of classification boundary according to the existing data.

⑵Multiayer Perceptrons (MLP): It has three parts: ①Input layer consists of training data.②Hidden layer contains several layers of neurons. ③Output layer is used to output results^[10].

⑶Support Vector Machine (SVM): It can find an optimal hyperplane to distinguish different types of samples. It has been widely used in binary classification problems ^[11].

⑷Random Forest: It uses Bootstrap re-sampling method to extract multiple samples from original data, and models each sample for decision tree using random node splitting technology. After summarizing the prediction results of multiple decision trees, the final result was drawn by voting.

⑸Extreme Gradient Boosting (XGBoost): It is an integrated learning algorithm based on gradient boosting methods ^[12]. One of the major advantages is that XGBoost provides L1 and L2 regularization. So, it can reduce overfitting and improve the generalization performance of models.

The predicting abilities of machine models were evaluated by five indicators: AUC (Area Under ROC Curve), Accuracy, Precision, Recall and F1-score. we used confusion matrix to illustrate these indicators. ^[13] (Table 2).

Table 2

confusion matrix
confusion matrix		True value
confusion matrix		Positive	Negative
Predicted value	Positive	True Positive (TP)	False Positive (FP)
	Negative	False Negative (FN)	True Negative (TN)
\({A}{c}{c}{u}{r}{a}{c}{y}=({T}{P}+{T}{N})/({T}{P}+{F}{N}+{F}{P}+{T}{N})\),\({P}{r}{e}{c}{i}{s}{i}{o}{n}=\left({T}{P}\right)/({T}{P}+{F}{P})\),\({R}{e}{a}{c}{l}{l}=\left({T}{P}\right)/({T}{P}+{F}{N})\),\({F}{1}_{{s}{c}{o}{r}{e}}=2\times\) (\({p}{r}{e}{s}{i}{s}{i}{o}{n}\times {r}{e}{c}{a}{l}{l})/({p}{r}{e}{c}{i}{s}{i}{o}{n}+{r}{e}{c}{a}{l}{l})\).

The better the prediction performance of the model, the better the method can improve the classification performance. The values of indicators can reflect the abilities of missing data processing methods.

The above procedures were repeated 10 times.

1.2 Series of simulated datasets 2

All variables in simulated datasets 1 have contribution to outcome variables. However, real-world datasets have many redundant variables. Thus, in order to simulated real-world datasets, we introduced redundant variables in the dataset whose number of samples was 500, number of features was 10, missing rate was 50% and distribution of missingness was exponential distribution. The multiples of redundant variables included 0,1,1.5,3,8.

OIS.method and five control methods (same as above) were adopted to handle the missing values. The remaining steps are the same as before.

In machine learning, the information provided by columns is more important when the dataset has a large size of samples. So, we added power exponents to\({f}_{subset}\) after using OIS.method to calculate Cell. \(Cell={n}_{subset}\times {f}_{subset}^{{e}}\), the values of e included:1,1.2,1.5,1.8,2.

The above procedures were repeated 10 times.

2. Validation in real-world datasets

OIS.method and three other control methods(simple impute, random forest, modified random forest) were applied in two real-world classification tasks. Results were presented in the form of mean ± standard deviation.

2.1 Task 1: risk prediction of hypotension during dialysis in dialysis patients

We collected the data of all renal dialysis patients in Sichuan Provincial People's Hospital. The dataset contained the records of 314, 534 patients, each made up of 496 features. Set missing rate = the number of missing values/total amount of data. The missing rate of this dataset was 42.54%.

After handling missing data. We have established three machine learning models: Random Forest, Gradient Boosting, Logistic Regression. Similarly, we also used five indicators (same as above) to reflect the performance of different methods.

2.2 Task 2: risk prediction of drug adverse reaction in elderly patients with type 2 diabetes

We collected the data of all elderly patients with type 2 diabetes in Sichuan Provincial People's Hospital. The dataset contained the records of 1,224 patients, each made up of 279 features. The missing rate of this dataset was 21.75%.

After handling missing data. We have established two machine learning models: Ensemble Learning and Passive Aggressive. The remaining steps were the same as before.

3. Statistical Analysis

Statistical analysis was performed using pycharm2021.2.3. Single variable multigroup data were analyzed by a one-way ANOVA. Spearman rank test was used to evaluate correlations between continuous covariates. Kruskal-Wallis test was used to evaluate correlations between discrete variables. A multivariate analysis was performed with multi-linear regression analysis. We have provided exact P values and 95% confidence intervals.

Algorithm

We have invented a novel missing data processing method. It can identify an optimal deleting order of columns to obtain the optimal intact subset. The intact subset is completely generated from original data set without imputing, which can reflect the actual information of original datasets. Using it for downstream statistical analysis can obtain more efficient conclusions.

Validation results in computer generated datasets

1. Series of simulated datasets 1

1.1 Validation results of OIS. Method

OIS. Method was used to handle missing data of 700 datasets, then established and evaluated machine learning models. In order to explore the conditions under which OIS. Method can improve the classification performance best, we used univariate analysis and multivariate analysis to find the variables that had influence on OIS.Method(Supplementary Table 4–5).

The influence of features number and missing rates on the OSI.method was shown in Fig. 3. Results showed that: ⑴OIS. Method worked best when features number was 11(AUC: 0.8137 ± 0.1993, accuracy༚0.8695 ± 0.1188, precision༚0.8709 ± 0.1181, recall༚0.8695 ± 0.1291, f1-score༚0.8691 ± 0.1231). ⑵OIS.Method worked worst when the missing rate was 50%(AUC༚0.6826 ± 0.2112, accuracy༚0.8637 ± 0.1245, precision༚0.8638 ± 0.1318, recall༚0.8632 ± 0.1415, f1-score༚0.8615 ± 0.1365).

1.2 The comparison results of OIS. Method and control methods

700 datasets were handled by OIS.Method and five control methods. Then, we established and evaluated machine learning models as described previously. Through univariate analysis and multivariate analysis, we identified a set of variables that had impacts on the performance of methods.(Supplementary Table 6–7)

Figure 4 showed the effect of four different variables on six missing data processing methods. Compared with the control methods, OIS. Method worked relatively bad.

2. Series simulated dataset 2

2.1 Validation results of OIS. Method

This series generated missing datasets containing different multiples of redundant variables. OIS.method was applied to handle missing data. Through univariate analysis and multivariate analysis, we found that the multiple of redundant variables and the power exponent of columns will affect the performance of OIS.method(Supplementary Table 8–9).

The influence of the power exponent of columns on the OIS.method was shown in Fig. 5. OIS.method works best when the power exponent of columns is 1 (mean AUC: 0.9320, mean accuracy༚0.8610, mean precision༚0.8613, mean recall༚0.8622, mean f1-score༚0.85989).

2.2 The comparison results of OIS. Method and control methods

We compared OIS. Method with five other control methods in series of simulated datasets 2. Through univariate analysis and multivariate analysis, we found that the multiple of redundant variables will affect the performance of all methods(n Supplementary Table 10–11).

Figure 6 illustrated that with the increase of redundant variables, the evaluation indicators generally showed a downward trend. It means that the more redundant variables were added to the datasets, the worse the performance of methods.

When introducing different multiples of redundant variables, we sorted the different methods according to the value of the five evaluation indicators respectively. Top 10, Top20 and Top50 methods were selected, and we counted the proportion of different methods respectively. The higher the ranking and the larger the proportion, the better the method. Radar plots (Fig. 7) were created to show the proportion of different methods in Top 10, Top20 and Top50. OIS. Method had the largest area in situation. Therefore, we could conclude that OIS.method performed best in simulated dataset 2.

Validation results in real-world datasets

1.Risk prediction of hypotension during dialysis in dialysis patients

We used OIS. Method and three other control methods to handle missing data, and then we used intact datasets to build three machine learning models. We sorted the results according to the values of evaluating indicators from largest to smallest respectively. The results were shown in Table 3. It was obviously that OIS.method performed best. Take AUC for an example: OIS.Method VS Simple Impute VS Random Forest VS Modified Random Forest, 0.8179 ± 0.0005VS0.8116 ± 0.0002VS0.8087 ± 0.0009VS0.8093 ± 0.0014.

2.Risk prediction of drug adverse reaction in elderly patients with type 2 diabetes

We applied the same methods to handle missing data and built two machine learning models using intact datasets. We ranked the results, too. The results were shown in Table 4. In Ensemble Learning, OIS. Method could acquire best AUC, Recall and f1-score (0.7028 ± 0.0126, 0.43599 ± 0.0597, 0.3423 ± 0.0348). In Passive Aggressive, OIS.method worked best at all evaluation indicators. (AUC:0.5831 ± 0.0513, Accuracy:0.6505 ± 0.0747, Precision:0.2652 ± 0.0734, Recall:0.4102 ± 0.1038, f1-score:0.3181 ± 0.0799).

In summary, in real-world study, OIS. Method showed best performance.

Table 3

Comparison of the performance of different missing data processing methods in dialysis patient dataset
AUC	Accuracy	Precision	Recall	F1-Score	Imputing methods	Machine learning method
0.8179 ± 0.0005	0.7459 ± 0.0020	0.7370 ± 0.0027	0.6797 ± 0.0031	0.7072 ± 0.0025	OIS.method	Random Forest
0.8116 ± 0.0002	0.7405 ± 0.0010	0.7333 ± 0.0015	0.6696 ± 0.0003	0.7000 ± 0.0008	Random Forest	Random Forest
0.8087 ± 0.0009	0.7367 ± 0.0079	0.7302 ± 0.0012	0.6625 ± 0.0013	0.6947 ± 0.0009	Modified Random Forest	Random Forest
0.8093 ± 0.0014	0.7375 ± 0.0012	0.7297 ± 0.0017	0.6662 ± 0.0026	0.6965 ± 0.0016	Simple impute	Random Forest
0.7468 ± 0.0026	0.6854 ± 0.0011	0.6674 ± 0.0009	0.6041 ± 0.0025	0.6342 ± 0.0017	OIS.method	Logistic Regression
0.7370 ± 0.0022	0.6771 ± 0.0016	0.6591 ± 0.0018	0.5923 ± 0.0023	0.6239 ± 0.0020	Random Forest	Logistic Regression
0.7374 ± 0.0026	0.6781 ± 0.0022	0.6601 ± 0.0025	0.5940 ± 0.0030	0.6253 ± 0.0027	Modified Random Forest	Logistic Regression
0.7362 ± 0.0029	0.6765 ± 0.0021	0.6585 ± 0.0025	0.5914 ± 0.0029	0.6231 ± 0.0027	Simple impute	Logistic Regression
0.7527 ± 0.0021	0.6897 ± 0.0020	0.6937 ± 0.0024	0.5600 ± 0.0030	0.6197 ± 0.0028	OIS.method	Gradient Boosting
0.7415 ± 0.0009	0.6807 ± 0.0011	0.6819 ± 0.0017	0.5510 ± 0.0016	0.6095 ± 0.0013	Random Forest	Gradient Boosting
0.7428 ± 0.0009	0.6815 ± 0.0007	0.6823 ± 0.0012	0.5535 ± 0.0010	0.6112 ± 0.0008	Modified Random Forest	Gradient Boosting
0.7415 ± 0.0011	0.6796 ± 0.0010	0.6797 ± 0.0017	0.5511 ± 0.0011	0.6087 ± 0.0011	Simple impute	Gradient Boosting
OIS. method worked best in three machining learning models.

Table 4

Comparison of the performance of different missing data processing methods in type 2 diabetes dataset
AUC	Accuracy	Precision	Recall	F1-Score	Imputing methods	Machine learning method
0.7028 ± 0.0126	0.7293 ± 0.013	0.3078 ± 0.0335	0.43599 ± 0.0597	0.3423 ± 0.0348	OIS.method	Ensemble Learning
0.6963 ± 0.0231	0.7118 ± 0.0696	0.2787 ± 0.1451	0.3875 ± 0.2506	0.2903 ± 0.1487	Random Forest	Ensemble Learning
0.6957 ± 0.0247	0.7483 ± 0.0706	0.3664 ± 0.1509	0.3055 ± 0.2556	0.2791 ± 0.1541	Modified Random Forest	Ensemble Learning
0.6699 ± 0.0249	0.7339 ± 0.0708	0.2921 ± 0.1532	0.2903 ± 0.2479	0.2454 ± 0.1498	Simple impute	Ensemble Learning
0.5831 ± 0.0513	0.6505 ± 0.0747	0.2652 ± 0.0734	0.4102 ± 0.1038	0.3181 ± 0.0799	OIS.method	Passive Aggressive
0.5507 ± 0.0625	0.6011 ± 0.0838	0.2124 ± 0.0605	0.3681 ± 0.1383	0.2609 ± 0.0695	Random Forest	Passive Aggressive
0.5625 ± 0.042	0.6215 ± 0.0552	0.2224 ± 0.0312	0.3792 ± 0.1401	0.2733 ± 0.0533	Modified Random Forest	Passive Aggressive
0.5554 ± 0.0354	0.6267 ± 0.0628	0.2317 ± 0.033	0.3875 ± 0.1317	0.2828 ± 0.0491	Simple impute	Passive Aggressive
OIS.method had best performance in AUC, Recall and f1-score in Ensemble Learning model, and OIS.method had best performance in all evaluation indicators in Passive Aggressive.

The main contribution of this paper is to propose a novel missing data processing method, and it is more suitable for the large-samples and multiple-variables real-world datasets for its small calculation. Compared with other methods, OIS.method can obtain more efficient results of downstream statistical analysis for the reason that the optimal intact subset was completely generated from original datasets without imputing. Additionally, OIS.method has been validated in our study.

OIS.method worked best in simulated datasets 2, however, the mean values of evaluating indicators were lower in simulated datasets 1. It doesn’t mean the ability of OIS.method is poor. Differing from that real-world dataset have redundant variables, each variable in simulated datasets 1 had contribution to outcome variables. Consequently, no matter deleting which variable will cause negative effect on downstream statistical analysis. Additionally, at the same missing rate, all methods works best when the number of variables was 11. This is because that with the number of variables increasing, the number of missing data will increase, too. So, the difficulty of determining the variable having contribution to outcome variables will increase.

Currently, novel missing data processing methods based on machine learning have been developed. Colin Pawlowski. et al ^[7] proposed a framework for imputing missing data which incorporated KNN, decision tree, and. SVM. Khaled M. Fouad. et al ^[14] proposed a method incorporating KNN and iterative imputation algorithms which can impute missing data depended on the similarity between records.

Moreover, there are three missingness mechanisms: ⑴Missing completely at random (MCAR): the missing values don’t depend on the observed data. ⑵Missing at random (MAR): the missing values depend on observed data only. ⑶Missing not at random (MNAR): the missing values the missing values depend on missing values themselves ^[15]. Dexin Shi. et al ^[16] discussed the performance of different methods under different missingness mechanisms Our study didn’t discuss it. In future studies, we can validate OIS.method in datasets with different missingness mechanisms.

Finally, missing data is unavoidable in health care real-world study. The selection of appropriate missing value processing method is crucial for downstream statistical analysis. OIS.method can be taken as one of the options when handle complex real-world datasets.

AUC：area under ROC curve

OIS.Method：Optimal Intact Subset Method

SVD: singular value decomposition

KNN: K-Nearest Neighbor

MLP: Multiayer Perceptrons

SVM: Support Vector Machine

XGBoost: Extreme Gradient Boosting

TP: True Positive

FP: False Positive

FN: False Negative

TN: True Negative

MCAR: Missing completely at random

MAR: Missing at random

MNAR: Missing not at random

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable.

Availability of data and materials

Simulated datasets underlying this article will be shared on reasonable request to the corresponding author. Real-world datasets underling this article cannot be shared publicly due to the privacy of patients who participated in this study.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable

Funding

This work was supported by National Natural Science Foundation of China. [grant number:72004020]

Author contributions

Wu Jingxian authored drafts of the paper, prepared the figures and tables. Chang Huan analyzed the data. Tong Rongsheng managed and coordinated responsibility for the research activity planning and execution. Wu Xingwei designed the experiment, and approved the final draft. All authors approved the final manuscript.

Acknowledgements

Not applicable

Larkins NG, Craig JC, Teixeira-Pinto A. A guide to missing data for the pediatric nephrologist. Pediatr Nephrol. 2019;34(2):223–31.
Pedersen AB, Mikkelsen EM, Cronin-Fenton D, et al. Missing data and multiple imputation in clinical epidemiological research. Clin Epidemiol. 2017;9:157–66.
Austin PC, White IR, Lee DS, van Buuren S. Missing Data in Clinical Research: A Tutorial on Multiple Imputation. Can J Cardiol. 2021;37(9):1322–31.
Marshall A, Altman DG, Royston P, Holder RL. Comparison of techniques for handling missing covariate data within prognostic modelling studies: a simulation study. BMC Med Res Methodol. 2010;10:7.
Zhang Z. Missing data imputation: focusing on single imputation. Ann Transl Med. 2016;4(1):9.
van Ginkel JR, Linting M, Rippe RCA, van der Voort A. Rebutting Existing Misconceptions About Multiple Imputation as a Method for Handling Missing Data. J Pers Assess. 2020;102(3):297–308.
Bertsimas D, Pawlowski C, Zhuo YD. From Predictive Methods to Missing Data Imputation: An Optimization Approach. J Mach Learn Res. 2018;18:1–39.
Wang G, Lu J, Choi KS, Zhang GA, Transfer-Based. Additive LS-SVM Classifier for Handling Missing Data. IEEE Trans Cybern. 2020;50(2):739–52.
Hong S, Lynn HS. Accuracy of random-forest-based imputation of missing data in the presence of non-normality, non-linearity, and interaction. BMC Med Res Methodol. 2020;20(1):199.
Chasiotis V, Nadi F, Filios A. Evaluation of multilayer perceptron neural networks and adaptive neuro-fuzzy inference systems for the mass transfer modeling of Echium amoenum Fisch. & Mey CA. J Sci Food Agric. 2021;101(15):6514–24.
Huang S, Cai N, Pacheco PP, Narrandes S, Wang Y, Xu W. Applications of Support Vector Machine (SVM) Learning in Cancer Genomics. Cancer Genomics Proteomics. 2018;15(1):41–51. https://doi.org/10.21873/cgp.20063.
Sun X, Xu Z, Feng Y, et al. RBC Inventory-Management System Based on XGBoost Model. Indian J Hematol Blood Transfus. 2021;37(1):126–33.
Wu MT. Confusion matrix and minimum cross-entropy metrics based motion recognition system in the classroom. Sci Rep. 2022;12(1):3095.
Fouad KM, Ismail MM, Azar AT, Arafa MM. Advanced methods for missing values imputation based on similarity learning. PeerJ Comput Sci. 2021;7:e619.
Fernández-García MP, Vallejo-Seco G, Livácic-Rojas P, Tuero-Herrero E. The (Ir)Responsibility of (Under)Estimating Missing Data. Front Psychol. 2018;9:556.
Shi D, Lee T, Fairchild AJ, Maydeu-Olivares A. Fitting Ordinal Factor Analysis Models With Missing Data: A Comparison Between Pairwise Deletion and Multiple Imputation. Educ Psychol Meas. 2020;80(1):41–66.

Supplementaryfile.docx

Download PDF

Version 1

posted

You are reading this older preprint version

Read the latest preprint version →

A novel method for handling missing data in health care real-world study: Optimal Intact Subset Method

Status:

Version 1

Abstract

Background

Method

Results

Conclusions

Figures

Introduction

Method

Algorithm

1. Find the optimal intact subset by traversing method

2. The derivation of the total number of deleting ways

3. The basic principle and steps of OIS.method

Step1: Compute the value of\({a}\)

Step2: Replace

Step 3 Compute the value of Sm

Step 4: Delete columns

Step 5: Choose the optimal deleting method

Model validation

1. Validation in computer-generated datasets

1.1 Series of simulated datasets 1

Step1: generate datasets with missing values

Step2: handle missing values

Step3: Build and evaluate machine learning models

1.2 Series of simulated datasets 2

2. Validation in real-world datasets

2.1 Task 1: risk prediction of hypotension during dialysis in dialysis patients

2.2 Task 2: risk prediction of drug adverse reaction in elderly patients with type 2 diabetes

3. Statistical Analysis

Result

Algorithm

Validation results in computer generated datasets

1. Series of simulated datasets 1

1.1 Validation results of OIS. Method

1.2 The comparison results of OIS. Method and control methods

2. Series simulated dataset 2

2.1 Validation results of OIS. Method

2.2 The comparison results of OIS. Method and control methods

Validation results in real-world datasets

1.Risk prediction of hypotension during dialysis in dialysis patients

2.Risk prediction of drug adverse reaction in elderly patients with type 2 diabetes

Discussion

Conclusion

Abbreviations

Declarations

References

Supplementary Files

Status:

Version 1

Step 3 Compute the value of S_m