Introduction and Comparison of Novel Decentral Learning Schemes with Multiple Data Pools for Privacy-preserving ECG Classification

doi:10.21203/rs.3.rs-1955846/v1

Download PDF

Research Article

Introduction and Comparison of Novel Decentral Learning Schemes with Multiple Data Pools for Privacy-preserving ECG Classification

https://doi.org/10.21203/rs.3.rs-1955846/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 17 Aug, 2023

Read the published version in Journal of Healthcare Informatics Research →

You are reading this latest preprint version

Purpose: Artificial intelligence and machine learning have led to prominent and spectacular innovations in various scenarios. Application in medicine, however, can be challenging due to privacy concerns and strict legal regulations. Methods that centralize knowledge instead of data could address this issue.

Methods: In this work, 6 different decentralized machine learning algorithms are applied to 12-lead ECG classification and compared to conventional, centralized machine learning.

Results: The results show that state-of-the-art federated learning leads to reasonable losses of classification performance compared to a standard, central model (-0.054 AUROC) while providing a significantly higher level of privacy. A proposed weighted variant of federated learning (-0.049 AUROC) and an ensemble (-0.035 AUROC) outperformed the standard federated learning algorithm. Overall, considering multiple metrics, the novel batch-wise sequential learning scheme performed best (-0.036 AUROC to baseline).

Conclusion: Although, the technical aspects of implementing them in a real-world application are to be carefully considered, the described algorithms constitute a way forward towards preserving-preserving AI in medicine.

Decentral Learning

Privacy-preserving Artificial Intelligence

Machine Learning

Deep Learning

Decision-support

1.1 Artificial Intelligence (AI) in healthcare

Artificial intelligence (AI), in particular the fields of machine learning and its advancement deep learning, has led to prominent and spectacular innovations in various medical fields such as radiology pathology [1], genomics [2], injury risk assessment [3] and disease prognosis [4, 5]. AI applications are expected to play an increasing role in the future of everyday medicine, based on a) superior processes as compared to the state-of-the-art with better outcomes for patients and b) non-inferior processes which are less expensive in terms of costs, time and/or resources.

Various techniques exist in the field of machine learning, which are currently dominated by artificial neural networks (ANNs). Most recently, the introduction of residual neural networks by He et al. in 2016 [6] has revolutionized this field. Residual models have shown astonishing results in various fields, often outperforming other architectures. At the Computing in Cardiology / PhysioNet Challenge in 2020 [7], 9 out of the 10 [8–17] best performing competing teams have used some type of residual network or skip connections. Regardless of the chosen technology or architecture, there is one aspect all AI algorithms have in common: the need for data. The correlation between data availability and model quality has been documented [18–20] and is now broadly accepted by the research community. Recently, there appears to be a shift in the literature to focus more on the data aspect of AI. In 2022, AI pioneer Andrew Ng shared this sentiment by stating that data should be the central element of AI applications, not the models per se [21].

1.2 Artificial intelligence (AI) and clinical data

Any data, and especially health data, however, are subject to rigorous legal regulations. Additionally, medical data are often collected in decentralized settings. Institutions like hospitals or medical universities collect data from their patients for routine care applications and/or for clinical trials and other research activities. However, beyond the scope of this primary use, the data are rarely used for other purposes (secondary use) let alone shared with other institutions. Due to legal regulations, sharing of data with other institutions is often related to certain risks for the data holders and owners. However, if the highest level of privacy preservation was applied to all AI applications, the utility of the data would be reduced and severe impairments of the clinical outcome would need to be conceded. This applies especially for applications that either require extensive amounts of data or areas where data are extremely sparse, such as rare diseases. Therefore, methods that balance data protection against data availability to optimize the overall outcome (“privacy-preserving AI”) are urgently needed.

1.3 Privacy-preserving Artificial Intelligence (PPAI)

Typically, clinical data are anonymized or pseudonymized prior model development. However, it has been shown in multiple studies, that removing obvious identifying elements (e.g. names, date of birth, addresses, etc.) is not sufficient to protect the patients’ privacy, since these datasets are still highly vulnerable to re-identifying attacks [22–25]. Latanya Sweeney found that with the three basic quasi-identifiers of date of birth, zip code and gender, 87% of individuals can be successfully re-identified [26]. Re-identification is possible by cross-referencing the remaining information with other, publicly available or leaked data, which is ever increasing. One could theoretically remove even more information from the datasets to address this, but at the same time, utility of that data is decreased. In her speech at the Differential Privacy Symposium in 2016, Cynthia Dwork famously stated “De-identified data isn’t” [27], aptly summarizing this dilemma of utility versus privacy. The concepts of k-anonymity [28] and l-diversity [29] are allowing for a gradual removal of sensitive information to address this, but are still vulnerable to attacks (e.g. skewness or similarity attacks) [30].

Various alternative methods have been explored in recent publications to centralize the data, but still sufficiently preserve privacy. However, a discrepancy between promising academic ideas and practically applicable solutions exists. There are prominent examples to this issue:

Homomorphic encryption is a compelling technique, which allows operations on fully encrypted data without prior decryption. Craig Gentry published the first fully homomorphic encryption scheme in 2009 [31]. While the technology is certainly promising, it is currently still computationally too expensive for widespread practical application in most clinical applications.
Dwork et al. introduced differential privacy [32], which has been successfully implemented in a wide range of applications since [33]. To be differentially private, a database is transformed so that the individual records are obscured but the underlying statistical information is retained. However, this approach might not be applicable on small datasets [34, 35].
Another approach that results in a similar solution is the application of generative adversarial networks (GANs) [36], which produce synthetic data samples derived from original examples. Those samples exhibit the same statistical properties while not containing real private information. This approach has already been applied on medical data [37, 38]. However, GANs are computationally expensive, time-consuming and their output is notoriously difficult to validate.

Instead of trying to find secure methods to aggregate data, federated learning (FL) is aiming to centralize knowledge without ever collecting data in a central infrastructure. The concept was proposed by Google researchers McMahan et al. in 2015 for improving typing predictions in their Android operating system [39].

1.4 Federated Learning (FL) – Principles

The core principle of FL is to avoid the pooling of data from different participating clients (nodes) in a central instance. This eliminates the need of secure data transfer and storage, which always come with a high risk of data leakages. In FL, data stay in the secure local environment where they were collected, and knowledge exchange is realized by transferring models and models only. In its original proposal [39], the FL workflow is comprised of five steps:

a central model is created
the model is distributed to all nodes,
this model is trained at the clients with the respective local data only,
the changes in the models’ parameters are securely averaged
the central model is updated with the new parameters.

Steps 2-5 are repeated multiple times until the model converges. Neural networks are well-suited for this as they are comprised of large matrices of weights and biases, for which standard mathematical operations like calculating a mean are easily applicable. McMahan et al. proposed averaging the parameters in a buffer, to obfuscate the individual nodes’ contribution even further [40].

A 2020 Nature publication investigated the possibility of applying federated learning in medicine and underlined its importance and potential [41]. However, the application of existing FL approaches in different scenarios might come with new challenges. In this paper, we focus on a simulated scenario, in which learning is not delegated to individual patients, but to various institutions holding pooled sets of data. In this scenario, only a few data nodes are used for training, while in its original application, potentially millions of Android smartphones were available. Furthermore, the different nodes can potentially provide quite homogenous datasets as not only the type of health data provider can vary (hospitals, research institutions, sports rehabilitation centers, geriatric homes, …) but also the population from which the data were collected (healthy subjects, patients, elderly adults, …). This raises the question if averaging the models’ weights is truly sufficient in such a setting, as described by Rieke et al. [41]. Sheller et al. proposed three alternative methods of federated learning for application in medicine: Federated Learning, Institutional Incremental Learning, and Cyclic Institutional Incremental Learning [42]. In their experiments, they compared a sequential learning scheme to the conventional federated learning approach and centralized machine learning. They found that cycling over institutions during learning achieved results comparable to conventional federated learning and even to centralized machine learning. However, this approach was less stable.

1.5 Aims and scope

The aim of the present work is to develop new decentral learning schemes for scenario in which data is distributed across different sources (i.e. nodes), each containing individual datasets of different size and with characteristics. These learning schemes are applied to ECG datasets in a complex multi-label, multi-class classification task. Their performance is compared to a standard machine learning approach, in which data is centralized. The main questions to be answered in this study are as following: A) Can standard federated learning as a form of privacy-preserving AI be applied to medical data with few nodes and B) Do methods exists, that might improve the standard federated learning algorithm as described in the chapter above?

2.1 Data description

We used the data provided for the 2020 Computing in Cardiology / PhysioNet challenge [7], which consisted of six publicly available 12-lead ECG datasets (CPSC, CPSC-Extra, INCART, PTB, PTB-XL, Georgia). Datasets collected by the same institutional source (CPSC and CPSC-Extra; PTB and PTB-XL) were merged to simulate a realistic distributed learning setting, resulting in a total number of four nodes: 1) INCART, 2) CPSC, 3) Georgia and 4) PTB (Table 1):

Table 1. Data description of the four ECG datasets used for our analyses

Node ID	Name	Number of samples	Duration [s]	Sampling rate[Hz]	Mean age [y]	Female [%]	Number of classes	Reference
1	CPSC	10,330	[6 - 144]	500	61.36	46.35	73	[43]
2	Georgia	10,344	[5 - 10]	500	60.52	46.34	67	[7]
3	INCART	74	[1,800]	257	55.99	45.95	37	[44]
4	PTB	22,353	[10 - 120]	500, 1000	59.76	47.41	60	[45][44]

The ECG recordings in these databases were heterogenous in terms of signal length, sampling rate, demographic properties and the number of classes. In total, 111 different classes represented by SNOMED codes were present. Each ECG could be labelled with one or multiple classes. For this publication, medically related classes were joined or merged into parent categories, resulting in 13 classes as described in Table 2.

Table 2. Considered ECG classes and frequency of occurrence in each data source

Class	CPSC	Georgia	INCART	PTB	Total
Sinus rhythm	922	1,752	0	18,172	20,846
ST interval abnormal	2,985	3,053	10	2,188	8,236
Myocardial infarction	1,544	7	9	5,629	7,189
T wave abnormal	27	3,118	1	2,639	5,785
Myocardial ischemia	545	1,635	0	2,580	4,760
Right bundle branch block	2,057	977	2	1,660	4,696
Left ventricular hypertrophy	158	1,232	10	2,359	3,759
Atrial fibrillation	1,374	570	2	1,529	3,475
Bradycardia	316	1,683	11	637	2,647
Ventricular ectopics	896	398	49	1,154	2,497
Tachycardia	303	1,261	11	827	2,402
1st degree AV block	828	769	0	797	2,394
Atrial ectopics	742	640	7	555	1,944

2.2 Pre-processing

All recordings were resampled to 250 Hz. Subsequently, from each signal, a 10-second sequence was extracted to generate a uniform data sample for the machine learning model. The first 5 seconds of a signal were ignored in this selection, if excessive data was available. The ECG data was filtered by a bandpass filter (3-30 Hz, Butterworth bandpass, 2^nd order).

2.3 Model architecture description

For this multi-class, multi-label classification task, a deep convolutional neural network with five one-dimensional convolutional blocks and a concluding global average pooling prior to the classification layer was applied [46]. Figure 1 graphically summarizes the model architecture:

The model was trained with the binary cross-entropy loss function and the Adam optimizer [47]. The number of training epochs and learning rate decay, as suggested by Kingma et al. [47], are described in the individual methods’ descriptions. All implementations were executed in Python 3.7.4 and modelling was realized with Tensorflow 2.4 [48].

2.4 Learning schemes

Eleven different learning schemes were applied (1 centralized baseline, 4 node-individual, 6 decentral), which are summarized in Table 3. The following chapters describe each method in detail.

Table 3. Learning method descriptions: 1 baseline model with centralized data B, 4 individual models for each node (I1-I4), 6 decentral learning schemes: M1 – M3

Notation	Name	Description	Reference
B	Baseline Model	1 model trained with centralized data of all nodes	standard
I1-I4	I1: Individual Model: CPSC	1 individual model trained with CPSC data only	standard
	I2: Individual Model: Georgia	1 individual model trained with Georgia data only	standard
	I3: Individual Model: INCART	1 individual model trained with INCART data only	standard
	I4: Individual Model: PTB	1 individual model trained with PTB data only	standard
M1a	Regression Ensemble	Classification obtained by averaging the results of I1-I4	[49]
M1b	Weighted Regression Ensemble	Classification obtained by averaging the results of I1-I4 with weights	[49]
M2a	Node-wise Sequential Learning	1 model trained on full data of nodes in one sequence	[42]
M2b	Batch-wise Sequential Learning	1 model trained on mini batches of data in multiple sequences	new
M3a	Federated Learning	1 model trained with standard federated learning (all nodes contribute equally)	[39]
M3b	Weighted Federated Learning	1 model trained with weighted federated learning (according to performance)	new

B: Baseline Centralised Model

I1-I4: Individual Models

One individual model was trained for each of the four data nodes, resulting in four additional models (I1: CPSC, I2: Georgia, I3: PTB and I4: INCART), which were trained the same way as the combined model, but only with training data from the respective nodes.

M1a: Regression Ensemble

The first method to aggregate knowledge from federated data sources was to calculate the average of all classification results from the individual models I1-I4. All models trained on individual nodes were queried to classify the common test set. Subsequently, the result was determined by calculating the mean of each class-specific regression result. Finally, a threshold of 0.5 (= 50% probability) was applied to derive the classification result of M1a from the regression values.

M1b: Weighted Regression Ensemble

In M1a, all four individual models contributed equally to the final classification. However, in M1b, the individual regression results were weighted according to two factors (see Equation 2): a) their training set size proportion in relation the total dataset size (sample size divided by the sum of sample sizes of all nodes) and b) their node-internal AUROC performance.

AUROC scores were interpolated to a range of [0, 1] and the final weights were normalized, so that the sum of all four weights was equal to 1.

M2a: Node-wise Sequential Learning

A combined model was trained by progressively exposing the initially untrained model to the data of one node after the other, so that knowledge was gathered sequentially. This method was comparable to Institutional Incremental Learning as proposed by Sheller et al [42]. For method M2a, a single model was sent to all nodes in the following order: 1) CSPC, 2) Georgia, 3) INCART and 4) PTB, as depicted in Figure 2.

At first, a model was initialised and sent to the first node, where the model is trained with the data of this specific node. After training, the model was sent to the next node in order, where its already partly optimized weights were the initial condition for continuing the training with the next pool of data. At each node, the model was trained for 50 epochs and the learning rate was decayed after each epoch as described in Equation 1. After training at a node, the learning rate was reset to the initial value of 0.001 and sent to the next node in order. This was repeated until the model was trained at all nodes once.

M2b: Batch-wise Sequential Learning

To take the idea of sequential learning even further, we applied a novel method called Batch-wise Sequential Learning (M2b). Instead of fully completing training at a node like in M2a, the model was trained only on a randomly selected single mini-batch of one node’s training data before sending it to the next node. The batch size of these mini-batches was set to 2% of a node’s training set size. This meant that each sample contributed equally to the model in M2b (which is equivalent to larger nodes consisting of more samples contributing more, as achieved with the weighted approaches). One epoch was considered completed, when the model was exposed to each training sample exactly once. The model was trained for 50 epochs in total and the learning rate was decayed after each epoch according to Equation 1. Method M2b is illustrated in Figure 3.

M3a: Federated Learning

In M3a, a model was trained in update cycles as depicted in Figure 4. Each of these cycles repeated the steps as described in chapter 1: 1) distribute central model, 2) train locally at the nodes, 3) average the weights of the trained models and 4) update central models with new parameters. The newly updated model from step 4 was then re-distributed as the central model in step 1 for the next update cycle. This method follows the original proposal for federated learning [39].

50 update cycles were completed. The epoch number for one cycle was set to 1. The learning rate was decayed according to Equation 1, where is the current update cycle iteration.

M3b: Weighted Federated Learning

As an advancement to Federated Learning (M3a), Weighted Federated Learning (M3b) was implemented. Therefore, a weighted average was used to calculate the new parameters in step 3 according to node-internal performance and dataset size as described in Equation 2 for model M1b.

2.5 Cross validation and evaluation metrics

We trained models with a central dataset, with local datasets and in decentral schemes. Therefore, only data available for training in the respective scheme were provided to the respective models during training. To find out how well all these models perform, each model was applied to a “global” dataset, containing data from all nodes. Therefore, a 10-fold cross-validation scheme was applied.

During training in each fold N, 90 percent of each dataset were applied to the respective learning scheme. Depending on the learning scheme, training was done based on data from single nodes or from all nodes as described in the Learning schemes chapter.

While training of fold N was done with different datasets depending on the learning scheme, all resulting models in fold N were evaluated with one and the same test-set-N. Therefore, the respective 10-percent shares of data from each dataset were aggregated to form one common test dataset N per fold. All models and decentral schemes described in the following chapters were tested on this test-set-N within fold N.

Predicted classes were compared to the known reference classes for each ECG and each model was evaluated with four standard metrics for a complete assessment of classification performance: accuracy, area under the receiver operator curve (AUROC), Jaccard score, and F1 score. To correctly address the multi-label classification problem, the metrics (except accuracy) were derived from a weighted average according to the frequency of occurrence in the test set [50].

To combine the results achieved with each of these evaluation metrics in a representative way, we ranked the models by each of the four evaluation-metrics and calculated the mean ranks of all metrics for each model, i.e., the best model ended up with the lowest mean rank.

3.1 Model performance

Table 4 summarizes average values obtained during the 10-fold cross-validation process as achieved for the four evaluation-metrics (accuracy, AUROC, Jaccard score, F1 score). Every model was ranked within each metric and the average rank for each model was calculated.

Table 4. Detailed performances of all models and methods. Displayed numbers are mean values of metrics achieved during the 10-fold cross-validation scheme. Numbers in brackets note the achieved rank of performance and right column displays the average achieved rank.

Model	Accuracy		AUROC		Jaccard Score		F1 Score		Average Rank
B	0.915	(1)	0.872	(1)	0.392	(1)	0.532	(1)	1
I1	0.880	(10)	0.754	(10)	0.204	(10)	0.296	(10)	10
I2	0.888	(9)	0.769	(9)	0.273	(9)	0.345	(9)	9
I3	0.661	(11)	0.500	(11)	0.010	(11)	0.081	(11)	11
I4	0.890	(8)	0.817	(7)	0.336	(4)	0.487	(2)	5.25
M1a	0.903	(4)	0.835	(4)	0.302	(7)	0.376	(8)	5.75
M1b	0.903	(3)	0.837	(2)	0.344	(3)	0.441	(4)	3
M2a	0.893	(7)	0.802	(8)	0.300	(8)	0.415	(6)	7.25
M2b	0.906	(2)	0.836	(3)	0.350	(2)	0.472	(3)	2.5
M3a	0.900	(5)	0.818	(6)	0.327	(6)	0.391	(7)	6
M3b	0.895	(6)	0.823	(5)	0.328	(5)	0.441	(5)	5.25

3.2 Average rank per model

Figure 5 illustrates the average rank per model. As expected, the baseline model with all data pooled centrally during learning performed best. From all decentral learning methods, models taking the size of the different nodes into account performed best (M1b, M2b, M3b), while models derived on data from small nodes only performed worst (especially I3).

3.3 Evaluation metrics per model

The evaluation metrics are displayed as box-whisker-plots for graphical comparison in Figure 6 below.

Privacy-preserving Artificial Intelligence (PPAI) is widely discussed nowadays. Federated learning is commonly used for training models based on data from large user groups (e.g. mobile phone users). The goal is to aggregate knowledge without centralising data to protect personal information. Clinical scenarios are different from the usual federated learning approach (e.g. less data nodes, less data at the nodes, non-iid data, …) and thus implementing this principle requires adaptions. We have implemented six different decentral learning schemes for ECG classification and compared the results of these learning schemes with each other and with a centralized approach.

As expected, the baseline model (B), which was trained on all available and centralized data performed the best and all individual models (I1-I4) performed worse than the baseline model. The INCART dataset (I3) was arguably too small (n = 74) to produce any sensible deep classification model. I4 performed the best of all models. This can be explained by the fact that I4 was trained on the largest data set (PTB-XL & PTB, 51.86% of the entire dataset) and therefore the test data set also consisted of more than 50 % from this data set, constituting a bias in the test set towards the I4 model.

As shown in Table 4, the novel Batch-wise Sequential Learning (M2b) and Weighted Regression Ensemble (M1b) performed best of all algorithms (see Figure 5). Although M1b achieved a higher AUROC score, M2b was overall the better performing model when considering all metrics. M2b was designed to mimic standard machine learning as close as possible in a decentral scenario and thus it appears sensible, that it performed best out of the tested algorithms.
Furthermore, our weighted variant of the federated learning scheme (M3b) outperformed the standard unweighted algorithm (M3a). The benefit of weighting is assumed to be related to the imbalance of the node size and the heterogeneity of the different datasets (i.e. cancelling out the negative impact of bad performing models trained on small datasets). Since in our experiments one of the nodes was significantly smaller than the others, and since the number of events per class varied a lot among the nodes, weighting led to significantly better results. In a more homogeneous and balanced setting, however, the effect is expected to be smaller.

The novel methods of Batch-wise Sequential Learning and the Weighted Federated Learning both outperformed standard federated learning, although the latter was exceeded by the Weighted Regression Ensemble. All three methods appear to be stable and reliable, showing only minor variance across the 10 cross-validation folds. However, the limitations and technical implications are varied and need to be discussed.

4.1 Limitations and technical implications

As compared to centralized approaches, all decentral schemes require computational power at the nodes, which comes with some challenges for the respective healthcare providers: Firstly, the nodes need to be online and ready for training simultaneously (especially true for M2b, M3a and M3b) and secondly, models are exchanged at a high frequency (most notably M2b), which might cause significant network loads at the nodes. In a real application this could be addressed by nightly routines, where network and computational loads are typically lower.

For scheme M2a and M2b, the order of nodes may have a detrimental influence on the final results, potentially leading to two possible extremes: A) The first node applied on the model might have already optimised the model's parameters towards a local optimum in such a way that other nodes cannot re-adjust them to the global optimum anymore. B) The influence of the first node might be completely extinguished by the other nodes at the end and cause its gradients' change to be irrelevant. This effect is expected to be more prominent in M2a as compared to M2b, as each node is only used once in M2a. Whether extreme A or extreme B is more likely to occur can be adjusted by the learning rate decay. In this experiment, we have reset the learning rate whenever the model was passed over to a new node, making M2a more prone to extreme B. Not resetting the learning rate and decaying it smoothly from node to node might lead to better overall results although this variant would be more susceptible to extreme A. This matter still needs to be explored in the future. Both versions of learning rate decay are explained schematically in Figure 7 below.

For a real-world implementation, the Weighted Regression Ensemble scheme seems highly attractive, since it performed well, is technically simple and requires little network load. However, all the models were currently trained with the same setting (model architecture, number of epochs, learning rate, batch size, etc.) and method of calculating the weights per node (based on size and intra-node AUROC). After the training, the averaging weights represent the only meta-parameter that can still be adapted. However, since only the models' regression outputs are used for M1b, individualising the models and training routine for each specific node could ultimately lead to better results. Therefore, more experiments with different settings for the individual datasets are possible subjects for further research.

4.2 Outlook

Future studies might focus on the influence of hyper-parameters on optimization for these parameters. Additionally, different application scenarios with regression, single label classification, more nodes, more and/or less imbalanced and/or heterogeneous nodes, would allow to further elucidate the advantages and disadvantages of all learning schemes.

Depending on the application scenario, different learning schemes are suitable. While a central approach should be preferred whenever legally and ethically possible, decentral schemes carry considerable potential in scenarios where privacy is of utmost priority. We have shown that the principle of federated learning as a form of privacy-preserving AI is indeed applicable to decentral ECG data. Since federated learning was designed for the application with millions of nodes (e.g. smartphones), its application to healthcare data might require adaptions due to the different characteristics (e.g. less nodes, more heterogenous, etc.). We have demonstrated such adaptions and variations of standard federated learning can improve performance. Especially the properties of each data node should be taken into consideration by the decentral algorithms (e.g. weighting the impact of each individual node), since this increased performance, which is necessary to develop successful decentral learning applications, which could constitute a valuable step towards privacy-preserving applications of AI in healthcare data.

The authors declare that they do not have competing interests. This research has not received a specific grant by any external funding agency in the public or private sector.

Author contributions

Martin Baumgartner and Sai Veeranki conceptualised the study and have contributed inputs of equal value. Martin Baumgartner executed code development and wrote the manuscript’s draft. Dieter Hayn offered in-depth methodological advice and extensive formal, technical and language-related reviews over all chapters in the manuscript. Günter Schreier provided elaborate scientific overview. Sai Veeranki, Dieter Hayn and Günter Schreier provided valuable review of the writing style, formatting and figure design.

Competing interests and funding statement

The authors declare that they do not have competing interests. This research has not received a specific grant by any external funding agency in the public or private sector.

Data availability statement

All data used in this study is open access. Links to the six data sets can be found here:

CPSC

CPSC-Extra

INCART

PTB

PTB-XL

Georgia Database

(from: https://physionetchallenges.org/2020/)

Code availability statement

All programming code (Python) will be made available upon request. Additionally, all code will be released as open-source on GitHub in case of successful publication.

Hosny, A. et al. Artificial intelligence in radiology. Nature Reviews Cancer. 18. 500–510. (2018). https://doi.org/10.1038/s41568-018-0016-5
Dias, R. and Torkamani, A. Artificial intelligence in clinical and genomic diagnostics. Genome Medicine. 11. 70. (2019). https://doi.org/10.1186/s13073-019-0689-8
Claudino, J. G. et al. Current Approaches to the Use of Artificial Intelligence for Injury Risk Assessment and Performance Prediction in Team Sports: a Systematic Review. Sports Medicine - Open. 5. 28. (2019). https://doi.org/10.1186/s40798-019-0202-3
Schmidt-Erfurth, U. et al. Prediction of Individual Disease Conversion in Early AMD Using Artificial Intelligence. Investigative Ophthalmology & Visual Science. 59. 3199–3208. (2018). https://doi.org/10.1167/iovs.18-24106
Makino, M. et al. Artificial intelligence predicts the progression of diabetic kidney disease using big data machine learning. Scientific Reports. 9. 11862. (2019). https://doi.org/10.1038/s41598-019-48263-5
He, K. Zhang, X. Ren, S. and Sun, J. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778. (2016).
Perez Alday, E. A. et al. Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020. Physiological Measurement. (2020).
Zhao, Z. et al. Adaptive Lead Weighted ResNet Trained With Different Duration Signals for Classifying 12-lead ECGs. Computing in Cardiology. 1–4. (2020). https://doi.org/10.22489/CinC.2020.112
Fayyazifar, N. et al. Impact of Neural Architecture Design on Cardiac Abnormality Classification Using 12-lead ECG Signals. Computing in Cardiology. 12–15. (2020). https://doi.org/10.22489/CinC.2020.161
Jia, W. et al. Automatic Detection and Classification of 12-lead ECGs Using a Deep Neural Network. Computing in Cardiology. (2020). https://doi.org/10.22489/CinC.2020.035
Bos, M. N. et al. Automated Comprehensive Interpretation of 12-lead Electrocardiograms Using Pre-trained Exponentially Dilated Causal Convolutional Neural Networks. Computing in Cardiology. 2–5. (2020). https://doi.org/10.22489/CinC.2020.253
Chen, J. et al. SE-ECGNet: Multi-scale SE-Net for Multi-lead ECG Data The First Affiliated Hospital of Chongqing Medical University, Chongqing, China Data processing. Computing in Cardiology. 1–4. (2020).
Zhu, Z. et al. Classification of Cardiac Abnormalities from ECG Signals Using SE-ResNet. Computing in Cardiology. 0–3. (2020). https://doi.org/10.22489/CinC.2020.281
Min, S. et al. Bag of Tricks for Electrocardiogram Classification with Deep Neural Networks. Computing in Cardiology. (2020). https://doi.org/10.22489/CinC.2020.328
Oppelt, M. P. Riehl, M. Kemeth, F. P. and Steffan, J. Combining Scatter Transform and Deep Neural Networks for Multilabel Electrocardiogram Signal Classification. Computing in Cardiology. (2020). https://doi.org/10.22489/CinC.2020.133
Natarajan, A. et al. A Wide and Deep Transformer Neural Network for 12-Lead ECG Classification. Computing in Cardiology. 1–4. (2020). https://doi.org/10.22489/CinC.2020.107
Hasani, H. Bitarafan, A. and Baghshah, M. S. Classification of 12-lead ECG Signals with Adversarial Multi-Source Domain Generalization. Computing in Cardiology. (2020). https://doi.org/10.22489/CinC.2020.445
Halevy, A. Norvig, P. and Pereira, F. The Unreasonable Effectiveness of Data. IEEE Intelligent Systems. 24. 8–12. (2009). https://doi.org/10.1109/MIS.2009.36
Obermeyer, Z. and Emanuel, E. J. Predicting the Future - Big Data, Machine Learning, and Clinical Medicine. The New England Journal of Medicine. 375. 1216–1219. (2016). https://doi.org/10.1056/NEJMp1606181
Vali-Betts, E. et al. Effects of Image Quantity and Image Source Variation on Machine Learning Histology Differential Diagnosis Models. Journal of Pathology Informatics. 12. 5. (2021). https://doi.org/10.4103/jpi.jpi_69_20
Andrew, N. Andrew Ng: Unbiggen AI. IEEE Spectrum. (2022).
Malin, B. and Sweeney, L. Re-identification of DNA through an automated linkage process. Proceedings. AMIA Symposium. 423–427. (2001).
Sweeney, L. Abu, A. and Winn, J. Identifying participants in the personal genome project by name (a re-identification experiment). arXiv.org. (2013).
Narayanan, A. and Shmatikov, V. How To Break Anonymity of the Netflix Prize Dataset. arXiv:Cryptography and Security. (2006).
Na, L. et al. Feasibility of Reidentifying Individuals in Large National Physical Activity Data Sets From Which Protected Health Information Has Been Removed With Use of Machine Learning. JAMA Network Open. 1. e186040–e186040. (2018). https://doi.org/10.1001/jamanetworkopen.2018.6040
Sweeney, L. Simple demographics often identify people uniquely. Health (San Francisco). 671. 1–34. (2000).
Dwork, C. The Definition of Differential Privacy, Institute for Advanced Study, 2016. YouTube [Online]: https://www.youtube.com/watch?v=lg-VhHlztqo
Sweeney, L. k-anonymity: A model for protecting privacy. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems. 10. 557–570. (2002).
Machanavajjhala, A. Kifer, D. Gehrke, J. and Venkitasubramaniam, M. l-diversity: Privacy beyond k-anonymity. ACM Transactions on Knowledge Discovery from Data (TKDD). 1. 3-es. (2007).
Li, N. Li, T. and Venkatasubramanian, S. t-closeness: Privacy beyond k-anonymity and l-diversity. 2007 IEEE 23rd International Conference on Data Engineering. 106–115. (2007).
Gentry, C. A fully homomorphic encryption scheme. (Stanford University: 2009)
Dwork, C. and Roth, A. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science. 9. 211–407. (2014).
Desfontaines, D. and Pejó, B. Sok: differential privacies. arXiv preprint arXiv:1906.01337. (2019).
Cormode, G. Procopiuc, C. Srivastava, D. and Tran, T. T. L. Differentially private summaries for sparse data. Proceedings of the 15th International Conference on Database Theory. 299–311. (2012).
Gondara, L. and Wang, K. Differentially Private Small Dataset Release Using Random Projections. Conference on Uncertainty in Artificial Intelligence. 639–648. (2020).
Goodfellow, I. et al. Generative Adversarial Nets. Advances in Neural Information Processing Systems 27. 2672–2680. (2014)Available: http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf
Shin, H.-C. et al. Medical Image Synthesis for Data Augmentation and Anonymization Using Generative Adversarial Networks. 2018 Workshop on Simulation and Synthesis in Medical Imaging. 1–11. (2018).
Baumgartner, M. et al. Experimenting with Generative Adversarial Networks to Expand Sparse Physiological Time-Series Data. Studies in Health Technology and Informatics. 271. 248–255. (2020).
Konečný, J. McMahan, B. and Ramage, D. Federated optimization: Distributed optimization beyond the datacenter. arXiv.org. (2015).
Bonawitz, K. et al. Practical Secure Aggregation for Privacy-Preserving Machine Learning. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 1175–1191. (2017). https://doi.org/10.1145/3133956.3133982
Rieke, N. et al. The future of digital health with federated learning. npj Digital Medicine. 3. 119. (2020). https://doi.org/10.1038/s41746-020-00323-1
Sheller, M. J. et al. Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Scientific reports. 10. 12598. (2020). https://doi.org/10.1038/s41598-020-69250-1
Liu, F. et al. An open access database for evaluating the algorithms of electrocardiogram rhythm and morphology abnormality detection. Journal of Medical Imaging and Health Informatics. 8. 1368–1373. (2018).
Goldberger, A. L. et al. PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals. Circulation. 101. e215–e220. (2000).
Bousseljot, R. Kreiseler, D. and Schnabel, A. Nutzung der EKG-Signaldatenbank CARDIODAT der PTB über das Internet, Biomedizinische Technik / Biomedical Engineering, 40. 317–318, 1995.
Chen, T. M. et al. Detection and classification of cardiac arrhythmias by a challenge-best deep learning neural network model. bioRxiv. (2019). https://doi.org/10.1101/766022
Kingma, D. P. and Ba, J. Adam: A Method for Stochastic Optimization. arXiv.org. (2014).
Abadi, M. et al. Tensorflow: A system for large-scale machine learning. OSDI’16: Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation. 265–283. (2016).
Ren, Y. Zhang, L. and Suganthan, P. N. Ensemble Classification and Regression-Recent Developments, Applications and Future Directions. IEEE Computational Intelligence Magazine. 11. 41–53. (2016). https://doi.org/10.1109/MCI.2015.2471235
Pedregosa, F. et al. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research. 12. 2825–2830. (2011).

No competing interests reported.

Download PDF

Journal Publication

published 17 Aug, 2023

Read the published version in Journal of Healthcare Informatics Research →

Editorial decision: Major revision
10 Feb, 2023
Reviews received at journal
09 Feb, 2023
Reviews received at journal
18 Nov, 2022
Reviewers agreed at journal
15 Nov, 2022
Reviewers agreed at journal
07 Nov, 2022
Reviewers invited by journal
07 Nov, 2022
Editor assigned by journal
16 Aug, 2022
Submission checks completed at journal
16 Aug, 2022
First submitted to journal
12 Aug, 2022

You are reading this latest preprint version

Introduction and Comparison of Novel Decentral Learning Schemes with Multiple Data Pools for Privacy-preserving ECG Classification

Status:

Journal Publication

Version 1

Abstract

Figures

1. Introduction

1.1 Artificial Intelligence (AI) in healthcare

1.2 Artificial intelligence (AI) and clinical data

1.5 Aims and scope

2. Methods

2.1 Data description

2.3 Model architecture description

2.4 Learning schemes

B: Baseline Centralised Model

I1-I4: Individual Models

M1a: Regression Ensemble

M1b: Weighted Regression Ensemble

M2a: Node-wise Sequential Learning

M2b: Batch-wise Sequential Learning

M3a: Federated Learning

M3b: Weighted Federated Learning

2.5 Cross validation and evaluation metrics

3. Results

3.1 Model performance

3.2 Average rank per model

3.3 Evaluation metrics per model

4. Discussion

4.2 Outlook

5. Conclusion

Statements and Declarations

References

Additional Declarations

Status:

Journal Publication

Version 1