This preprint is under consideration at BMC Bioinformatics. A preprint is a preliminary version of a manuscript that has not completed peer review at a journal. Research Square does not conduct peer review prior to posting preprints. The posting of a preprint on this server should not be interpreted as an endorsement of its validity or suitability for dissemination as established information or for guiding clinical practice.
Learn more about In Review
Research Article
Improving Biomedical Named Entity Recognition with Label Re-correction and Knowledge Distillation
Huiwei Zhou
Dalian University of Technology
Corresponding Author
License:
This work is licensed under a CC BY 4.0 License. Read Full License
Background: Biomedical named entities recognition is one of the most essential tasks in biomedical information extraction. Previous studies suffer from inadequate annotation datasets, especially the limited knowledge contained in them.
Methods: To remedy the above issue, we propose a novel Chemical and Disease Named Entity Recognition (CDNER) framework with label re-correction and knowledge distillation strategies, which could not only create large and high-quality datasets but also obtain a high-performance entity recognition model. Our framework is inspired by two points: 1) named entity recognition should be considered from the perspective of both coverage and accuracy; 2) trustable annotations should be yielded by iterative correction. Firstly, for coverage, we annotate chemical and disease entities in a large unlabeled dataset by PubTator to generate a weakly labeled dataset. For accuracy, we then filter it by utilizing multiple knowledge bases to generate another dataset. Next, the two datasets are revised by a label re-correction strategy to construct two high-quality datasets, which are used to train two CDNER models, respectively. Finally, we compress the knowledge in the two models into a single model with knowledge distillation.
Results: Experiments on the BioCreative V chemical-disease relation corpus show that knowledge from large datasets significantly improves CDNER performance, leading to new state-of-the-art results.
Conclusions: We propose a framework with label re-correction and knowledge distillation strategies. Comparison results show that the two perspectives of knowledge in the two re-corrected datasets respectively are complementary and both effective for biomedical named entity recognition.
Keywords
Biomedical named entity recognition, Knowledge distillation, Label re-correction
Figure 1
Figure 2
Figure 3
Figure 4
Due to technical limitations, full-text HTML conversion of this manuscript could not be completed. However, the manuscript can be downloaded and accessed as a PDF.
Badges
Prescreen
This manuscript passed our Prescreen assessment
Complete author information
Appropriate declaration statements
Potential risks to human health
Prescreen
Peer Review Timeline
CURRENT STATUS: Under Review
Version 1
Posted 15 Dec, 2020
No community comments so far
Review #5 received
Received 04 Jan, 2021
Review #3 received
Received 04 Jan, 2021
Review #4 received
Received 04 Jan, 2021
Review #1 received
Received 04 Jan, 2021
Review #2 received
Received 04 Jan, 2021
Reviewer #2 agreed
On 17 Dec, 2020
Reviewer #7 agreed
On 17 Dec, 2020
Reviewer #5 agreed
On 17 Dec, 2020
Reviewer #6 agreed
On 17 Dec, 2020
Reviewer #3 agreed
On 17 Dec, 2020
Reviewer #4 agreed
On 17 Dec, 2020
Reviewer #1 agreed
On 17 Dec, 2020
Reviewers invited
Invitations sent on 14 Dec, 2020
Editor assigned
On 14 Dec, 2020
Editor invited
On 14 Dec, 2020
Submission checks complete
On 14 Dec, 2020
First submitted
On 10 Dec, 2020
Metrics
Comments: 0
PDF Downloads: ...
HTML Views: ...
Learn more about our company.
This preprint is under consideration at BMC Bioinformatics. A preprint is a preliminary version of a manuscript that has not completed peer review at a journal. Research Square does not conduct peer review prior to posting preprints. The posting of a preprint on this server should not be interpreted as an endorsement of its validity or suitability for dissemination as established information or for guiding clinical practice.
Learn more about In Review
Research Article
Improving Biomedical Named Entity Recognition with Label Re-correction and Knowledge Distillation
Huiwei Zhou
Dalian University of Technology
Corresponding Author
Badges
Prescreen
This manuscript passed our Prescreen assessment
Complete author information
Appropriate declaration statements
Potential risks to human health
Prescreen
Peer Review Timeline
CURRENT STATUS: Under Review
Version 1
Posted 15 Dec, 2020
No community comments so far
Review #5 received
Received 04 Jan, 2021
Review #3 received
Received 04 Jan, 2021
Review #4 received
Received 04 Jan, 2021
Review #1 received
Received 04 Jan, 2021
Review #2 received
Received 04 Jan, 2021
Reviewer #2 agreed
On 17 Dec, 2020
Reviewer #7 agreed
On 17 Dec, 2020
Reviewer #5 agreed
On 17 Dec, 2020
Reviewer #6 agreed
On 17 Dec, 2020
Reviewer #3 agreed
On 17 Dec, 2020
Reviewer #4 agreed
On 17 Dec, 2020
Reviewer #1 agreed
On 17 Dec, 2020
Reviewers invited
Invitations sent on 14 Dec, 2020
Editor assigned
On 14 Dec, 2020
Editor invited
On 14 Dec, 2020
Submission checks complete
On 14 Dec, 2020
First submitted
On 10 Dec, 2020
Metrics
Comments: 0
PDF Downloads: ...
HTML Views: ...
License:
This work is licensed under a CC BY 4.0 License. Read Full License
Background: Biomedical named entities recognition is one of the most essential tasks in biomedical information extraction. Previous studies suffer from inadequate annotation datasets, especially the limited knowledge contained in them.
Methods: To remedy the above issue, we propose a novel Chemical and Disease Named Entity Recognition (CDNER) framework with label re-correction and knowledge distillation strategies, which could not only create large and high-quality datasets but also obtain a high-performance entity recognition model. Our framework is inspired by two points: 1) named entity recognition should be considered from the perspective of both coverage and accuracy; 2) trustable annotations should be yielded by iterative correction. Firstly, for coverage, we annotate chemical and disease entities in a large unlabeled dataset by PubTator to generate a weakly labeled dataset. For accuracy, we then filter it by utilizing multiple knowledge bases to generate another dataset. Next, the two datasets are revised by a label re-correction strategy to construct two high-quality datasets, which are used to train two CDNER models, respectively. Finally, we compress the knowledge in the two models into a single model with knowledge distillation.
Results: Experiments on the BioCreative V chemical-disease relation corpus show that knowledge from large datasets significantly improves CDNER performance, leading to new state-of-the-art results.
Conclusions: We propose a framework with label re-correction and knowledge distillation strategies. Comparison results show that the two perspectives of knowledge in the two re-corrected datasets respectively are complementary and both effective for biomedical named entity recognition.
Figure 1
Figure 2
Figure 3
Figure 4
Due to technical limitations, full-text HTML conversion of this manuscript could not be completed. However, the manuscript can be downloaded and accessed as a PDF.