See the published version of this preprint at BMC Medical Informatics and Decision Making.
This is a preprint, a preliminary version of a manuscript that has not completed peer review at a journal. Research Square does not conduct peer review prior to posting preprints. The posting of a preprint on this server should not be interpreted as an endorsement of its validity or suitability for dissemination as established information or for guiding clinical practice.
Learn more about In Review
A new preprint design is coming!
We have improved readability, clarity, accessibility, and opportunities for engagement.
Research article
Daniel B Hier
University of Illinois at Chicago College of Medicine
Corresponding Author
ORCiD: https://orcid.org/0000-0002-6179-0793
License:
This work is licensed under a CC BY 4.0 License. Read Full License
Background: When patient distances are calculated based on phenotype, signs and symptoms are often converted to concepts from an ontological hierarchy. There is controversy as to whether patient distance metrics that consider the semantic similarity between concepts can outperform standard patient distance metrics that are agnostic to concept similarity. The choice of distance metric often dominates the performance of classification or clustering algorithms. Our objective was to determine if semantically augmented distance metrics would outperform standard metrics on machine learning tasks.
Methods: We converted the neurological signs and symptoms from 382 published neurology cases into sets of concepts with corresponding machine-readable codes. We calculated inter-patient distances by four different metrics (cosine distance, a semantically augmented cosine distance, Jaccard distance, and a semantically augmented bipartite distance). Semantic augmentation for two of the metrics depended on concept similarities from a hierarchical neuro-ontology. For machine learning algorithms, we used the patient diagnosis as the ground truth label and patient signs and symptoms as the machine learning features . We assessed classification accuracy for four classifiers and cluster quality for two clustering algorithms for each of the distance metrics.
Results: Inter-patient distances were smaller when the distance metric was semantically augmented. Classification accuracy and cluster quality were not significantly different by distance metric.
Conclusion: Using patient diagnoses as labels and patient signs and symptoms as features, we did not find improved classification accuracy or improved cluster quality with semantically augmented distance metrics. Semantic augmentation reduced inter-patient distances but did not improve machine learning performance.
Keywords
ontology, patient similarity, distance metric, neurological findings, signs and symptoms, UMLS, SNOMED CT, classification, diagnostic error, proximity
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
The full text of this article is available to read as a PDF.
Loading...
Badges
Prescreen
This manuscript passed our Prescreen assessment
Complete author information
Appropriate declaration statements
Potential risks to human health
Prescreen
Peer Review Timeline
CURRENT STATUS: Published
View the published article at BMC Medical Informatics and Decision Making.
No community comments so far
Editorial decision: Accept
On 12 Aug, 2020
Editor assigned
On 10 Aug, 2020
Submission checks complete
On 09 Aug, 2020
Editor invited
On 09 Aug, 2020
No comments provided
Review #2 received
Received 03 Aug, 2020
Reviewer #4 agreed
On 23 Jul, 2020
Review #3 received
Received 21 Jul, 2020
Reviewer #1 agreed
On 21 Jul, 2020
Reviewer #2 agreed
On 21 Jul, 2020
Reviewer #3 agreed
On 21 Jul, 2020
Review #1 received
Received 21 Jul, 2020
Editor assigned
On 20 Jul, 2020
Reviewers invited
Invitations sent on 20 Jul, 2020
Editor invited
On 19 Jul, 2020
Submission checks complete
On 17 Jul, 2020
Version (no preprint)
Posted 21 Jul, 2020
Editor assigned
On 06 Aug, 2020
Submission checks complete
On 05 Aug, 2020
Editor invited
On 05 Aug, 2020
This version was not preprinted
Version 2
Posted 19 Jun, 2020
No comments provided
Editorial decision: Minor revision
On 06 Jul, 2020
Review #4 received
Received 03 Jul, 2020
Review #3 received
Received 03 Jul, 2020
Review #1 received
Received 03 Jul, 2020
Review #2 received
Received 02 Jul, 2020
Review #5 received
Received 30 Jun, 2020
Reviewer #6 agreed
On 22 Jun, 2020
Reviewer #7 agreed
On 22 Jun, 2020
Reviewer #5 agreed
On 22 Jun, 2020
Reviewer #2 agreed
On 21 Jun, 2020
Reviewer #3 agreed
On 21 Jun, 2020
Reviewer #4 agreed
On 21 Jun, 2020
Reviewer #1 agreed
On 20 Jun, 2020
Reviewers invited
Invitations sent on 19 Jun, 2020
Editor assigned
On 18 Jun, 2020
Submission checks complete
On 17 Jun, 2020
Editor invited
On 17 Jun, 2020
No comments provided
Editorial decision: Major revision
On 17 May, 2020
Review #3 received
Received 05 May, 2020
Review #4 received
Received 05 May, 2020
Review #1 received
Received 02 May, 2020
Review #2 received
Received 01 May, 2020
Reviewer #4 agreed
On 14 Apr, 2020
Reviewer #3 agreed
On 13 Apr, 2020
Reviewer #2 agreed
On 12 Apr, 2020
Reviewer #1 agreed
On 11 Apr, 2020
Reviewers invited
Invitations sent on 10 Apr, 2020
Editor assigned
On 30 Mar, 2020
Submission checks complete
On 29 Mar, 2020
Editor invited
On 29 Mar, 2020
First submitted
On 27 Mar, 2020
Metrics
Comments: 0
PDF Downloads: ...
HTML Views: ...
Learn more about our company.
A new preprint design is coming!
We have improved readability, clarity, accessibility, and opportunities for engagement.
See the published version of this preprint at BMC Medical Informatics and Decision Making.
This is a preprint, a preliminary version of a manuscript that has not completed peer review at a journal. Research Square does not conduct peer review prior to posting preprints. The posting of a preprint on this server should not be interpreted as an endorsement of its validity or suitability for dissemination as established information or for guiding clinical practice.
Learn more about In Review
Research article
Daniel B Hier
University of Illinois at Chicago College of Medicine
Corresponding Author
ORCiD: https://orcid.org/0000-0002-6179-0793
Badges
Prescreen
This manuscript passed our Prescreen assessment
Complete author information
Appropriate declaration statements
Potential risks to human health
Prescreen
Peer Review Timeline
CURRENT STATUS: Published
View the published article at BMC Medical Informatics and Decision Making.
No community comments so far
Editorial decision: Accept
On 12 Aug, 2020
Editor assigned
On 10 Aug, 2020
Submission checks complete
On 09 Aug, 2020
Editor invited
On 09 Aug, 2020
No comments provided
Review #2 received
Received 03 Aug, 2020
Reviewer #4 agreed
On 23 Jul, 2020
Review #3 received
Received 21 Jul, 2020
Reviewer #1 agreed
On 21 Jul, 2020
Reviewer #2 agreed
On 21 Jul, 2020
Reviewer #3 agreed
On 21 Jul, 2020
Review #1 received
Received 21 Jul, 2020
Editor assigned
On 20 Jul, 2020
Reviewers invited
Invitations sent on 20 Jul, 2020
Editor invited
On 19 Jul, 2020
Submission checks complete
On 17 Jul, 2020
Version (no preprint)
Posted 21 Jul, 2020
Editor assigned
On 06 Aug, 2020
Submission checks complete
On 05 Aug, 2020
Editor invited
On 05 Aug, 2020
This version was not preprinted
Version 2
Posted 19 Jun, 2020
No comments provided
Editorial decision: Minor revision
On 06 Jul, 2020
Review #4 received
Received 03 Jul, 2020
Review #3 received
Received 03 Jul, 2020
Review #1 received
Received 03 Jul, 2020
Review #2 received
Received 02 Jul, 2020
Review #5 received
Received 30 Jun, 2020
Reviewer #6 agreed
On 22 Jun, 2020
Reviewer #7 agreed
On 22 Jun, 2020
Reviewer #5 agreed
On 22 Jun, 2020
Reviewer #2 agreed
On 21 Jun, 2020
Reviewer #3 agreed
On 21 Jun, 2020
Reviewer #4 agreed
On 21 Jun, 2020
Reviewer #1 agreed
On 20 Jun, 2020
Reviewers invited
Invitations sent on 19 Jun, 2020
Editor assigned
On 18 Jun, 2020
Submission checks complete
On 17 Jun, 2020
Editor invited
On 17 Jun, 2020
No comments provided
Editorial decision: Major revision
On 17 May, 2020
Review #3 received
Received 05 May, 2020
Review #4 received
Received 05 May, 2020
Review #1 received
Received 02 May, 2020
Review #2 received
Received 01 May, 2020
Reviewer #4 agreed
On 14 Apr, 2020
Reviewer #3 agreed
On 13 Apr, 2020
Reviewer #2 agreed
On 12 Apr, 2020
Reviewer #1 agreed
On 11 Apr, 2020
Reviewers invited
Invitations sent on 10 Apr, 2020
Editor assigned
On 30 Mar, 2020
Submission checks complete
On 29 Mar, 2020
Editor invited
On 29 Mar, 2020
First submitted
On 27 Mar, 2020
Metrics
Comments: 0
PDF Downloads: ...
HTML Views: ...
License:
This work is licensed under a CC BY 4.0 License. Read Full License
View the published article at BMC Medical Informatics and Decision Making.
Background: When patient distances are calculated based on phenotype, signs and symptoms are often converted to concepts from an ontological hierarchy. There is controversy as to whether patient distance metrics that consider the semantic similarity between concepts can outperform standard patient distance metrics that are agnostic to concept similarity. The choice of distance metric often dominates the performance of classification or clustering algorithms. Our objective was to determine if semantically augmented distance metrics would outperform standard metrics on machine learning tasks.
Methods: We converted the neurological signs and symptoms from 382 published neurology cases into sets of concepts with corresponding machine-readable codes. We calculated inter-patient distances by four different metrics (cosine distance, a semantically augmented cosine distance, Jaccard distance, and a semantically augmented bipartite distance). Semantic augmentation for two of the metrics depended on concept similarities from a hierarchical neuro-ontology. For machine learning algorithms, we used the patient diagnosis as the ground truth label and patient signs and symptoms as the machine learning features . We assessed classification accuracy for four classifiers and cluster quality for two clustering algorithms for each of the distance metrics.
Results: Inter-patient distances were smaller when the distance metric was semantically augmented. Classification accuracy and cluster quality were not significantly different by distance metric.
Conclusion: Using patient diagnoses as labels and patient signs and symptoms as features, we did not find improved classification accuracy or improved cluster quality with semantically augmented distance metrics. Semantic augmentation reduced inter-patient distances but did not improve machine learning performance.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
The full text of this article is available to read as a PDF.
Loading...