Screening Children at Risk for Developmental Disabilities Based on Face Landmark from Video Data of Mobile-Based Application: Preliminary Cross-Sectional Study

doi:10.21203/rs.3.rs-993056/v1

Download PDF

Research Article

Screening Children at Risk for Developmental Disabilities Based on Face Landmark from Video Data of Mobile-Based Application: Preliminary Cross-Sectional Study

https://doi.org/10.21203/rs.3.rs-993056/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background

Early detection and intervention of developmental disabilities (DDs) are critical for improving the long-term outcomes of the afflicted children. Mobile-based applications are easily accessible and may thus help the early identification of DDs.

Objective

We aimed to identify facial expression and head pose based on face landmark data extracted from face recording videos and to differentiate the characteristics between children with DDs and those without.

Methods

Eighty-nine children (DD, n=33; typically developing, n=56) were included in the analysis. Using the mobile-based application, we extracted facial landmarks and head poses from the recorded videos and performed Long Short-Term Memory (LSTM)-based DD classification.

Results

Stratified k-fold cross-validation showed that the average values of accuracy, precision, recall, and f1-score of the LSTM based deep learning model of DD children were 88%, 91%,72%, and 80%, respectively. Through the interpretation of prediction results using SHapley Additive exPlanations (SHAP), we confirmed that the nodding head angle variable was the most important variable. All of the top 10 variables of importance had significant differences in the distribution between children with DDs and those without (p<0.05).

Conclusions

Our results provide preliminary evidence that the deep-learning classification model using mobile-based children’s video data could be used for the early detection of children with DDs.

Health Economics & Outcomes Research

Health Policy

developmental disabilities

mobile-based application

cross-validation

LSTM

SHapley Additive exPlanations (SHAP)

Developmental disabilities (DDs) are conditions that show impairments in physical, learning, language, or behavioral development, and encompass autism spectrum disorder (ASD), language disorder (LD), and intellectual disability (ID). The prevalence of children with DD showed a marked increase worldwide, affecting 17.8% of children in the US from 2015 to 2017.¹ According to a recent study, the prevalence of DD in South Korea steadily increased by more than four times from 2003 to 2017.² DDs significantly and negatively affect the quality of life for both the afflicted individuals and their families³ as a large amount of medical and social support is needed for children with DD. The social costs of DD are significant and expected to grow in size considering the increasing prevalence of DD.^4,5

Early identification of DDs is crucial for children to receive early evidence-based interventions,^6–8 which have been shown to be highly effective in improving the outcomes of children with DD. Although many studies emphasized the importance of early detection and intervention, difficulties in universal screening lead to the delay in the age of identification and diagnosis.⁹ Currently, ASD, ID, and LD are diagnosed by standardized instruments that evaluate the language, cognition, or social development, including Bayley Scales of Infant Development (BSID), Psychoeducational Profile - Revised (PEP-R), Autism Diagnostic Observation Schedule (ADOS) and the Autism Diagnostic Interview-Revised (ADI-R).^8,10,11 However, those standardized tests usually take much time and need to be conducted by trained professionals.¹² This highlights the need for user-friendly and time-efficient mobile screening tools for children with DD.

As mobile devices are becoming widely distributed, the possibility that mobile-based screening tools could be useful in the early identification of DD was suggested. Mobile-based screening programs are easily accessible and are less time-consuming, do not require trained professionals, and may thus help speed up the identification of DD. Different types of mobile-based screening measures for DD are emerging, including Cognoa ASD Screener, a platform that examines home video by analysts,¹³ iTracker, an eye-tracking algorithm,¹⁴ and ASDTest, an application based on standardized tests such as the Autism Quotient 10 (AQ) and Quantitative Checklist for Autism in Toddlers (Q-CHAT).¹⁵ Video-analyzing platforms^13,16 and eye-tracking algorithms¹⁷ are the most frequently studied mobile device-based methods for identifying DDs. In contrast, screening tools based on facial emotion recognition and expression have only been recently introduced.

Facial emotional recognition and expressions are indispensable in the sharing of emotions and human-to-human interactions. People with DD, especially ASD, have difficulties understanding and expressing facial emotions.^18–20 Although much research has been done on screening DD using facial emotion recognition, not much research has been done on screening ASD with facial emotion production.²¹ Similarly, only few studies have been performed on facial emotion expression in individuals with ID despite there being several studies regarding facial emotion recognition in individuals with ID.^22,23 Moreover, only few studies have been conducted on LD and facial emotion recognition and expression. Recently, some studies reported a significant difference in facial expression between children with DD and those without using models that categorize facial expression with facial landmarks.^24,25 Manfredonia and colleagues²⁵ recorded videos of participants and divided them into image frames. These studies suggest that programs examining the facial expression of children based on face recording video data could be useful for identifying children with DD. Thus, we aimed to identify facial expressions and head pose based on face landmark data extracted from face recording videos and to differentiate the characteristics between children with DD and those without.

Mobile-based application

This study aimed to examine the effectiveness of a deep learning classification model from facial video data of children collected from a mobile-based application in identifying the facial expressions and head poses of children with developmental disabilities and those without. The mobile-based application records the faces of children staring at the screen and saves them as videos. Games in mobile applications have been redesigned for this study. Among them, an animated film for eye tracking tests is from "D-kit" previously used by investigators. In previous studies, D-kit animation has been demonstrated to increase children's participation in training sessions. The eye tracking test consists of animation content and non-animated content. Non-animated content tracks the subject's eye movements to test the subject's attention and participation level when exposed to the content. The test shows four different paintings, friendly animal characters, blocks, different animal characters, and animal characters with blocks. The content tests whether children's eyes follow the target object and whether they are more interested in the social characteristics that children can sympathize with. Animation content shows four different pictures from the previous test, but it shows the animation method, and tests whether more children are involved when objects move and whether the results of non-animation tests are the same. (Supplementary Figure A).

Participant enrollment

From May 2020 to July 2020, a total of 124 children were recruited from community-based daycare centers, kindergartens, and special education centers. The children were between 34 and 77 months of age. Children were not included if they had (i) a history of neurologic diseases such as cerebral palsy, (ii) any sensory disturbances (i.e., vision, hearing, taste, or smell), and (iii) severe gross or fine motor problems that prevented them from participating in the psychometric tests. Of the 124 children, data from 35 children could not be analyzed because (i) facial data could not be extracted due to the use of facial masks (n=25), (ii) facial data during video games were not available (n=5), and (iii) landmark points were incorrectly extracted (n=5). As a result, a total of 89 children were analyzed in this study (Figure 1).

The participants of this study underwent comprehensive developmental assessment. The children performed the Psychoeducational Profile-Revised (PEP-R), and their caregivers completed Vineland Adaptive Behavior Scales (VABS), the Korean version of the Childhood Autism Rating Scale (K-CARS), Social Responsiveness Scale (SRS) and Child Behavior Checklist (CBCL). Diagnoses of DD were made by clinicians according to the Diagnostic and Statistical Manual of Mental Disorders Fifth Edition (DSM-5). Among typically developing children, children with confusing test results or developmental concerns were seen by clinicians and confirmed as typically-developing controls. The study was approved by the institutional review board of Asan Medical Center and informed consent was obtained from the parents of each child. In addition, this study was performed in accordance with the principles of Good Clinical Practice and the Helsinki Declaration.

Data collection and preprocessing

Face landmark extraction

Face landmarks are standard reference points, such as the inner and outer corner of the eye fissure where the eyelids meet. In many cases, the landmarks used in computational face analysis are very similar to the anatomical soft tissue landmarks used by physicians. These extracted landmark points can be applied in various fields such as human emotion recognition, gaze detection, and face conversion. In our work, we use 2D-FAN (Face Alignment Net), a convolutional neural network-based method, to recognize children's faces in videos and extract 68 landmarks. The corresponding algorithm has been trained with an LS3D-W dataset consisting of approximately 230,000 face photographs including adult men and women, children, and showed higher performance when extracting the face landmarks compared with other algorithms.²⁶ We decomposed the video into a frame unit image of 33 ms for each child, recognizing the child's face from each image and extracting the face landmark points (Figure 2).

Preprocessing

The extracted face landmark points are stored in 68 coordinate values. For frames in which the extraction algorithm did not recognize faces, we interpolated the coordinate values of the landmark points by reflecting the information of the previous and subsequent frames of those frames. In video data, when a child bows his head or goes out of the video screen, the face land mark may not be properly extracted. In this case, the video frame was regarded as an abnormal frame and removed (Figure 2).

Feature extraction

We utilized the OpenCV library to estimate the head pose in that frame through the landmark coordinate values. We measured three head poses (Pitch, Roll, Yaw) by specifying six landmark points (i.e., eyes, nose, chin, and left and right mouth) and measuring the Euler angle of how much the points were rotated in the frame by assuming 3D coordinates with the coordinates facing forward. Pitch was measured at an angle of nodding the head up and down, Roll was archwise and left and right, Yaw was the angle of rotation of the head from left to right, and each value was measured at an angle of -90 to 90 degrees (Figure 2).

To measure how much a child's face has moved over the previous frame in the current frame, we assumed the average position of 68 landmark coordinate values in the frame as the center point of the face, and calculated the Euclidean distance to determine the distance traveled. Since each frame was captured with a frame rate of 33 ms, the distance the face traveled was measured by reflecting the frame rate.

The changes between 68 landmark points were measured in the video. A combination of all landmark points allows the construction of a total of 2284 combinations of distance variables. We calculated all Euclidean distances of those combinations and selected frames staring forward in each child's video. Based on the distance of the corresponding frame's landmark combination, the ratio of the remaining frames to the normal frame was obtained. Among all the obtained proportion variables, the top 40 variables were used as derivatives by selecting those with significant differences in the distribution between children with developmental disabilities and those without.

Model algorithm

The data used in the current analyses were time-series data consisting of frames of video data recording the faces of children. Accordingly, we used the long short-term memory (LSTM) model for the binary classification of developmental disability.²⁷ As a recurrent neural network (RNN) model, the LSTM model determines whether the weight value is maintained by adding cell states in an LSTM cell. The state obtained from an LSTM cell is used as input to the next LSTM cell, so the state of an LSTM cell affects the operation of the subsequent cells. The final target output at the end of the sequence represents a label classifying the developmental disability. The LSTM model has the ability to remove or add information to the cell state, carefully regulated by structures called the gates, which are a way of optionally letting information through. The LSTM model is more persistent than the existing RNN because it is possible to control long-term memory.²⁸ Since the lengths of each of the seven videos are different, we generated an LSTM model corresponding to each video, utilized variables such as sex and age as input variables in additional deep neural network (DNN) models, and built a model to combine the results of seven LSTM models with the results of one DNN model to finally predict developmental disability (Figure 3).

We also performed stratified K-cross fold validation for the robustness of the model. Stratification is the process of rearranging the data to ensure that each fold is a good representative of the whole. The stratified K-cross fold validation technique splits the dataset into K sets and the model uses K-1 folds for training and is validated on the K^th fold. This is continued until all the folds are used to validate the model once. Stratification ensures that each fold is a good representation of the whole dataset, which leads to parameter finetuning and helps the model in better classifying the developmental disability.²⁹ The K used in our study was five. For the evaluation of the trained model, the following standard machine learning metrics were used:

1. Accuracy: Percentage of correctly classified data frames in the given test dataset.

$$Accuracy= \frac{True positive+True negatives}{True positives+True negatives+False positives+False negatives}$$

2. Precision: Measure of exactness or quality of model prediction.

$$Precision= \frac{True positives}{True positives+False positives}$$

3. Recall: Measure of completeness or quantity of model prediction.

$$Recall=\frac{True positives}{True positives+False negatives}$$

4. F1 score: Harmonic mean of the precision and the recall.

$$F1 score=2\times \frac{Precision \times Recall}{Precision +Recall}$$

True positives are data samples that are classified as positive by the model and are actually positive. False positives are data samples that are classified as positive by the model but are actually negative. False negatives are data samples that are classified as negative by the model but are actually positive. When false positives and false negatives are close to zero, both the precision and the recall are closer to 1 and hence the F1-score is 1. In the worst case, the F1-score is 0, which means that the model is not performing any correct classification.

Statistical analysis

To compare the differences in the distribution of variables in the two groups, the normalities of the variables were tested using the Shapiro–Wilk test. Variables satisfying normality were examined using Student’s t-test, and those not satisfying normality were compared between groups using the Mann–Whitney U test. All statistical analysis was conducted using Python (version 3.7) software.

Overall population

A total of 89 children were included in this study. Among them, 33 children had DD and 56 were typically developing children. Children with DD were significantly older than those without (56.7±9.4 vs. 52.3±8.2 months; P=.024) (Table 1). Sex distribution was not significantly different between the two groups, as 20 (61%) of children with DD and 28 (50%) children without were boys (P=.33). Of the children with DD, 25 (76%) were diagnosed with ASD, 23 (70%) with ID, and 7 (21%) with LD. The mean scores of the PEP-R Developmental Quotient, VABS Adaptive Behavior Composite score, SRS total score, and K-CARS were all significantly higher in children with DD (P<.001). The mean score of the Internalizing, Externalizing, and Total Problems score of the CBCL of children with DD were not significantly different from those of typically developing children, indicating that comorbid psychiatric symptoms are not different between two groups.

Table 1

Demographic characteristics of children with developmental disabilities and typically developing children
Variable	Children with	Typically developing	P value
Variable	development disabilities (n=33)	children (n=56)	P value
Age, mean (SD), months	56.7 (9.4)	52.3 (8.2)	0.024
Male sex, n (%)	20 (60.9%)	28 (50.0%)	0.33
Diagnosis, n (%)
ASD	25 (75.8%)
ID	23 (69.7%)
LD	7 (21.2%)
PEP-R DQ, mean (SD)	52.0 (27.0)	107.1 (10.8)	<0.001
VABS ABC, mean (SD)	58.1 (12.2)	90.2 (11.7)	<0.001
SRS total score,	69.2 (15.7)	46.4 (8.1)	<0.001
mean (SD), [n=86]	69.2 (15.7)	46.4 (8.1)	<0.001
K-CARS, mean (SD)	29.4 (6.7)	15.9 (1.4)	<0.001
CBCL total behavior problems, mean (SD) [n=86]	56.9 (8.0)	45.9 (8.8)	0.907
CBCL Internalizing score, mean (SD) [n=86]	55.5 (8.3)	46.5 (8.3)	0.794
CBCL Externalizing score, mean (SD) [n=86]	54.9 (8.2)	45.9 (9.1)	0.533
ASD, Autism Spectrum Disorder; CBCL, Child Behavior Checklist; ID, Intellectual Disability; K-CARS, Korean version of the Childhood Autism Rating Scale; LD, Language Disorder; PEP-R DQ, Psychoeducational Profile Revised Developmental Quotient; SD, Standard Deviation; SRS, Sentence Recognition Score; VABS ABC, Vineland Adaptive Behavior Scale Adaptive Behavior Composite

Model evaluation

We conducted a stratified five-fold cross-validation, in which the mean value of accuracy was 0.88 (ranges, 0.78–1.00). The mean value of precision was 0.91 (range, 0.75–1.00). Recall showed the lowest performance among the evaluation metrics, with a mean value of 0.72 (range, 0.50–1.00). The mean value of the F1-score was 0.80 (range, 0.60–1.00) (Table 2). When only landmark variables around the eyes and nose were selected to learn the model in order to reflect the Covid-19 situation of wearing a mask, the average values of accuracy, precision, recall, and f1-score were 77%, 83%, 67% and 73%, respectively (Supplementary Table A).

Table 2

Performance of the stratified five-fold cross validation
Fold	Accuracy	Precision	Recall	F1-score
1	0.78	0.75	0.50	0.60
2	0.78	0.80	0.57	0.67
3	0.89	1.00	0.71	0.83
4	1.00	1.00	1.00	1.00
5	0.94	1.00	0.83	0.89
Average	0.88	0.91	0.72	0.80

Model interpretation

After checking the performance of the model through cross-validation, we sought to find variables that significantly contribute to the prediction of DDs through the interpretation process of the model using SHAP.³⁰ We calculated the SHAP values from the DeepExplainer of the SHAP package on five folds divided from train dataset and calculated the mean values of absolute SHAP values in all folds. In the obtained SHAP values, we found that the nodding head angle variable had the highest results by a large margin when ranking the high-contributing variables, thus making the most contribution to predicting DDs (Figure 4).

In addition, for the top 10 variables with high contributions, the differences in the distribution of these variables between children with DD and those without were analyzed. The Mann–Whitney U-test showed that there were significant differences in the distributions of all variables between the two groups (P<.05) (Figure 5) (Table 3).

Table 3

Mann-Whitney U test results for top 10 variables of importance
Feature	Mann-Whitney U	P value
Nodding head angle	823.0	<.001
Left lip and left face corner angle	612.5	.01
Right eye and right lip area	673.0	<.001
Moving distance of face center point compared to previous time	592.0	<.001
Tilting head angle	649.5	.03
Nose tip and left nose area	522.0	<.001
Right nose and right lip area	582.0	.01
Shaking head angle	664.0	<.001
Moving speed of face center point compared to previous time	724.0	<.001
Right eyebrow area	649.0	.03
Mann-whitney U, The values of variables in both groups are listed in order of size to calculate their rankings. The U value is then calculated by considering the sum of the ranks of each group, the rank average, and the number of data.

In this study, we present the utility of deep learning classification model using a mobile-based video data that predicts the presence of DD by distinguishing the facial characteristics between children with DD and those without by extracting 68 facial landmarks from the faces and generating derivatives such as head pose estimation (pitch, yaw, roll) and landmark point distance. The deep learning classification model using mobile-based video data predicted the presence of DD with an average accuracy of 88% and found that in the pitch (head nodding) variable, children with DD have a significantly wider distribution than those without. Through the model's interpretation process, we identified important predictive variables, including the pitch variables, which all showed statistically significant differences in the distribution between children with DD and those without.

Caregiver questionnaires including Quantitative Checklist for Autism in Toddlers (Q-CHAT), Autism Behavior Checklist (ABC), Ages & Stages Questionnaire (ASQ), and the Parents’ Evaluation of Developmental Status (PEDS) are the most commonly used screening methods for DD. Of them, Q-CHAT showed a sensitivity of 71–75% and a specificity of 63–65%.^31,32 ABC was reported to have a sensitivity of 78.4%.³³ PEDS, which consists of two open-ended questions and eight yes/no questions completed by parents, showed sensitivities of 78.9% and 54.9% in severe and moderate-to-severe delays, respectively, and a specificity of 79.6%.³⁴ ASQ-3 showed sensitivities of 60.0% and 53.1% in severe and moderate-to-severe delays, respectively, and a specificity of 89.4%.³⁵ Thus, in terms of the accuracy of detection, our classification model seems to have comparable performance (88%) compared with the existing methods for screening.

Several digital screening methods for DDs were suggested in previous studies. Most web-based developmental surveillance programs are trials of online versions of established questionnaires. The Web-Based Modified Checklist for Autism in Toddlers with Follow-up interview (M-CHAT/F) is a checklist scored by parents and implemented as a two-stage screening test, in which a positive result prompted follow-up interview to clarify or correct the failed items. When administered by primary care pediatricians, the web-based M-CHAT/F had a sensitivity of 59% and a specificity of 71%.³⁶ In another study that used the digital M-CHAT-revised with Follow-up, accurate documentation in the electronic health record of screening results increased from 54–92% and appropriate action for children screening positive increased from 25–85% compared with the results from the paper form of the M-CHAT.³⁷ In addition, the smartphone application PEDS operated by community healthcare workers was shown to have a close correspondence with the gold standard paper-based PEDS tools operated by health professionals.³⁸ Most smartphone screening applications also focus on developing questionnaires answered by parents or medical professionals.³⁹ ASDTests is an application that is based on the Autism-Spectrum Quotient and Quantitative Checklist for Autism in Toddlers and evaluates the possibility of having autistic traits.¹⁵ Cognoa is a mobile screening application that consists of both parental questionnaires and home video recording, and has a sensitivity of 75% and a specificity of 62%.^13,39 These studies suggest that web-based or mobile-based screening tools could be reliably used for screening DD. Because web-based or mobile-based screening tools are quicker, cheaper, and more accessible, they could be helpful in improving the early identification of DD.

Some recent studies evaluated DD using digital observational methods via analyzing gazes, faces, or behaviors. Eye-tracking algorithms have shown progress in their potential use for screening ASD in rural areas.^16,17 Vargas-Cuentas and colleagues¹⁶ recorded videos of participants watching social or non-social videos and analyzed the image frames from the video. Fujioka and colleagues¹⁷ used infrared light sources and cameras to record the eye position. In one study from Bangladesh, a machine learning classifier trained by data from ADOS and ADI-R was able to detect developmental delay and autism by analyzing the behavior portrayed in home videos, and showed a sensitivity and accuracy of 76%.⁴⁰ Strobl and colleagues¹⁴ also developed a smartphone application in which the participants’ gaze was analyzed by an eye-tracking algorithm. These studies show that digital methods could be used for the screening of DD.

Our study showed that facial landmark analysis, among mobile-based methods, could play a significant role in the detection of DD. In previous studies examining head pose and facial expressions, Happy and Routray²⁴ used the Facial Action Coding System (FACS), which classifies facial expressions using salient facial patches and showed a 94.14% of expression recognition accuracy. This study differs from ours in that the FACS extracts a maximum of 19 facial patches, while our kit extracts 68 face landmarks; in addition, they used facial expression databases using 329 images in total, while our study directly collected data from 89 children. Another study used a computer vision-based head tracking program (Zface) to demonstrate the differences between typically developing children and children with ASD.⁴¹ their results differ from our research because they were able to find the differences in the speed and quantity of head movement in yaw and roll but not in pitch. In another study, children with ASD and those with ADHD were differentiated with an accuracy of 94% via a Red-Green-Blue-Depth sensor from a depth measurement camera.⁴² this study is similar to our work in that there was a difference in facial expressions using FACS, but at the same time different from our results in that the study targeted adults aged 18 and older and there was a difference in head movements in yaw. While these studies are computer-based programs and require special-purpose equipment, our study used a mobile-based application and can thus be more convenient and easy to use. In one study, children watched movies on a smart tablet while the embedded camera recorded their facial expressions. Then, the computer vision analysis automatically tracked the facial landmarks and used them to classify the facial expressions into three types (Positive, Neutral, and Other) with a maximum sensitivity of 73%, with different results depending on the type of movie being shown; notably, children with ASD displayed neutral expressions more often than children without ASD.⁴³ This study differs from ours in that we evaluated not only children with ASD but those with DD.

Based on our results, we carefully suggest that facial landmarks and head poses may be used as screening tools for children with DD. A recent study that quantified head movement dynamics (displacement and velocity) showed that children with ASD had greater head movement dynamics than those without ASD.⁴¹ Several papers hypothesized that turning away may be an adaptive strategy for individuals with ASD to regulate the overwhelming amount of information,^44,45 which may explain the atypical head movement of individuals with ASD. Therefore, using facial landmarks as a method of screening could aid the early identification of children with DD.

There are several limitations to this study. First of all, we were unable to find significant differences in facial landmarks or head pose when children were shown social videos and non-social videos. Second, our study did not analyze the results of the subgroups of DD (i.e., ASD, ID, LD). Third, since children with incorrect data were excluded, the sample size is relatively small and thus has limited generalizability. Fourth, we do not know whether these findings are limited to certain age groups. Fifth, our study did not consider body motion information because we used videos that only recorded the children's faces.

Despite these caveats, our study evaluated the utility of digital methods, especially mobile-based methods, in the screening of DD in community-based preschool children. Our results provide preliminary evidence that the deep-learning classification model using mobile-based children’s video data could be used for the early detection of children with DD.

Acknowledgement

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Ministry of Science and ICT (NRF-2020R1A5A8017671).

Zablotsky, B. et al. Prevalence and trends of developmental disabilities among children in the United States: 2009-2017. Pediatrics 144, doi:10.1542/peds.2019-0811 (2019).
Rah, S. S., Hong, S. B & Yoon, J. Y. Prevalence and incidence of developmental disorders in Korea: a nationwide population-based study. J Autism Dev Disord 50, 4504-4511 (2020).
Rattaz, C., Michelon, C., Roeyers, H. & Baghdadli, A. Quality of life in parents of young adults with ASD: EpiTED Cohort. J Autism Dev Disord 47, 2826-2837, doi:10.1007/s10803-017-3197-y (2017).
Cakir J, Frye, R. E. & Stephen, J. W. The lifetime social cost of autism: 1990–2029. Res Autism Spectr Disord Volume 72 (2020).
Genereaux, D., van Karnebeek, C. D. & Birch, P. H. Costs of caring for children with an intellectual developmental disorder. Disabil Health J 8, 646-651, doi:10.1016/j.dhjo.2015.03.011 (2015).
Zwaigenbaum, L. et al. Early identification of autism spectrum disorder: recommendations for practice and research. Pediatrics 136 Suppl 1, S10-40, doi:10.1542/peds.2014-3667C (2015).
von Suchodoletz, W. Early identification of children with developmental language disorders - when and how?. Z Kinder Jugendpsychiatr Psychother 39, 377-384; quiz 384-375, doi:10.1024/1422-4917/a000136 (2011).
McKenzie, K. & Megson, P. Screening for intellectual disability in children: A review of the literature. J Appl Res Intellect Disabil 25, 80-87, doi:10.1111/j.1468-3148.2011.00650.x (2012).
Shattuck, P. T. et al. Timing of identification among children with an autism spectrum disorder: Findings from a population-based surveillance study. J Am Acad Child Adolesc Psychiatry 48, 474-483, doi:10.1097/CHI.0b013e31819b3848 (2009).
McCarty, P. & Frye, R. E. Early detection and diagnosis of autism spectrum disorder: Why is it so difficult? Semin Pediatr Neurol 35, 100831, doi:10.1016/j.spen.2020.100831 (2020).
Tippelt, S., Kuhn, P., Grossheinrich, N. & von Suchodoletz, W. Diagnostic accuracy of language tests and parent rating for identifying language disorders. Laryngorhinootologie 90, 421-427, doi:10.1055/s-0031-1271655 (2011).
Akshoomoff, N., Corsello, C. & Schmidt, H. The role of the autism diagnostic observation schedule in the assessment of autism spectrum disorders in school and community settings. Calif School Psychol 11, 7-19, doi:10.1007/BF03341111 (2006).
Kanne, S. M., Carpenter, L. A. & Warren, Z. Screening in toddlers and preschoolers at risk for autism spectrum disorder: Evaluating a novel mobile-health screening tool. Autism Res 11, 1038-1049, doi:10.1002/aur.1959 (2018).
Strobl, M. A. R. et al. Look me in the eye: evaluating the accuracy of smartphone-based eye tracking for potential application in autism spectrum disorder research. Biomed Eng Online 18, 51, doi:10.1186/s12938-019-0670-1 (2019).
Thabtah, F. An accessible and efficient autism screening method for behavioural data and predictive analyses. Health Informatics J 25, 1739-1755, doi:10.1177/1460458218796636 (2019).
Vargas-Cuentas, N. I. et al. Developing an eye-tracking algorithm as a potential tool for early diagnosis of autism spectrum disorder in children. PLOS ONE 12, e0188826, doi:10.1371/journal.pone.0188826 (2017).
Fujioka, T. et al. Gazefinder as a clinical supplementary tool for discriminating between autism spectrum disorder and typical development in male adolescents and adults. Mol Autism 7, 19, doi:10.1186/s13229-016-0083-y (2016).
Deutsch, S. I. & Raffaele, C. T. Understanding facial expressivity in autism spectrum disorder: An inside out review of the biological basis and clinical implications. Prog Neuropsychopharmacol Biol Psychiatry 88, 401-417, doi:10.1016/j.pnpbp.2018.05.009 (2019).
Shanok, N. A., Jones, N. A. & Lucas, N. N. The nature of facial emotion recognition impairments in children on the autism spectrum. Child Psychiatry Hum Dev 50, 661-667, doi:10.1007/s10578-019-00870-z (2019).
Zaja, R. H. & Rojahn, J. Facial emotion recognition in intellectual disabilities. Curr Opin Psychiatry 21, 441-444, doi:10.1097/YCO.0b013e328305e5fd (2008).
Faso, D. J., Sasson, N. J. & Pinkham, A. E. Evaluating posed and evoked facial expressions of emotion from adults with autism spectrum disorder. J Autism Dev Disord 45, 75-89, doi:10.1007/s10803-014-2194-7 (2015).
Scotland, J. L., Cossar, J. & McKenzie, K. The ability of adults with an intellectual disability to recognise facial expressions of emotion in comparison with typically developing individuals: A systematic review. Res Dev Disabil 41-42, 22-39, doi:10.1016/j.ridd.2015.05.007 (2015).
Memisevic, H., Mujkanovic, E. & Ibralic-Biscevic, I. Facial emotion recognition in adolescents with disabilities: The effects of type of disability and gender. Percept Mot Skills 123, 127-137, doi:10.1177/0031512516660781 (2016).
Happy, S. & Routray, A. Automatic facial expression recognition using features of salient facial patches. IEEE Trans Affect Comput 6, 1-12 (2014).
Manfredonia, J. et al. Automatic recognition of posed facial expression of emotion in individuals with autism spectrum disorder. J Autism Dev Disord 49, 279-293, doi:10.1007/s10803-018-3757-9 (2019).
Bulat, A. & Tzimiropoulos, G. How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks) In Proc. of the IEEE International Conference on Computer Vision, ICCV'17, 1021-1030 (IEEE, New York, NY, USA, 2017).
Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Comput 9, 1735-1780 (1997).
Kwon, D. H., Kim, J. B., Heo, J. S., Kim, C. M. & Han, Y. H. Time series classification of cryptocurrency price trend based on a recurrent LSTM neural network. J Inf Process Syst 15, 694-706 (2019).
Sinha, J. & Manollas, M. Efficient deep CNN-BiLSTM model for network intrusion detection. In Proc. of the 2020 3rd International Conference on Artificial Intelligence and Pattern Recognition, AIPR'20, 223-231 (ACM, New York, NY, USA, 2020).
Lundberg, S. M. & Lee, S. I. A unified approach to interpreting model predictions. In Proc. of the 31st International Conference on Neural Information Processing Systems, NIPS'17, 4768–4777 (ACM, New York, NY, USA, 2017).
Thabtah, F. & Peebles, D. Early Autism screening: A comprehensive review. Int J Environ Res Public Health 16, 3502, doi:10.3390/ijerph16183502 (2019).
Raza, S. et al. Brief report: Evaluation of the short quantitative checklist for autism in toddlers (Q-CHAT-10) as a brief screen for autism spectrum disorder in a high-risk sibling cohort. J Autism Dev Disord 49, 2210-2218, doi:10.1007/s10803-019-03897-2 (2019).
Juneja, M., Sharma, S. & Mukherjee, S. B. Sensitivity of the autism behavior checklist in Indian autistic children. J Dev Behav Pediatr 31, 48-49, doi:10.1097/DBP.0b013e3181c7241a (2010).
Hamilton, S. Screening for developmental delay: reliable, easy-to-use tools. J Fam Pract 55, 415-422 (2006).
Sheldrick, R. C., Marakovitz, S., Garfinkel, D., Carter, A. S. & Perrin, E. C. Comparative accuracy of developmental screening questionnaires. JAMA Pediatr 174, 366-374, doi:10.1001/jamapediatrics.2019.6000 (2020).
Sturner, R. et al. Autism screening with online decision support by primary care pediatricians aided by M-CHAT/F. Pediatrics 138, e20153036, doi:10.1542/peds.2015-3036 (2016).
Campbell, K. et al. Use of a digital modified checklist for autism in toddlers - revised with follow-up to improve quality of screening for autism. J Pediatr 183, 133-139 e131, doi:10.1016/j.jpeds.2017.01.021 (2017).
Maleka, B. K., Van Der Linde, J., Glascoe, F. P. & Swanepoel, W. Developmental screening-evaluation of an m-health version of the parents evaluation developmental status tools. Telemed J E Health 22, 1013-1018, doi:10.1089/tmj.2016.0007 (2016).
Abbas, H., Garberson, F., Glover, E. & Wall, D. P. Machine learning approach for early detection of autism by combining questionnaire and home video screening. J Am Med Inform Assoc 25, 1000-1007, doi:10.1093/jamia/ocy039 (2018).
Tariq, Q. et al. Detecting developmental delay and autism through machine learning models using home videos of Bangladeshi children: Development and validation study. J Med Internet Res 21, e13822, doi:10.2196/13822 (2019).
Martin, K. B. et al. Objective measurement of head movement differences in children with and without autism spectrum disorder. Mol Autism 9, 14, doi:10.1186/s13229-018-0198-4 (2018).
Jaiswal, S., Valstar, M. F., Gillott, A. & Daley, D. Automatic detection of ADHD and ASD from expressive behaviour in RGBD data. In Proc. of the 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), 762-769. (IEEE, New York, NY, USA, 2017).
Carpenter, K. L. H. et al. Digital behavioral phenotyping detects atypical pattern of facial expression in toddlers with autism. Autism Res 14, 488-499, doi:10.1002/aur.2391 (2021).
Mottron, L. et al. Lateral glances toward moving stimuli among young children with autism: Early regulation of locally oriented perception? Dev Psychopathol 19, 23-36, doi:10.1017/S0954579407070022 (2007).
Hellendoorn, A. et al. The relationship between atypical visual processing and social skills in young children with autism. Res Dev Disabil 35, 423-428, doi:10.1016/j.ridd.2013.11.012 (2014).

No competing interests reported.

SupplementaryFigureASupplementaryTableA.docx

Download PDF

Version 1

posted

You are reading this latest preprint version

Screening Children at Risk for Developmental Disabilities Based on Face Landmark from Video Data of Mobile-Based Application: Preliminary Cross-Sectional Study

Status:

Version 1

Abstract

Background

Objective

Methods

Results

Conclusions

Figures

Introduction

Methods

Mobile-based application

Participant enrollment

Data collection and preprocessing

Face landmark extraction

Preprocessing

Feature extraction

Model algorithm

Statistical analysis

Result

Overall population

Model evaluation

Model interpretation

Discussion

Declarations

Acknowledgement

References

Additional Declarations

Supplementary Files

Status:

Version 1