In this paper, we provide a systematic literature review that explores the Internet of Things (IoT) application to support people affected by Autism Spectrum Disorder (ASD). This disorder represents several limitations to learning, interacting with others, and communicating. Research has shown the great potential of the IoT in providing approaches that target individuals with ASD and help them in learning and their daily activities. Several studies on the topic of IoT were reviewed. The main aim of this review is to recognize relevant studies about the use of IoT to support people affected by ASD, particularly children. In addition, a classification is presented to classify the existing papers.
Research Article
A systematic literature review on the internet of things applications used by people with autism spectrum disorder
https://doi.org/10.21203/rs.3.rs-2503204/v1
This work is licensed under a CC BY 4.0 License
Version 1
posted
You are reading this latest preprint version
Autism
IoT
Learning
Supporting Autism
Sensing Technology
Autism Spectrum Disorder (ASD)
Children with autism may have difficulties with social interactions and communication [1]. They develop many skills through distinctive progressions from typically developing (TD) children. Hence, they need special interventions to enhance their cognitive ability, communication skills, daily living skills, and social interaction. Autism not only affects children, but their caregivers are also impacted. Therefore, the development of the Internet of Things (IoT) is advantageous for health field systems to deliver better services [2]. Thus, this growth of IoT and the advancement of sensing technologies make it possible to create an environment that supports children with autism in their daily activities, contributes to developing their skills, and helps them on their learning paths.
Several reviews have taken place in the field of IoT and autism spectrum disorders (ASD). Investigations concerning the application of sensing technologies for supporting autism were presented in [39]. The taxonomy of the reviewed studies was based on what the smart device could measure. The main categories were eye trackers, movement trackers, electrodermal activity monitors, prosody and speech detection, and sleep quality assessment detection. The study did not consider devices that track environmental parameters. However, autism is characterized by variable sensory sensitivities. For example, some individuals with autism may be frustrated and react in an excessive way to environmental stimuli, such as fluorescent lights and sounds [40]. Therefore, environmental trackers are necessary for monitoring and interventions. In addition to this work, the review presented in [41] also highlights the use of sensing technologies to support autism. This review distinguishes between technologies used for assessment, intervention systems, and monitoring approaches. It introduces studies that use IoT for processing data concerning facial expressions, voices, body movement, cognition, social behavior, multimodal information, etc. The authors demonstrated the potential of this information in the assessment. However, some limitations should be noted. The review relies on a small number of studies for each category. Only four studies were presented in the monitoring category, although this field is promising for IoT applications.
On the other hand, a systematic review was presented by [42] to investigate how wearables and mobile technologies are useful in autism interventions. The study followed the Systematic Literature Review (SLR) method, and it selected 77 articles that met the criteria. The authors noted the impressive evolution of mobile technologies that can be beneficial for therapies and diagnoses. Moreover, they revealed that the developed projects target different groups (85.2% of patients, 5.7% of parents, 2.8% of teachers, 2.6% of carers, and 1.5% of specialists). Despite the fact that they identified 21 main categories and calculated their percentages, the authors did not align each paper with the category. Similarly, the authors of [43] reported that the progress of mechatronic devices promises a useful solution for ASD detection and rehabilitation. Mechatronic devices were categorized into screening and therapeutic groups. This review indicated that non-wearable devices are more commonly used for recording behavioral data than wearable ones in the screening field. Furthermore, therapeutic devices can help in the therapy process and provide a great opportunity for parents to implement rehabilitation even at home. However, this categorization is insufficient since mechatronic devices can assume a key role in many other fields, such as education and developing social skills.
This work is reated to the two reviews presented in [44] and [45]. Prevously, we analyzed how IoT can be deployed for autism. We have identified three main categories based on the user of the proposed IoT-based approach. The figure bellow describes these three categories.
The first category was traited in [44], the work presented in [45] consider the second category, in this review we adresse the third category and we explore IoT approaches that can be used by people with autism themselves.
To the best of our knowledge, this is the first work that reviews how IoT can support autism and how IoT solutions can be used directly by the affected person or with a small intervention. In this review, we explore several studies that demonstrate how IoT can be used to help affected individuals in their everyday lives. To this end, we use the SLR process introduced in [3]. The main contributions of this review are:
-
Considering how IoT can help people with ASD;
-
Presenting a categorization of papers based on IoT intervention areas for autism support;
-
Analyzing and discussing the results of SLR.
The remaining sections of this survey are organized as follows: Section 2 presents the methodology of this systematic review. Section 3 provides the proposed categories for the classification of selected papers. Section 4 presents a discussion by answering research questions. Finally, in section 5, we present the conclusion.
In this section, we will detail the application of SLR. According to [3], the SLR is a methodology that consists of identifying, collecting, and analyzing relevant articles within the literature. First, we identified the Goal-Question-Metric approach (GQM) [4]. According to [4], the GQM approach is an intensive method to specify useful metrics by identifying goals and objectives for the process, specifying goals with data or metrics, and finally, using a framework to interpret the data according to the defined goals.
The research questions are an essential part of any SLR [5] since they drive the search methodology. To designate pertinent studies concerning IoT and to sense technology research to support people with autism, we proposed the following research questions:
- RQ1: Which type of action can be currently performed for the affected individuals by autism using IoT?
- RQ2: Which IoT approaches for autism are examined and evaluated in this work?
- RQ3: Which techniques and technologies are combined with IoT?
- RQ4: What are the validation measures for ASD approach?
- RQ5: What types of wearable/non-wearable devices and sensors have been implemented?
In particular, our purpose was to define the current state-of-art of autism support using IoT (RQ1), the proposed approach in the paper (RQ2), techniques and technologies reported in action, the techniques that were implemented with IoT in the approach, and the kind of metrics used for testing and validating the approach, and (RQ3) and (RQ4), respectively. In addition, we examined the types of wearable/non-wearable devices and sensors that are applied (RQ5).
To answer these questions and identify relevant papers for this review, we adopted the SLR guidelines presented in [5]. In the next section, “Search strategy,” we identify the search terms and the source of resources
2.1 Search Strategy
Based on [5], it is important to define a search strategy; accordingly, in this section, we will identify the search terms and the search resources. For the identification of the search terms, a typical method is to divide the subject into separate components, such as population, intervention and outcomes, and context. A list of synonyms, abbreviations, and alternate spellings should then be created. By looking at the subject headings used in journals and databases, more keywords can be found. Sophisticated search words can then be developed using Boolean ANDs and ORs.
Search terms
Five instructions are defined in [5]:
- The identification of population, intervention, and outcome is:
- Population: autism;
- Intervention: IoT;
- Outcome: Kind of action.
For example, the research question can be formulated like this:
- The identification of synonyms:
- Autism (“autism” OR “Autism Spectrum Disorder” OR “ASD” OR “autistic people” OR “Children affected by autism” OR “individuals on the spectrum disorder”);
- IoT (“Internet of Things” OR “IoT” OR “Sensing technology” OR “sensors”);
- The analysis of keywords in each research study
- The formulation of search queries using “OR” and “AND” operators. The operator “AND” is for concatenation. For example :
The formulation of a search query with a limited number of Boolean operators. For example:
Search resources
The main search resources used were Google Scholar, Springer, IEEE, ACM, Science Direct, SCOPUS, PubMed, and Wiley.
In the next section, “Papers selection process,” we specify the steps of retrieving pertinent papers, the search criteria, and the quality assessment to select the relevant research.
The keyword selection procedure
We searched for relevant papers concerning significant IoT applications for supporting autism. Search keywords were identified according to the research questions and all associated topics addressed in this review, and four keywords related to IoT and autism were specified in the search term. The obtained list of keywords was: Autism, IoT, Learning, Supporting Autism, Sensing Technology, and Autism Spectrum Disorder (ASD). However, this list was not exhaustive; indeed, some additional or alternative keywords were changed to reduce the risk of missing important and related publications.
2.2 Papers selection process
In this section, we will identify how the research articles were selected for this review. The process of the selection of related studies is illustrated as follows:
More than 50 articles were retrieved from the initial selection using the selected databases. Meanwhile, some papers (for example, papers that were referenced in a retrieved paper from a database) were searched manually.
In the succeeding stage, we applied inclusion and inclusion factors. We detail this part in the “Search Criteria” paragraph. The result of this selection was 42. The quality of the rest of the papers was evaluated using the criteria specified in the paragraph: “Quality assessment.”
Search Criteria
Following SLR, we applied two types of criteria:
Inclusion Criteria:
- Research not concerned with autism;
- Research not concerned with IoT or sensing technology;
- Research not written in English;
- Research not in peer-reviewed journals or conferences, books, and lecture notes;
- Research written before 2012.
Exclusion Criteria:
- Research published between 2012 and 2022;
- Research that used keywords;
- Research that exploited IoT to support and diagnose autism;
- Full version (not abstract) research.
The result of the “Search Criteria” step gathered 53 research papers.
Quality Assessment
In this phase, we verify the collected papers from the previous step in detail. According to SLR, we conduct this phase with a set of questions. We asked the following questions as they pertained to each research article’s approach:
- Q1: Is the approach presented?
- Q2: Is the proposed solution specified in detail?
- Q3: Is the IoT applied?
- Q4: Are the validation strategies and performance measures explicitly detailed?
- Q5: Has the approach been tested in a real environment?
Each selected paper had a score that corresponded to the quality. According to the scoring procedure specified by SLR, each question was scored as: “YES” = 1, “Partially” = 0.5, “NO” = 0, or Unknown. The score of the paper was the sum of the five questions scored. Then, we categorized the research quality as “High (score ≥ 4),” “Medium (3≤ score < 4),” and “Low (score < 3).” Hence, a paper with the highest quality had five. Only papers having medium or high scores were selected.
Data collection
The final studies that passed all selection steps (Table 1) were examined to reply to the research questions. For this purpose, we propose the classification of papers in the next section. Then, we aim to answer the research questions in the “Results” sections.
Deploying IoT to assist and support autistic people was the main issue we sought to explore in the research. We selected relevant papers that suggested IoT as a technology that supported ASD. Then, we analyzed those papers to recognize how IoT can enhance the quality of life of affected individuals, especially children. We classified studies according to the following categories:
-
Learning
-
Autonomy and safety.
In the following section, we will describe proposed systems, frameworks, and prototypes based on IoT that help children with ASD in their learning paths. For that purpose, we distinguish between three types of skills: academic and cognitive skills, social interaction and communication skills, and motor development skills.
3.1 Learning
Academic and cognitive skills
Autistic children have difficulties in communication and social interaction, which limit them in their learning path. An assistive framework based on IoT was presented for children with ASD [6]. This framework relies on a Juxtapose (JXTA)-Overlay platform and a smart box that manages the surrounding environment. JXTA-Overlay incorporates the JXTA protocol that allows the connection of devices to a network to exchange messages [84]. Also, JXTA-Overlay is a middleware based on the JXTA specification. The system described in [6] controls the light, sounds, and smells to keep a proper and calming condition for children when learning or playing.
A way to get a high level of student engagement is by customizing and adapting the learning procedure. The purpose of the MaTHiSiS project’s platform is to personalize the learning path. The platform enables teachers to set goals and graphs for each learner. The learning process is affected by the user’s emotional state, which is collected through sensors [7]. Machine learning techniques were applied to multimodal intelligence systems to create personalized learning content. This approach aims to provide aid to ASD children and develop their performance. Wireless sensors were used for collecting data that was integrated into the machine learning algorithm [8]. In addition to this study, an IoT-based affective computing framework named “Affect-learn” was presented in [37] to assist teachers in dealing with hyperactivity or inattention issues in children with autism. This framework was built upon various IoT devices, including an accelerometer to monitor activities, a heartbeat sensor to track heart rate data, and other sensors that collect environmental data. All extracted features were used to construct a pattern to recognize hyperactivity. Support Vector Machine (SVM) was used for the pattern recognition methodology. The accuracy of the framework reaches 90.23% when evaluating the framework with the participation of three volunteers.
Furthermore, play is important for children with autism. By playing, children enhance their skills and develop their ability to learn and interact with peers. Three IoT-based gaming applications for ASD children were designed and developed by [9]. The goal of those applications was to ameliorate their cognitive skills. The proposed games involved an object-finding game, a puzzle game, and a road-crossing game. Therapists and parents could create an account via the application to follow the progress of the child. In addition, Virtual Reality (VR) gaming is growing fast in the educational sector. An interactive VR-based solution was proposed to help children with autism improve their knowledge [10]. The proposed approach allowed the child to play with a game divided into four levels. The levels were ordered by complexity to ensure the child’s learning progress. Another study found that children learn words better if they are pronounced by their parents. Thus, the “let’s play” application was developed to teach autistic children pronunciation and new words. Parents selected objects around their house, recorded their names, and tagged them with a beacon. Once the object was detected by the application, its name was pronounced by the parent’s voice [11].
Homework is an important part of the educational system. In [12], the authors conducted several experiments to show the major impact of homework on the rate of acquisition of children with disabilities. According to the authors, practicing fundamental skills is not expected only within the classroom, but children with autism are supposed to learn those skills and apply them outside the school. The authors of [13] consented to this point. However, they supposed that homework is challenging for children with ASD. Then, they proposed using IoT and robotic assistance to construct a smart environment that supports children through homework activities. In this system, intelligent devices were used to monitor the child’s reactions. The robotic assistance was implemented to support the child when doing homework. It played a role similar to a therapist. This system was developed using three stages: the prototype design, the training and experiment design, and the clinical validation.
Applied Behavior Analysis (ABA) is an effective method designed for people with autism and is based on several nonscientific guidelines [36]. In [35], the authors aimed to demonstrate the possibility of using IoT-based technologies to escalate ABA treatment in the classroom setting for children with special needs. For this purpose, they presented a 3i-learning system based on IoT sensors to collect data and detect how the student felt about ABA. The proposed system also helped children enhance their skills by providing appropriate training. The system was evaluated by ten teachers who appreciated the impact on the learner’s behaviors.
Social interaction and communication
The “NAO humanoid” robot was used for motor learning. This robot was developed by the Aldebaran Robotics Company, and it was autonomous and programmable. The proposed learning method was based on the interaction between children with ASD and the robot. The robot gave instructions to the child and asked for instructions from the child. Although this technique could not handle complex conversations, it improved the children’s communication and social skills [14]. Instead of a robot with a fixed appearance, a virtual agent named “AVATAR” was customized to teach children interaction skills by imitation. The main feature of AVATAR was its capability to change its appearance by conforming to the needs [15]. The authors of [38] also proposed using human-robot interaction to enhance the communication skills of children with ASD. Indeed, they invited four children affected by autism from the Society for the Welfare of Autistic Children (SWAC) to participate in the interaction with the NAO robot program. The proposed program was based on four sessions, and the interactions were recorded by cameras. In the first session, the robot was presented to the children and their caregivers. The NAO robot asked simple questions to the child in the second session. Then, physical activities were planned in the third session, and the last session consisted of collecting feed feedback related to previous sessions. The authors noticed an improvement in the communication skills of three children.
A virtual reality system with three levels was proposed as a therapy for children with autism [16]. In the first level, the child’s attention is cached using sounds, colors, and pictures. Then, in the second level, the child is asked to complete activities and interact with friends to improve their social skills. The third level is for decision-making. The environment may change at any stage if the child is not comfortable with it. On the other hand, a study yielded by [17] considered the encouraging results of existing studies around the humanoid robot, Kaspar, and proposed using this robot as a tool for therapy sessions. First, the authors evaluated how ASD children were influenced by Kaspar. The findings showed that children tend to imitate the pose, sequence of movements, and facial expressions of the robot, which can improve their social and emotional, sensory, and motor development. Then, the authors proposed that Kaspar could be used by teachers/professionals in the context of a classroom to develop children’s social skills and communication.
The authors of [18] considered that the current autism-therapy systems are intended to overlook the emotional cognition problem. To deal with this problem, they designed the first-view emotion care system (First-ECS) as a useful solution for developing the perception and expression ability of children with ASD. The architecture of this system was built upon a carrier system represented by a wearable robot and sensors (microphone, camera, temperature, and heart rate sensor). Emotions were recognized via the analysis of sensor data, and they were sent through the system network. There were two applications of the system: the first application “First-ECS emotion interaction mechanism,” helped children recognize other people’s emotions. In this case, the system received audio and image data, then executed the emotion cognition based on a deep learning-based model. After this, the system described the emotion of the child after using mapping rules that integrated the interest of the child. The second application was the “emotion expression engine,” and it allowed people surrounding the child to understand their inner emotions.
In addition, the approach described in [34] aimed to help children suffering from epilepsy by using the IoT module. The architecture of the proposed prototype was based on a set of sensors: heart rate sensor, accelerometer, tilt sensor, associated with a motor, a microcontroller, and driver relays. The authors described the used hardware and software components, but they did not specify how the system could be used in real environments and experiments.
Motor development
To develop the academic, sensory, and motor skills of ASD children, research introduced in [19] investigated the implementation of games using Game Kinect assistance. The proposed game consisted of two steps: training and testing. In the training phase, the child was taught to count from 0 to 10 and try to remember numbers by vision and hearing. Then, in the testing, the child was asked to click on a particular number using his hand. This game had several levels, so it could help autistic children improve their cognitive and motor skills.
According to [20], IoT applications have significant potential in securing the environment of many industries. On the other hand, children with autism may suffer from abnormal repetitive behaviors [21]. In [21], the authors specified that self-injury is very common in those children, which can be dangerous. The next section reveals existing studies that show the possibility of using IoT to provide autonomy and safety for those children.
3.2 Autonomy and safety
Reducing the dependence of ASD children on their caregivers is an essential purpose. A type of smart glasses was developed by [22]. The glasses were augmented reality-based and aimed to help children to develop daily living skills. The smart glass worn by the child sends visual instructions based on the context and helps them to achieve a task. A web platform is developed to enable caregivers and therapists to define steps for each task.
Furthermore, an emotional stimulus can make people affected by ASD impulsive and furious. They may lose control of their emotions. Emotion regulation begins with the understanding of problematic behaviors and works on the modification of the emotional status that promotes a purposeful behavior. Smartwatches can be equipped with sensors that detect disturbing situations and then send instructions and notifications that assist people with autism. This approach for emotional self-regulation using a smartwatch was introduced by [23]. The intervention was provided without the need for external devices.
An IoT approach can be implemented in the patient’s house for remote health care [24] by using affordable devices such as cameras and smartphones. Images and facial expressions were collected for emotion recognition. In addition, I-Rise is an IoT-based effective framework for monitoring anxiety [25]. The system tracks daily activities, ambient temperature, eye events, and heart rate to recognize the emotional status. If a stressful situation is detected, the system provides audio feedback and manages the surrounding environment. Another study from [26] proposed using wearable technology for emotion self-regulation by teaching mindfulness techniques. The wearable device is equipped with several sensors to record data. The system sends reminders of coping skills to the wearer to make them aware of the panic situation to be able to manage them.
Improving the autonomy of people with autism in their daily living was a purpose for [27], and they proposed a system for children with autism to reduce their dependence on their caregivers and parents. The system was based on a fuzzy assistive method. It generated messages and sent them to parents, caregivers, and children with ASD after examining the collected data through sensors. In [28], the authors aimed to enhance the scope of monitoring. They designed a Service Oriented Architecture (SOA) based approach to support ASD individuals. The system implemented several sensors, and they collected physiological data from three different zones: home, community/neighborhood, and away from home. For planning and stress management, the study presented in [29] came with prototyped ideas that helped parents with ADHD and autism in their daily activities. Mainly energy path and life balance, these ideas were based on IoT. They correlated between pulse and calendar events to help manage stress, equilibrate energy, etc. In addition to that, people with autism may face a considerable number of hurdles and difficulties; therefore, the authors of [30] presented a concept of a smart home that provides health services for disabled people to stay comfortable. They developed an IoT-based mobile gateway solution to establish an interactive wireless system at home. The system was built upon three models: monitoring system, decision-making system, and controlling system. The first model ensured the observation of the health status and professional diagnosis. Data was acquired using a Wireless Sensor Network (WSN) and was sent to a decision-making model that provided the intervention of caregivers. The third module was conceived to control the entire house: over the fan, light devices, door, and windows.
Sports training improves ASD children’s motor skills and their level of fitness [31]. The authors of [31] demonstrated this finding by observing 16 children with autism over 12 weeks. They observed all movements and processed data with SPSS. At the end of the experience, they noticed a positive impact of sports on their motor skill. Aquatic activities allowed the children with autism to be active and practice physical activities. Since swimming is a physical activity that develops motor skills, it is highly recommended for children affected by autism. “iBall to swim” is a serious game developed by [32] to help those children practice swimming autonomously and safely. The components of this IoT-based interactive game were a ball with lighting, a heartbeat monitor attached to a wetsuit, wireless bone conduction headphones, and a mobile application that uses these components to qualify the performance and the abilities of the child during the swim session. Eleven children participated in validating the game. Results of the test could validate the improvement of children’s performance using the ball. In addition, the authors of [33] presented a low-power wireless wearable device that allowed the recording of physical signs of autonomic activities. The smart device was also compact, comfortable, and low-cost. The authors proposed a website called iCalm.org to store and process data. This project was designed to develop a low-cost, comfortable, and multisensory wearable device that could work in a wireless network for collecting, sharing, and processing data.
4.1 RQ1: Which type of action can be currently performed for autism using IoT?
According to the 25 selected papers and the presented taxonomy, the use of IoT to assist people with ASD is demonstrated to be a powerful tool. Indeed, the existing articles emphasize the role of the adaptation of the teaching process according to the state of the child. In addition, they revealed how IoT could be deployed to improve the safety and autonomy of people with autism.
Based on the number of the surveyed articles for each category (learning, autonomy, and safety), we calculated the percent of each area where IoT is implemented for supporting and assisting people with autism, as depicted in Fig. 2.
In Table 1, we summarize the main idea of each paper, the used IoT devices or sensors, and some of their limitations.
| Study | Description | Devices/sensors Used | Issues/Limitations |
|---|---|---|---|
| [33] | The development of a low-power wearable device to monitor autonomic activities | Customized wearable device | - The accuracy of this system is undefined |
| [38] | This study allows the indirect observation of children with autism while playing. Toys equipped with sensors were provided to identify movements in real-time and allow specialists to examine the behavior of children from different views. | Accelerometer, gyroscope, and magnetometer | - Experiments did not rely on enough data to provide an accurate clinical diagnosis. |
| [41] | The transformation of eye-tracking scan paths into a visual representation to classify autism | The SMI remote eye-tracker | - A small number of participants. - The duration of video scenarios is short |
| [36] | The application of SGIN on eye-tracking data to classify ASD and TD children. | SR Eyelink 1000 Plus | - Much work is needed for optimization. - Small sample of children participated in experiments |
| [35] | An interactive eye-tracking system that enables the visualization and the interpretation of eye movement data | The Tobii X2-60 eye tracker | - Small dataset. - Another dynamic saliency model needs to be explored to compare the performance |
| [30] | The conception of an affordable smart home is based on IoT, to provide health services to disabled people | Smart devices network | - The sample size is relatively small |
| [29] | The assistance of parents with ADHD and autism in stress management and planning time. | Heart rate sensor | - The development of the prototyped ideas is not detailed |
| [28] | An SOA-based system to support people with autism using IoT | Multiple sensors depending on different zones | - The study needs to include more participants |
| [26] | The implementation of mindfulness techniques and wearable technology for emotional self-regulation | Customized wearable device | - A small database used for experiments and validation |
| [32] | The development of a serious game based on IoT to help children with autism in their daily activities. | The game is equipped with several sensors | - Few participants |
| [39] | A system to diagnose autism through the RTN protocol. | RGB cameras Microsoft Kinect SDK | - The system needs to improve the accuracy |
| [43] | The approach aims to measure, identify, and classify stereotypical motor movements | Accelerometers | - The amount of training data is insufficient |
| [42] | The use of musical instruments to qualify autism sensory disorders | AMI | - The performance of the proposed method is not indicated |
| [34] | An IoT approach to help children with autism suffering from epilepsy | accelerometer, tilt sensor, heart rate sensor, PIC, etc. | - Lack of a description of how the system works in a real environment. - Lack of evaluation results |
| [2] | A system was used to monitor the interaction between children and collect data | Wearable badge | - Sensor Blockage - Lack of Accuracy - The loss of data during interaction |
| [60] | The application of IoT to develop a customized learning experience | JXTA-Overlay system Smart Box WPAN | - The effectiveness of learning new abilities is not reported - Few datasets |
| [17] | The conception of Play sessions with Kaspar to promote the social and communication skills of children with autism by imitation | Kaspar robot | - Questionnaire data is unreliable - The child’s mood is unreliable |
| [19] | The development of a game using Kinect to teach children with autism several skills | Kinect | - Short duration of validation |
| [7] | The development of an IoT-based method for the learning process using a serious game | Serious gaming | - The perception of the child’s emotional state is not detailed |
| [8] | The conception of a personalized learning approach using IoT infrastructure and machine learning | WSN | - The description of the IoT infrastructure is not detailed |
| [9] | The development of an IoT based games to enhance cognitive skills | piezo sensor, RFID tag | - Few datasets for validation - Games are not evaluative |
| [6] | The development of a virtual reality-based game that attracts the child and enhances his interaction skills | leap motion sensor | - The target group of children is small - Leap motion controller requires daylight - VR game is Expensive |
| [13] | A smart home environment that apply homework in supporting children | Smart home environment | - The approach is expensive - The feedback of users (children) is not provided |
| [11] | The use of AT to provide a platform for enhancing the language and social skills of children with autism using their parent’s voices | Estimote Beacon sensors, Mobile system | - The progress of the child is not tracked |
| [14] | Building a platform using NAO humanoid robot for motor learning | NAO humanoid robot, The Kinect camera | - The approach is not tested in a real environment |
| [15] | Using VR to improve the social interaction of the child | VR | - The use of AVATAR as a therapeutic tool is not detailed |
| [37] | « Affect-learn » is an IoT-based framework conceived to deal with human issues during learning | Accelerometer, heartbeat sensor, and other sensors that collect environmental data | - Experiment duration (30min) is short - Only 3 participants - Few data to evaluate the system |
4.2 RQ2: Which IoT approaches for autism are examined and evaluated in this work?
We identified two main areas where IoT is exploited to support autism. More than 25 selected articles investigated methods and approaches designed for people with autism to enhance their quality of life and assist them in several activities, including learning.
4.3 RQ3: Which techniques and technologies are combined with IoT?
Several technologies were combined with IoT to prototype and construct a system that assists people with autism. In the learning field, IoT has been exploited with serious gaming ([7]), VR ([10], [14]), and robot technology (humanoid robot, wearable robot, etc.) To enhance the autonomy of people with autism and their safety, mobile technology was used for visualization and intervention.
4.4 RQ4: What are the validation measures for ASD approach?
It is worth mentioning that the validation datasets in almost all the papers were small. However, accuracy, sensitivity, specificity, and latency were the most calculated metrics for the validation of the proposed systems. Still, the quality of the devices was not evaluated.
4.5 RQ5: What types of wearable/non-wearable devices and sensors have been implemented?
To monitor physiological signs and environmental parameters, many types of wearable and non-wearable sensors were used to collect data that will be processed for visualization and decision-making to make the appropriate intervention. Then, sensors need to be strong, comfortable, invasive, and well-implemented to avoid the loss of data and the distraction of children with ASD when they wear the device.
The application of IoT significantly improves the support of people with autism. Some researchers suggest helping children in their learning path; others emphasize the role of IoT in the safety and autonomy of ASD individuals. Different sensors were used, including wearable and non-wearable ones. Each type of sensor was designed to acquire measures that can be processed and analyzed to collect data. Furthermore, most of the studies proposed ML or DL classifier algorithms to process the collected data. However, several challenges may face the algorithms, such as accuracy and efficiency.
Ethical Approval
Not applicable
Funding
None
Conflicts of interest/Competing interests
None
Availability of data and material
Authors can confirm that all relevant data are included in the article and/or its supplementary information files.
Code availability (software application or custom code)
Not applicable
Authors' contributions
E.F. and E.K. conceived of the presented idea. They investigate the application of IoT in presenting assistance that can be used by autistic people, and supervised the findings of this work. All authors discussed the results and contributed to the final manuscript.
- Maenner MJ,et al. Prevalence of Autism Spectrum Disorder Among Children Aged 8 Years - Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2016. MMWR Surveill Summ. 2020 Mar 27;69(4):1–12. doi: 10.15585/mmwr.ss6904a1. Erratum in: MMWR Morb Mortal Wkly Rep. 2020 Apr 24;69(16):503. PMID: 32214087; PMCID: PMC7119644.
- Kelly JT, Campbell KL, Gong E, Scuffham P. The Internet of Things: Impact and Implications for Health Care Delivery. J Med Internet Res. 2020 Nov 10;22(11):e20135. doi: 10.2196/20135. PMID: 33170132; PMCID: PMC7685921.
- Kitchenham, Barbara & Brereton, Pearl & Budgen, David & Turner, Mark & Bailey, John & Linkman, Stephen. (2009). Systematic literature reviews in software engineering-A systematic literature review. Information and Software Technology. 51. 7–15. 10.1016/j.infsof.2008.09.009.
- Van Solingen R, Basili, V., Caldiera, G., Rombach HD (2002) Goal question metric (gqm) approach. Encycl Softw Eng
- Keele S et al (2007) Guidelines for performing systematic literature reviews in software engineering. Tech. rep., Technical report, Ver.2.3 EBSE Technical Report. EBSE
- Sula A et al (2013) An IoT-Based Framework for Supporting Children with Autism Spectrum Disorder. Information Technology Convergence. Springer, New York, (pp 193–202)
- Spyrou E et al (2018) Exploiting IoT Technologies for Personalized Learning. Conference on Computational Intelligence and Games (CIG). IEEE.
- Banik K, Bhuiyan M (2015) Personalized Learning Materials for Children with Special Needs using Machine Learning. Internet Technologies and Applications (ITA), 2015At: Wrexham, United Kingdom
- Hasan U et al (2020) Towards Developing an IoT Based Gaming Application for Improving Cognitive Skills of Autistic Kids. Intelligent Information and Database Systems, 12th Asian Conference, ACIIDS. (pp 411–423).
- Vasudevan S et al (2019) Learn quest – a virtual reality-based system for training autistic kids. International Journal of Medical Engineering and Informatics 11(2):103
- Wojciechowski A et al (2017) Assistive technology application for enhancing social and language skills of young children with autism. Multimedia Tools and Applications, 76(4):5419–5439.
- Rosenberg MS. The effects of daily homework assignments on the acquisition of basic skills by students with learning disabilities. J Learn Disabil. 1989 May;22(5):314–23.
- J. Berrezueta-Guzman, I. Pau, M. -L. Martín-Ruiz and N. MáximoBocanegra, "Smart-Home Environment to Support Homework Activities for Children," in IEEE Access, vol. 8, pp. 160251–160267, 2020
- Liu X et al (2016) An Interactive Training System of Motor Learning by Imitation and Speech Instructions for Children with Autism. 9th International Conference on Human System Interactions (HSI). (pp 56–61)
- Guerrero et al (2017) AVATAR "Autism: Virtual Agents To Augment Relationships in Children". Conference: 2017 IEEE XXIV International Conference on Electronics, Electrical Engineering and Computing (INTERCON)
- Manju T et al (2018) A Rehabilitation Therapy for Autism Spectrum Disorder Using Virtual Reality. Springer, Singapore. (pp 328–336)
- Karakosta, Efstathia & Dautenhahn, Kerstin & Syrdal, Dag Sverre & Wood, Luke & Robins, Ben. (2019). Using the humanoid robot Kaspar in a Greek school environment to support children with Autism Spectrum Condition. Paladyn, Journal of Behavioral Robotics. 10. 298–317. 10.1515/pjbr-2019-0021.
- Xiao, Wenjing & Li, Miao & Chen, Min & Barnawi, Ahmed. (2020). Deep interaction: Wearable robot-assisted emotion communication for enhancing perception and expression ability of children with Autism Spectrum Disorders. Future Generation Computer Systems. 108. 709–716. 10.1016/j.future.2020.03.022.
- H. Y. Mir and A. K. Khosla, "Kinect Based Game for Improvement of Sensory, Motor and Learning Skills in Autistic Children," 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS), 2018, pp. 1670–1674, doi: 10.1109/ICCONS.2018.8662894.
- Montbel Thibaud, Huihui Chi, Wei Zhou, Selwyn Piramuthu, Internet of Things (IoT) in high-risk Environment, Health and Safety (EHS) industries: A comprehensive review, Decision Support Systems,Volume 108, 2018,Pages 79–95,ISSN 0167–9236,https://doi.org/10.1016/j.dss.2018.02.005.(https://www.sciencedirect.com/science/article/pii/S0167923618300344)
- Bodfish JW, Symons FJ, Parker DE, Lewis MH. Varieties of repetitive behavior in autism: comparisons to mental retardation. J Autism Dev Disord. 2000 Jun;30(3):237 – 43. doi: 10.1023/a:1005596502855. PMID: 11055459.
- Machado, E (2019) An Assistive Augmented Reality-based Smartglasses Solution for Individuals with Autism Spectrum Disorder. IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech). (pp. 245–249)
- Shi Y et al (2017) An Experimental Wearable IoT for Data-driven Management of Autism. 9th International Conference on Communication Systems and Networks (COMSNETS). IEEE.
- Mano LY et al (2016) Exploiting IoT technologies for enhancing Health Smart Homes through patient identification and emotion recognition. Computer Communications (pp 178–190)
- Sundaravadivel P et al (2020) i-RISE: An IoT-based Semi-Immersive Affective monitoring framework for Anxiety Disorders. IEEE International Conference on Consumer Electronics (ICCE).
- Lisa R. Ellrodt, Tilak Agerwala (2020) Leveraging Wearable Technology to Mitigate Anxiety. Proceedings of Student-Faculty Research Day, CSIS, Pace University
- Sumi AI et al (2018) f ASSERT A Fuzzy Assistive System for Children with Autism Using Internet of Things. brain Informatics. Springer, (pp 403–412)
- Eshetu Y, Bhuyan P, Behura A (2018) A Service-Oriented IoT System to Support Individuals with ASD. International Conference on Communication, Computing, and Internet of Things (IC3IoT). IEEE, New York
- Einarson D, et al (2016) IoT-support systems for parents with ADHD and autism. International Conference on Computation System and Information Technology for Sustainable Solutions (CSITSS), (pp, 198–203).
- Aryadi, Rahmad & Naren, J. (2017). A Framework on Health Smart Home using IoT and Machine Learning for disabled people.
- Huseyin, Oktay R. “The impact of sport activities on basic motor skills of children with autism.” Pedagogics, psychology, medical-biological problems of physical training and sports (2019): n. pag.
- Berardina De Carolis and Domenico Argentieri. 2020. IBall to Swim: a Serious Game for Children with Autism Spectrum Disorder. In Proceedings of the International Conference on Advanced Visual Interfaces (AVI '20). Association for Computing Machinery, New York, NY, USA, Article 65, 1–5. https://doi.org/10.1145/3399715.3399917
- R. R. Fletcher et al., "iCalm: Wearable Sensor and Network Architecture for Wirelessly Communicating and Logging Autonomic Activity," in IEEE Transactions on Information Technology in Biomedicine, vol. 14, no. 2, pp. 215–223, March 2010, doi: 10.1109/TITB.2009.2038692.
- Sabarish, P. & a T, Sankara & Gayathri, A. & praveen, A. (2019). A Novel wearable therapeutic aid with intelligent information processing systems. IOP Conference Series: Materials Science and Engineering. 623. 012011. 10.1088/1757-899X/623/1/012011.
- Wong, Chun Man Victor et al. “Internet of Things (IoT)-Enhanced Applied Behavior Analysis (ABA) for Special Education Needs.” Sensors (Basel, Switzerland) vol. 21,19 6693. 8 Oct. 2021, doi:10.3390/s21196693
- Richard M. Foxx, Applied Behavior Analysis Treatment of Autism: The State of the Art, Child and Adolescent Psychiatric Clinics of North America, Volume 17, Issue 4, 2008, Pages 821–834, ISSN 1056–4993,
- Yadav, Ghazal & Sundaravadivel, Prabha & Kesavan, Lokeshwar. (2020). Affect-Learn: An IoT-based Affective Learning Framework for Special Education. 1–5. 10.1109/WF-IoT48130.2020.9221339.
- Farhan, Sk & Khan, Mohammad Nasfikur & Razzak, M. & Shukhon, Ramendra. (2021). Improvement of Verbal and Non-Verbal Communication Skills of Children with Autism Spectrum Disorder Using Human Robot Interaction.. 10.1109/AIIoT52608.2021.9454193.
- Benoit E et al (2019) Musical instruments for the measurement of autism sensory disorders. Journal of Physics Conference Series 1379:012035.
- Uljarević, M., Carrington, S. & Leekam, S. Brief Report: Effects of Sensory Sensitivity and Intolerance of Uncertainty on Anxiety in Mothers of Children with Autism Spectrum Disorder. J Autism Dev Disord 46, 315–319 (2016). https://doi.org/10.1007/s10803-015-2557-8
- Plötz T et al (2012) Automatic assessment of problem behavior in individuals with developmental disabilities. In Proceedings of the 2012 ACM conference on ubiquitous computing, (pp 391–400).
- Kang, J.Y et al (2016). Automated Tracking and Quantification of Autistic Behavioral Symptoms Using Microsoft Kinect; IOS Press: Amsterdam, The Netherlands, (pp 167–170).
- Wedyan, M.& Al- Jumaily, A (2016). Early diagnosis autism based on upper limb motor coordination in high-risk subjects for autism. In Proceedings of the 2016 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS), (pp 13–18).
- El Arbaoui F.E.Z., El Hari K., Saidi R. A Survey on the Application of the Internet of Things in the Diagnosis of Autism Spectrum Disorder. In: Advanced Technologies for Humanity. ICATH 2021. Lecture Notes on Data Engineering and Communications Technologies, vol 110. Springer, Cham. (2022)
- El Arbaoui F.E.Z., El Hari K., Saidi R. A Review on The Application of The Internet of Things in Monitoring Autism and Assisting Parents and Caregivers. In: Computational intelligence for medical internet of things MIoT application systems. Elsevier. (2022)
No competing interests reported.