This study found that children with high tenacity in the serious game have high score improvement regardless of developmental disabilities. This is also the first study to measure tenacity in the data-driven index, unlike traditional qualitative studies that measured tenacity in survey-based indicators. We measured user behavior with the data-driven index that was acquired and found that user tenacity is associated with improvement in gameplay skills, known as the effect of children’s cognitive development in the serious game.
Effect of tenacity for improving serious game driven cognitive index.
Tenacity, descriptively defined in our study, was consisted of two factors: (1) having obstacles, (2) playing the serious game without giving up. We identified the obstacle as checking the incorrect answer count record, and persistency as checking whether the children complete the task. Through this process, we could estimate tenacity objectively and quantitively. It could also minimize observer effect, Hawthorne effect, since children did not feel that they are observed to measure the tenacity during playing the serious game.
Our analysis was conducted separately according to the presence of developmental disabilities because of the fundamentally different characteristics of each group. Further analyses were conducted according to the game levels since children who played at a different game level showed statistically significant differences.
We observed differences in the score change distributions between tenacity groups in some of the categories depending on the game level and the presence of developmental disabilities. The meaning of the larger gap in the group is that the gameplay skills have been improved than the other group, and the absolute value or the number sign did not matter.
Do children with high tenacity score better than others? Since each group had different baseline scores, a DID estimation was performed. As a result, we found that children with high tenacity had higher gameplay score improvements in their specific cognitive sub-categories - Inference, Numerical, Organizing – regardless of any developmental disabilities.
Future work to develop non-cognitive factors for children with low tenacity.
The serious game used in this study was a mobile application for cognitive development. As it is in the process of FDA pre-submission, it is related to digital therapeutics . From the perspective of cognitive therapy, it is possible to focus only on the effects of cognitive development. However, we found that true cognitive therapy is possible when stimulating both cognitive development and non-cognitive development such as tenacity together. In other words, it is necessary to provide functions in the serious game to restore tenacity of the users. For instance, there were 312 children who were included in the study by performing up to episode 10 but designated low tenacity. Even for the 2,072 children who were excluded because they could not perform until episode 10, low tenacity may be one of the reasons for stopping the game. If the serous game included tenacity reinforcing elements, it would have been possible to check the cognitive development of more children without excluded participants.
Through our study, a child’s tenacity and its effects can be linked to the issue of selecting a target group in digital therapy Digital therapeutics can be effective when the players achieve the characterizing goal, and when the effects of learning or training are sustainable . This sustainability is related to tenacity in terms of persistence and perseverance, trying to reach to the end. This indicates that the potential of clinical efficacy of this serious game is derived from user compliance effect depending on the human factors, regardless of the presence of developmental disabilities.
It is not clear whether the tenacity we technically defined and measured represents the actual tenacity of the child. We observed that children with specific gameplay patterns showed higher play-skill improvement in the serious game, but we cannot be sure if it is a real indicator of the tenacity in children.
We did not evaluate the accuracy of the algorithm that defines the tenacity. As a future study, we will conduct a study comparing the child's evaluation by an expert or other educational interventions to our tenacity index in order to be generalizable to other populations in other contexts.
Moreover, there is a possibility that the gameplay patterns were created based on parental intervention, and not from the child himself. As our future work, we will study to find the elements of the game that can distinguish whether the child played the game on their own or with the help of their parents, and based on this, the algorithm will be advanced.