WearNControl: Textile-Based Multimodal Customizable Activators

doi:10.21203/rs.3.rs-3138447/v1

Download PDF

Research Article

WearNControl: Textile-Based Multimodal Customizable Activators

https://doi.org/10.21203/rs.3.rs-3138447/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

This paper presents early prototypes of WearNControl, textile-based multimodal customizable activators. The prototypes were targeted at turning our ordinary clothing and accessories assistive. The activators are part of a system with a web-based interface that allows the user to tailor the functionalities of the activators. These first prototypes can be activated by three different methods; pushing, touching, and pulling, and the interface allows the following functionalities: sending text messages and emails, making phone calls, and playing music. Based on preliminary prototype testing and system evaluation by five testers, an overall success rate of 98% was achieved for all activator inputs and text message outputs. Next, more types of activator prototypes will be created, and user opinions will be gathered for desired activator functionalities. Further, by using better servers and services, optimal system performance can be achieved.

Assistive technology

Multimodality

Wearables

Smart clothing

Smart accessories

Three textile-based wearable multimodal activator prototypes were created that the user can wear and activate using touch, pull, or press. The prototypes were targeted at turning our ordinary clothing and accessories assistive.
We introduced three activator prototypes: 1) Bracelet Prototype - activated by pressing, 2) Hoodie Prototype - activated by pulling, and 3) Touch Prototype - activated by light touch. All activators were tested and proved to be functional.
A web-based interface was made to customize the functionality of the activators. From the interface, an option to send emails and text messages, make phone calls, or play music can be selected.

Approximately 1.6 billion people worldwide experience disabilities, e.g., activity limitations or participation restrictions, and inclusion and empowerment of these people are critical global goals [1]. Further, statistics show aging as a global challenge, and even though aging itself doesn’t cause disabilities, morbidity is related to old age [2–4]. Decreased independence reduces quality of life, which further increases the need for assistance for the elderly and people with disabilities, in a way, that does not make them to be seen or to feel as a person requiring constant assistance [5–7]. Assistive technology (AT) is technology that increases, maintains, or improves the functional capabilities of the user [8], providing enabling solutions to those who have disabilities and to the caregivers around them [9–11].

Ambient assisted living (AAL) uses technology that enables people to keep living independently in their own environment. The goal is to promote well-being and help users to maintain their independence. AAL systems, e.g., use automated decision-making processes to analyze and interpret complex information from various devices and sources to support the user’s living [12]. Wearable devices along with in-ear devices (known as hearables) and nearby devices that interact with the wearables (called nearables) have a significant impact on the future of healthcare and lifestyle [13, 14].

Recently, work has been done to implement activators, such as buttons, into clothing and into surroundings for wearable wellbeing, home-automation, and assistive solutions. A few examples include passive ultra-high frequency (UHF) radio frequency identification (RFID) tags combined with 3D-printed pushbuttons, which are battery-free, lightweight, and easy to integrate into surroundings [15], Internet of Things (IoT)-based wearable devices and artificial intelligence (AI)-based tools for human-computer interaction in assisted living, and a control system to provide a platform using the power of AI to interact with diverse technologies for improving user’s daily life [16]. Further, [17] has proposed a home panic button that uses a NodeMCU ESP8266 microcontroller to send a message via Message queuing telemetry transport (MQTT) to a web subscriber, notifying them of the emergency and providing a map of the area to ensure a secure, flexible, and reliable system for public security. Another recent article [18] described the design and implementation of an adaptive panic button prototype for older adults, specifically those suffering from Alzheimer's, that can be monitored through geolocation data, and a processing unit to send the user's location data to the management system. Technical tests were performed with older adults to evaluate message arrival times, bandwidth, and system stability [18]. Finally, [19] introduced ClothFace, a battery-free, unnoticeable human-technology interface platform integrated into a shirtsleeve, consisting of passive UHF RFID components.

The aim in this study is to integrate multimodal customizable activators into everyday clothing and accessories in a discreet and fashionable way. We introduce three prototypes: 1) Bracelet Prototype - activated by pressing, 2) Hoodie Prototype - activated by pulling, and 3) Touch Prototype - activated by lightly touching. We also present a web-based interface to customize the functionality of the activators. From the interface, an option to send emails and text messages, make phone calls, or play music can be selected.

2.1 WearNControl System Overview

Our system is built around a web-based interface and three types of activators. The interface allows the user to select the desired activator and the wanted functionality for that activator. Each activator has a wireless module that can communicate with the interface through Wi-Fi. The interface is hosted on a cloud server, and from there it is listening to any incoming requests from the activators and performing the pre-selected functionality. For establishing the functionalities, i.e., making phone calls, sending emails and text messages, and playing music, If this then that service (IFTTT) was used. Figure 1 shows an overview of the different parts of the system, while Fig. 2 presents the system steps from user input to desired output.

2.2 WearNControl Activator Prototypes

For this study, we developed three types of activators: 1) Bracelet Prototype - activated by pressing, 2) Hoodie Prototype - activated by pulling, and 3) Touch Prototype - activated by lightly touching. Once activated, these activators will perform the functionality that is set through the created web-based interface.

The aim was to create three types of activators with different modes of activation. This was achieved by using a momentary push-button for sensing press, force sensing resistor (FSR) sensor for sensing touch, and an analog stick for pull activation. Considering the other electronics needed, ESP8266 Huzzah feather development board had the needed properties by being extremely small, lightweight. The mode of wireless communication is through Wi-Fi. This gives the activators far better range and easy adaptation into indoor and outdoor environments. The ESP8266 connects to a Wi-Fi network that allows it to communicate with a server. For this prototype, we set up a server in a private cloud, so the activators can be accessed from anywhere. Figure 3 shows a wireless module during fabrication of a Hoodie Prototype.

A lithium-polymer battery was used for power, as it can provide power to the prototype for two days. Finally, the microcontrollers were programmed to connect to a specified Wi-Fi network and to establish a connection to the interface server. Once connected, the ESP-8266 is ready to be activated when the buttons are pressed or a certain analog value from a sensor is achieved. The main protocol of communication is done using hypertext transfer protocol (HTTP) requests. The two types of HTTP methods used were GET and POST. GET requests are made to retrieve data from server and POST requests are made to send a new resource or data to the server [20]. Once the wireless modules are activated, the module sends a HTTP POST request with the activator ID as data. Each time the activator is activated, the POST request is made to the ‘submit-form’ endpoint of the interface host. The flowchart for the wireless module programming is shown in Fig. 4.

The Hoodie Prototype - activated by pulling, was created using a 2-axis analog joystick attached to the end of a hoodie string. By pulling on the string, the potentiometer of the joystick provides a changing analog reading to the ESP-8266, which is then recognized by the system; a functionality was activated when the analog input reaches 1023, i.e., the string must be pulled until the joystick reaches the final position. Figure 5 shows the Hoodie Prototype and a person using it.

The Touch Prototype - activated by lightly touching, was implemented using a FSR sensor, which is activated with pressure. Here, the threshold analog input value was chosen as 150, i.e., 146.6 mV, that is required to activate the module. The value was optimized for touching over clothing, as presented in Fig. 6. The prototype was created using a small pouch, which has the FSR along with the microcontroller. This pouch has magnets and can be attached to any cloth having a magnetic part. Thus, it can be easily attached and removed. Figure 6 shows the Touch Prototype and a person using it.

The Bracelet Prototype - activated by pressing, was created simply by using a click button to activate the wireless module. A bracelet-brooch looking accessory was created, and the wireless module was placed inside. Figure 7 shows the Bracelet Prototype and a person using it.

2.3 Web-based WearNControl interface

For the web-based interface, the frontend communicates with the backend that performs all the necessary functionalities in the background. Frontend was developed using HTML first. Next, CSS was used to give the application a better appearance. Finally, JavaScript was used to make the frontend interactable. The backend was created with NodeJS. Figure 8 shows a screenshot of the interface. To host the backend in a server, a free cloud hosting service called Render was used. The whole repository of frontend and backend was uploaded to GitHub, and then deployed onto render. Figure 9 presents the flowchart of the web interface. Every time the ‘submit-form’ endpoint receives the HTTP request with the activator ID, it checks the ‘selectedfunction’ for which functionality was selected by the user and sends trigger to IFTTT for output.

2.4 IFTTT Service used in WearNControl

For establishing the functionalities, IFTTT service was used. In this platform, applets are individual condition-based triggers, which can be activated. Once the applet receives an appropriate trigger, it launches the action associated with it. This trigger is sent from the interface once an activator is activated. For WearNControl, we chose Gmail applet for sending emails, Telegram applet for sending text messages, and automated call bot for making phone calls. Another action was added, which can play music through an Android phone. The messages to be sent as text or as email can be inserted through the applets together with the destination number or email address. The automated call bot makes a call to the number provided in its applet and speaks out the message provided inside the applet. For playing music, the applet uses the IFTTT app to open the selected music player app, in this case Spotify, and starts playing a specific track from the app. IFTTT is very easily customizable by anyone, and it also has smartphone app. Another reason for using this service is that it allows free service for testing purposes.

A preliminary evaluation was carried out for the activator prototypes and the system by five testers. Each tester activated each prototype ten times in order to send a text message to the assigned phone number. The overall success rate for all activator prototypes was calculated by how often the expected outcome, i.e., text message, is achieved from the input, i.e., from activation, and it turns out to be 98%.

For further analysis, by checking the console monitor of ESP8266, it was possible to record how many times the prototypes were unsuccessfully activated and how many times they were not activated. Next, we opened the IFTTT applets, where it shows how many times the applets were triggered. This tells us how many activations got lost between the prototype and the IFTTT service. This includes the interface and the internet connection. It also tells us how many times the IFTTT service failed to generate text messages from the triggers it received. Table 1 provides these results.

Table 1

Reliability of the Prototype
Prototype	Input given	Successful output, i.e., text message as output	Unsuccessful input, i.e., prototype was not activated	System loss, i.e., loss between prototype and IFTTT	IFTTT fails to generate output	Success rate
Bracelet	50	48	0	0	2	100%
Hoodie	50	50	0	0	0	100%
Touch	50	47	2	1	0	96%

The simple click button in the Bracelet prototype and the pulled Hoodie Prototype were both activated by all inputs. Since there was reliable connection, everything worked 100% until the IFTTT. Here, twice the IFTTT failed to generate outputs, i.e., text messages. This is probably since the free version of the IFTTT provides service at a lower priority. During peak demand times, some actions are not performed as urgently and thus get lost. Optimal server conditions and uninterrupted internet connection together with fully functional IFTTT service are thus needed for the system. In general, the Bracelet Prototype was the easiest to activate. One thing that was noticed with the Hoodie Prototype is that the string did not always go back to its original position and had to be manually shifted by shaking the string. This can be easily fixed for the next prototype by using a higher quality analog stick.

The Touch Prototype took a bit longer for the users to understand, as it technically used pressure instead of actual touch. Still, the users only needed to be shown once how to touch the prototype. From Table 1, it can be seen that three times it failed to generate text messages. Out of those three times, two were due to the touch not registering in the prototype. This prototype thus needs further tuning for optimal results. Nevertheless, for an early-stage prototype, the number is decent. Further, one of the successful attempts made from the Touch Prototype failed to reach the IFTTT service. HTTP requests can get lost if there is a connection problem between the server and client. Losing connection may have contributed to this. However, a more likely reason is the server not registering the request at the given time due to a server overload or limitation in the free service.

Next, more types of activator prototypes will be created, and by using better servers and services, optimal functionality results can be achieved. Also, this work will be extended by developing the interface further for allowing phone numbers, email addresses, and choice of music input right from the interface rather than from the IFTTT applets. The interface will also be developed to host more options for functionalities.

Three textile-based wearable multimodal activator prototypes were presented that the user can wear and activate by three different methods: pressing, touching, and pulling. The prototypes were targeted at turning our ordinary clothing and accessories assistive. A web-based interface was made to customize the functionalities of the activators. At this first stage, the interface allows the user to send text messages and emails, make calls, and play music. However, the application possibilities are not limited to these. With the interface, these activators can be made to perform versatile actions, such as controlling smart home applications and communicating through different media. Although the results achieved were promising, the prototypes and web-based interface are at a very early stage of development. Next, more types of activator prototypes will be created and by using better servers and services, optimal functionality results can be achieved. The next system version will also have user authentication. Our goal is to gather data on user experience of the actual users, i.e., elderly people and people with special needs. By integrating assistive technology into clothing and accessories, we can greatly promote inclusion. Our prototypes are a great example of what is possible in the future.

Statements & Declarations

Funding

This work has been funded by Finland Ministry of Education and Culture (TEKOS project) and the Academy of Finland (decision 332168).

Competing Interests

The author(s) declared no conflicts of interest regarding the research, authorship, and/or publication of this article.

Data availability

There is no additional data other than the data presented in this manuscript.

Ethical approval

The research was conducted under the ethics guidelines provided by the Ethical principles of research with human participants and ethical review in the human sciences in Finland under the Finnish National Board on Research Integrity (TENK) in 2019. In accordance with the guidelines of Tampere University, further ethical review was not necessary for this non-medical research.

Consent to participate

All participants gave informed consent to take part in this study.

WHO (2023), Disability and health. https://www.who.int/news-room/fact-sheets/detail/disability-and-health. Accessed 10 May 2023
Ritchie H, Roser M (2019) Age Structure. Our World Data
Kennedy BK, Berger SL, Brunet A, Campisi J, Cuervo AM, Epel ES, Franceschi C, Lithgow GJ, Morimoto RI, Pessin JE, Rando TA, Richardson A, Schadt EE, Wyss-Coray T, Sierra F (2014) Geroscience: Linking Aging to Chronic Disease. Cell 159:709–713. https://doi.org/10.1016/j.cell.2014.10.039
Phiriyapokanon T (2011) Is a big button interfaceenough for elderly users?: Towards user interface guidelines for elderly users
Massimi M, Baecker RM, Wu M (2007) Using participatory activities with seniors to critique, build, and evaluate mobile phones. In: Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility. Association for Computing Machinery, New York, NY, USA, pp 155–162
Oleson M, Heading C, Shadick KM, Bistodeau JA (1994) Quality of life in long-stay institutions in England: nurse and resident perceptions. J Adv Nurs 20:23–32. https://doi.org/10.1046/j.1365-2648.1994.20010023.x
Kochera A, Straight A, Guterbock T (2005) Beyond 50.05: A Report to the Nation on Livable Communities - Creating Environments for Successful Aging
Dave L. Edyburn (2004) Rethinking assistive technology, Special Education Technology Practice 5.4
Carver J, Ganus A, Ivey JM, Plummer T, Eubank A (2016) The impact of mobility assistive technology devices on participation for individuals with disabilities. Disabil Rehabil Assist Technol 11:468–477. https://doi.org/10.3109/17483107.2015.1027295
Gasteiger N, Ahn HS, Fok C, Lim J, Lee C, MacDonald BA, Kim GH, Broadbent E (2022) Older adults’ experiences and perceptions of living with Bomy, an assistive dailycare robot: a qualitative study. Assist Technol Off J RESNA 34:487–497. https://doi.org/10.1080/10400435.2021.1877210
Nicolson A, Moir L, Millsteed J (2012) Impact of assistive technology on family caregivers of children with physical disabilities: a systematic review. Disabil Rehabil Assist Technol 7:345–349. https://doi.org/10.3109/17483107.2012.667194
Jovanovic M, Mitrov G, Zdravevski E, Lameski P, Colantonio S, Kampel M, Tellioglu H, Florez-Revuelta F (2022) Ambient Assisted Living: Scoping Review of Artificial Intelligence Models, Domains, Technology, and Concerns. J Med Internet Res 24:e36553. https://doi.org/10.2196/36553
Kirk S (2014) The Wearables Revolution: Is Standardization a Help or a Hindrance?: Mainstream technology or just a passing phase? IEEE Consum Electron Mag 3:45–50. https://doi.org/10.1109/MCE.2014.2345996
Loncar-Turukalo T, Zdravevski E, Silva JM da, Chouvarda I, Trajkovik V (2019) Literature on Wearable Technology for Connected Health: Scoping Review of Research Trends, Advances, and Barriers. J Med Internet Res 21:e14017. https://doi.org/10.2196/14017
Valo P, Leino M, Lehtinen T, Virkki J, Tchuenbou-Magaia F, Poberznik A, Kortelainen J, Koivisto A, Merilampi S (2022) Additional use cases for RFID tags by implementing 3D printed push-button functionalities. In: 2022 IEEE 9th Electronics System-Integration Technology Conference (ESTC). pp 576–580
Jaboob S, Chauhan MS, Dhanasekaran B, Natarajan SK (2022) Multimodal Assistive Technologies Prototype for the Disabled in Oman. In: 2022 6th International Conference on Computer, Software and Modeling (ICCSM). pp 70–75
Prayogo SS, Rafi FA, Mukhlis Y (2019) Design and Built IoT Home Panic Button for Smart City. J Phys Conf Ser 1175:012097. https://doi.org/10.1088/1742-6596/1175/1/012097
Jara JD, Lima J, Valdez C, Barbecho M, Bermeo JP, Chacón D (2020) Design of a Mobile Panic Button for Older Adults for Monitoring through the ECU911 System. In: Proceedings of the 3rd International Conference on Vision, Image and Signal Processing. Association for Computing Machinery, New York, NY, USA, pp 1–5
Mehmood A, He H, Chen X, Vianto A, Vianto V, Buruk O ‘Oz’, Virkki J (2020) ClothFace: A Passive RFID-Based Human-Technology Interface on a Shirtsleeve. Adv Hum-Comput Interact 2020:e8854042. https://doi.org/10.1155/2020/8854042
What is HTTP? Protocol Overview for Beginners. (2023) In: freeCodeCamp.org. https://www.freecodecamp.org/news/what-is-http/. Accessed 9 Jun 2023

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

WearNControl: Textile-Based Multimodal Customizable Activators

Status:

Version 1

Abstract

Figures

Article Highlights

1 Introduction

2 System Development

2.1 WearNControl System Overview

2.2 WearNControl Activator Prototypes

2.3 Web-based WearNControl interface

2.4 IFTTT Service used in WearNControl

3 Practical testing

4 Conclusions

Declarations

Statements & Declarations

Funding

Competing Interests

Data availability

Ethical approval

Consent to participate

References

Additional Declarations

Status:

Version 1