Precision and Accuracy of a Simple Consumer-oriented Wristband Activity Tracker During Walking at Different Speeds

doi:10.21203/rs.3.rs-742950/v1

Download PDF

Research note

Precision and Accuracy of a Simple Consumer-oriented Wristband Activity Tracker During Walking at Different Speeds

https://doi.org/10.21203/rs.3.rs-742950/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Objective: FAIT Tag is a wrist-worn consumer-oriented pedometer composed of a small and lightweight core and silicone wrist band. The aim of this study was to validate the measurement precision and accuracy of the FAIT Tag during walking.

Results: To assess intra-subject precision, five subjects wearing a FAIT Tag performed a repeated 200-step walking test in which they walked 200 steps a total of 10 times. To assess inter-subject accuracy, 30 subjects wearing a FAIT Tag and ActiGraph walked for 3 minutes at two different speeds. Step count measured by these devices was compared with the actual step count counted visually and simultaneously by an investigator. The average intra-subject precision was 2.2. For the inter-subject precision of FAIT Tag, the concordance correlation coefficient and the absolute percentage error ranged from 0.35 to 0.39 and 5.4% to 10.0%, respectively. In conclusion, the FAIT Tag is a practical pedometer with low intra-subject error and acceptable measurement accuracy. It may be a useful tool for long-term patient monitoring and digital biomarker acquisition.

Cardiac & Cardiovascular Systems

pedometer

precision

accuracy

validation

concordance correlation coefficient

absolute percentage error

mean percentage error

Bland-Altman plot

Digital devices equipped with various types of sensors have become increasingly more popular over recent years. Wearable, non-invasive vital sensors are now readily available ^1-3. The data obtained by these devices, known as digital biomarkers ⁴, reflect daily life at home. Digital biomarkers are expected to play a complementary and enhancing role to traditional medical data acquired at hospitals.

The number of steps an individual takes each day as part of their daily life can be considered a digital biomarker. Step count is generally measured using an accelerometer. The waist-mounted ActiGraph is known to be a highly reliable accelerometer; however, it requires frequent recharging (about once a week) and is cumbersome to handle. In contrast, the FAIT Tag is a lightweight, 5 bar water-resistant pedometer with longer battery life, and is considered suitable for long-term wear. Therefore, the FAIT Tag has the potential to be a digital biomarker measurement device for long-term monitoring of disease status; however, its measurement reliability has not been fully verified.

The purpose of this study was to validate the measurement precision and accuracy of the FAIT Tag during walking.

1. Device

FAIT Tag (Sony Network Communications, Tokyo, Japan) is a wrist-worn battery-driven activity tracker composed of a core and silicone wrist band. The core contains a tri-axial accelerometer and a consumer button battery, and has a diameter of 25 mm, thickness of 8 mm, and weight of 15 g. It is a consumer-oriented activity tracker for health promotion ^5,6. Although it has no display, measured data can be transmitted via Bluetooth to a dedicated app on a smartphone or tablet, and further to a dedicated secure cloud. In this study, we used firmware that was updated in 2020.

2. Pilot Study

In the pilot study, we assessed the intra-subject precision of step count. Five researchers performed a repeated 200-step walking test in which they walked 200 steps a total of 10 times at their own pace around a 150-m circuit or along a corridor over 100 m in length. FAIT Tag was worn on the non-dominant wrist and the actual step count was counted visually by another researcher.

3. Clinical Study

Healthy staff and students of Osaka University and Sony Corporation participated as volunteers in this study. Participants were eligible for inclusion if they were 20 years old or older, and could walk without assistance.

Data were obtained from participants enrolled in the current study between June and November 2020. Each participant completed a walking test on a single visit at two different speeds: a normal pace and a brisk pace. The participants walked on treadmill or along a corridor over 50 m in length. Each participant wore a FAIT Tag on the non-dominant wrist and an ActiGraph (ActiGraph, Pensacola, FL, United Sates) on the waist. The normal and brisk paces were set at 4.0±1.0 km/hr and 6.0±1.0 km/hr for males, and 3.5±1.0 km/hr and 5.5±1.0 km/hr for females, respectively. The step count was measured using the two activity trackers, and also visually by a researcher.

4. Statistical Analysis

All statistical analyses were performed using JMP Pro 15.1.0 and R 4.0.3. Continuous variables were expressed as mean ± standard deviation (SD). Intra-subject precision was defined as the SD of measured step count/actual step count x 100, with 0 as the best score, in each subject ⁷. The concordance correlation coefficient (CCC) was defined as CCC = 2σ₁₂ / [(μ₁-μ₂)² +σ₁² +σ₂² )], where σ₁₂ is the covariance, μ₁andμ₂ are the mean, and σ₁² and σ₂² are the variance ⁸. Mean percentage error (MPE) was calculated as [(measured step count – actual step count) / actual step count] x 100 [%]. Absolute percentage error (APE) was calculated as [｜(measured step count – actual step count)｜/ actual step count] x 100 [%].

The five researchers who participated in the pilot study consisted of three males and two females whose average age was 34±6 years old. The measured step count and intra-subject precision in the repeated 200-step walking test are shown in Table 1. The average intra-subject precision was 2.2.

Subsequently, a total of 30 participants were recruited to participate in the clinical study. The average age was 36±10 years old, height was 168±9 cm, body weight was 62±12 kg and body mass index was 22.1±3.1 kg/m².　Eighteen (60%) were male and 27 (90%) were right-handed. Regression analyses showed no significant relationship between APE and age, height or gender in each condition (data not shown).

We measured step count in a 3-minute walking test at two different speeds (Table 2). At the normal walking speed, the step count measured by FAIT Tag was 322.4±31.4 and that by ActiGraph was 347.6±69.9; the actual step count was 337.0±22.3. At the brisk walking speed, step count measured by FAIT Tag was 356.4±48.7 and that by ActiGraph was 403.6±46.7; the actual step count was 396.0±28.4.

To assess agreement between the different measurement methods, we plotted step count measured by FAIT Tag or ActiGraph on the y-axis against actual step count on the x-axis (Fig. 1a-d). The CCC between the actual step count and step count measured by FAIT Tag was 0.39 at the normal walking speed (Fig. 1a) and 0.35 at the brisk walking speed (Fig. 1b). The CCC between the actual step count and step count measured by ActiGraph was 0.13 at the normal walking speed (Fig. 1c) and 0.23 at the brisk walking speed (Fig. 1d).

Bland-Altman plots for the actual and measured steps in each condition are showed in Fig. 1e-f. The plots demonstrate that the FAIT Tag was likely to underestimate step count. There was no obvious proportionality effect at either the normal or the brisk walking speed.

The MPE and mean APE under each condition are shown in Fig. 1i-j. The MPE ranged from -10.0% to -4.2% for FAIT Tag, and 2.2% to 3.3% for ActiGraph, while the APE ranged from 5.4% to 10.0% for FAIT Tag, and 5.6% to 9.4% for ActiGraph.

We evaluated the precision and accuracy of FAIT Tag for counting steps during walking. FAIT Tag is a simple, lightweight wrist-worn pedometer that may be suitable for long-term monitoring for months or years.

First, we performed a repeated 200-step walking test to assess intra-subject precision. There are limited reports on the precision of pedometers. Furthermore, to our knowledge, no report has examined the intra-subject precision of wrist-worn pedometers. Foster et al. reported that the precision of two commercially available waist-worn pedometers ranged between 2.1 and 2.7 at 4.8 km/h and between 2.9 and 7.3 at 3.2 km/h ⁷. Our result of 2.2 is comparable to the precision of these waist-worn pedometers.

Next, we assessed the inter-subject accuracy of FAIT Tag, together with ActiGraph, at two different speeds. We expressed the measurement accuracy using four typical methods or parameters: CCC, Bland-Altman plot, MPE and APE. Few studies have reported the CCC for pedometers and there is currently no established CCC threshold. Daligadu et al. measured the accuracy of a Fitbit, a typical wrist-worn consumer device, using CCC and APE. They showed that the CCC and APE of the Fitbit was 0.46 and 16.2 for slow walking, and 0.33 and 21.9 for fast walking, respectively ⁹. Both MPE and APE are known indicators of pedometer accuracy. APE is a more conservative estimate of error, because it considers both overestimation and underestimation. Because FAIT Tag tended to underestimate the actual step count (Fig. 1e, f), which is also true of other devices ¹⁰, we found no obvious difference in the percentage error between MPE and APE. In contrast, APE was larger at the brisk walking speed than at the normal walking speed. MPE is known to vary with walking speed for some pedometers ^11,12. Considering that APE was smaller at the normal walking speed than the brisk walking speed, FAIT Tag might be more suitable for assessing usual daily activities. Although there is no absolute consensus on the acceptable error of pedometers, an error of 3% in controlled walking and up to 10% in free living are considered acceptable in some previous reports ^13,14. Although the FAIT Tag performed relatively poorly in terms of CCC, it may be an acceptable pedometer for long-term monitoring of daily life, given its lack of a proportional effect, acceptable APE and MPE, and lightweight and small build.

The mechanism underlying the variable accuracy of pedometers among subjects is not fully understood. In our study, neither gender nor age affected APE (data not shown). Similarly, BMI had no effect on APE, which is consistent with previous reports ¹². Acceleration sensed by a pedometer on the wrist is affected by the way an individual walks, including their arm swing, kick off from the ground and landing. Evaluating raw acceleration during walking in different subjects is very important for interpreting pedometer data. Therefore, we additionally examined acceleration patterns during walking. Three members of our research team walked 200 steps at their own pace along a corridor wearing Shimmer (Shimmer Sensing, Dublin, Ireland) and FAIT Tag on the non-dominant wrist. Shimmer is a research-grade inertial measurement unit ^15,16. We acquired acceleration values during walking at a sampling rate of 256 Hz. Step counts measured by FAIT Tag were close to the actual step count in Subject 4 and 6 (189 steps and 195 steps, respectively), while the measured step count was an underestimate of the actual step count in Subject 3 (162 steps). Supplemental Figure 1 shows representative waveforms of acceleration norm and acceleration along the X, Y and Z axes recorded by Shimmer. The orientation of the X, Y and Z axes are illustrated in Suppl Fig. 1g. Subjects 4 and 6 showed constant, periodic peaks of acceleration norm (Suppl Fig. 1b, c). In contrast, the waveform for Subject 3 was small and jagged, and periodic peaks were difficult to identify (Suppl Fig. 1a). When comparing the waveforms in each axis, differences between the subjects were most evident in the Y axis (Suppl Fig. 1d-f). These results suggest that wave patterns of acceleration during walking differ among subjects, which may partially explain the varied accuracy of pedometers among subjects. Furthermore, a pedometer is equipped with a number of functions to avoid over-sensing caused by wrist movement when not walking or contact between the wrist and trunk during walking. One such function is the dynamic threshold. FAIT Tag continuously updates the threshold based on acceleration norm of the most recent series of steps ¹⁷. Another function is to limit the frequency of acceleration norm. FAIT Tag continuously monitors the frequency of acceleration norm with every 4 peaks without overlap. When the frequency is outside the pre-defined range, the 4 peaks are excluded from step recognition to avoid over-sensing. Moreover, FAIT Tag requires eight continuous norm peaks above the dynamic threshold to begin to detect walking ¹⁷. Therefore, FAIT Tag is unable to start counting steps if some peak(s) of the acceleration norm fail to reach the dynamic threshold at the initiation of walking. Because the acceleration norm waveform during walking varies from subject to subject, these overestimation avoidance functions may conversely reduce the measurement accuracy of the FAIT Tag for some subjects. In general, there is often a trade-off between methods for avoiding over-sensing and underestimating step count.

FAIT Tag is a small, lightweight pedometer with low intra-subject error and acceptable measurement accuracy. It may be a useful tool for long-term patient monitoring and digital biomarker acquisition.

The study had several limitations. First, the sample size was relatively small and the participants were all healthy and young or middle-aged adults, which could limit the generalizability of the study findings to aged populations or patients with diseases ¹⁸. Second, some of the participants walked on a treadmill while others walked along a corridor. However, this difference in protocol does not seem to affect the measured outcomes ¹¹. Third, we assessed the accuracy of activity trackers only under controlled conditions, but not under free-living conditions.

APE: absolute percentage error; CCC: concordance correlation coefficient; MPE: mean percentage error; SD: standard deviation.

Ethics approval and consent to participate

Written informed consent was obtained from all participants in the clinical study before testing. The study protocol was approved by the institutional review board at Osaka University and Sony Corporation before testing. All procedures in this study were in accordance with the 1964 Declaration of Helsinki.

Consent for publication

Not applicable

Competing interests

YS received a research grant from Sony Group Corporation. The other authors declare that they have no competing interests.

Funding

This research was supported by MEXT “Innovation Platform for Society 5.0” Program Grant Number JPMXP0518071489 and Sony Corporation.

Authors’ contributions

All authors have been personally and actively involved in substantive work in the lead up to writing the manuscript and hold themselves jointly and individually responsible for its content. SK, DN and MK contributed to the design and implementation of the research, to the analysis of the results and to the writing of the manuscript. KN and MS contributed to the data collection and writing of the manuscript. KY contributed to data collection. HK contributed to the statistical analysis of the acquired data. TT, YM and YS contributed to the revision and quality check of the manuscript. All authors read and approved the final manuscript.

Acknowledgments

We appreciate Ms. Mikako Kataoka for her dedicated support with data collection.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

1. Wilbanks JT, Topol EJ. Stop the privatization of health data. Nature 2016;535:345-8.

2. Pevnick JM, Birkeland K, Zimmer R, Elad Y, Kedan I. Wearable technology for cardiology: An update and framework for the future. Trends Cardiovasc Med 2018;28:144-50.

3. Dunn J, Runge R, Snyder M. Wearables and the medical revolution. Per Med 2018;15:429-48.

4. Babrak LM, Menetski J, Rebhan M, Nisato G, Zinggeler M, Brasier N, et al. Traditional and Digital Biomarkers: Two Worlds Apart? Digit Biomark 2019;3:92-102.

5. Sony Network Communications Inc.: https://iot.sonynetwork.co.jp/case/fait/ Accessed on 04 May 2021.

6. PR TIMES: https://prtimes.jp/main/html/rd/p/000001241.000000196.html Accessed on 04 May 2021.

7. Foster RC, Lanningham-Foster LM, Manohar C, McCrady SK, Nysse LJ, Kaufman KR, et al. Precision and accuracy of an ankle-worn accelerometer-based pedometer in step counting and energy expenditure. Prev Med 2005;41:778-83.

8. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989;45:255-68.

9. Daligadu J, Pollock CL, Carlaw K, Chin M, Haynes A, Thevaraajah Kopal T, et al. Validation of the Fitbit Flex in an Acute Post-Cardiac Surgery Patient Population. Physiother Can 2018;70:314-20.

10. Fuller D, Colwell E, Low J, Orychock K, Tobin MA, Simango B, et al. Reliability and Validity of Commercially Available Wearable Devices for Measuring Steps, Energy Expenditure, and Heart Rate: Systematic Review. JMIR Mhealth Uhealth 2020;8:e18694.

11. Coffman MJ, Reeve CL, Butler S, Keeling M, Talbot LA. Accuracy of the Yamax CW-701 Pedometer for measuring steps in controlled and free-living conditions. Digit Health 2016;2:2055207616652526.

12. Feito Y, Bassett DR, Thompson DL, Tyo BM. Effects of body mass index on step count accuracy of physical activity monitors. J Phys Act Health 2012;9:594-600.

13. Chen MD, Kuo CC, Pellegrini CA, Hsu MJ. Accuracy of Wristband Activity Monitors during Ambulation and Activities. Med Sci Sports Exerc 2016;48:1942-9.

14. Tudor-Locke C, Sisson SB, Lee SM, Craig CL, Plotnikoff RC, Bauman A. Evaluation of quality of commercial pedometers. Can J Public Health 2006;97 Suppl 1:S10-5, s-6.

15. Damji F, MacDonald K, Hunt MA, Taunton J, Scott A. Using the VERT wearable device to monitor jumping loads in elite volleyball athletes. PLoS One 2021;16:e0245299.

16. Jung S, Michaud M, Oudre L, Dorveaux E, Gorintin L, Vayatis N, et al. The Use of Inertial Measurement Units for the Study of Free Living Environment Activity Assessment: A Literature Review. Sensors (Basel) 2020;20.

17. STMicroelectronics: https://www.st.com/resource/en/application_note/dm00472670-lsm6ds3trc-alwayson-3d-accelerometer-and-3d-gyroscope-stmicroelectronics.pdf Accessed on 04 May 2021.

18. Modave F, Guo Y, Bian J, Gurka MJ, Parish A, Smith MD, et al. Mobile Device Accuracy for Step Counting Across Age Groups. JMIR Mhealth Uhealth 2017;5:e88.

Table 1

Heading: Measured Step Count in Repeated 200-step Walking Test

Legends: Intrasubject precision was calculated as the SD of measured step count/ actual steps x 100, with 0 as the best score.

Subject	#1	#2	#3	#4	#5	Average
Mean step count, steps	199.1	196.8	151.3	194.8	183.4	185.1
Intra-subject precision	1.0	1.3	3.2	2.3	3.3	2.2

Table 2

Heading: Actual and Activity Tracker-measured Step Count during 3-minute Walking Test at Two Different Speeds

Legends: Step count is expressed as average ± SD.

	Normal Walking Speed	Brisk Walking Speed
	n=30	n=30
Actual, steps	337.0±22.3	396.0±28.4
FAIT Tag, steps	322.4±31.4	356.4± 48.7
ActiGraph, steps	347.6±69.9	403.6±46.7

FAITValidationBMCSupplFig0526.pptx
Supplementary Figure 1. Representative waveforms of acceleration norm (a-c) and acceleration in each axis (d-f) recorded by Shimmer during walking. In Subjects 4 (b) and 6 (c), constant, periodic peaks of acceleration norm are evident (arrows). In contrast, in Subject 3 (a), the waveforms are small (whited arrow heads) or split (black arrow heads). In terms of the acceleration waveform in each axis, waveforms in the Y axis in Subject 3 (d) are markedly different from those in Subjects 4 (e) and 6 (f). (g) The X-axis of Shimmer corresponds to the radial-ulnar axis, the Y-axis to the long axis of the radius and ulna, and the Z-axis to the palmar-dorsal axis.

Download PDF

Editorial decision: Reject after review
12 Sep, 2021
Review #2 received at journal
07 Sep, 2021
Review #1 received at journal
05 Sep, 2021
Review #3 received at journal
05 Sep, 2021
Reviewer #3 agreed at journal
23 Aug, 2021
Reviewer #2 agreed at journal
22 Aug, 2021
Reviewers invited by journal
19 Aug, 2021
Reviewer #1 agreed at journal
19 Aug, 2021
Editor assigned by journal
13 Jul, 2021
Submission checks completed at journal
13 Jul, 2021
Editor invited by journal
13 Jul, 2021
First submitted to journal
21 Jun, 2021

You are reading this latest preprint version

Precision and Accuracy of a Simple Consumer-oriented Wristband Activity Tracker During Walking at Different Speeds

Status:

Version 1

Abstract

Figures

Introduction

Methods

Results

Discussion

Conclusions

Limitations

Abbreviations

Declarations

References

Tables

Supplementary Files

Status:

Version 1