Polymeric Foam Pressure-sensing Pens for Measuring Written Language Production


 To understand human cognition, cognitive and behavioral scientists measure external behavior using a variety of tools. However, many of these tools are not sensitive enough to detect small changes in behavior, they are too costly, or they can only be used in dedicated lab space, thus limiting behavioral science from studying many populations. Here, we present a reliable, robust, cost-effective device that can measure small modulations in human handwriting behavior through pressure sensing on the writing instrument itself. This is made possible through a cross-disciplinary approach, combining advantages of new, high-sensitivity pressure sensors and experimental psycholinguistics. We show that this instrument is reliable and sensitive to the typical pressure range in writing. Then, we present a proof of concept from an experimental replication and demonstrate the utility of handwriting pressure measurement in a classic experimental paradigm, thus opening new research directions in psycholinguistics, cognitive science, and psychology.


Introduction
In order to understand mental processes, most methods in the cognitive and behavioral sciences measure relative changes in people's reactions to different categories of stimuli. Psycholinguists build models of how words are stored and organized in a human mind, and evaluate those models by measuring external human behavior, such as reaction or reading times, 1-4 brain responses (e.g., using ERP or fMRI [5][6][7][8], or physiological effects originating in the autonomic nervous system like pupil size or skin conductivity (e.g., using pupillometry or skin conductivity [9][10][11][12][13][14]. Each of the existing methods has its advantages, but also limitations, be it cost, lack of portability, or poor temporal and spatial resolution (see Table 1). 13,15,16 The development of new research instruments promises to help researchers circumvent many of these limitations. For instance, language production research mostly studies spoken language, even though the cognitive processes involved in language production are known to be quite different between writing and speaking, 17,18 because one difficulty in studying writing is the current lack of reliable, affordable research instruments. Finally, the current COVID-19 crisis prohibiting in-lab experimentation 19 highlights the particular need for a simple, robust, and conveniently deployable instrument that can be used to obtain highly sensitive behavioral data outside the lab.  Empirical relationship between recorded capacitance and pressure, D) Electronics card for data acquisition, prototype with rubber grip, and prototype with PVC grip.
Here we present a novel research instrument to study variations in mental effort by recording muscular pressure exerted laterally onto a pen (Fig. 1). Intuitively, grip onto a pen varies depending on the difficulty of a writing task; and indeed, there is substantial evidence indicating that variations in mental effort -such as when multitasking or performing a difficult or stressful cognitive task -leads to changes in muscle tension and applied force, including during handwriting. [20][21][22] Tools for measuring lateral ('barrel' 23 ) pressure on a pen have been developed before, but existing approaches have different crucial shortcomings: Their measurement may be binary, lacking the ability to measure differences of changes in pressure; 24 the system may rely on pressure sensors that need to be replaced often and are marked by high variability, making comparisons across participants harder; 25,26 or the setup may require a pre-determined grip that may not be commensurate with a natural way of holding a pen, potentially leading writers to use an unnaturally strong or weak grip while writing. 27 . In addition, many electronic pressure sensors require a large, rigid instrument.
To address concerns with existing tools, we created a pressure-sensing pen that takes advantage of state-of the-art soft materials and insights from soft matter robotics 28,29 allowing for a structure that is light, durable, and adapts naturally to a writer's grip. The last decade has seen the development of many different types of flexible pressure sensors. [30][31][32] Here, we have chosen to use soft capacitive sensors based on carbon polymer composites, which have a very good sensitivity to low pressure. 33 We built a novel instrument by affixing a pen with a pressure-sensitive dieletric material: a polymer foam whose pores are covered by carbon particles, and whose dielectric constant varies with pressure ( Fig. 1b and d). For each pen, we obtain a calibration curve specific to the individual instrument that allows conversion to absolute pressure units (pascal). The calibration is repeated for each new pen to guarantee precise measurements, without the frequent re-calibration and renewal of sensors required by existing force-sensitive setups. 25,26 When pressure is applied to the sensor by the grip forces holding the pen, the capacitance of the foam capacitor varies, allowing us to measure the absolute barrel pressure applied in writing by using the unique calibration curve converting capacitance (see Supplemental Material and Fig. 1c). By adjusting the formulation, 33 we can maximize sensitivity in a pressure range typical for handwriting.
In the following, we show how this system allows us to record and establish the typical range of pressure during handwriting with sensitivity in the pascal range. In addition, we introduce a novel instrument to the psycholinguistics toolkit: By integrating our pens into a classic experimental task, we not only extract traditional reaction time measures directly from a signal, but we also find differences in barrel pressure depending on the category of stimulus the participants heard. This constitutes a proof of concept for the use of barrel pressure as a useful research measurement.

Pressure ranges in handgrip
We recorded and analysed the barrel pressure exerted by 16 participants using either of two pens while copying two short stories onto a normal letter-sized paper (for examples of two randomly selected 20s recordings, see Fig. 2a and b). We found that either of the pens, PVC or rubber tip, was equally sensitive to the variability in pressure during handwriting (see Supplemental Material for analysis details). The average variability was 17.5 kPA (SD: 7.5 kPA). The lack of differences in pressure exerted on the PVC versus rubber pen grips and the similar distribution of pressure exerted on both pens (Fig. 2c) indicates that studies using pressure-sensitive pens, built with our design, can be conducted across multiple sites, and with different pens, with stable In a and b, 20s snippets of sample pressure signals during writing extracted from recordings taken from two participants (purple, one of eight participants using the PVC grip; green, one of ten participants using the rubber grip). In c, a normalized histogram of pressure values exerted by participants using the PVC or rubber grip pens is calculated using kernel density estimation. In d, the range of pressures exerted by individual participants is represented (purple and green correspond to the examples shown in a and b). Center lines represent medians; inner hinges represent first and third quartiles; whiskers represent most extreme point within 1.5 × the inter-quartile range; ticks represent the maximum pressure value.
precision across individual instruments. Furthermore, the empirical distribution of pressure shown in Fig. 2c confirms the adequacy of our pressure sensor configuration, which targets variations in pressure ranging up to 150 kPa.
Thus, our instrument is sensitive and reliable, and an adequate measurement tool. However, while the median pressure across participants is around 50 kPa, we predictably 34 observe strong variability across individual participants in how much pressure they apply (Fig. 2d). Adjustments to the polymer foam ensured our device allows for sensitive measurement of a significant portion of the range of pressure exerted by any specific individual, and the polymer foam can be further adjusted to account for different pressure ranges if found necessary in future research with, for instance, clinical populations. 33

Experiments A and B: Benchmarking against known effects
Having established the validity of our tool, and established typical adult handgrip pressure, we now benchmark the use of our pens against known, reliable effects in psycholinguistics, using the lexical decision task, one of the most widely used paradigms in this field. This task has been used for decades to understand how people store words in memory, how they detect words, and how they distinguish them from possible words that don't exist. In this paradigm, participants are presented with strings of sounds or letters, and they decide whether the string represents a word (for instance, the real word motor) in the testing language, or not (for instance, the non-word fifful). Responses to lexical decision tasks are typically conducted using button-presses. One can draw inferences about the organization of the internal lexicon by measuring the reaction time it takes participants to make their decision depending on manipulated parameters within and across the stimuli, for instance frequency 35 , phonological neighborhood density 36 , or the transitional probabilities between sounds both in the non-words and words 37 .
For our purposes, we rely on a widely replicated effect as a benchmark: It has consistently been shown that frequentlyoccurring (EASY) words like motor are faster to recognize than either infrequent (HARD) words, such as spoilsport, or non-words, such as the EASY-to-categorize fifful, and the HARD-to-categorize steeree. 36,38,39

Reaction times
We compare data collected in Experiment A and B with previously collected reaction time results from the Massive Auditory Lexical Decision Database (MALD; Fig. 3a) on 48 two-syllable words and nonwords (for full list of stimuli, see Supplementary  Table S1). 38 We automatically extracted reaction times directly from large changes in the pressure signal following presentation of each stimulus, and then compared average reaction times for each stimulus type to the benchmark data reported in MALD (Fig. 3). In Experiment A (Fig. 3b), eight participants used the PVC-grip pen and wrote their answer ('yes' for word, 'no' for non-word); in Experiment B (Fig. 3c), ten participants used the rubber-grip pen and filled out a scantron-style optical mark recognition (OMR) multiple-choice answer sheet with their responses. The pattern of reaction times from the MALD data was quantitatively replicated in both these experiments.  38 for the stimuli used in our experimental task. In b, reaction times from the handwritten yes/no lexical decision task (completed using the PVC pen grip). In c, reaction times in the OMR bubble-filling task (completed using the rubber pen grip). Number in bar represents the average mean reaction time in seconds. All error bars are standard error.
In Experiment A, we found an interaction between WORDS vs. NON-WORDS, and EASY vs. HARD trials, with greater differences in reaction times between EASY and HARD trials in WORD trials compared to the NON-WORD trials. Reaction times were faster in EASY relative to HARD trials. While reaction times were numerically faster in WORD trials relative to NON-WORD trials, the difference was not statistically significant. For Experiment B, the pattern was similar, except that the interaction between WORDS vs. NON-WORDS, and EASY vs. HARD trials did not reach significance (for analyses, see Supplemental Materials). When compared to results reported in MALD, reaction times in the pressure pen task were longer, and there was less variability in reaction times across the four conditions. Exploration of these differences will be left to future studies, but we hypothesize that increased latencies are largely due to the time constraints of motor planning required to perform the more complicated actions in handwriting versus simple button pressing. Importantly, in both Experiment A and B, we replicated the pattern of results recorded using button-press times. 38

Change in handgrip pressure
The above analysis replicated patterns of reaction times using our novel device, establishing our instrument as a method comparable with current technology in the field. While such a replication is promising, the use of a pen which collects continuous barrel pressure allows for more subtle investigation of sensorimotor control -and its potential relationship with cognitive factors -during handwriting tasks. Here, we compare the change in pressure exerted on the pen across the different task conditions over time. Fig. 4 shows the average change in pressure (P d ) across participants for each stimulus condition. We find that EASY WORDS (dotted blue lines) elicit not only fastest reaction times (see above), but also the highest average pressure during the lexical decision phase, both in Experiment A and in Experiment B. This condition presumably requires the least cognitive resources. These data thereby present difficulties for models which predict increased exertion during cognitively stressful activities. 21,40 These data thus provide a valuable empirical test of theories of the relationship between mental effort and physiological effort, and future studies will contribute to deciphering the mechanistic link underlying this relationship.

Discussion
We have presented a novel device that can measure modulations in human handwriting behavior through pressure sensing on the writing instrument itself. We first established that typical handwriting pressures range between 10 kPa and 150 kPa, with similar ranges observed in recordings obtained using two different styles of pen grip. These pressure ranges fall within the range of high sensitivity afforded by our pressure pen design.
Second, we provided proof of concept, replicating a task often used in psycholinguistics. Consistent with previous results, we find that people are faster to recognize highly frequent words in an auditory lexical decision task, compared to low-frequency words, or made up words. In addition to replicating reaction time measures, however, we also added to this literature a new way of measuring mental effort: highly sensitive pressure data, which show that pressure onto the pen is higher following a highly . Event-related changes in pressure (P d ) during lexical decision tasks. In a and b, P d for each condition averaged across participants in the handwritten Y/N task (completed using prototype I) and in the OMR bubble task (completed using prototype II), respectively. The timescale is locked to the onset of the stimulus. In c and d, the average mean P d in the lexical decision period across all participants for each of the four conditions. Error bars are standard error. frequent, easy word than a word that is hard, or a made up nonsense word. We also provided evidence that these findings are generalisable from one mode of answer (writing yes or no) to another (filling in an optical mark recognition multiple-choice answer sheet). This is useful because many of the phenomena studied in the cognitive and behavioral sciences are either not amenable to reaction time measurements, reaction times alone do not provide information about time course of effort, or the differences between experimental manipulations are too subtle to be registered by analyzing reaction times. [41][42][43] Here, we provide proof of concept that not only can we replicate previously found reaction time data, but our pen can provide additional information to understand human behavior.
The main limitation of our device stems from the novelty of the measurement itself. While there is evidence that variations in mental effort affects muscle tension during handwriting [20][21][22] , the directionality of the pressure changes and the mechanisms behind them, are insufficiently studied: Does higher pressure indicate easier processing, higher confidence, or increased mental effort? One reason for this lack of research is that so far, measuring small changes in grip pressure could not be established as a valid method, due to lack of adequate instrumentation. However, now that the validity and sensitivity of our device is established, the road to research applications is wide open. Our pens enable research not only on understanding the effect of mental effort onto muscle tension, but we can also start to understand, for instance, the time-course of fine motor planning in writing, how people learn to write in a second language or second writing system, or how literacy affects cognition.
The low cost of our system (<$400), its durability, and its ability to record in absolute units, enables researchers to reliably collect data from large population samples, which is typically required in order to reach adequate statistical power in psycholinguistic studies. 44,45 . Research will also be comparable between laboratories and its portability enables research in remote locations, allowing for investigations with speakers of lesser-studied languages, who may live far from laboratory settings. Furthermore, the applicability of this device is not restricted to research: such a tool may prove useful in rehabilitation medicine, for example in post-stroke rehabilitation, or in therapeutic treatments of relevant focal dystonias such as writer's cramp 46 .

Hardware design and fine-tuning to typical grip
The setup involves three parts: the pressure-sensitive pen itself, the acquisition card, and a computer. The pen converts a pressure signal, produced by a finger on the grip, into a capacitive electrical signal, which the acquisition card collects digitizes to be sent, stored, and converted into pressure by the computer on the basis of a previously established calibration curve.
The pressure-sensitive pen consists of three components: the pen itself, which is commercially available (here, we used a standard Bic brand pen) for writing, a grip area for holding the pen and positioning the fingers, and a custom-made pressure sensor incorporated between the grip and the pen. Two pens have been developed (Fig. 1), A); the same sensor technology is used in both, with the only differences being in the construction of the pen grip, i.e. the contact area between the finger and the pen. The challenge here is to successfully position the grip while maintaining the pressure sensitivity of the sensor.

Sensor design.
The pressure sensor itself is based on a state-of-the-art pressure sensitive material 33 . Between two flexible electrodes, we place a dielectric material: a polymer foam, whose pores are covered by carbon particles, and whose dielectric constant varies with pressure ( Fig. 1), B). When pressure is applied to the sensor by the grip forces holding the pen, the capacitance of the foam capacitor varies systematically and reliably. This constitutes our dependent variable.
We used the same sensor material (Polydimethylsiloxane-PDMS, RTV615 from Momentive-with 10 wt% of carbon black particles from Alfa Aesar) and the same copper electrodes (from Radiospare) for both prototypes. Our data show that pressures on the order of 10-150 kPa are applied in a conventional way when writing (additional figure see Supplemental Materials). We adapted the material to this range by manipulating foam density, pore size, and Young's modulus of the polymer foam, to achieve extraordinarily high sensitivity.

Grip design.
The two prototypes differ in the material and shape of the grip associated with them. Prototype I has a grip made of cylindrical PVC where the pressure sensor is directly flush with the cylinder. When a finger is placed on the sensor, there is a noticeable difference in mechanical resistance between the PVC, which is very slightly deformable, and the highly elastic PDMS. Prototype II, on the other hand, is based on a commercial soft rubber grip (The Pencil Grip, TPG-11106) with a circular opening to accommodate the pressure sensor. This grip is usually used to help children and adults maintain a normative grip style while holding a pencil. The mechanical properties between rubber and PDMS are very similar, making the tactile distinction between the sensor and the grip much more subtle than in prototype I. In addition, the shape of the rubber grip allows a more stable position of the fingers, and may prevent fingers from slipping off the sensor. In both prototypes, the grip is fixed after placing the sensor on the pen. To do this, it is slipped over the sensor and then liquid PDMS is inserted between the grip and the pen. The cross-linking of the PDMS when hot (1h, 60 • C) allows total maintenance of the structure.

Electronic design.
The electronic board (designed by ESPCI Paris) allows for analog measurement of the electrical capacitance signal (between 0 and 1 nF) and its conversion into digital values. The card is connected directly to the pen via the coaxial cable, whose end is connected to the top and bottom electrodes of the pressure sensor. The electronic card/pen connection has been optimized to make the system "plug and play", enabling a maximum data acquisition frequency of 1000 Hz. Its power supply and data transfer are ensured by a serial link via the micro USB port.

Data acquisition for typical handgrip pressure ranges
We recorded and analysed the barrel pressure exerted by participants using either prototype I (N = 8) or prototype II (N = 8) while they copied two stories, matched in length (101 words) and word frequency (see Supplemental Material for the stories). The study was approved by the University of California, San Diego Institutional Review Board, and written informed consent was obtained from each participant prior to testing. All research was performed in accordance with the relevant guidelines and regulations. Participants were instructed to orient the pen during writing so that the thumb was located directly above the pressure sensor -participants were otherwise allowed to use whatever handgrip felt most comfortable and natural. Capacitance was recorded for one second prior to participants picking up the pen to write for each short story. The median capacitance value in this period (c 0 ) is used to convert raw capacitance values to pressure for each recording, taking into account the possibility that the structure of the sensing material and environmental conditions (and therefore the capacitance recorded with ambient room pressure) may vary slightly across time. In the case of one participant, the c 0 recording was only recovered for one story. In that case, the c 0 value recorded prior to the second story copied by the participant was used for calculating pressure values in both short stories.
All data was downsampled to 100Hz prior to analysis. The capacitance signal was occasionally lost, and recorded as 0pf, due to data acquisition issues. This occurred in recordings from two participants, with 19 artifactual samples (<1% of data) in one recording and 3071 (8% of data) samples in the second.
In order to exclude outliers in the pressure times series and to analyze only periods of time when the participant was gripping the pen, the following protocol was followed: A rolling median pressure value was calculated with a window extended 2.5s before and after each point. A non-causal averaging window was selected to focus outlier detection on short-term spikes in the pressure signal, rather than on stable increments or decrements to pressure that may be caused by occasional adjustment of the handgrip or the intensity of tonic handgrip pressure. This window allows for the identification local outliers for rejection, while allowing for smooth increases or decreases in pressure over time. The window length was selected to allow for at least one full word writing cycle to be complete for typical participants. 47,48 A median average deviation from median pressure (MAD) was calculated for each point using the same window. Any point which occurred outside a confidence interval of 2.5 MADs was selected for rejection. 49,50 To include only periods when participants are gripping the pen and writing, we also excluded points where pressure dipped below 5 kPa. We recorded a mean of 35,626 samples of gripping and writing per participant. Of these, approximately 3% of samples were rejected as outliers in both pens, with a range in rejections across participants with a minimum participant rejection rate of less than one percent and a maximum of 7%.

Experiments A & B
For the lexical decision tasks, we selected the easiest and the hardest two-syllable words, as well as the easiest and the hardest non-words from a large database of previous lexical decision task findings 38 . In Experiment A, using the PVC pen, eight participants responded to auditory stimuli by writing 'yes' or 'no' on an answer sheet to make the lexical decision, and then spell what they heard, while handgrip pressure was recorded. In Experiment B, ten participants using the rubber-grip pen did the same, but they used an optical mark recognition multiple-choice answer sheet (OMR sheet). This custom designed OMR sheet includes response fill-in options for 'yes' and 'no' and spaces to spell each word or non-word (for an illustration of both paradigms, see Supplementary Fig. S4). The study was approved by the University of California, San Diego Institutional Review Board, and written informed consent was obtained from each participant prior to testing. All research was performed in accordance with the relevant guidelines and regulations.
We recorded one second of ambient-pressure capacitance (c 0 ) immediately prior to each participant picking up the pen to begin the study. Participants were then presented stimuli and prompts using over-the-ear headphones. To ensure that participants attended to stimuli, a 750ms pure tone was played one second prior to stimulus presentation. Participants responded with their lexical decision immediately after the stimulus was presented. A second, lower-pitched tone was played after 4s (in Experiment B, 6s) to direct participants to attempt to spell the stimulus. Recording continued for 8s after the onset of spelling. 48 trials were presented in randomized order.
All data were downsampled to 100Hz prior to analysis. To characterize changes in barrel pressure associated with stimuli, all analyses were conducted using change over baseline pressure (P d ) as the dependent variable. Baseline pressure was calculated by taking the mean pressure exerted during the 1000ms of silence between the preparation tone and the onset of stimulus presentation ('BL' in Fig. 4). Mean P d was calculated both during lexical decision response and during spelling. Occasionally, participants failed to begin writing during the lexical decision period, or begin writing before a stimulus was presented. To discard these incidences, we rejected all trials in which the mean lexical decision period handgrip pressure was not greater than the baseline (a total of 33 or 9% of trials across all participants in Exp. A, 33 or 7% in Exp. B). To avoid extreme outliers in pressure, we also rejected individual samples with P d values more than 100 kPa greater or lesser than the baseline.
Response reaction time was automatically detected for each trial, within the lexical decision period. The reaction time was detected as the first instance where P d remained above 5kPa for at least 50ms. If this criterion was not met within the lexical decision period, the trial was marked for rejection from the reaction time analysis. In Experiment A, 2% of trials were rejected on these grounds; in Experiment B, 3%.

Data Availability
The datasets generated during and/or analysed during the current study are available in the Open Science Framework repository "PASCAL Pen", https://osf.io/aexkv/, in de-identified format and in accordance with UC San Diego's Human Subjects Research Protection Program's guidelines. Figure 1 Schematic views of the two pressure sensitive pens. A) Overview. B) Focus on the pressure sensor design. C) Empirical relationship between recorded capacitance and pressure, D) Electronics card for data acquisition, prototype with rubber grip, and prototype with PVC grip. In a and b, 20s snippets of sample pressure signals during writing extracted from recordings taken from two participants (purple, one of eight participants using the PVC grip; green, one of ten participants using the rubber grip). In c, a normalized histogram of pressure values exerted by participants using the PVC or rubber grip pens is calculated using kernel density estimation. In d, the range of pressures exerted by individual participants is represented (purple and green correspond to the examples shown in a and b).

Figures
Center lines represent medians; inner hinges represent rst and third quartiles; whiskers represent most extreme point within 1.5 the inter-quartile range; ticks represent the maximum pressure value.

Figure 3
Average of mean reaction times in lexical tasks for the four conditions across participants. a: The mean reaction time in lexical decision tasks using button-press responses as reported in the MALD Database38 for the stimuli used in our experimental task. In b, reaction times from the handwritten yes/no lexical decision task (completed using the PVC pen grip). In c, reaction times in the OMR bubble-lling task (completed using the rubber pen grip). Number in bar represents the average mean reaction time in seconds. All error bars are standard error.

Figure 4
Event-related changes in pressure (Pd) during lexical decision tasks. In a and b, Pd for each condition averaged across participants in the handwritten Y/N task (completed using prototype I) and in the OMR bubble task (completed using prototype II), respectively. The timescale is locked to the onset of the stimulus. In c and d, the average mean Pd in the lexical decision period across all participants for each of the four conditions. Error bars are standard error.

Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download. PASCALSupplement.pdf