A cutaneous receptors-mimicking system for real-time and multimodal detection of tactile stimuli

doi:10.21203/rs.3.rs-1420019/v1

Download PDF

Article

A cutaneous receptors-mimicking system for real-time and multimodal detection of tactile stimuli

https://doi.org/10.21203/rs.3.rs-1420019/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

A human can intuitively perceive and comprehend complicated tactile information, when interacting with objects, owing to the different cutaneous receptors distributed in the fingertip skin. Many research groups have attempted to mimic the structure and receptors of the skin to develop next-generation tactile sensors that can precisely and seamlessly deliver the overall tactile sensation. In this study, we propose a real-time multimodal tactile system that mimics the sensing qualities of cutaneous receptors entirely by simultaneously acquiring four types of decoupled tactile information in real time using multiple sensors integrated into three dimensions (3D), a signal-processing module, and a transmission module. The interconnections between 3D-integrated sensors and the signal-processing module were manufactured by 3D printing methods to have an adaptable shape. Furthermore, the proposed system can differentiate between various tactile stimuli, texture characteristics, and consecutive complex motions depending on the decoupled tactile sensing signals of pressure, shear force, vibration, and temperature. We believe that the results of this study can provide a novel design for a skin-like, perceivable, tactile sensing system for application in soft robotics, human-machine interfaces, health monitoring systems, and biomedical devices.

Human skin is considered the most efficient and powerful tactile sensor^1,2, and hence many studies have been conducted to develop flexible electronic skin that emulate the properties of human skin and provide various tactile information, such as force feedback and texture classification, which would allow a robot to perform precise and delicate motions^3–9. Advances in skin-mimicking tactile sensing technology could provide diverse human–machine interfaces in virtual/augmented reality and the metaverse^10,11.

However, to fully mimic the sensing capability of human skin, it is crucial to develop a sensing system that can detect different physical quantities while simultaneously considering tactile information in real time, in a manner similar to the cutaneous receptors located under the human skin¹. Cutaneous receptors comprise thermoreceptors that respond to temperature changes and mechanoreceptors that detect normal force, shear force, and vibration in real time, and the detect multiple tactile signals are then transmitted to the brain, allowing humans to perceive and comprehend sophisticated tactile information².

Early research on electronic skin mainly focused on improving the flexibility and sensitivity of these tactile sensing devices that could detect a single physical quantity, such as normal force^12–18. Although a few research groups have developed multifunctional electronic skins in recent years, these devices, unlike cutaneous receptors, detected chemical and physical qualitative information such as humidity, ultraviolet light, and magnetic field^19–21. While some electronic skins exhibited sensing characteristics of multiple tactile forces such as normal force, shear force, bending, and torsion, they require a subjective classification for each force because the forces could be distinguished only by the shape of the signal waveform^22–25. Furthermore, multiple signals were acquired from a single device; however, they were mixed and overlapped, making it difficult to distinguish the applied multiple stimuli simultaneously and independently.

However, these issues can be resolved by assembling different sensing devices and reading the signals from each device separately, similar to the multiple cutaneous receptors present in the human skin^26–35. Most of the sensors reported in literature have detected two of the four tactile stimuli: normal force, shear force, dynamic force, and temperature. Many studies have proposed integrated sensors comprising two devices that can detect static and dynamic stimuli^26–30. These studies mimicked mechanoreceptors by classifying them into slow-adapting and fast-adapting receptors, but they were still not able to detect various complicated tactile stimuli. Furthermore, it is difficult to read signals from multiple sensors simultaneously, in real time, within one system, because each sensor signal is usually obtained individually using commercial measuring instruments. Therefore, a new design for a tactile sensing system is needed to detect multiple cutaneous qualities within a compact system, in real time, in order to recognize and classify complicated and diverse tactile information

In this study, we propose a cutaneous receptors-mimicking tactile (CRMT) system that can simultaneously detect four different types of tactile stimuli namely normal force, shear force, vibration, and temperature in real time. The CRMT system is composed of three parts: a multi-functional sensing unit, a signal processing module, and a tactile analysis module.

To detect multiple tactile stimuli at a single sensing node, the multifunctional sensing unit was developed by vertically stacking and integrating four types of sensors, all of which are flexible and thin. In the signal processing module, tailored electronic integrated circuits with different output characteristics were designed for each sensor to obtain real-time signals without any interference between the multiple tactile stimuli, without using expensive and bulky measureingt instruments. Finally, the tactile analysis module was developed using a programming language to enable real-time monitoring and signal analysis. Additionally, to improve the system usability in the real world, stretchable three-dimensional (3D) interconnections were fabricated between the 3D-integrated sensors and signal processing module using direct ink writing, which is a micro-extrusion 3D printing technology. The proposed CRMT system managed to successfully measure and distinguish various tactile stimuli, such as dynamic touch, deep pressure, shear force, and temperature variation in real time. Furthermore, the system acquired texture information, such as pattern pitch and roughness, and recognized eight consecutive complex motions. Therefore, we believe the proposed CRMT system can be potentially used in a wide range of applications, such as humanoid robot arms and human-machine interfaces.

Multimodal sensors mimicking cutaneous receptors

Figure 1A shows the schematic of human and artificial tactile systems. In the human tactile system, sensory receptors in the skin detect external stimuli and transduce them into neural impulse signals, which are then transmitted to the brain through sensory nerves and processed for humans to perceive and comprehend tactile information. Similarly, the artificial tactile system (the CRMT system) detects external tactile stimuli and transduces them into multiple electrical signals, which are then transmitted to a tactile analysis module through a signal processing module without signal interference in real time, and processed to determine the type of tactile stimuli. Therefore, various tactile information such as texture perception, grasp control, and slip detection can be obtained from the tactile system, as illustrated in Fig. 1B.

As shown in Fig. 2A, the cutaneous receptors comprise thermoreceptors and four mechanoreceptors, which include the Meissner, Pacinian, Ruffini, and Merkel receptors. The thermoreceptor measures the environmental temperature. The Meissner and Pacinian receptors, known as rapidly adapting receptors, respond to dynamic forces and vibrations, and thus play essential roles in texture discrimination and slip detection. The Merkel and Ruffini receptors, also known as slowly adapting receptors, detect sustained normal and shear forces, respectively, and continue to respond as long as the stimulus is applied.

Similarly, the multimodal sensing unit in the proposed CRMT system comprises four types of tactile sensors, presenting the different sensing characteristics of cutaneous receptors. Each sensor was designed to be flexible and thin, as shown in Fig. S1. The four sensors were stacked vertically, which allowed the simultaneous detection of multiple stimuli obtained from a single sensing node.

A pressure sensor, wherein the resistance changes as static normal force is applied, was positioned at the bottom-most layer of the multimodal sensing unit to serve as the Meissner and Merkel corpuscles. Piezoresistive films, with microdome patterns was selected as the pressure sensor. The piezoresistive films were composed of polyvinylidene fluoride (PVDF) and reduced graphene oxide (rGO). Owing to the conductive property of rGO, the piezoresistive property of the PVDF/rGO composite reduces the electrical resistance of the film when external pressure is applied to the sensor. The detailed mechanism is reported in previous study³⁶. The sensor formation was optimized to embeded in CRMT system by tuninning the electrode deposition and the gap between the films.

The second layer from the bottom is a new type shear force sensor, which comprises a permanent magnet embedded in a thin elastomeric film and a magneto-resistive (MR) sensing element in a hybrid Wheatstone bridge, mimics the Ruffini corpuscle that detects tangential force. When an external shear force is applied and the 3D integrated structure becomes flexibly deformed, the relative distance between the MR sensing element and the magnet changes, resulting in variations in the output signal from the sensor. Compared to existing shear force sensors, the proposed shear force sensor can be fabricated as a very thin and flexible film. Furthermore, the sensor can detect the magnitude and direction of the applied shear force using a pair of electrical output lines. The development of the compact 3D integrated tactile sensing units was made possible owing to these unique features.

Additionally, a vibration sensor was placed above the shear force sensor to effectively sense dynamic tactile stimuli, similar to Meissner and Pacinian receptors. A thin film of a single crystal of piezoelectric material was used as the sensing material for the vibration sensor owing to its excellent electromechanical properties. Furthermore, to improve flexibility, the piezoelectric thin film was diced into fibers, and polymer epoxy was filled in between the fibers.

A new temperature sensor made using a composition of rGO/PVDF/polyethylene oxide (PEO) in a fingerprint pattern was placed on the uppermost layer of the sensors for direct contact with an object by eliminating the heat conduction loss. A thermistor material rGO, which has a thermo-resistive effect, was used as the sensing material for the temperature sensor owing to its possessing a high thermal coefficient and excellent stability, thereby resulting in high-resolution temperature detection. Especially, PEO enhances the sensitivity by transforming from a semi-crystalline to amorphous state, inducing volume expansion. The volume change in PEO destroys the conductive network in the composite material, helping to create a sharp increase in resistivity change. The fingerprint pattern also improves the thermal sensitivity of the temperature sensor by increasing the surface area and amplifying the vibrational input when scanning over a surface. The detailed fabrication process for each sensor is described in the Materials and Methods section.

3D integration of multimodal sensors

To develop a fully functional robust 3D integrated sensing unit, the four sensors must be assembled vertically and connected with the analog electronics of the signal processing module without undergoing mechanical and electrical failures caused by the elastic modulus between the sensors, adhesive elements, and electrical wires. In particular, because the tactile sensors are exposed to physical contacts, they are susceptible to interconnection failures. Moreover, all four sensors are comprised of two or four vertically separated electrical pads. Therefore, a customized electrical interconnection system should be developed.

Figure 2B shows the flexible 3D electrical interconnection system for 3D tactile sensors developed using a flexible conducting polymer and 3D printing techniques. The system comprises an interconnection interface (ICI), interconnection lines (ICL), and a customized flexible printable circuit board (FPCB), as shown in Fig. S2A.

Silver-based elastomer composites were printed on a flexible and stretchable thin film using direct ink writing 3D printing to ensure mechanical and electrical stability of the ICI and ICL. The ICI and ICL designs can be easily modified depending on the geometry of the sensors and applications, as shown in Fig. S2B. The 3D-printed interconnection system enabled successful connection between the sensors and the electronics unit, and overcame the maximum height difference between the sensors in the multi-modal sensing unit, which was approximately 1.6 mm, as shown in Fig. S2C.

The 3D assembled design of the interconnection system realized a compact electrical connection of the multiple signal lines from the four sensors to a single analog signal processing module, as shown in Fig. 2B. Additionally, based on the flexibility and robustness of the developed interconnection system, the 3D integrated sensing unit can be mounted on a robotic finger, as shown in Fig. 2C. More details of the fabrication process of the sensor assembly and the 3D printed flexible interconnection system are provided in the Materials and Methods section.

Tactile signal processing and transmission module configuration

Given that all four sensors in the CRMT system exhibit different output characteristics, configuring a signal conditioning circuit suitable for each sensor by considering the sensing mechanism for real-time and simultaneous monitoring of tactile information is crucial. As shown in Fig. 3A, each output signal is adequately modified using the signal conditioning circuit and digitized by an analog-to-digital converter (ADC). The digital signals are then sent to the PC through serial communication.

It is advantageous to develop a customized signal conditioning circuit for practical applications considering the circuit enables simultaneous detection of multiple tactile signals without requiring bulky and expensive measurement equipment, such as an oscilloscope, an inductance/capacitance/resistance meter, or an impedance analyzer.

The electronic components used for signal conditioning circuits comprised operational amplifiers, an instrumentation amplifier, and discrete passive elements. Because the temperature and pressure sensors detect external stimuli through resistance change, the voltage divider circuit, which converts the value of resistance change into voltage, in accordance with Ohm’s law, was selected as the main part of the signal conditioning circuit, as shown in Fig. S3A. The fixed resistance value in the voltage divider was determined from the output resistance range of the sensor under external stimuli. Because the resistance value of the temperature sensor increases/decreases as the temperature increases/decreases, the fixed resistance value of the voltage divider was set to be similar to that of the temperature sensor at room temperature (~ 20°C). By contrast, as the resistance of the pressure sensor decreased monotonically when external pressure was applied, the fixed resistance value in the voltage divider was lower than that of the initial resistance of the sensor.

Under dynamic mechanical deformation, piezoelectric-based vibration sensors generate output voltages in pairs of positive and negative pulses. Considering that the microcontroller unit can only read positive values, a DC level shifter was introduced to convert the entire waveform into the positive range, as shown in Fig. S3B. Furthermore, because piezoelectric sensors usually exhibit high internal impedance, the most suitable impedance matching circuit was considered. Additionally, the conditioning circuits for temperature, pressure, and vibration sensors included a voltage buffer to avoid signal interference. The signal conditioning circuit for the shear force sensor comprised an offset controller, an instrumentation amplifier, and a current source, as shown in Fig. S3C. In contrast to other sensors, the shear force sensor was composed of only one electrode pair to measure voltage change, whereas the other electrode pair was used to supply a constant current, as shown in Fig. S3D.

A constant current was used as the operating source for the shear force sensor to convert the sensor output signal into voltage, so that all the output signals could be processed simultaneously in the signal processing module. However, the magnitude of variation in the output signal was too small for the ADC to convert the analog signal to digital because the overall shear sensor had a very thin form factor, which limits the relative movement of the permanent magnet and MR sensing unit. Therefore, we configured an instrumentation amplifier with low input noise and high accuracy to obtain a gain of 2100 to match the span of the ADC. Additionally, an offset controller comprising a variable resistor was constructed to eliminate the small difference in the initial signal to prevent saturation of the output signal.

Furthermore, low-pass filters were configured at the end of all the signal conditioning circuits to reduce the external electrical noise. Four output voltage signals from the signal conditioning circuits were digitalized using the ADC, which were then sent to the PC (data processing unit) through serial communication. Finally, a Python-based program was developed to process digital signals, which enabled real-time monitoring and analysis of various tactile information.

Sensing properties of the 3D-integrated CRMT system

Figures 3B–3G show the sensing characteristics of the proposed CRMT system. As shown in Fig. 3B, the amount of voltage change in the temperature sensor increases linearly as the temperature increases from 0 to 50°C. The actual sensing material has a negative temperature coefficient, but the sign of the value was reversed so that the temperature change through the tactile analysis module could be intuitively understood.

The amount of voltage change in the vibration sensor was measured when the applied force ranged from 0.1–10 N using load weights. The output voltage increased as the dynamic load increased and saturated in the high-force region, which is when a dynamic load is larger than approximately 6 N, as shown in Fig. 3C. Furthermore, the dynamic characteristics of the vibration sensor are demonstrated in the time domain. Light dynamic forces of 0.5 N and 1 N were each applied eight times to the sensor. As shown in Fig. 3D, the vibration sensor successfully detected consecutive gentle touches.

Figure 3E shows the measured output voltage of the shear sensor. The output voltage changed linearly as the applied shear force increased from 0.1 N to 7.0 N. Furthermore, the dynamic shear force was applied six times in each opposite direction to present the bidirectional sensing capability of the sensor as shown in Fig. 3F. This characteristic can be obtained owing to the specific design of the MR sensing element, which has a combination of quadrant circles.

Finally, as shown in Fig. 3G, the pressure sensor located at the bottom of the sensor unit responds to an external load of over 3.5 N. The change in voltage increased sharply under external loads ranging from 3.5 N to 6 N, owing to the drastic increase in the contact area between the two interlocking films with microdome structures. Under applied loads of over 6 N, the interlocking microstructures were now in full contact and the two flat films were squeezed, and hence the value increased slowly because the contact area changed slowly. The available pressure range and sensitivity can be further improved by controlling the elastic properties of the upper sensor layers.

Pattern pitch and texture discrimination

Usually, humans perceive textures by moving their fingers across surfaces. Similarly, the CRMT system differentiates between surface structures based on different periodic pitches (p) by scanning over surfaces. The 3D integrated sensing unit was attached to a flat plate mounted on a linear motorized stage, and a constant distance was maintained between the plate and the samples that had different surface structures, as shown in Fig. 4A. The stage moved in the x-axis direction at a fixed scanning speed (v) of 5 mm/s. Figure 4A shows the normalized output voltages obtained from the vibration sensor when scanning periodic grating structures with p values of 1, 1.5, and 2 mm. The output signals showed distinguishable periodicity that reflected the pattern form. When p was 1, 1.5, and 2 mm, the number of single waveforms in 1 s was approximately 5, 3, and 2.5, respectively. Figure 4B shows the power spectra graph for the obtained output signals, which are the squared frequency spectra transformed using fast Fourier transform (FFT). The characteristic peak frequency (f), considered as the frequency value with the largest magnitude in the power spectrum, is related to the sample pitch. As expected, the f values of the samples with p values of 1, 1.5, and 2 mm were 5, 3.33, and 2.5 Hz, respectively, which were equivalent to the values calculated using the equation:

$$f=\frac{v}{p}$$

In addition to the periodic structures, different materials with random roughness can also be discriminated by scanning the surfaces. To demonstrate this, we measured the voltage responses of the vibration sensor while scanning three laboratory supplies, namely a poly glove, latex glove, and disposable tissue. Contrary to the output signals of the periodic structures, whose pattern pitches were distinguished easily in the time-voltage graphs, the voltage signals of the different textures were difficult to distinguish, considering the signal amplitude was small and the waveform had an irregular shape, as shown in Fig. 4C. Figure 4D shows the FFT results of the voltage signals. The peak frequencies for the poly glove, disposable tissue, and nitrile glove were approximately 1, 7, and 12 Hz, respectively. The poly glove exhibited the largest average roughness, whereas the nitrile glove exhibited the smallest roughness, because smaller levels of average roughness in the samples leads to larger peak frequencies in the output signals at the same scanning speed. These results are consistent with the photographic and microscopic images of the three materials shown in Figs. S4A–S4C. The FFT spectrum of the disposable tissue was slightly broader considering it had a more irregular surface morphology than the other materials. The calculated results were consistent over repeated experiments, as shown in Figs. S4D–S4F.

Real-time monitoring for various tactile stimuli

To demonstrate the simultaneous monitoring capability of various tactile stimuli in real time, the variations in the output voltage of the sensor system were recorded under four different physical stimuli: normal force, tangential (shear) force, gentle touch, and hot temperature, as illustrated in Fig. 5A.

The characteristic responses for fast consecutive finger taps are shown in the first region of Fig. 5B, indicated by a flat blue line. Considering that the piezoelectric-based vibration sensor used in the CRMT system can detect rapid variations in dynamic forces, the output voltage of the vibration sensor responds appropriately to the dynamic variation, as shown in Fig. 5B. In addition, while the normal and shear force sensors did not respond under the light dynamic touch condition, the output voltage of the temperature sensor increased slightly considering the temperature of the skin of the finger was approximately 36°C, which was higher than the room temperature which was approximately 20°C.

The second region, shown in yellow in Fig. 5B, shows the electrical responses of the sensor system under three consecutive normal pressures applied to the device by a fingertip. The normal force sensor responded well to the static pressure. The vibration sensor responded to the dynamic stimuli and generated a bipolar voltage pulse during the press and release moments only. Furthermore, a small amount of electrical response was obtained from the shear force sensor because the deep pressure applied by the fingertip deformed the device owing to the flexibility of the 3D integrated sensors, resulting in a tangential force on the sensor. Similarly, the temperature sensor detected the finger temperature when dynamic forces were applied.

The curves in the gray region in the figure depict the electrical responses when the forces were applied repeatedly, three times, to the side of the CRMT sensing unit sensors. It was seen that only the shear force sensor responded drastically to the shear force of the stimuli.

To verify the ability of the sensors to detect temperature variations, a glass of hot water was placed closer to and away from the surface of the sensing unit three times. Considering no mechanical stimuli were applied to the device, the vibration, shear, and normal force sensors had no electrical response, whereas the temperature sensor a showed variation in voltage, as shown in the red region in Fig. 5B. Furthermore, the magnitude of the voltage gradually decreased over time, as heat from the glass was released into the atmosphere, causing the temperature of the glass to decrease.

Perception of complex tactile motion

To further demonstrate the capability of the CRMT system to classify and understand complicated tactile information, eight consecutive tactile event situations were performed using a gripper and a cuboid-shaped acrylic bottle, as graphically represented in Fig. 6A.

As the gripper began to grasp the bottle (t₁) firmly, a short electrical response for the dynamic motion occurred, and the electrical outputs from the normal and shear force sensors increased sharply. Here, the shear force arose from the gravitational force of the bottle. When hot water was poured into the bottle (t₂), the shear force increased steadily, and the temperature sensor responded to the temperature change at the contact area, as shown in Fig. 6B.

Next, the hot water was drained off from the valve opening (t₃) and cool water was poured into the bottle (t₄). As a result, the shear force decreased owing to weight loss and increased when the weight increased, whereas the temperature decreased. As more water was poured (t₅), the bottle began to slip, and the amplitude of the shear force decreased similar to that during drain off (t₃). Therefore, it would be impossible to distinguish between the two motions (t₃ and t₅) if the system comprised a single sensor. However, because the CRMT system comprises a vibration sensor that detects the dynamic motion that occurs when a slip occurs, it becomes possible to discriminate between two different complex situations. The last operation was to tighten the gripper to hold the falling bottle (t₆), pour more cold water (t₇), and finally, loosen the gripper to separate the gripper and bottle (t₈).

This study attempted to mimic the human skin form and the functions of the cutaneous receptors present within the skin. Although all the sensors were embedded within each encapsulation layer in the proposed CRMT system, they were designed and fabricated to be thin and flexible, approximately 1.5 mm, similar to the dermis layer of human skin. The spatial resolution of the sensors can be further improved by optimizing the sensor components.

Additionally, multiple contact stimuli were measured simultaneously in real time like human tactile perception. The detectable functions of the cutaneous receptors were further subdivided into four categories: normal force, shear force, vibration, and temperature. While most existing sensors can detect two types of tactile qualities such as vibration and pressure, the proposed system can distinguish all four tactile qualities, which is essential for understanding more sophisticated tactile information and feedback action behavior such as texture perception, slip detection, differentiation between hot and cool, and grasp control similar to human skin.

Moreover, the CRMT system does not require bulky measuring instruments and is appropriate for practical applications because a customized signal processing module was developed to tailor the output characteristics of each sensor. The signal processing module was used for the standalone measurement for the four sensors with only a 3.3 V voltage supply. In the future, an additional wireless communication module can be introduced to enhance the proposed CRMT system for applications in diverse mobile and wearable devices. We believe the proposed 3D integrated sensor system, inspired by the cutaneous receptors and human skin structures, provides a solution for multimodal tactile sensing and can improve the functionality of robots and prostheses.

Fabrication of the temperature sensor

The temperature sensor was developed using reduced graphene oxide (rGO)/polyvinylidene fluoride (PVDF)/polyethylene oxide (PEO). A 4-inch silicon wafer mold was fabricated using conventional photolithography and deep reactive ion etching to create a fingerprint-shaped pattern on the surface of the temperature sensor. The rGO/PVDF/PEO solution was poured onto a silicon wafer mold and cured at 160°C for 30 min.

Fabrication of the vibration sensor

A piezoelectric single-crystal plate was made of P_b(Mg_1/3Nb_2/3)O₃-Pb(Zr,Ti)O₃ (PMN-PZT) (Ceracomp Co. Ltd) and fabricated using the solid-state single-crystal growth method. The piezoelectric plate was diced to resemble macrofibers, and the gaps between the macrofibers were filled with an epoxy to improve flexibility. The sputtered Au electrode film on a 25 µm polyimide (PI) sheet was laminated onto the piezoelectric single crystal-based fiber composite film to form a flexible electrical contact. The Au-PI sheets were placed on both sides of the piezoelectric composite film with an active area of 6 × 6 mm². The piezoelectric film was poled under a 1 kVmm^− 1 electric field at 80°C for 30 min.

Fabrication of the shear sensor

The MR sensing element comprised thin layers of Ta/NiFe/Cu/NiFe/IrMn/Ta with a spin-valve structure and the thicknesses of the layers were 5, 10, 10, and 5 nm, respectively. The MR sensing element was deposited on a 25 µm polyimide film using DC magnetron sputtering and patterned through a conventional photolithography process with a ring shape of diameter 10 mm.

Fabrication of the pressure sensor

The PVDF/GO composite material was mixed with PVDF and GO, and dispersed in dimethylformamid (DMF) at 70°C for 2 h. The composite solution was then poured into a PDMS mold with a microdome structure, and heated at 160°C to eliminate residual DMF in the composite material and reduce GO to rGO. During this process, the composite exhibited sufficient electrical conductivity owing to rGO. The pressure sensor comprised two 100 µm PVDF/rGO composite films in parallel, whose inward side had microdome-shape patterns. A 200 nm thin gold layer was deposited as an electrical pad on the outward side of the PVDF/rGO films through a thermal evaporation. The gold-deposited PVDF/rGO composite films were sandwiched using patterned Kapton tape to form an interlocking structure.

3D printing of interconnections

The customized 3D printer, used to print the interconnections, comprised a pneumatic controller, three-dimensional manipulator, and syringe. The syringe was mounted on the manipulator, and a camera was fixed to the syringe holder to visually detect the height of the needle from the printing substrate. The pneumatic controller had a wide pressure range of 10–700 kPa. The interconnection interface was fabricated as a part of the bottom-most layer of the entire 3D integrated tactile sensor. Furthermore, the degassed PDMS of a base prepolymer and crosslinking agent (10:1 weight ratio) was spin-coated at 250 rpm for 60 s on a silicon wafer, and 180 µm PDMS was cured in an oven at 100°C for 15 min. The silver paste was printed on the PDMS film using a dispenser, and ten interconnection lines 0.3 mm wide, including twenty 1(W) × 2(L) mm² contact pads, were printed for the entire sensor assembly. To fabricate the interconnection lines, the degassed PDMS (Sylgard 184, Dow Corning Co.) of a base prepolymer and crosslinking agent (10:1 weight ratio) was spin-coated at 1,000 rpm for 60 s on a silicon wafer, and 50 µm thick PDMS was cured in an oven at 100°C for 10 min. Two 0.3 mm wide silver paste line patterns were printed on the substrate using a dispenser.

Assembly of the sensors

In the first step of the assembly, the 3D integrated tactile sensor was aligned and the pressure sensor was placed on the ICI. The top and bottom gold layers of the pressure sensor were connected to contact pads using a silver paste and the entire pressure sensor was covered with a 100 µm thick PDMS layer using a film applicator and cured at 100°C for 15 min. The MR sensing element of the shear force sensor was placed on the 100 µm thick PDMS cover layer. A 250 µm thick Ecoflex 0030 layer was applied to the MR sensing element using a film applicator and cured in an oven at 100°C for 10 min. A cylindrical permanent magnet of diameter 2 mm and thickness 0.5 mm was aligned over the center of the ring shape of the MR sensing element. The permanent magnet was fixed with a 100 µm thick PDMS layer. The electrical pads of the MR sensing element were connected directly to the contact pad of the ICI using the silver paste. The ICL for the vibration sensor was attached to the edge of the vibration sensor using a drop of liquid PDMS as an adhesive, whereas the electrical pad of the vibration sensor was connected to the ICL using the silver paste. Then, the prepared vibration sensor was placed over the permanent magnet layer, covered with a 50 µm thick PDMS layer, and cured. The ICL for the temperature sensor was placed under the temperature sensor and connected, mechanically and electrically, in a manner similar to the vibration sensor. The ICL-connected temperature sensor was then attached to the vibration sensor layer using PDMS solution. The ICLs for the vibration and temperature sensors were connected successfully to the contact pads of the ICI owing to its good flexibility; however, a maximum gap of 1.6 mm still existed between the temperature sensor and the bottom-most layer.

Data acquisition and processing

In the signal processing module, the transformed signals passing through the signal conditioning circuits were sent to the Arduino microcontroller (Arduino due or nano). The digitalized data were then transmitted to a PC using the serial port with a baud rate of 19200 and received using software written in Python, one of the most widely used programming languages, for further processing and plotting.

Measurements

The customized tactile tester comprised a load cell, x-, y-, and z-stages, and a controller computer. A load cell with a square-shaped 10 mm² tip and a load capacity of 500 g was mounted on the z-stage, and pressure was applied at a speed of 0.1 mm/min. The customized thermoelectric element controller (Peltier Technology Co.) changed the temperature from 0 to 50 ºC from − 4 to 6 V. The minimum measurement resolution of the temperature change was set to 1°C.

Acknowledgments:

Funding:

This research was supported by the Convergence Technology Development Program for Bionic Arm through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (No. 2014M3C1B2048177) and Korea Institute of Machinery and Materials (Basic Research Program, No. NK236C), Republic of Korea.

Author contributions:

Conceptualization: BYL, SK, YJ, HL

Methodology: BYL, SK

Investigation: BYL, SK, SO, YL, JP, HK, JCK

Visualization: BYL

Funding acquisition: HL

Project administration: HL

Supervision: YJ, HL

Writing – original draft: BYL

Writing – review & editing: SK, YJ, HL

All authors have read and agreed to the published version of the manuscript.

Competing Interests:

No potential competing interest was reported by the authors

1. Liu, X. The more and less of electronic-skin sensors. Science 370, 910–911 (2020). 10.1126/science.abe7366, Pubmed:33214265.

2. Chortos, A., Liu, J. & Bao, Z. Pursuing prosthetic electronic skin. Nat. Mater. 15, 937–950 (2016). 10.1038/nmat4671, Pubmed:27376685.

3. Drimus, A., Kootstra, G., Bilberg, A. & Kragic, D. Design of a flexible tactile sensor for classification of rigid and deformable objects. Robot. Auton. Syst. 62, 3–15 (2014). 10.1016/j.robot.2012.07.021.

4. Wu, Y. et al. A skin-inspired tactile sensor for smart prosthetics. Sci. Robot. 3, 1–9 (2018). 10.1126/scirobotics.aat0429, Pubmed:33141753.

5. Zhao, H., O’Brien, K., Li, S. & Shepherd, R. F. Optoelectronically innervated soft prosthetic hand via stretchable optical waveguides. Sci. Robot. 1, 1–10 (2016). 10.1126/scirobotics.aai7529, Pubmed:33157858.

6. Yamada, D., Maeno, T. & Yamada, Y. Artificial finger skin having ridges and distributed tactile sensors used for grasp force control. J. Robot. Mechatron. 14, 140–146 (2002). 10.20965/jrm.2002.p0140

7. D’Anna, E. et al. A closed-loop hand prosthesis with simultaneous intraneural tactile and position feedback. Sci. Robot. 4 (2019). 10.1126/scirobotics.aau8892, Pubmed:33137741.

8. Li, G., Liu, S., Wang, L. & Zhu, R. Skin-inspired quadruple tactile sensors integrated on a robot hand enable object recognition. Sci. Robot. 5, 1–12 (2020). 10.1126/scirobotics.abc8134, Pubmed:33328298.

9. Su, Z. et al., (2015). Force estimation and slip detection/classification for grip control using a biomimetic tactile sensor. IEEE-RAS international conference Humanoid Robot 297–303. 10.1109/HUMANOIDS.2015.7363558.

10. Kim, K. et al. Tactile avatar: Tactile sensing system mimicking human tactile cognition. Adv. Sci. (Weinh) 8, 2002362 (2021). 10.1002/advs.202002362, Pubmed:33854875.

11. Pyo, S., Lee, J., Bae, K., Sim, S. & Kim, J. Recent progress in flexible tactile sensors for human-interactive systems: From sensors to advanced applications. Adv. Mater. 33, e2005902 (2021). 10.1002/adma.202005902, 33887803.

12. Huang, Y. C. et al. Sensitive pressure sensors based on conductive microstructured air-gap gates and two-dimensional semiconductor transistors. Nat. Electron. 3, 59–69 (2020). 10.1038/s41928-019-0356-5.

13. Lee, B. Y., Kim, J., Kim, H., Kim, C. & Lee, S. D. Low-cost flexible pressure sensor based on dielectric elastomer film with micro-pores. Sens. Actuators A 240, 103–109 (2016). 10.1016/j.sna.2016.01.037.

14. Zang, Y., Zhang, F., Di, C. A. & Zhu, D. Advances of flexible pressure sensors toward artificial intelligence and health care applications. Mater. Horiz. 2, 140–156 (2015). 10.1039/C4MH00147H.

15. Dagdeviren, C. et al. Conformable amplified lead zirconate titanate sensors with enhanced piezoelectric response for cutaneous pressure monitoring. Nat. Commun. 5, 4496 (2014). 10.1038/ncomms5496, Pubmed:25092496.

16. Kim, H. et al. Transparent, flexible, conformal capacitive pressure sensors with nanoparticles. Small 14, 1–10 (2018). 10.1002/smll.201703432, Pubmed:29372583.

17. Schwartz, G. et al. Flexible polymer transistors with high pressure sensitivity for application in electronic skin and health monitoring. Nat. Commun. 4, 1859 (2013). 10.1038/ncomms2832, Pubmed:23673644.

18. Mannsfeld, S. C. B. et al. Highly sensitive flexible pressure sensors with microstructured rubber dielectric layers. Nat. Mater. 9, 859–864 (2010). 10.1038/nmat2834, Pubmed:20835231.

19. Ahmed, A. et al. Multifunctional smart electronic skin fabricated from two-dimensional like polymer film. Nano Energy 75 (2020). 10.1016/j.nanoen.2020.105044.

20. Xu, Z. et al. Bio-inspired smart electronic-skin based on inorganic perovskite nanoplates for application in photomemories and mechanoreceptors. Nanoscale 13, 253–260 (2021). 10.1039/d0nr06550a, Pubmed:33331373.

21. Hua, Q. et al. Skin-inspired highly stretchable and conformable matrix networks for multifunctional sensing. Nat. Commun. 9, 244 (2018). 10.1038/s41467-017-02685-9, Pubmed:29339793.

22. Choi, D. et al. A highly sensitive tactile sensor using a pyramid-plug structure for detecting pressure, shear force, and torsion. Adv. Mater. Technol. 4 (2019). 10.1002/admt.201800284, 1800284.

23. Pang, C. et al. A flexible and highly sensitive strain-gauge sensor using reversible interlocking of nanofibres. Nat. Mater. 11, 795–801 (2012). 10.1038/nmat3380, Pubmed:22842511.

24. Park, J. et al. Tactile-direction-sensitive and stretchable electronic skins based on human-skin-inspired interlocked microstructures. ACS Nano 8, 12020–12029 (2014). 10.1021/nn505953t, Pubmed:25389631.

25. Sun, Q. et al. Fingertip‐skin‐inspired highly sensitive and multifunctional sensor with hierarchically structured conductive graphite/polydimethylsiloxane foams. Adv. Funct. Mater. 29 (2019). 10.1002/adfm.201808829, 1808829.

26. Chun, S. et al. Self-powered pressure- and vibration-sensitive tactile sensors for learning technique-based neural finger skin. Nano Lett. 19, 3305–3312 (2019). 10.1021/acs.nanolett.9b00922, Pubmed:31021638.

27. Navaraj, W. & Dahiya, R. Fingerprint‐enhanced capacitive‐piezoelectric flexible sensing skin to discriminate static and dynamic tactile stimuli. Adv. Intell. Syst. 1 (2019). 10.1002/aisy.201900051, 1900051.

28. Chun, K. Y., Son, Y. J., Jeon, E. S., Lee, S. & Han, C. S. A self-powered sensor mimicking slow- and fast-adapting cutaneous mechanoreceptors. Adv. Mater. 30, e1706299 (2018). 10.1002/adma.201706299, 29424032.

29. Chun, S. et al. An artificial neural tactile sensing system. Nat. Electron. 4, 429–438 (2021). 10.1038/s41928-021-00585-x.

30. Park, J., Kim, M., Lee, Y., Lee, H. S. & Ko, H. Fingertip skin–inspired microstructured ferroelectric skins discriminate static/dynamic pressure and temperature stimuli. Sci. Adv. 1, e1500661 (2015). 10.1126/sciadv.1500661, Pubmed:26601303.

31. You, I. et al. Artificial multimodal receptors based on ion relaxation dynamics. Science 370, 961–965 (2020). 10.1126/science.aba5132, Pubmed:33214277.

32. Boutry, C. M. et al. A hierarchically patterned, bioinspired e-skin able to detect the direction of applied pressure for robotics. Sci. Robot. 3, 1–10 (2018). 10.1126/scirobotics.aau6914, Pubmed:33141713.

33. Kim, T. et al. Heterogeneous sensing in a multifunctional soft sensor for human-robot interfaces. Sci. Robot. 5, 1–14 (2020). 10.1126/scirobotics.abc6878, Pubmed:33328297.

34. Chen, H. et al. Hybrid porous micro structured finger skin inspired self-powered electronic skin system for pressure sensing and sliding detection. Nano Energy 51, 496–503 (2018). 10.1016/j.nanoen.2018.07.001.

35. Seok Jo, H. S., An, S., Kwon, H. J., Yarin, A. L. & Yoon, S. S. Transparent body-attachable multifunctional pressure, thermal, and proximity sensor and heater. Sci. Rep. 10, 1–12 (2020). 10.1038/s41598-020-59450-0.

36. Kim, S. W. et al. A triple-mode flexible E-skin sensor interface for multi-purpose wearable applications. Sensors (Basel) 18, 1–11 (2017). 10.3390/s18010078, Pubmed:29286312.

There is NO Competing Interest.

cutaneousreceptorsmimickingsensorsystemsuppl.docx
Supplementary MaterialsFig. S1. Photographs of the four types of tactile sensors.Fig. S2. Photographs of the flexible 3D interconnection system.Fig. S3. Signal conditioning circuits for the proposed CRMT sensor.Fig. S4. Detection of various textures.

Download PDF

Version 1

posted

You are reading this latest preprint version

A cutaneous receptors-mimicking system for real-time and multimodal detection of tactile stimuli

Status:

Version 1

Abstract

Figures

Introduction

Results

Multimodal sensors mimicking cutaneous receptors

3D integration of multimodal sensors

Tactile signal processing and transmission module configuration

Sensing properties of the 3D-integrated CRMT system

Pattern pitch and texture discrimination

Real-time monitoring for various tactile stimuli

Perception of complex tactile motion

Discussion

Methods

Fabrication of the temperature sensor

Fabrication of the vibration sensor

Fabrication of the shear sensor

Fabrication of the pressure sensor

3D printing of interconnections

Assembly of the sensors

Data acquisition and processing

Measurements

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1