4.1 Materials
Tetrabutylammonium hydroxide (25% in water), hexadecyltrimethylammonium bromide, m-cresol purple, glycine, cobalt chloride hexahydrate, isopropanol, ethanol, and ethylene glycol were purchased from Aladdin Inc. Silicon dioxide nanopowder (5–15 nm) was purchased from Sigma‒Aldrich Inc. Thermochromic microcapsules (color-change temperature: 20, 35, 50, 65°C; particle size: 2–10 µm) were purchased from Dongfang Color-change Tech Inc. (Shenzhen, China). Clear resin (Model: RS-F2-GPCL-04) was purchased from Formlabs Inc. All chemical reagents were used directly without further purification. Ultra-pure water (18 MΩ) was produced by Millipore Direct-Q.
4.2 Preparation of the colorimetric inks M-cresol purple (36 mg), glycine (45 mg) and silicon dioxide nanopowder (800 mg) were first dissolved in 6 ml of mixed solution (the volume ratio of water, ethanol and ethylene glycol was 5:2:5) and shaken for 30 minutes. Subsequently, 2 ml of tetrabutylammonium hydroxide (25% in water) and 2 ml of hexadecyltrimethylammonium bromide (0.1 mol L− 1 in ethanol) were added and stirred evenly to obtain a carbon dioxide-sensitive ink.
Cobalt chloride hexahydrate (600 mg) and silicon dioxide nanopowder (300 mg) were dissolved in 2.2 ml of the abovementioned mixed solution and shaken for 30 minutes to obtain humidity-sensitive ink.
Thermochromic capsules (800 mg) were dissolved in 1 ml isopropanol. Then, 2.5 ml of clear resin was added and stirred evenly to obtain thermosensitive ink.
4.3 Fabrication of the M-imager chip Five-megapixel CMOS imagers (ov5640, OmniVision Technologies) with a 1.4 µm pixel size and 3.7×2.7 mm imaging area were used to fabricate the M-imager chips. An inkjet deposition system (Sonoplot, Microplotter Proto) was employed for the colorimetric ink printing. Before printing, the lens of the CMOS imager was disassembled to expose the imaging area. As shown in Supplementary Figure S2, the colorimetric inks were preserved in a 96-well plate and aspirated by a nozzle needle. The inks were ejected from the 50 µm nozzle needle by ultrasonic resonance. The printing process is shown in Supplementary Movie S1. First, a square area was patterned on the imaging area under the control of the built-in software “SonoGuide”, and the square was moistened with colorimetric inks. Second, the nozzle needle was manually moved over the printing area, and a 3–10 V voltage was applied to form a droplet on the wet area. After the solvent evaporated, an M-imager chip with multiple sensing units was obtained. As shown in Fig.
1D, we fabricated a total of 16 sensing units on the CMOS imager chip. The upper 4 units were sensitive to CO2, and the bottom 4 units were sensitive to humidity. The left and right 4 units with different colors were sensitive to temperature (From top to bottom: black-65°C, blue-50°C, red-35°C, and green-20°C). 4.4 Assembly of the M-imager module Supplementary Figure S3 shows the assembly of the M-imager module. A sampling fan (UB393-500) with a size of 9×9×3 mm was purchased from SUNON Inc., (Kaohsiung, China Taiwan). We designed a PCB board with 4 white light LEDs to provide a light source for the sensing unit lensless imaging. A 3D printed sensor shell was employed to separate the LED and ambient light sources, which achieved dual-focus imaging on the M-imager chip surface. Finally, a custom lens with a radius of 2 mm, focal length of 3.6 mm, and viewing angle of 20° was embedded in the shell for visual imaging.
4.5 Optimization and calibration of the sensing units To evaluate the sensing performance of the colorimetric units, we manufactured CO2, humidity and temperature calibration apparatuses. The structures of these apparatuses are illustrated in Supplementary Figure S4. The MFC controller (ACU10FD-LC) was purchased from AccuFlow Technology Inc. (Beijing, China). The CO2 sensor (SCD30) and humidity sensor (SHT31) were purchased from Sensirion Inc. To calibrate the temperature, a thermoelectric cooler (TEC) component with the Peltier effect was attached to a heat dissipation fan. By applying forward or reverse current on the TEC component, a temperature difference was created to realize cooling or heating. An RTD sensor (Heraeus, PT100) was employed to detect the temperature of the chamber. In addition, CO2 (4% CO2, 96% N2) and standard air (20% O2, 80% N2) calibration gases were supplied by Jingong Gas Inc. (Hangzhou, China).
To validate the enhancement effect of the nanoparticles on the sensing unit performance, we deposited three CO2-sensitive inks with different silica concentrations (0, 66, and 133 mg ml− 1) on a CMOS imager, and 6 units were fabricated for each ink (Fig.
2A). The CMOS imager was placed in the calibration apparatus. Then, 310, 1330, 2670 and 5880 ppm CO2 gases were sequentially flowed into the chamber. The intensity responses of the units were recorded and analyzed. To evaluate the sensing performance of colorimetric units with different sizes, we deposited 6 units with different sizes ranging from 40–250 µm on a CMOS imager, and 6 units were printed for each size. Then, the CO2, humidity and temperature sensing performances were assessed.
Finally, referring to the assessment of sensing performance of different size units, we employed 200 µm as a typical unit size and calibrated the CO2, humidity and temperature colorimetric responses.
The morphologies of the CMOS imaging area and sensing units were characterized by a scanning electron microscope (Phenom, XL). The optical microscope images were captured by a stereomicroscope (Nikon, SMZ18).
4.6 Demonstrations: microrobot for environmental sensing and mapping To validate the applicability of the M-imager in microrobot applications, we designed a 1.4×2.2 cm M-imager data processing PCB. The M-imager module and data processing board were equipped in a coin-sized (2.0×4.0 cm) remote-controlled car. A circuit schematic diagram of the data processing board is shown in Supplementary Figure S5. The CMOS imager was connected to a DSP module via an MIPI interface to convert the digital signal to an image stream. The stream was transmitted to a computer- or smartphone-based terminal through an MCU (MT7268DAN) with an integrated WiFi module. A Li-ion battery was employed to power the WiFi, DSP and LED array. Figures
5A, B and C demonstrate the data processing PCB and the wireless remote-controlled car equipped with the M-imager module. In the proof of concept demonstrations, the car was controlled to pass the alcohol lamp while monitoring the breath cycles and atomizer. The video stream was transmitted to a smartphone, and the colorimetric responses of the sensing units were recorded. A stuffed doll named “Xiaozhi” (Mascot of Zhejiang Lab) was placed in front of the remote-controlled car to determine whether the M-imager could capture visual information.
In another demonstration, we placed 4 components, including a fan heater, humidifier, ice bags and alcohol lamp, at each corner of a black and white checkered blanket. The central area was a rectangle of size 200×160 cm (Supplementary Figure S11). We divided the area into 5×4 grids (Fig.
5K). The micro car was controlled to move around the map in an S-shape and to stop and take a photo at the center of each grid. The field distributions of the temperature, humidity and CO2 in the area were analyzed. 4.7 Demonstration: multimodal endoscope A Bama miniature pig (male, 40 kg, 12 months) was employed to validate the applicability of the M-imager in endoscopy. The Bama pig was under quarantine and observation for two weeks before the experiment and was given good care in a separate cage during the quarantine period. After the quarantine period ended, the Bama pig was used for the endoscopic experiment. All experimental procedures were approved by the Laboratory Animal Management and Ethics Committee of Zhejiang Chinese Medical University (animal experiment ethics approval number: IACUC-20220328-16).
We assembled the M-imager module in an endoscope, and the assembly diagram is shown in Fig.
6B. The CMOS imager and DSP module were combined in a 3D printed shell. A transparent acrylic sheet and 0603 patch LED were placed on the front of the shell for convenience during CMOS imaging. PTEF membranes (11×14 mm, IP64) were encircled along the shell to allow air to enter and prevent liquid leaks. We also integrated a fan into the endoscopic probe to accelerate the gas exchange and improve the response speed of the sensing units. In addition, 4 wires connected the DSP module to a computer through a USB protocol. In total, 8 sensing units were incorporated on the CMOS imager (Fig.
6C). The left 4 units (Group 1) used the same formula as the abovementioned carbon dioxide-sensitive ink. Considering the high concentration of acid gas in the digestive tract of pigs, for the right 4 units (Group 2), we modified the ink formula and added 4 ml of tetrabutylammonium hydroxide to extend the sensitivity range. The Bama pig fasted for 12 hours before the beginning of the experiment. A 2 mg kg− 1 dose of propofol was injected to induce anesthesia. After the Bama pig was anesthetized, a breathing tube for oxygen containing 2% isoflurane was inserted into the trachea of the Bama pig. The Bama pig was kept in a prone position to ensure that its esophagus was not compressed (Supplementary Figure S12). We slowly passed the endoscope through the pig's mouth into the digestive tract. The video data were instantly transferred and recorded on a computer. The whole experiment lasted 15–20 min, and vital signs (e.g., blood oxygen, breathing, the BIS index) were monitored to confirm the safety and anesthesia state of the Bama pig. After all operations were completed, the Bama pig was euthanized.