An Artificial Intelligence-Assisted Digital Microfluidic System for Multistate Droplet Control

doi:10.21203/rs.3.rs-3782688/v1

Download PDF

Article

An Artificial Intelligence-Assisted Digital Microfluidic System for Multistate Droplet Control

https://doi.org/10.21203/rs.3.rs-3782688/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Digital microfluidics (DMF) is a versatile technique for parallel and field-programmable control of individual droplets. Given the high freedom in droplet manipulation, it is essential to establish self-adaptive and intelligent control methods for DMF systems with informed of the transient state of droplets and their interactions. However, most related studies focus on the localization and shape recognition of droplets. Here, we develop an AI-assisted DMF framework named "µDropAI" for multistate droplet control based on droplet morphology. Semantic segmentation model is integrated into our custom-designed DMF system to recognize the droplet states and their interactions for feedback control with a state machine. The proposed model has a strong flexibility and can recognize droplets of different colors and shapes with an error rate of less than 0.63%. It enables control of droplets without user intervention. The coefficient variation (CV) of the volumes of split droplets can be limited to below 2.8%, which is lower than the CV of traditional dispense, contributing to an improvement in the precision of droplet split. The proposed system will inspire the development of semantic-driven DMF systems which can interface with multimodal large language models (MLLM) for fully automatic control.

Physical sciences/Engineering/Electrical and electronic engineering

Physical sciences/Chemistry

Digital microfluidics (DMF) has become a prevalent technique for handling liquid due to its unique flexible and discrete fluid control capabilities.^1-3 In particular, DMF systems based on electrowetting-on-dielectric (EWOD) principles utilize voltage modulation to control the wettability of solid surfaces, facilitating the manipulation of droplets across scales from nL to μL.^4-7 This technique has advantages such as miniaturization, programmability, parallelization, and low power consumption.^8,9 Due to these benefits, DMF devices have been used in a wide range of applications, including point-of-care testing,^10,11 cell research,^12,13 biomedicine,^14,15 and environmental monitoring.¹⁶

Typically, DMF devices accept a standard set of basic instructions to perform droplet manipulations. Users can create droplet actuation sequences for each actuating electrode,^17,18 thereby allowing the droplets to move, dispense, merge, split, and mix.^19-21 Typically, an automated DMF system incorporates sensors liking capacitance or impedance to monitor droplet operations in real-time.^22-25 The control algorithm is also integrated into the DMF system to manage electrode switching and sequencing for subsequent operations, including droplet tracking and path planning based on feedback signals.^26,27 These sensing and control methods not only enhance the droplet manipulation stability but also optimize the motion performance of the droplet.²⁶ Imaging technologies, which include background subtraction methods, edge detection, object detection, and image segmentation,^28-31 are applied to droplet motion recognition. They offer a non-contact, real-time solution for signal acquisition, resulting in rapid and accurate detection and tracking of droplets, even transparent ones.³² In practice, the operations of droplets often result in changes of droplet appearance such as shapes and colors. However, few studies have focused on the automated control of droplets with recognition of their states based on the appearance changes.

In the last decade, the advance of convolutional neural network paved the way for the development of semantic segmentation techniques, which is a pixel level classification that provides the understanding of images.³³ Compared to the image processing such as edge detection and YOLO,³⁴ the semantic segmentation techniques stands out for its outstanding advantages in complex scenarios and multi-category object segmentation tasks, including precise object segmentation, support for object multiple categories or states, environmental awareness, and advanced image analysis applications.^35-37 Therefore, they have been widely applied in AI-assisted technology, such as autonomous driving, robotics, and smart healthcare.^38-40 Therefore, semantic segmentation techniques can provide DMFs with accurate recognition of droplet multistate, complemented by control methods that enable the development of fully automated and flexible control systems to perform complex tasks.

In this paper, we develop an artificial intelligence-assisted digital microfluidics system named "μDropAI" for highly automated multistate droplet control (Fig. 1). The framework contains four parts: 1) a hardware system for droplet actuation and video capture; 2) a semantic segmentation model trained to detect the droplet states; 3) a region growth algorithm to extract the position and morphological information of droplets; and 4) automated control processes for user-programmed automatic control. An open-source dataset was established with labelled droplet images consisting different states, colors and shapes. The proposed DMF system had a semantic segmentation model trained on the dataset to recognize droplet multistate, controlled the electrode energization sequence, and was successfully used for automatic droplet splitting, movement, and dispensing. To our best knowledge, it is the first time to use the semantic segmentation for the control of DMF systems. Both the dataset and the μDropAI are open source. In addition, the proposed control system is also compatible with existing DMF devices that integrated with digital camera such as DropBot, openDrop, and DropLab.^41-43 Therefore, it can be widely used in DMF-based applications and it will promote the integration of more AI-based techniques to be utilized in DMFs such as reinforcement learning and ChatGPT.⁴⁴

2.1 Encoder-decoder semantic segmentation model

A deep learning-based algorithm which automatically segments droplet multistate is proposed. Our model is based on the U-net model with an encoder-decoder structure.⁴⁵ The encoder unit of the proposed model performs feature extraction and the decoder unit performs upsampling operations. Shortcut connections connect the corresponding feature maps between the encoder and decoder units (Fig. 2). By incorporating skip connections, the model can capture both high-level and low-level features.

In the encoder stage, the model comprises 16 convolutional layers and 3 fully connected layers to extract crucial feature information from input images, which is later used in the subsequent decoder stage to generate semantic segmentation masks. These convolution layers all use a 3x3 convolution kernel, a step length is 1, and the same filling method to keep the size of the feature graph unchanged. After each convolution layer, the ReLU activation function is applied to introduce nonlinearity.

The convolution layer part can be divided into 5 convolution blocks. There are 2 convolution layers within each convolution block, followed by a 2x2 maximum pooling layer, which is used to reduce the spatial size of the feature graph. By stacking multiple convolutional blocks, the depth of the network can be gradually increased, thus improving the feature expression ability of the model.

In the decoder stage, 8 convolutional layers corresponding to the encoder are used, followed by upsampling and transpose convolution operations after each block. These operations restore the feature map size to the original input image size, and skip connections are utilized to connect the feature maps of the same resolution from the encoder and decoder stages. These shortcut connections help retain more spatial information and details, thus assisting the network in obtaining more accurate segmentation results.

The final layer of the network is a 1x1 convolutional layer with 5 channels. This layer maps the feature maps from the decoder to the same number of channels as the number of label categories considered in this paper. This approach ensures that the predicted image segmentation results are accurate.

2.2 Region growing algorithm

The region growing algorithm (†ESI S1 and †Fig. 1) is introduced to obtain a pixel-level division of the droplet state in each frame. Additionally, these segments can be connected based on pixel similarity to obtain a more accurate representation of the droplet. This algorithm has a strong ability to segment and compute regions based on pixel similarity.⁴⁶

2.3 Automated control process based on droplet states

The control process is executed using a state machine implementation, enabling automated manipulations, such as droplet movement and splitting, following user-provided manipulation instructions (Fig. 3). For example, by inputting the move process steps, the droplet moves in the direction of the specified electrode, and the control method determines whether the droplet has reached the specified position according to the feedback information (Fig. 3a). When the split command is input, the droplet is controlled to split. During the splitting process, the droplet states are recognized, and the next manipulation is determined based on the feedback information to ensure that the droplet has completely split (Fig. 3b). When the initial state contains two droplets, a merge command can be applied to merge the two droplets into one large droplet (Fig. 3c).

3.1 Chip fabrication

A double-plate DMF chip was utilized, with both the top and bottom plates constructed from ITO glass. The electrodes were directly etched onto the bottom plate (52 mm x 50 mm) using a UV laser operating at a wavelength of 355 nm and a pulse duration of 20 µs.⁴⁷ Then, the glass plates were cleaned with ethanol for 10 minutes to remove residual material and exposed to plasma treatment for 1 minute. Following the application of a vacuum-deposited Parylene C film (~ 2 µm thick) onto the bottom plate surface as a dielectric layer, CYTOP films were spin-coated (2000rpm, 30s) onto both the top and bottom plates to serve as hydrophobic layers. A double-sided adhesive with a thickness of 0.5 mm was employed to securely attach the upper and lower plates. Then, silicon oil (0.65 cs) was introduced to fill the gap between the two plates, effectively diminishing the friction encountered during liquid droplet movement and deterring evaporation.

3.2 Setup of DMF hardware

A homemade DMF system was used to manipulate the DMF chip and control the droplet manipulation processes. Optocoupler switch array controlling the high voltage (ATG-2081, Aigtek, China) applied to the electrodes on the chip was configured by a microcontroller unit (STM32L432, ST, Italy), which receives the instructions from a python script embedded with the proposed AI assisted control model. A microscope equipped with a CCD camera (IUA20000KPA, ToupTek, China) was positioned above the DMF system to observe and capture the droplet status. The resolution of the CCD camera was set to 2560x1440 to obtain clear images. Subsequently, the images were transmitted to a computer with an R7 5700X CPU, 16 GB RAM, and RTX 3070 GPU for further processing and analysis.

3.3 Dataset establishment

A multistate droplet dataset was established with a digital camera. Images of various droplet manipulations were captured with frame rate of 30 frames per second. The captured videos were converted into a series of images (1920x1080) with OpenCV. These droplet manipulation images were labelled into four types of states: "unsplit", "splitting", "split", and "merging" with LabelMe (†ESI Table 1), and converted to PASCAL VOC data format. The dataset was divided into training and test sets in a ratio of 0.8:0.2, including 1751 images in the training set and 438 images in the test set. The training set was used to train the model, and the test set was used to evaluate the model's performance. The dataset had been made public for others (https://github.com/Eric1105/-DropAI.git). This protocol can be used as a reference for establishing other DMF droplet manipulation datasets.

3.4 Model training parameters and evaluation metrics

The training GPUs were dual RTX3090 12G. The experiment was conducted using the PyTorch framework, with the programming language Python version 3.8 on a Windows system.

To adapt to the dataset, which has a large proportion of background and a small proportion of objects, the following optimized parameter was set to train the semantic segmentation model. After our testing, Adam was chosen as the optimizer it has the ability to quickly identify the optimal parameter configuration to achieve better training performance. The learning rate was set to 0.0001, with a minimum learning rate of 0.000001. The learning rate was reduced every 5 epochs using the cosine annealing learning rate scheduling strategy.

The Dice loss was employed as the loss function (Eq. 1). The Dice loss ranges from 0 to 1, with lower values indicating greater similarity between the predicted and ground truth results.

$$loss{\kern 1pt} ={\kern 1pt} 1 - 2 \times \frac{{\left| {x \cap y} \right|}}{{\left| x \right|+\left| y \right|}}$$

where x represents the predicted segmentation mask and y represents the ground truth segmentation mask.

The mean intersection over union (mIoU) was employed as the main evaluation indicator for the semantic segmentation performance (Eq. 2). This metric is the average of the ratio of the intersection and union of various real labels and prediction results.

$$mIoU{\kern 1pt} ={\kern 1pt} \frac{1}{{n+1}}\sum\limits_{{i=0}}^{n} {\frac{{{p_{ii}}}}{{\sum\nolimits_{{j=0}}^{n} {{p_{ij}}} +\sum\nolimits_{{j=0}}^{n} {{p_{ji}} - {p_{ii}}} }}}$$

where n represents the total number of categories, ${p_{ii}}$represents the number of correct predictions, ${p_{ij}}$ represents false negatives, in which i is predicted as j, and ${p_{ji}}$represents false positives.

4.1 Model comparison

To validate the segmentation performance of the proposed deep learning algorithm, a comparative experiment with the traditional segmentation model U-net and DeeplabV3 + ⁴⁸ on our open-source dataset was designed to compare their segmentation results, mIoU, and loss. The mIoU indicates the extent to which the model-predicted area overlaps with the actual area. The loss is used to measure the difference between the model prediction and the actual target. In the recognition process, red represents the "unsplit" state, green indicates the "splitting" state, yellow represents the "split" state, and blue signifies the "merging" state.

Figure 4a-d illustrates the process of a droplet splitting. As can be seen, it transitions from the initial red “unsplit” state to the green “splitting” state, culminating in the yellow “split” state. In our evaluation, the DeeplabV3+, U-net, and our proposed model accurately recognize droplets in the "unsplit" and "split" states (Fig. 4b1-d1 and b3-d3). In instances where droplets assume an hourglass shape, the DeeplabV3 + fails to segment the connected regions within the "splitting" state (Fig. 4b2). In contrast, both U-net and our proposed model effectively segment the "splitting" droplet (Fig. 4c2 and d2). Accurate recognition of the "splitting" state is crucial for the successful execution of the splitting operation. When two droplets merge, our proposed model accurately discriminates between the blue "merging" and "split" states (Fig. 4d4). In contrast, the DeeplabV3 + erroneously categorizes them as "split" (Fig. 4b4), while U-net detects parts of the droplets as "split" (Fig. 4c4). These results indicate that our model consistently outperforms the other models in most scenarios.

The mIoU of the proposed is in the range of 85–91% (Fig. 4j), while the mIoU of DeeplabV3 + and U-net are in the range of 80–89% and 83–89% (Fig. 4e and g). The mIoU of the proposed 1.1-6% better than that of the DeeplabV3+, and 1.1-2% better than that of the U-net. A higher mIoU indicates better segmentation performance, with the proposed predicted regions overlapping more closely with the actual regions. Moreover, the loss of the proposed stabilizes at 0.08–0.15 (Fig. 4k), which is approximately 50–60% lower than the DeeplabV3 + loss of 0.2–0.3 (Fig. 4f), and 20–37% lower than the U-net loss of 0.11–0.18 after the 75th epoch (Fig. 4h). A lower loss indicates that the proposed predictions are closer to the true droplet states. Furthermore, the loss and mIoU of the proposed tend to stabilize after convergence, indicating that it has learned and mastered effective semantic segmentation ability.

The pixel-level evaluation metrics on a test set of 438 images were also computed, including the mean precision (mPrecision), mean pixel accuracy (mPA), and mean recall (mRecall) values. Three metrics are used to evaluate the performance of the model's semantic segmentation results in terms of category accuracy, pixel-level classification accuracy, and positive sample detection (†ESI S2 Eqs. 1–5). The mPrecision is 95.72%, the mPA is 93.30%, and the mRecall is 93.30% (†Fig. 2). The three metrics of the proposed method are greater than 91.76%, 89.80%, and 88.76% of the DeeplabV3+, and greater than 92.96%, 90.97% and 90.37% of the U-net, with higher values indicating that the proposed can accurately recognize objects in the image and generate segmentation results that highly match the actual labelled objects.

4.2 Influence of luminous environment

To investigate the impact of potential variations in luminous environment within various DMF systems, OpenCV was utilized to adjust the brightness levels of real-time videos (Fig. 5a) and the accuracy of recognizing states, positions, and overall recognition under different lighting conditions was analyzed across eight videos (†ESI S2 Eqs. 6–8).

With a 0% and 50% increase or decrease in brightness, the overall accuracy decreased to 100% and 96.7% (Fig. 5b). Meanwhile, the mean accuracy for recognizing both states and positions consistently ranges between 97% and 100% (Fig. 5c and d). Obviously, the DMF's lighting just changed the droplet color from blue to cyan or dark blue, and the droplet on the DMF chip are still visible. The change in lighting intensity is consistent with the lighting conditions of the vast majority of DMFs, and in this case, the proposed system still maintains a high recognition accuracy of over 96.7%. With a 60% and 80% increase or decrease in brightness, the overall mean accuracy decreases to 91.7% and 67.5% (Fig. 5b). Likewise, the mean accuracy for states and positions recognition decreases to 95.6% and 73.8% (Fig. 5c), 95.7% and 76.3% (Fig. 5d). It is evident that when the brightness variation approaches 80%, overexposure or underexposure occurs. The droplet's color has changed, and its edges are no longer distinct, leading to a noticeable drop in recognition accuracy. Furthermore, as the brightness varies from 0–50%, the standard deviation (SD) of overall accuracy consistently remains within the range of 0-0.3%, which is lower by 0.5–2% compared to the observed SD fluctuating between 0.5–2.3% when brightness varies from 60–80%. This indicates that the proposed system exhibits stronger accuracy and robustness, especially in the brightness variation range of 0–50%.

4.3 Influence of droplet color and shape

Considering the difference in color and shape of droplets due to differences in their composition and handling methods in DMF system, experiments for recognizing droplets with different colors and shapes were established. Droplets of different colors can be recognized as "unsplit" by the different shapes they appear to take during moving, such as L-shaped, rectangular, circular, and triangular. (Fig. 6a1-b1 and †Fig. 3a). In real-time operation, the state transitions of the splitting of different colored droplets from "unsplit" to "splitting" to "split", and the merging of two droplets "merging" can also be recognized (Fig. 6a2-5, Fig. 6b2-5, †Fig. 3b-e, and †video 1). In conclusion, the proposed system not only accurately recognizes various shapes and states of blue droplets but also extends its recognition to red, yellow, black, and even transparent droplets (†ESI S4). This proves that our system has strong generalization ability to recognize different colors and shapes droplets of instantaneous changes in operating states.

Considering the merging of droplets with various substances during DMF experimental operations to observe reaction phenomena, an experiment for the merging of droplets of different colors was devised. When red and blue "unsplit" droplets merge (Fig. 6b1), they enter a "merging" state (Fig. 6b2), returning to "unsplit" state after merging (Fig. 6b3). Even as they mix further, they still stay "unsplit" and trackable (Fig. 6b4-b5)). The successful recognition of the merging of droplets with different colors indicates the ability of the proposed DMF system for merging, subsequent splitting, and moving of droplets with various reagents or materials, extending beyond the constraints of single-substance droplets.

The error rate is the error caused by the droplets being recognized as other states while remaining in the continuous and same state. It can be found that the average error rates for the 4 states of different colored droplets are maintained at around 0.63%, 0.57%, 0.43% and 0.44% respectively, which means that the average number of incorrectly recognized frames is less than 1 frame in continuous and same state (Fig. 6c). This demonstrates the system's capability to perform real-time recognition of multiple states of differently colored droplets with a low error rate. When applied in real-time recognition during DMF experimental operations, it ensures the automation of droplet handling processes, preventing operational failures due to misrecognized states. For example, in a control sequence that transitions from a "splitting" state to a "splitting" state that is incorrectly identified as "splitting," the system incorrectly determines that the droplet has completed the splitting process.

4.4 AI-assisted multistate droplet control

To validate the applicability of the AI-assisted multistate feedback control system named µDropAI, droplet manipulations including moving, splitting, and dispensing on the DMF platform were performed.

Droplet moving, and splitting on digital microfluidic are common biological and chemical experimental operations. A flow chart of automated feedback control involving droplet moving and splitting is performed (†ESI S5 and †Fig. 4). In the droplet moving manipulation, µDropAI recognizes the droplets and determines their locations on the platform. Subsequently, the activated electrodes drive the droplets to reach their destination. When the droplet reaches the target electrode, µDropAI confirmed that the droplet has successfully moved. (Fig. 7a4-a5, Fig. 7b, and †video 2). When the movement fails, the electrode needs to be reactivated to drive the droplet. Compared to existing image-based methods,²⁸ our system not only recognizes the contours of the droplets, but also distinguishes the state and position of the droplets and is adaptable to a variety of environments. In the droplet splitting manipulation, the droplet undergoes a total of three state transitions, from the "unsplit" state of a large droplet to the hourglass-shaped "splitting" state, and finally to the "split" state of two small droplets (Fig. 7a1-a3, Fig. 7b, and †video 2). When two electrodes are energized sequentially, the droplet may be pulled to one side by the first energized electrode, leading to a failure in splitting. In such cases, the control system reverts the droplet to its initial state and re-split (†video 3). Additionally, the system can automatically merge the droplets and to recognize this droplet. The proposed control system can accurately monitor the droplet state, and guide the droplet operation according to the droplet state, extending beyond mere location-based operation determination.

Droplet dispensing is an operation to generate droplets from a reservoir. A flow chart of automated feedback control experiments for dispensing is performed (†ESI S5 and †Fig. 5). The initial position and state of droplet in reservoir is recognized (Fig. 7d1), an automatic process is executed by the µDropAI to activate the electrodes to split the droplet from the reservoir, (Fig. 7d2-d3). During the process, an hourglass-shaped structure is formed and recognized as the “splitting” state until the droplet is totally spitted from the reservoir (Fig. 7d4, Fig. 7c, and †video 4). To create a droplet of one unit size, simply activate one electrode on one side of the reservoir. AI-assisted methods can adjust the ideal size of droplets on demand, and can be used to enhance the precise of droplet generation for high-throughput experiments.

The volume precision of droplets is crucial in various applications such as drug discovery and quantitative analysis. Hence, a feedback closed-loop control method based on pixel values is proposed to enable the precise splitting of droplets by monitoring droplet volumes and controlling electrodes (†ESI S5 and †Fig. 6). During the process of droplet splitting, the proposed semantic segmentation model is also utilized to calculate the pixel values of the split droplets, determining their respective volumes. If the volume error of the split droplets exceeds 3%, the droplet will be moved to another electrode for re-split. The volume of a droplet covering two electrodes is approximately 4 µL, corresponding to a pixel value of 8430 (The pixel value of the droplet in a stationary state is averaged over 70 experiments). The coefficients of variation (CV) are calculated for split droplets under the proposed close-loop and open-loop control methods in 70 experiments. A larger CV indicates a greater volume error in the split droplets. It is observed that the proposed method yields more evenly split droplets, with CV value being limited to below 2.8%, which is lower than CV value 5.62% (The CV value is basically consistent with that of traditional droplet dispense.⁴⁹) (Fig. 7e and †Fig. 7).

The fundamental droplet operations shown above demonstrate the unprecedented advantages of the AI-assisted feedback control system for DMF. It can autonomously conduct experimental operations with various forms of reagents, such as protein-rich droplets, on DMF chips without human intervention. Moreover, it can combine with reinforcement learning to enhance the precision and automation of droplet manipulation by recognizing, tracking, discriminating states, and automating droplet control, thereby expanding the capabilities and applications of DMF systems.

In summary, an AI-assisted digital microfluidic system named µDropAI is presented in this paper. The system utilizes the U-net model for multistate droplet recognition and segmentation and employs a closed-loop feedback control strategy for real-time control during multistate droplet manipulations. The U-net-based model is robust enough to recognize droplets at different brightnesses, while it can also segment droplets of different colors and shapes, different states, and even the merging of droplets of different colors. The AI-assisted DMF platform demonstrates the ability to automatically correct failed droplet manipulations by recognizing the droplet position and judging the manipulation state. Meanwhile, the AI-assisted platform also brings an improved uniformity of the split droplet volume. We envision that this AI-assisted feedback approach will be widely applied in other DMF platforms to achieve precise and automatic droplet manipulations in a wide range of automated DMF applications. Moreover, with the advancement of large-scale models such as ChatGPT, the AI-assisted digital microfluidic system can be integrated with ChatGPT to enable more intelligent manipulations.

Nonetheless, this study is a preliminary proof-of-concept, and we note several limitations of the current implementation. For one, the model is too large to be integrated for computation on, for example, embedded devices, as this is more computationally time and resource intensive. This is a problem that we plan to start with model reduction to ensure accurate recognition while reducing the number of parameters. In addition, the manipulation of multiple droplets is still a difficult point, and we intend to design algorithms to track the manipulation process of droplets and realize automatic feedback control of the manipulation of multiple droplets.

Acknowledgements

This research was funded by the National Key R&D Program of China (2021YFC2101101) and the National Natural Science Foundation of China (No. 62103148).

Author details

¹Key Laboratory of Smart Manufacturing in Energy Chemical Process Ministry of Education, East China University of Science and Technology, Shanghai, China.

Author contributions

Kunlun Guo: investigation, methodology, validation, data curation, visualization, writing – original draft. Zerui Song: investigation, validation, data curation, instrument designing, writing – original draft. Jiale Zhou: investigation, methodology. Bin Shen: investigation, resources. Bingyong Yan: investigation, writing – review & editing. Zhen Gu: conceptualization, methodology, supervision, writing – review & editing, resources, funding acquisition. Huifeng Wang: writing – review & editing, resources, funding acquisition. All authors discussed the results and contributed to the final manuscript.

Conflict of interest

The authors declare no competing interests.

Zhong, J. et al. When robotics met fluidics. Lab Chip 20, 709-716 (2020).
Campos, R. P. S. et al. "Plug-n-Play" Sensing with Digital Microfluidics. Anal. Chem. 91, 2506-2515 (2019).
Guo, J. et al. Auto-affitech: an automated ligand binding affinity evaluation platform using digital microfluidics with a bidirectional magnetic separation method. Lab Chip 20, 1577-1585 (2020).
Atabakhsh, S. & Jafarabadi Ashtiani, S. Thermal actuation and confinement of water droplets on paper-based digital microfluidics devices. Microfluid. Nanofluid. 22, 1577-1585 (2018).
Torabinia, M., Asgari, P., Dakarapu, U. S., Jeon, J. & Moon, H. On-chip organic synthesis enabled using an engine-and-cargo system in an electrowetting-on-dielectric digital microfluidic device. Lab Chip 19, 3054-3064 (2019).
Geng, H., Feng, J., Stabryla, L. M. & Cho, S. K. Dielectrowetting manipulation for digital microfluidics: creating, transporting, splitting, and merging of droplets. Lab Chip 17, 1060-1068 (2017).
Jin, K. et al. "One-to-three" droplet generation in digital microfluidics for parallel chemiluminescence immunoassays. Lab Chip 21, 2892-2900 (2021).
Wan, L. et al. A digital microfluidic system for loop-mediated isothermal amplification and sequence specific pathogen detection. Sci Rep 7, 14586 (2017).
Wang, J. X. et al. Auto-Panning: a highly integrated and automated biopanning platform for peptide screening. Lab Chip 21, 2702-+ (2021).
Zhai, J. et al. A digital microfluidic system with 3D microstructures for single-cell culture. Microsyst Nanoeng 6, 6 (2020).
Ruan, Q. Y. et al. Digital-WGS: Automated, highly efficient whole-genome sequencing of single cells by digital microfluidics. Sci. Adv. 6, eabd6454 (2020).
Zhai, J. et al. A digital microfluidic system with 3D microstructures for single-cell culture. Microsyst Nanoeng 6, 6 (2020).
Samlali, K., Alves, C. L., Jezernik, M. & Shih, S. C. C. Droplet digital microfluidic system for screening filamentous fungi based on enzymatic activity. Microsyst Nanoeng 8, 123 (2022).
Sklavounos, A. A., Nemr, C. R., Kelley, S. O. & Wheeler, A. R. Bacterial classification and antibiotic susceptibility testing on an integrated microfluidic platform. Lab Chip 21, 4208-4222 (2021)..
Kanitthamniyom, P. et al. A 3D-printed modular magnetic digital microfluidic architecture for on-demand bioanalysis. Microsyst Nanoeng 6, 48, doi:10.1038/s41378-020-0152-4 (2020).
Sathyanarayanan, G., Haapala, M., Dixon, C., Wheeler, A. R. & Sikanen, T. M. A Digital‐to‐Channel Microfluidic Interface via Inkjet Printing of Silver and UV Curing of Thiol–Enes. Adv. Mater. Technol. 5, 2000451 (2020).
Fobel, R., Fobel, C. & Wheeler, A. R. DropBot: An open-source digital microfluidic control system with precise control of electrostatic driving force and instantaneous drop velocity measurement. Appl. Phys. Lett. 102, 193513 (2013).
Kaminski, T. S. & Garstecki, P. Controlled droplet microfluidic systems for multistep chemical and biological assays. Chem. Soc. Rev. 46, 6210-6226 (2017).
Abdelgawad, M. & Wheeler, A. R. The Digital Revolution: A New Paradigm for Microfluidics. Adv. Mater. 21, 920-925 (2009).
Jebrail, M. J. & Wheeler, A. R. Let's get digital: digitizing chemical biology with microfluidics. Curr. Opin. Chem. Biol. 14, 574-581 (2010).
Jebrail, M. J., Bartsch, M. S. & Patel, K. D. Digital microfluidics: a versatile tool for applications in chemistry, biology and medicine. Lab Chip 12, 2452-2463 (2012).
Shih, S. C. C., Fobel, R., Kumar, P. & Wheeler, A. R. A feedback control system for high-fidelity digital microfluidics. Lab Chip 11, 535-540 (2011).
Sadeghi, S. et al. On Chip Droplet Characterization: A Practical, High-Sensitivity Measurement of Droplet Impedance in Digital Microfluidics. Anal. Chem. 84, 1915-1923 (2012).
Lederer, T., Clara, S., Jakoby, B. & Hilber, W. Integration of impedance spectroscopy sensors in a digital microfluidic platform. Microsystem Technologies - Micro- and Nanosystems. Information Storage and Processing Systems 18, 1163-1180 (2012).
Zhang, C. J. et al. An Impedance Sensing Platform for Monitoring Heterogeneous Connectivity and Diagnostics in Lab-on-a-Chip Systems. ACS Omega 5, 5098-5104 (2020).
Gao, J. et al. An intelligent digital microfluidic system with fuzzy-enhanced feedback for multi-droplet manipulation. Lab Chip 13, 443-451 (2013).
Gong, J. & Kim, C. J. All-electronic droplet generation on-chip with real-time feedback control for EWOD digital microfluidics. Lab Chip 8, 898-906 (2008).
Vo, P. Q. N., Husser, M. C., Ahmadi, F., Sinha, H. & Shih, S. C. C. Image-based feedback and analysis system for digital microfluidics. Lab Chip 17, 3437-3446 (2017).
Shin, Y.-J. & Lee, J.-B. Machine vision for digital microfluidics. Rev. Sci. Instrum. 81, 014302 (2010).
Ghosh, S., Rahaman, H. & Giri, C. Test diagnosis of digital microfluidic biochips using image segmentation. Proceedings of the IEEE 27th Asian test symposium 185-190 (2018).
Tang, Y. X. et al. Image-based real-time feedback control of magnetic digital microfluidics by artificial intelligence-empowered rapid object detector for automated in vitro diagnostics. Bioeng. Transl. Med. 8, e10428 (2022).
Li, L. B. et al. Intelligent droplet tracking with correlation filters for digital microfluidics. Chin. Chem. Lett. 32, 3416-3420 (2021).
Long, J., Shelhamer, E., Darrell, T. & Ieee. Fully convolutional networks for semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition 3431-3440 (2015).
Shafiee, M. J., Chywl, B., Li, F. & Wong, A. Fast YOLO: A fast you only look once system for real-time embedded object detection in video. Preprint at https://doi.org/10.48550/arXiv.1709.05943 (2017).
Taghanaki, S. A., Abhishek, K., Cohen, J. P., Cohen-Adad, J. & Hamarneh, G. Deep semantic segmentation of natural and medical images: a review. Artificial Intelligence Review 54, 137-178 (2021).
Wang, P. et al. Understanding convolution for semantic segmentation. Proceedings of the IEEE winter conference on applications of computer vision 1451-1460 (2018).
Mo, Y., Wu, Y., Yang, X., Liu, F. & Liao, Y. Review the state-of-the-art technologies of semantic segmentation based on deep learning. Neurocomputing 493, 626-646 (2022).
Wang, H. et al. SFNet-N: An Improved SFNet Algorithm for Semantic Segmentation of Low-Light Autonomous Driving Road Scenes. IEEE Transactions On Intelligent Transportation Systems 23, 21405-21417 (2022).
Lin, A. et al. DS-TransUNet: Dual Swin Transformer U-Net for Medical Image Segmentation. IEEE Trans. Instrum. Meas. 71, 1-15 (2022).
Wang, L., Wu, J. J., Liu, X. Y., Ma, X. L. & Cheng, J. Semantic segmentation of large-scale point clouds based on dilated nearest neighbors graph. Complex & Intelligent Systems 8, 3833-3845 (2022).
Fobel, R., Fobel, C. & Wheeler, A. R. DropBot: An open-source digital microfluidic control system with precise control of electrostatic driving force and instantaneous drop velocity measurement. Appl. Phys. Lett. 102, 193513 (2013).
Alistar, M. & Gaudenz, U. OpenDrop: An Integrated Do-It-Yourself Platform for Personal Use of Biochips. Bioengineering 4, 45 (2017).
Hu, X., Gao, X., Chen, S., Guo, J. & Zhang, Y. DropLab: an automated magnetic digital microfluidic platform for sample-to-answer point-of-care testing—development and application to quantitative immunodiagnostics. Microsyst Nanoeng 9, 10 (2023).
Radford, A., Narasimhan, K., Salimans, T. & Sutskever, I. Improving language understanding by generative pre-training. Preprint at https://api.semanticscholar.org/CorpusID:49313245 (2018).
Ronneberger, O., Fischer, P. & Brox, T. U-net: convolutional networks for biomedical image segmentation. Proceedings of the medical image computing and computer-assisted intervention 234–241 (2015).
Pavlidis, T. & Liow, Y. T. Integrating region growing and edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 225-233 (1990).
Song, Z.-R. et al. Optimization of electrode patterns for an ITO-based digital microfluidic through the finite element simulation. Micromachines 13, 1563 (2022).
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F. & Adam, H. Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the European conference on computer vision 801-818 (2018).
Wang, W. Precise Droplet Dispensing in Digital Microfluidics with Dumbbell-Shaped Electrodes. Micromachines 13, 484 (2022).

(Not answered)

ESI.docx
video1.mp4
Influence of droplet color and shape- video1
video2.mp4
AI-assisted multistate droplet control-video2
video3.mp4
AI-assisted multistate droplet control-video3
video4.mp4
AI-assisted multistate droplet control-video4

Download PDF

Editorial decision: revise
04 Mar, 2024
Review #3 received at journal
15 Feb, 2024
Review #2 received at journal
23 Jan, 2024
Reviewer #3 agreed at journal
17 Jan, 2024
Reviewer #2 agreed at journal
12 Jan, 2024
Review #1 received at journal
28 Dec, 2023
Reviewer #1 agreed at journal
21 Dec, 2023
Reviewers invited by journal
21 Dec, 2023
Submission checks completed at journal
21 Dec, 2023
Editor assigned by journal
20 Dec, 2023
First submitted to journal
20 Dec, 2023

You are reading this latest preprint version

An Artificial Intelligence-Assisted Digital Microfluidic System for Multistate Droplet Control

Status:

Version 1

Abstract

Figures

1 Introduction

2 Methodology

2.1 Encoder-decoder semantic segmentation model

2.2 Region growing algorithm

2.3 Automated control process based on droplet states

3 Experimental methods

3.1 Chip fabrication

3.2 Setup of DMF hardware

3.3 Dataset establishment

3.4 Model training parameters and evaluation metrics

4 Results and discussion

4.1 Model comparison

4.2 Influence of luminous environment

4.3 Influence of droplet color and shape

4.4 AI-assisted multistate droplet control

5 Conclusions

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1