Learning control of a laser-driven locomotive microrobot for dry environments

In this paper, we introduce the SerpenBot, a microrobot smaller than 1 mm in size, powered by laser energy, and designed to operate in dry environments. The microrobot achieves locomotion on a silicon substrate by selective coupling of laser energy between two legs that move the robot forward and steer it. The laser energy coupling mechanism is achieved by selective energy-time irradiation of the microrobot’s body and legs to initiate the gait. Complex multi-physics models developed in our past work are simplified to a differential drive kinematic model that approximates the robot behavior and is used to formulate a deep learning controller that can steer the robot to desired locations on the substrate. Simulations predict the robot position regulation achieved by deep learning of the inverse robot Jacobian. Experimental results are presented confirming that the microrobot can be automatically steered to a goal location using a simple PD controller.


Introduction
Microrobotics has seen considerable progress in the last 30 years.Similar to macro-scale robots, microrobots can walk, swim, fly, change their position and affect their environment.However, the vast majority of mobile microrobots operate in "wet environments", where the surface forces dominate their operation [1].The nonlinear nature of these surface forces makes it challenging to design, simulate, and fabricate the microrobots with controllable behavior.
Due to the size limitations, most sub-millimeter robots must harvest energy from the environment [1][2][3][4].The environment can also be utilized for the robot's control, as in the case of physical fields, for instance electrical and magnetic [1][2][3][4].The most popular energy coupling method to power and control microrobots has used magnetic fields which are particularly suitable for in-vivo medical applications.[5][6][7][8][9][10][11][12][13][14][15] Electric field was also used to control the behavior of bacteria-powered microrobots (BPM) [16].Acoustic actuation is another method utilized in recent studies [17,18] focused on motion control by tuning the frequency of the ultrasound.
Finally, in the last two decades, several research groups considered and employed laser light as a way to supply power to and realize control of robots (or structures in general) of various sizes and designs.The choice of laser light as an actuation method is dictated by its inherent and diverse properties, such as directionality, optical coherence, high efficiency of power density (intensity per area), control of size and shape of the irradiated area, and waveform-continuous or pulsed mode of operation.For example, RoboBee is a flying robot with integrated solar cells, which when irradiated with a laser, power piezoelectric actuators coupled with wings [19].A similar principle was applied in the micro-swimmer also using solar cells to set the robot into motion [20].In another study, Opto-thermal effects have been utilized for flow-addressed bubble (OFB) microrobots [21,22]; light-tweezer effects have been used to drive micro-gripper [23], Banerjee et al. and Shakoor et al. applied optical tweezers for manipulation of biological cells [24,25]; light-pressure and polarization effects were harnessed for micro-drone operating in liquid [26].Dong et al. proposed the idea of stimulating and controlling the muscle cells of a nematode worm with the help of a laser beam [27].In summary, current studies prove a good control of the microrobots operating in a liquid environment by different actuation means with potential major applications in medicine and automated fabrication of microscale structures.However, mobile micro-robotic systems in dry environments are rarely considered in recent studies since the optical power harvested is insufficient to overcome surface forces.Continued investigation of the dry environment micro-robotic systems would allow us to advance knowledge in the bionic research area, helping us better understand the behavior of microorganisms such as Pyroglyphidae (dust mites, feature size 0.2mm to 0.3mm ).Also, this can bring us closer to the realization of the micro-factory or lab-on-a-chip concept, with novel applications such as micro/nanoparticle transportation.
In this paper, we present a laser-driven microrobot, SerpenBot, designed for operation in a dry environment.The microrobot was fabricated from a Silicon-on-Insulator (SOI) wafer using a novel, two-layer Deep Reactive Ion Etching process.SerpenBot is utilizing an opto-thermomechanical conversion effect for initiating locomotion on dry surfaces.It is driven by microscale thermal actuators which are sequentially engaged when irradiated by a pulsed laser.The laser enables motion and steering control as well acting as a power supply, [28][29][30].In our previous preliminary studies, we have shown that steering control of microrobots in dry environments can be realized in two ways: 1) tuning laser frequency [30,31], and 2) controlling the off-axis location for selective irradiation of the robot's body.In this work, we focused on the latter method and experimentally validate driving and steering motions of SerpenBots toward a goal.The proposed control scheme to achieve controlled steering consists of a novel Neural -Network Learning Controller (NNLC) based on deep learning.The NNLC is initially deployed through simulation using the microrobot's differential drive model [32].The controller was then tuned using experimental data, and results are presented in this paper demonstrating its performance.Results confirm that SerpenBot can travel to goals for significant distances greater than 1 cm, with translational velocity in the range of 1 ∼ 100 m∕s.
This paper is organized as follows: Sect. 2 includes details about SerpenBot design and fabrication; Section 3 describes the steering strategy and its modeling.Section 4 introduces a control scheme based on neural network solutions; Sect. 5 includes details on laser experimental setup; in Sect.6 we present experimental results and discussion; Sect.7 is a summary of the main findings and discussion of future work.

SerpenBot fabrication
SerpenBot is a laser-driven microrobot that shares locomotion principles with its predecessor, the ChevBot [29][30][31].The SerpenBot has two asymmetrical thermal actuators, called Elbow Thermal Actuator (ETA), implemented as legs shown in Figs. 1 and 2. Actuation by stick and slip principle is achieved by laser irradiation on the microrobot, at selective locations, power levels, and pulse frequencies.When the laser radiates on the surface of the microrobot, cyclic thermal expansion of the actuators sets into the motion robot's legs.The actuation serpentine springs of the two legs can be designed with different spring constants, so these two actuators have different oscillation frequencies.When the modulated laser pulse is close to the actuator resonant frequency, the motion of the actuator will reach the maximum amplitude, which can be used to frequency control the turning behavior of the microrobot.Alternatively, by varying the center of the laser beam selectively onto each leg, turning behavior can also be achieved regardless of oscillation resonant frequency differentiation.In this case, the microrobot gait is achieved by the selective amplitude of vibration of each leg, coupled with the substrate using one or more dimples that double the robot thickness from 20 to 40 microns, and helps tilt the robot relative to the substrate.
Recently, a preliminary study [33], identified both theoretical and experimental methods to evaluate 7 different

C)
SerpenBot designs and led to the selection of best-fitting candidates for further investigation regarding steering control.Typical design parameters for the dimensions of the SerpenBot and top and side views of the robot are presented in Fig. 3.In our past work with SerpenBots, the fabrication process included an assembly step to connect the robot body and dimple with the help of the UV adhesive.Unfortunately, this introduced significant difficulties in maintaining robot dimensional precision and required delicate steps of aligning the dimple with the microrobot's body using a dedicated micro-assembly station [31,34].Moreover, during operation, we often experienced decomposition of the UV adhesive at elevated temperatures during laser irradiation.
To improve the fabrication yield, we recently developed a new assembly-less fabrication process in which SerpenBot's dimple is manufactured directly on the body of the microrobot in the cleanroom, thus eliminating the need for additional assembly steps and the use of UV adhesive (Fig. 1).The new fabrication process uses a Silicon on Insulator (SOI) wafer with 40 µm thickness device layer, and selectively etches away the device layer through 10 process steps depicted in Fig. 2 and detailed below: 1. 300 nm SiO2 growth as a protection layer for the dimple area by Plasma-enhanced chemical vapor deposition for 10 mins at 300 W power using an Oxford PECVD system.2).9. Removal of the photoresist through an NMP bath for approximately 5 min.10.Release the microrobot from the wafer substrate by vapor hydrogen fluoride using a uEtch VHF system set to 6000 etch cycles taking approximately 90 min of process time.
After completing the successful fabrication of Serpen-Bot, microrobot tethers are broken, and the robot is removed from the release die, and placed on a transfer die.Finally, it is transferred on a 2 or 3-inch Si wafer substrate (arena) for experiments.
The resulting microrobot has a body with approximate dimension 647 μm by 445 μm, with serpentine width 5 μm and 9 turns (Fig. 3B).This design is based on the consideration of the laser spot size, power, and pulse frequency, as detailed in our previous publications [28-31, 33, 34].The design is shown in Fig. 3.

Opto-thermo-mechanical conversion
The motion of the SerpenBot is governed by complex optothermal-mechanical phenomena, where optothermal absorption effect can be described by a first order differential equation, and the thermo-mechanical part can be represented by a second order differential equations, leading to an overall third order system.The opto-thermal behavior can be described by the heat equation [31]: where R is surface reflectivity, E e is laser irradiation, A l is the laser spot heating area on the robot, h is the thermal conductivity constant, A r is the thermal diffusion area, T ∞ is the environment temperature, is the material density, c is the material specific heat capacity and V is the volume of the robot.
The transient temperature change is the solution of the Eq. ( 1), the thermal expansion for given mechanical structure is the linear relation with the temperature as: in which, is the thermal expansion coefficient, L is the original length of the mechanical structure, dT is the temper- ature change on the structure.Since the mechanical structure of the ETA actuator is complex, but the motion of the leg of ETA relative to the body is relatively small, for simplicity, we can assume the force generated by the ETA is also the linear relation with the temperature which is F ∝ T.
To actuate the Serpenbot, we apply pulsed laser as the energy source.When the laser irradiates the actuator of the SerpenBot, its temperature would increase due to the optothermal effects, producing thermal expansion and ultimately generating stick-and-slip motion.
Our previous study on laser-driven microrobots [31,33,34] by means of Finite Element Analysis (FEA) revealed that their motion is highly dependent on thermal boundary conditions and intermittent contact with the substrate.Practically, it is difficult to use such models to design a robot controller, since they are complex and computationally costly to implement using FEA.The physics-based dynamic modeling of ChevBot shows that these microrobots can be described as complex nonlinear third order systems with many unknown or hard to measure parameters.Nevertheless, we can use a linear kinematic model to simulate the approximated behavior of the robot.Since the original system is third order, referring to [34], we have showed that the dominant pole of the SerpenBot is from mechanical vibration.To approximate the behavior of the legs of the robot, ( 1) we can use a second order system function which represents the frequency response of the actuators.

Differential drive kinematics
The simplified kinematics of the robot can be described as a mapping between leg velocities in a local frame and robot velocity in a global frame depicted in Fig. 4A, and represented by states ̇qR = ̇xR ̇yR T , ̇qG = ̇xG ̇yG θG T , in which ̇xR and ̇yR are the velocity of the robot legs in the local frame, while ̇xG , ̇yG , θG are the linear and angular velocities of the robot in the global frame.The relationship between ̇qR and ̇qG can be further described through the Jacobian matrix of the robot J rob as: Through numerical difference expansion and integration, we can approximate the robot location in a discrete-time fashion according to: where k represents a time step index, l is distance between the legs of the robot, and Δt is the sampling time.Since the motion of the leg occurs only along one direction parallel to X R , we can use scalars ̇xR l and ̇xR r to represent the robot right leg and left leg velocity.These velocities can be described as a second order system, a function of pulsed laser energy and frequency inputs as: where f n is the actuator resonant frequency, is damping ratio, K l is the gain of the laser, an empirical value, and I is the laser current.By describing the motion of the SerpenBot as a differential drive robot, if we can realize a scenario where each leg of the robot moves at different pace (different velocity), we can change the turning behavior of the robot.Therefore, our control strategy involves laser beam heating of selected actuator of the robot each time we want to drive different leg, and in a result to run the whole SerpenBot in a desired direction.We called this scheme depicted in Fig. 4B Right-Forward-Left control (RFL).Furthermore, an important quantity in our scheme is an offset distance Δl, which is a continuous parameter, defined as a distance between center of laser beam and geometrical center of SerpenBot, along axis Y R (Fig. 4C).Δl represents not only precise location of the laser spot relative to the microrobot's body along Y R , but also describes selective irradiation of the actuators.It is an essential control parameter for steering the SerpenBot. (5)

SerpenBot controller design
To develop a controller for the SerpenBot and implement the steering strategy, we must consider a scheme in which the laser spot position is synchronized with respect to the motion of the microrobot on the substrate.In our microrobot driving system, described in more detail in Sect.5, the substrate, also called the arena, is moved via two micropositioning stages under a fixed laser spot delivered from a 532nm green pulsed 2W laser unit.Thus, our control system consists of two main parts: 1) visual servoing for laser spot control and 2) steering microrobot to accomplish goal position control (Fig. 5).Since the SerpenBot is moving on the arena and the position of laser spot is fixed, the visual servoing system is continuously keeping selected parts of the robot under the laser beam (Fig. 4B).Furthermore, the microscope camera provides feedback to always keep the robot legs and body irradiated by the laser in the center of the field of view.The robot steering behavior control system consists of PID Neural Right-Forward-Left (PID-N-RFL) scheme that via motorized stages ensures irradiation of selected microrobot's actuators (left actuator, right actuator, or both), while the varying intensity of the laser beam (laser gain-laser diode current) can be used to change the turning radius of the robot.To describe our controller, we refer to Fig. 4A showing what the microscope image (camera's field of view (FOV)) of the robot will look like, while the overall concept of controller design is illustrated in block diagram from Fig. 5.

Visual servoing
A visual servoing controller is necessary to track robot during its motion by keeping it irradiated, e.g., within the same area in a FOV of a camera, with the help of motorized x-y stages located under the robot arena.In this arrangement, control error is the difference between robot current location in pixel coordinates s P r , and desired point in FOV of camera-center of the FOV (center) s P c .Error mapping serves as a control value for the x-y stages and is realized with the help of inverse image Jacobian matrix J −1 img for transformation from pixel space to Cartesian space of x-y stages.(Fig. 5).
The image Jacobian matrix is described as follows: We can identify the image Jacobian matrix using recursive Broyden method [32,[35][36][37], by acquiring set of randomly selected points based on the motion of the motorized x-y stages and recording from the smart camera.The Jacobian identification procedure is accomplished in the following steps: 1. Initialize 1 st version of Jacobian matrix J img,k .2. Acquire initial robot location in camera system of coordinates (SC) s P,k and motorized stage SC q G,k using cam- era feed and stage's built-in position sensor.3. Generate a random set of points on arena and using motorized stages displace robot to each of this point in sequence determining coordinates in xy stage's and camera's SCs respectively q G,k+1 , and s P,k+1 .4. Calculate Δs P,k and Δq G,k for each point on the arena.
After identifying Image Jacobian matrix, we implemented a proportional (P) controller, determining the final control equation: where K img is the gain of the control value for visual servo- ing and s P c is the center coordinate of the camera in camera space.

Homogenous coordinates transformation for action switching
Through visual servoing we establish a method for centering the microrobot in the microscope field of view, which coincides with the center of the laser spot.We can then apply the visual servoing strategy to switching the location of the laser spot in different parts of the robot in particular on each of the Serpenbot legs.We employ a homogeneous coordinates transformation to redirect the laser spot onto desired parts of the SerpenBot for steering control.Figure 6 shows a (7)
where K v is the gain for linear velocity, PID parameter including K ,P , K ,I , and K ,D are the gain for the angu- lar velocity, The PID-N-RFL controller is very flexible with respect to applications, as it only requires one discrete variable as tag for decision making on what part of the robot to apply the laser spot, and one continuous variable for velocity control.To validate this controller, we began by selecting the laser current as the continuous control variable determining velocity and conducted simulations.However, through experiments we later found that the offset distance is a better choice of control variable since the SerpenBot's angular velocity is more responsive to this variable.
We d e f i n e t h e c o n t r o l v a r i a b l e C a s b e l o n g i n g t o a s e m i -d i s c r e t e d o m a i n , C ∈ {c|A ⊗ K, A ∈ {−1, 0, 1}, andK ∈ [0, 1]}, including discrete actions A (RFL) and normalized continuous laser current K .We propose to use a neural network to approxi- mate the robot inverse Jacobian J −1 rob as a bridge mapping the continuous value of the PID controller to be semi-discrete space of the actual control value which is The neural network approximating J −1 rob , is represented by diagram shown in Fig. 7.
The structure of our neural network is described in the diagram below (Fig. 7B), highlighting input layer with linear and angular velocities, activation functions tanh, and activation functions softmax and sigmoid for discrete and continuous control respectively.

Neural network-simulation and learning method
The choice of a learning method for our neural network controller, was dictated by characteristics of SerpenBot's steering behavior.We have mentioned previously that SerpenBot is a highly non-linear dynamic system with many unknown or hard to measure parameters, and for which experimental conditions (e.g.complex surface effects) have a time varying character.It was reported [33,34], and it will be discussed later in experimental section of this paper, that the behavior of the SerpenBot is not always repeatable at different locations on the substrate -especially in the case of continuous parameters such as laser gain and offset distance.When manually controlling the robot, a human operator continuously adjusts these parameters in order to achieve desired responses from the microrobot.With an autonomous controller, its operation will involve decision making process regarding specific action (left, right forward) and adjustment of the continuous parameter (laser gain/offset).
Based on these requirements, supervised learning was chosen to train a neural network controller [39] with tags defined as right, forward, and left (RFL) representing discrete parameters and the laser gain values as a continuous parameter for controller simulation, and offset distance during experimental testing of the controller.For training the system, these tags, and the normalized laser current (laser gain) are randomly selected as actions (Fig. 7B).After applying actions to the system, the system output is related to the microrobot state velocity ̇qG , expressed as a pair with scalar velocity v G = √ ̇xG 2 + ̇yG 2 and angular velocity  G = θG of the robot is sent back to the neural network.At the end of training process, the neural network acts as a classifier finds the occurrence probability of the input actions.However, during the training process, the difference of the tags and output probabilities with the laser gain will back propagate to update the parameters of the neural network.Finally, the neural network learns to get the current states of the microrobot as input and finds the laser behavior RFL as output.That makes the neural network J −1 rob a classifier as well as a tool to calculate the inverse kinematics of the robot.
In practice, we constructed our neural network using PyTorch (® open-source library with two hidden layers of 128 neurons each and selected "tanh", as activation function for the hidden layers, and selected a learning rate of 0.00017.The output layer includes four outputs: right, forward, and left (defined tags) as well as offset relative to the center of the SerpenBot in pixel.The activation functions for the NN output was "softmax" for discrete action switching, and "sigmoid" for the continuous laser gain in case of simulations and offset distance during experimental testing of the nn.(Fig. 7B).
Figure 8 presents training results of the proposed NN scheme.For this process, we initially used the kinematic model to train the neural network, then through transfer learning, we continued training process of the neural Fig. 8 The loss functions during the learning process represented by blue dots after 70000 training points.The red line is the average loss during learning network with experimental data collected during steering trials.Since a differential drive kinematic model is significantly idealized, it is critical to apply transfer learning based on respective experimental data collected with the given SerpenBot, to realize simulated control of the Ser-penBot as proof of concept (Fig. 9).Learning of our NN was realized offline, using previously collected experimental data for the given SerpenBot of specific design.Therefore, it cannot be translated for the control of the robot with different design-different geometry of the serpentine actuators or dimple location (frame or legs).
As a loss function we have used Mean Square Error MSELOSS() from PyTorch library which creates a criterion that measures the mean squared error (MSE) between each element in the input and target: where x is the given action with laser gain (or offset), and y is the neural network's output laser gain (or offset).Main reason behind application of MSE as loss function is its ease of interpretation and application within neural network, since it produces one accumulated value representing overall ( 14) prediction error.Furthermore, MSE is continuous and differentiable which makes it suitable as a measure in case of the continuous parameters -laser gain or offset.
Results show that the loss function converges to significantly smaller values under the pre-described threshold, demonstrating the promising ability of the neural network to model the robot inverse kinematics from image data (Fig. 8).Nevertheless, there is significant number of outliers, which could be due to the fact that in learning process has been utilized drop learning method to avoid overfitting.
We make a few additional clarifying notes regarding our training process.First, during training, we did not present the robot with ambiguous turn commands by more than 180 degrees.As a result, the learned behavior will always minimize the absolute value of the turning angle, e.g.pick turn left by 30 degrees as opposed to turn right by 330 degrees.Therefore, minimizing the orientation error would determine the optimal direction of the rotation-clockwise or counter-clockwise.
Second, while training our neural network we have simulated the robot's steering mechanism based on three distinct states: left, forward, or right depending on the laser irradiation of the selected part of SerpenBot and continuous parameter laser gain (Fig. 9).Simulation results show that we were able to realize three different trajectories necessary for microrobot control: clockwise/counterclockwise revolutions, and translational motion.As it can be seen in the Fig. 9B, C for a given action value (-1, or 1) SerpenBot follows curved trajectory turning clockwise and counter clockwise respectively.
Noticeable asymmetry of the trajectories 1 and 2 could have been most likely caused caused by our simulation approach, in which we approximate conditions during experiment by introducing asymmetry to the SerpenBot model, varying geometry of the left/right serpentine actuators -different dimensions of the features (Fig. 1C).The purpose was to test whether this design asymmetry would still allow realization of our steering method at the simulation stage of our study.On other hand, goal coordinates for point 2 are not symmetrical to goal coordinates #1 with respect to vertical axis which might be additional cause for the asymmetry between trajectories 1 and 2.
Simulation served as proof of concept for our NN controller, ensuring feasibility and experimental realization in realistic settings.However, as it will be discussed in the next chapter, the implementation of the NN based controller for experimental testing led to changes in the choice of the continuous parameter.Specifically, for the experimental tests of our NN based controller, laser gain parameter was replaced with offset distance, which produced more reliable turning behavior.

Experimental setup
A schematic view of the experimental setup to drive the SerpenBot is depicted in Fig. 10.As source of laser light for microrobot's actuation, we have used high power Nd:YAG pulse laser (Spectra-Physics Explorer® one) with wavelength of 532nm .The pulse frequency can be adjusted between 0.5 ∼ 60kHz , the pulse time width and average out- put power can be adjusted between 10 ∼ 40ns and 0 ∼ 2W respectively.Laser beam diameter (waist diameter) at the surface of arena was set to 400 μm.Position of the laser spot can be controlled with precision of 4 μm -this is a size of a single pixel in field of view of the NI smart camera.A variable neutral density filter (NDF) followed by the Uniblitz optical shutter was used to attenuate laser beam during experiments.A system of two lenses adjacent to NDF allows adjustment of the size of laser spot and position of the laser beam is adjusted by varying tilt of 4 mirrors (M1-M4).Two cameras integrated with beam splitter were attached to two tube lenses: one, a National Instrument smart camera (ISC-1772C) was used for visual servoing and equipped with optical notch filter to block laser light (532 nm); second, a Pixelink PL-D734, was used as real time visual feedback for SerpenBot.The microrobots are placed on an arena consisting of a 3-inch Si wafer, which was secured on a sample chuck and positioned on top of four cascaded linear stages.Two of the stages are manually controlled for calibrated adjustments, while the other two are motorized (Physik Instrumente PI Q-521), controlled by National Instrument LabVIEW, and used for automated tracking and control of SerpenBot's motion.A custom LabVIEW user interface (UI) allows adjustment of the laser spot position relative to the SerpenBot's body thus allowing irradiation of the selected actuator and results in automated or semi-automated steering of the robot (Fig. 6).

Neural network training process and controller testing
In Sect.4.1, we utilized the approximate kinematic model of the SerpenBot to validate our PID-N-RFL controller in simulation.We then utilized the real experimental data to train the forward and inverse kinematic model of the robot to validate the concept and then applied it to experiments with the robot.To efficiently collect data, we utilized the Monte Carlo method generating random action sequences (left -1, forward 0 or right + 1), offsets, and a random period of time as inputs to the robot.The velocity of the robot ̇x was calculated by backward differentiation from microscope images using: where Δt = t k+1 − t k , corresponds to time difference between two consecutive points (positions) recorded during robot's motion, where xk and xk + 1 are the positions of the Ser-penBot, at respective moment of time tk and tk + 1 during the motion.
The PID-N-RFL controller utilizing offset as a control variable is depicted in Fig. 11.Specifically, the control variable where A is the action representing decision process to irradiate the laser beam on the left side or the right side of the SerpenBot, s P o is the offset Δl relative to the center of the SerpenBot measured in pixels.In our experiment the maximum offset was 100 pixels, based on specific magnification and size of our robot, and the neural network type and size were similar to the simulation case (Sect.4.3).

Experimental results and discussion
We conducted a series of experiments to verify the proposed steering strategy involving selective actuation by irradiating specific parts of the SerpenBot.In a first stage of experimentation, we tested whether selective irradiation of the left or right or both actuators would result in an expected and repeatable behavior-motion in a specific (15) ̇xk+1 ≈ x k+1 − x k Δt direction.In a second stage of experimentation, we have driven SerpenBots along a planned path of specific shape.And, in the third stage of experimentation, we demonstrate automatic goal attainment by direct action with the PID-N-RFL controller.

SerpenBot steering control through selective irradiation
We conducted a series of experiments in order to verify our steering method.Using a user interface written in LabVIEW, we irradiated specific parts of the SerpenBot by manual laser spot control and observed the resulting behavior of our microrobot.Results show that we can realize clockwise and counterclockwise rotation depending on which actuator was under the laser beam (Fig. 12).
As it can be seen in Fig. 12E, recorded trajectories are elliptical in shape and for clockwise motions, we have noticed significant drift of the center of rotation.We presume that the main reason behind this is the fact that experimental data presented in Fig. 12 was collected for the SerpenBot with asymmetrical geometry of the actuator structures.Furthermore, also experimental conditions have to be taken in account, such as fabrication defects of the SerpenBot and surface effects of the arena.Nevertheless, motion is repeatable and relatively stable, with respect to uninterrupted propulsion through laser irradiation, making it possible to realize number of revolutions continuously (Fig. 12C, D, E, F).Combining these three types of Ser-penBot's motions allows us to direct the microrobot to a desired location.Hence, we verified our steering strategy that enables future experimental implementation of the proposed control scheme based on the neural network.

SerpenBot steering control through selective irradiation -motion along specific trajectory
For this experiment, we wanted the microrobot to follow a rectangular shape trajectory.The position of the laser beam relative to SerpenBot's body was manually controlled with the help of a LabVIEW UI. Figure 13 presents the recorded trajectory of the SerpenBot along an approximately rectangular shape.In this experiment, the laser frequency was kept constant at 1700 Hz with laser power of 430 mW, and we used the laser's burst mode with 50 pulses per burst, and a delay time between bursts of 300 ms.
To better understand proposed steering mechanism the trajectory shown in Fig. 13 can be described in the following way: 1. section AB and BC -microrobot is initially at rest and starts to move forward along straight line upon exposure to a laser light focused on the center of SerpenBot, as depicted in bottom diagram in Fig. 13.
2. point C -robot turns left after the laser beam was focused on left actuator of the SerpenBot, see top diagram in the plot, Fig. 13. 3. section CD -laser beam is focused back on center of SerpenBot which moves in forward direction; around ¾ of the CD section robot starts to drift slightly to the left.4. point D-robot turns left after the laser beam was focused on a left section of robot. 5. section DE -laser beam is focused on center againrobot moves in forward direction.6. point E -robot turns left again.7. section EB and BF -finally, the beam is focused on a center of the robot; at point B it intersects section AC and stops at F, after the laser beam is shut off.
Applying the same steering strategy, we were able to move and control SerpenBot of other designs along the trajectories of different shapes (Fig. 14).For rectangular, triangular, and trapezoidal shapes of paths (Fig. 14B, C, D) we followed the same receipt as in case of trajectory from Fig. 13, switching position of the laser beam depending on the part of the motion.However, for the circular trajectory (Fig. 14A) we have kept laser beam focused on one of the actuators continuously during the whole travel (Fig. 11).
Trajectories shown in the Fig. 14B, C, D are less stable than "circular" one (Fig. 14A), which is related to the fact that during motion along the straight path, robots tends to uncontrollably turn, thus continuing adjustments has to be made during the motion by human user inputs with the help of LabVIEW UI.These adjustments include switching the position of the laser beam from one side of the robot to the opposite in order to compensate for the uncontrollable turns to keep it traveling along the main trajectory.

Neural controller training and experimental validation
Using a Monte-Carlo approach to generate microrobot input action, we collected image sequence information and used it to train the inverse neural network approximation of the robot Jacobian.The input signals and output states of the SerpenBot are shown in Fig. 15, collected for a duration of over 10 min.It can be seen that when the input is a positive offset Δl, the angular velocity is also positive, while vice versa, a negative offset produces a negative value of the angular velocity.Furthermore, the loss function during the training process is shown in Fig. 16 and converges toward zero after 140, 000 data points, which corresponds to training session duration of approximately 25 min.Plot in Fig. 16 has multiple peaks as this is multi variable combination and episode training process, each episode will converge to zero gradually, as well as average loss function.Each new episode at specific moment of time involves change of the action and offset value which causes initial sharp increase of the loss function and subsequently its convergence to zero.
After NN training was completed, we applied the PID-N-RFL controller to realize automated steering of the microrobot.For that purpose, we conducted a series of It can be clearly seen that the path includes several relatively straight lines ( t 0 , t 1 ) and curved sections (arcs,t 0 , t 1 , t 3 , t 4 , t 5 ) with various arc radii.The behavior of the microrobot while traveling along first path (Fig. 17A) was recorded in the plots of Fig. 17B showing changes in time of the robot's controller motion parameters (offset, robot's orientation angle, angular velocity, and error) which correspond to specific section of the trajectory and respective time interval ( t 0 tot 6 ).The plots provide insight into our controller's operation as discussed below.1.In the first section of the trajectory (time interval t 0 ), SerpenBot travels along two arcs with large radii, and short straight path -first turning sightly clockwise then counterclockwise, which is represented by initial decreasing trend of the angle vs time and negative value angular velocity (Fig. 17B) (around 10s ∼ 20s ), then slight increase of the angle value -positive value of angular velocity ( 20s ∼ 25s ), and plateau with con- stant value of angle and angular velocity around zero ( 25s ∼ 30s ).In the meantime, the value of the offset is constant for the t 0 (Fig. 17B), and the error value is decreasing, as microrobot is moving towards goal point.2. At around 30s of the motion, at the beginning of the sec- ond section t 1 , SerpenBot takes a sudden turn clockwise, away from the goal, where angle is increasing, and angu-lar velocity is positive with a rising slope of the distinct peak ( ∼ 30s ).The controller introduces correction in the motion, by switching the offset from around positive 20 to -40 (Fig. 17B), consequently we observe that Ser-penBot takes a sharp turn counter-clockwise (Fig. 17A), as angle is decreasing with the time -falling slope of angular velocity distinct peak ( ∼ 30s).A similar pattern in SerpenBot's behavior can be observed for the time intervals t 3 , t 4 , t 5 .-similar shape of the trajectory (Fig. 17A).Sudden deviation of the microrobot from the approximate path leading towards goal, is corrected by switching the offset (laser beam position), which steers the robot towards the desired direction.5. Finally, SerpenBot reaches the goal location within the error tolerance after approximately 90 s and recorded an average speed of approximately 100 m∕s.
In Fig. 18A, B we have tested additional trajectories verifying controller's functionality for different goal points.By trial-and-error tuning of the PID gain values of our controller, we were able to find K v = 1 , K ,P = 10 , K ,I = 0.001 , and K ,D = 0.1 that seer the motion of the SerpenBot to the desired goals point.In Fig. 18A, the initial configuration was [0, 0, 6.188] and the goal was [−5, 5, 5.790] , while for Fig. 18B, the initial configuration was [0, 0, 2.280] and the goal was [−5, −5, 5.408].
In both cases microrobot reaches the vicinity of the goal coordinates (Figs.17, and 18).Because we used a PD controller to track our trajectories, the final steady state errors to the goal during motions recorded in our experiments varies from 0.1 mm to 0.5 mm.These values depend on robot conditions (fabrication defects) and environment (arena surface, system properly calibrated and clean).

Conclusion
In this paper we presented a novel design of assembly-less laser driven microrobot on dry surfaces, SerpenBot, along with new fabrication process, which simplified assembly process and optimized experimental process with laser irradiation.We have proposed new steering strategy for SerpenBot based on the selective irradiation of the microrobot's body, which was verified experimentally.Experimental results show that we have a sufficient steering control enabling realization of various planned trajectories.At average linear speed between 10 ∼ 100 m∕s .It is suggested that proposed steering mechanism is a robust approach which allows overcoming the difficulties related to complex surface effects at microscale in dry environments.We experimentally demonstrated controlled motion of the SerpenBot in all direction on the flat plane -Si substrate.Furthermore, we have conducted simulations of the steering control scheme based on the implementation of the neural network solutions, developed, and experimentally verified PID-N-RFL controller by demonstrating microrobot's motion in desired direction realized autonomously without major human user intervention.In future, we plan to refine our controller, optimize its operation, and continue experimental trials to accomplish SerpenBot's travel along more complex trajectories.Finally, we will integrate a galvanometer in our experimental set up which

Fig. 1
Fig. 1 SEM images of the assembly-less SerpenBot.A general view of the tethered microrobot, B fabricated circular dimple, C serpentine structures of the robot's actuator

Fig. 2 Fig. 3 A
Fig. 2 Flow chart of the cleanroom fabrication process for the assembly less SerpenBot

Fig. 4
Fig. 4 SerpenBot steering scheme based on selective irradiation.A kinematic model of the SerpenBot B.1 clockwise (counterclockwise) motion, B.2 forward (backward) motion, B.3 counterclockwise (clockwise) motion.(Red arrows represent orientation and direction of the motion.Dashed circles indicate which actuator pair and leg is

Fig. 5
Fig. 5 Proposed concept of PID-N-RFL controller design

Fig. 6
Fig. 6 Laser spot switching using visual servoing for microrobot steering.The laser spot to the left in each figure is a specular reflection from camera lens and substrate, and not a second laser spot

T
is proposed for motion control of SerpenBot in order to reach a commanded goal location on the substrate.The PID controller maps the error between current configuration x G c by setting the robot velocity in polar coordinateṡqG C = v G c  G cT , including its forward velocity v G c and turning velocity G c as: e y = y G c − y G d and e = G c − G d , are the controller position and orientation errors, and G d is defined as the desired directional angle toward the goal:

Gd=
atan2 e x , e y .

Fig. 7 A
Fig. 7 A Learning scheme of the SerpenBot's inverse kinematics by applying a set of random action and gains, collecting batch data, and fitting an inverse Jacobian mapping.B architecture of neural network

Fig. 9 A
Fig. 9 A simulation results of the controller reaching random goal locations, B, C, D the control parameters for reaching the goal locations in A

Fig. 10
Fig. 10 Schematic and actual image of experimental setup for SerpenBot steering control

Fig. 12 Fig. 13
Fig. 12 Experimental results representing three basic types of SerpenBot motion -on the left trajectories; on the right recorded position (XY) and angle variation with respect to time: A, B translational motion, C, D counterclockwise rotation, E, D clockwise rotation

Fig. 15 Fig. 16 A
Fig. 15 Angular velocity and input action sequence

3 .
After robot's course was corrected and set towards the goal -around the middle of the t 2 section, the controller switches offset value back to the positive ( ∼ 20s ), where SerpenBot move along approximately straight line, orientation angle is slightly decreasing, with minor fluctuation of the angular velocity around 0. Error value keeps decreasing as microrobot is closer to the goal.

Fig. 17 A
Fig. 17 A Trajectory of the SerpenBot with the PID-N-RFL controller.B Controller output sequence and SerpenBot angular displacement and angular velocity

Fig. 18
Fig.18 Additional experiment results with different starting orientations (B) or goal configuration (A).The steady-state controller error was between from 0.1 to 0.5 mm due to the surface conditions and visual servoing steady-state error diagram of our SerpenBot under the camera field of view, and we employ visual servoing to make the laser spot overlap with the geometric center of the robot, to the left leg center, and right leg center.Assuming a laser spot offset sP o T in normal camera space.Finally, to move the laser beam location to the desired spot, we simply implement Eq. (8).