Design of Pill Filming System and Automatic Pill Classication System

Background Invent Pill classication system that can detect and classify into one tablet unit by deep-learning and Pill lming system that generate comprehensive and multi-dimensional data for learning. Methods Pill lming system and Pill classication system, they have two chapters consisted of structure design, model introduction, and controller design. Pill classication system's structure categorization is Input box, Linear conveyor, and Output box. In Data preprocessing, a similarity map is obtained with Structure Similarity Index Measure(SSIM). And RetinaNet is used as a pill classication learning model. Mean Accuracy Precision (mAP) is 0.9842, and we take experiment about measuring the number and accuracy of the classied pills for each experimenter's classication time. Conclusion Pill lming system and Pill classication system are expected to reduce labour losses for simple tasks. It helps medical personnel focus on signicant and urgent tasks. And It can contribute to experiments about that deep-learning control the mechanical device.


Background
The hospital's drug department provides medicine to the patient according to the doctor's prescription.
However, depending on the patient's condition, the prescription changes frequently occur, resulting in prepared but not administered drugs [1][2][3]. These drugs are recovered and require reclassi cation [4]. The reclassi cation process is currently carried out through manual tasks directly checked by two or more pharmacists or nurses [5]. This process results in hours of labour loss for simple work [6].
Recently, COVID-19 broke out and increased the use of drugs in hospitals and also the number of recovered drugs [7]. It results in not satisfying the demand for reclassifying and spotlights the effectiveness of medical personnel's labour [8].
There are devices named pill classi cation in the market, but it can only classify boxes or pack units. Co.
Emtros patented about Classi cation apparatus of recovery pills in 21014. However, it was not based on deep-learning but can recognize based on class with shape: circle, ellipse, and capsule and with color: ivory, red, sky blue, orange, black, white, and blue [9]. So it has problem with classifying pills, which have similar look. Therefore there is never system that can detect and classify just one tablet by deep-learning. Therefore, a deep learning-based pill classi cation system is needed to classify pills automatically to reduce labour losses for simple tasks. It helps medical personnel focus on signi cant and urgent tasks.
Deep learning technology increases the effect of more data available for learning [10]. Malaya university had experiment about pill identi cation using deep convolutional network. It used 10 ~ 25 pictures for each pill, but it is not enough amount of data for learning and taking pictures of pills by hand is cumbersom [2]. However, securing large amounts of data sets is not easy [11]. As a result, many attempts operate to augment data through simple data expansion techniques such as rotation, reversal, and ltering [12][13]. However, these techniques are limited in scalability because they use only the data they already have. Therefore, to classify high-accuracy pills, Pill lming system is proposed to generate comprehensive and multi-dimensional data through the operation of motor-controlled-based seesaw structures. This paper presents the Pill lming system and Pill classi cation system in two stages, and they consist of struct design, model introduction, and controller design. Figure 1 is the prediction result of the pill image captured by the Pill classi cation system's camera. To verify the performance of the Pill classi cation system, we experimented with measuring the number and accuracy of the classi ed pills for each experimenter's classi cation time. The experiment was conducted on 100 of the ve pills used in this paper. In Fig. 2, when each graph ends, the speed was fast in the order of pharmacist, nurse, Pill classi cation system, and non-medical. However, the slope of the graph means that the device was the most constant. That is, the variation of concentration was slight.

Results
Also, the accuracy of pill classi cation by experimenter is shown in Table 2. The accuracy of classi cation by the pill classi cation system was the highest at 0.99. Therefore, it is more e cient to classify pills with Pill classi cation system when classifying large amounts of pills.

Conclusion And Discussion
Pills recovered to the pharmaceutical department require much labour to reclassify. COVID-19 broke out and spotlighted the effectiveness of medical personnel's labour. There are devices named pill classi cation in the market, but it can only classify boxes or pack units.
To address this problem, we suggest Pill classi cation system that can detect and classify just one tablet. Deep learning-based automatic pill classi er can solve this, but it needs image data taken from various angles. Therefore, in this paper, we also designed Pill lming system that can generate multiangle and multi-dimensional image data to be learned by an automatic pill classi cation system. Figure 3 shows fabrication of the Pill lming system and Fig. 4 shows fabrication of the Pill classi cation system we produced.
We take two experiments, one is about the number of classi ed pills per time(second), and the other is measuring accuracy. This system has the speed for running equally without acceleration and makes user calculating the time for a speci c amount of pill reclassifying. It has the highest accuracy at 0.99 in the experiment with pharmacist, nurse, and non-medical. But it is just that one person of job-pool taken and it has potential difference between personal. It can be supplemented by more experiments. From g.6, there are hills inside the Seesaw, and when the motor activates, the hill randomly changes the way the pill looks at the camera. When the pill places under the camera, the camera shoots the pill. The Pill lming system's overall size is 195 mm × 189 mm × 206 mm, and it weighs 0.52 kg, not including the pill. This system was manufactured with PLA using a 3D printer for a lightweight and easy manufacturing process. Fig.7 and Fig.8 show that the Pill lming system consists of power supply, board, motor controller and motor [14]. The motor controller consists of motor drive L293D, which can control the motor and the DC motor provides a major force to the device. The DC motor's speed is 15000 RPM, and the Gear Reduction Ratio is 150:1, and consequently, the number of rotations is 100 RPM. The motor speed was controlled by setting the Dutycycle to 13%, thus obtaining approximately 14 images per minute.

GUI Design
User interface was implemented using QT designer, as shown in g.5. Once you click on the UI button, 300 images like g.10 place in each folder.

Pill Classi cation System
Structure Design The Pill classi cation system consists of Input box, Linear conveyor, and Output box with different heights in Fig. 11, and they operate in regular order. User through unclassi ed pills into the Input box, it put off one pill on the Linear conveyor. Linear conveyor puts the pill under camera for classi cation and takes the pill to Output box. The Output box turns as tting the pill, and then the Output box gets the pill inside case. Step motor which is xed in Baseplate control Turnplate to turning. The Turnplate has six Pill case and each Pill case match for pills.
Baseplate xes every parts' position and has a control board inside. Pillar (A) suspend Pillbox, and Pillar (B) suspend the Camera holder.
All part excepting Baseplate was produced with material PLA using a 3D printer for the easy manufacturing process and Baseplate is made of aluminum. The Pill classi cation system's overall size is 680 mm × 220 mm × 302 mm, and it weighs 9.08kg, not including the pill.

Deep-learning Data
As shown in Fig. 13, a total of 5 pills are used from Gachon University Gil Hospital in the data set. After obtaining 300 images of multi-dimensional pills for each type by operating a manufactured pill lming system, 240 are used for training data, and 60 are used for validation data.

Data preprocessing
The image annotation process is performed to extract the pill that is the region of interest from the obtained pill image. After adding pill images for each type, average them to make a reference (background image). A similarity map is obtained with Structure Similarity Index Measure(SSIM), a structural similarity index that measures image quality. It is a structure that estimates the region with the lowest value by calculating the local Norm value corresponding to 170 pixel x 170 pixel in consideration of the pill size.
To speed up the operation, wavelet transform is on the similarity map to remove horizontal, vertical, and diagonal components, and then extrapolation to make the original size. Also, when calculating the local norm, the process of skipping the pixels exceeding the reference value is performed.

Modeling & Learning
In this paper, RetinaNet is used as a pill classi cation learning model, and it is selected as a model capable of simultaneous detection and classi cation in consideration of the differences in the environment of pill lming system and pill classi cation system.
RetinaNet is an integrated network consisting of a ResNet-based Backbone, and two subnetworks, a class subnet and a box subnet, as shown in Fig. 15 [15].
The epoch used for training is 100, the batch size is 12, and the learning rate is 0.00001. Fig.16 is a block diagram of Pill classi cation system, and Fig.17 is a circuit of Pill classi cation system.

Controller Design
Control board gets voltage 5v from power supply and makes system operating [14].
In Input box, the board sends signal about angle, which control servo motor for a pill dropped and gets detector switch's input value to nd out when the pill touches the detector switch.
External power supply gives voltage 12v to Linear conveyor's motor drive, L293D, which controls DC motor. The DC motor in the Linear conveyor provides a major force to conveyor belt for running. The DC motor's speed is 15000 RPM, and the gear reduction ratio is 150:1, and consequently, the number of rotation is 100RPM.
Output box has the same motor drive L293D, and it gets voltage 12v from the external power supply. The motor drive controls step motor, which operates 1.8° per 1 step. Fig. 18 shows the algorithm ow diagram of the executable code for pill classi cation system. By controlling the servo motor from 25° to 130°, hit the structure with a constant jaw between the input box and the linear conveyor, and drop the pill from the input box one by one. The frequency was set to 50 Hz and 30 % duty-cycle speed, controlling the DC motor through PWM control, and rotating the motor once to position the pill in the camera frame at the top. After that, the camera is captured by the detector switch's input signal, followed by detection and classi cation. As the predicted result, the step motor is controlled based on 60° between each pill container so that the container corresponding to each pill comes to the end of the linear conveyor. By setting the frequency to 50Hz and the duty-cycle to 65%, the DC motor is rotated three times through PWM control to operate the linear conveyor. Then the pills are dropped into the appropriate pill case. After all this process is over, it goes back to its original position.

Declarations
Ethics approval and consent to participate This article does not contain any studies with human participants or animals performed by any of the authors.

Consent for publication
This article does not contain patient data.

Availability of data and material
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.