Pill filming system
Structure Design
Fig. 5 shows that DC motor, Motor cap, Columns place on Base plate, and Crank connects with Seesaw and Camera guides support Camera holder.
The device operates with converting the DC motor's rotary motion to linear motion through the Crank, and it makes Seesaw running. When the Seesaw operates, the Columns next to each side of the Seesaw fix the Seesaw's motion axis.
From fig.6, there are hills inside the Seesaw, and when the motor activates, the hill randomly changes the way the pill looks at the camera. When the pill places under the camera, the camera shoots the pill.
The Pill filming system's overall size is 195 mm × 189 mm × 206 mm, and it weighs 0.52 kg, not including the pill. This system was manufactured with PLA using a 3D printer for a lightweight and easy manufacturing process.
Controller Design
Fig.7 and Fig.8 show that the Pill filming system consists of power supply, board, motor controller and motor[14]. The motor controller consists of motor drive L293D, which can control the motor and the DC motor provides a major force to the device. The DC motor's speed is 15000 RPM, and the Gear Reduction Ratio is 150:1, and consequently, the number of rotations is 100 RPM. The motor speed was controlled by setting the Dutycycle to 13%, thus obtaining approximately 14 images per minute.
GUI Design
User interface was implemented using QT designer, as shown in fig.5. Once you click on the UI button, 300 images like fig.10 place in each folder.
Pill Classification System
Structure Design
The Pill classification system consists of Input box, Linear conveyor, and Output box with different heights in Fig. 11, and they operate in regular order. User through unclassified pills into the Input box, it put off one pill on the Linear conveyor. Linear conveyor puts the pill under camera for classification and takes the pill to Output box. The Output box turns as fitting the pill, and then the Output box gets the pill inside case.
Fig. 12 shows each part’s name in Pill classification system. In the Input box, Pillbox gets lots of pills inside, and Servo motor under Pillbox hit Slider vertically. Slider has six Threshold which is different from each other, so when Slider gets hitting, pills slide according to Slider’s way, and one pill throw out itself onto the Linear conveyor (A). Detector SW has position Slider’s terminal.
When the pill attaches to Linear conveyor (A), DC motor at the Linear conveyor's side runs to move the pill under Camera holder. After classification about what pill is, the Output box turns as fitting to that pill’s Pill case. Then DC motor makes the pill dropping at Linear conveyor (B), and the Output box turns to its original position.
Step motor which is fixed in Baseplate control Turnplate to turning. The Turnplate has six Pill case and each Pill case match for pills.
Baseplate fixes every parts’ position and has a control board inside. Pillar (A) suspend Pillbox, and Pillar (B) suspend the Camera holder.
All part excepting Baseplate was produced with material PLA using a 3D printer for the easy manufacturing process and Baseplate is made of aluminum. The Pill classification system's overall size is 680 mm × 220 mm × 302 mm, and it weighs 9.08kg, not including the pill.
Deep-learning
Data
As shown in Fig. 13, a total of 5 pills are used from Gachon University Gil Hospital in the data set. After obtaining 300 images of multi-dimensional pills for each type by operating a manufactured pill filming system, 240 are used for training data, and 60 are used for validation data.
Data preprocessing
The image annotation process is performed to extract the pill that is the region of interest from the obtained pill image. After adding pill images for each type, average them to make a reference (background image). A similarity map is obtained with Structure Similarity Index Measure(SSIM), a structural similarity index that measures image quality. It is a structure that estimates the region with the lowest value by calculating the local Norm value corresponding to 170 pixel x 170 pixel in consideration of the pill size.
To speed up the operation, wavelet transform is on the similarity map to remove horizontal, vertical, and diagonal components, and then extrapolation to make the original size. Also, when calculating the local norm, the process of skipping the pixels exceeding the reference value is performed.
Modeling & Learning
In this paper, RetinaNet is used as a pill classification learning model, and it is selected as a model capable of simultaneous detection and classification in consideration of the differences in the environment of pill filming system and pill classification system.
RetinaNet is an integrated network consisting of a ResNet-based Backbone, and two subnetworks, a class subnet and a box subnet, as shown in Fig. 15 [15].
The epoch used for training is 100, the batch size is 12, and the learning rate is 0.00001.
Controller Design
Fig.16 is a block diagram of Pill classification system, and Fig.17 is a circuit of Pill classification system. Control board gets voltage 5v from power supply and makes system operating[14].
In Input box, the board sends signal about angle, which control servo motor for a pill dropped and gets detector switch’s input value to find out when the pill touches the detector switch.
External power supply gives voltage 12v to Linear conveyor’s motor drive, L293D, which controls DC motor. The DC motor in the Linear conveyor provides a major force to conveyor belt for running. The DC motor’s speed is 15000 RPM, and the gear reduction ratio is 150:1, and consequently, the number of rotation is 100RPM.
Output box has the same motor drive L293D, and it gets voltage 12v from the external power supply. The motor drive controls step motor, which operates 1.8° per 1 step.
Fig. 18 shows the algorithm flow diagram of the executable code for pill classification system. By controlling the servo motor from 25° to 130°, hit the structure with a constant jaw between the input box and the linear conveyor, and drop the pill from the input box one by one. The frequency was set to 50 Hz and 30 % duty-cycle speed, controlling the DC motor through PWM control, and rotating the motor once to position the pill in the camera frame at the top. After that, the camera is captured by the detector switch's input signal, followed by detection and classification. As the predicted result, the step motor is controlled based on 60° between each pill container so that the container corresponding to each pill comes to the end of the linear conveyor. By setting the frequency to 50Hz and the duty-cycle to 65%, the DC motor is rotated three times through PWM control to operate the linear conveyor. Then the pills are dropped into the appropriate pill case. After all this process is over, it goes back to its original position.