YOLOInvoiceWizard: Revolutionizing Invoice Generation through Object Detection

doi:10.21203/rs.3.rs-3577586/v1

Download PDF

Research Article

YOLOInvoiceWizard: Revolutionizing Invoice Generation through Object Detection

https://doi.org/10.21203/rs.3.rs-3577586/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

This research paper introduces a novel approach to streamline the customer checkout process in retail stores through the integration of computer vision technology using YOLOv8 for object detection. The project's focus lies in facilitating efficient invoice generation by detecting products, identifying them accurately, and seamlessly adding them to a web-based graphical user interface (GUI) shopping cart. The proposed system aims to change the conventional billing procedure by leveraging computer vision capabilities to expedite the checkout process, minimizing customer wait times, and reducing human errors during billing. By harnessing the power of YOLOv8, a state-of-the-art object detection model, the system can identify various products placed on the counter. The detected items are added into a user-friendly web GUI cart that allows customers to conveniently track their selections while navigating the store. The integration of computer vision and web technology creates a unique and delightful customer experience during the checkout process, contributing to increased customer satisfaction and loyalty. The potential benefits of this project are multifaceted, encompassing reduced wait times at the billing counter, enhanced accuracy in billing processes, and overall improved efficiency in retail operations. The research paper concludes with a detailed evaluation of the prototype system's performance, validating its effectiveness in enhancing customer experiences and optimizing billing procedures in a real-world retail environment. This paper contributes to the advancement of computer vision applications in the retail sector, setting the stage for future innovations in customer-centric shopping experiences.

YOLOv8

Object Detection

Retail Industry

Invoice Generation

Computer Vision

The supermarket invoice system is an essential tool for managing sales transactions and recording purchases in a retail environment. It serves as a digital or paper-based record of all items sold to customers, providing detailed information about each transaction. Some key details included in an invoice are – Itemized details, Payment information, Date and Time stamp, Customer information, Tax Calculations and many more.

Invoice Generation generates an invoice for products purchased by detecting the objects placed in front of the camera and sending them to a web GUI which is developed using ReactJS, Flask to carry out the transaction.

A. Rationale

The intention behind developing this project is to create an innovative invoice generation tool that reduces time at checkout for the customer. This removes the use of barcode scanning and RF-ID (Radio Frequency Identification) tag scanning and offers a seamless experience to the customer.

B. Goal

The goal of the project is to produce a detailed invoice of the products that are purchased using the web GUI in the form of a PDF file which includes itemized details of products, payment information, date and time stamp, customer information, prices of the products, tax calculations. This eliminates the need for barcode scanning and RFID tag scanning, providing customers with a unique shopping experience.

A. Python

Python is a high-level, versatile, and interpreted programming language known for its simplicity and readability. Developed by Guido van Rossum and first released in 1991, Python has become one of the most popular languages. The local webserver and the core summarizing logic were developed in Python using Flask [1, 2].

B. Object Detection

Object Detection [3–6] plays a key role in this project. It’s a computer vision task that involves identifying and localizing objects within an image/video [7, 8]. It is a critical component in various applications such as autonomous vehicles, robotics, surveillance systems and augmented reality.

C. Object Detection Tasks

Some of object detection tasks are:

• Image acquisition

• Preprocessing

• Object Detection Model

• Prediction

• Visualisation (bounding box)

• Output

D. Object Detection Tools

1) OpenCV: OpenCV [9] is a widely used open-source computer vision library that covers various aspects of computer vision, including image processing, feature detection, object recognition, and machine learning. It is used in this project to handle the live feed from the camera and process the video.

2) YOLOv8: YOLOv8 (You Only Look Once) [10–14] is the state-of-the-art computer vision model built by Ultralytics, who is also responsible for development of YOLOv5. This project utilizes this algorithm to train custom object detection model using a dataset of around 500 images for 10 classes. It has pre-trained weights such as yolov8n.pt, yolov8m.pt, yolov8l.pt that can be used for training the model. The yolov8m.pt weight was used to train the model for 50 epochs in this project.

3) Data Annotation: The process of labelling or tagging data to provide ground truth information for training models is known as data annotation [3, 4, 15]. Dataset of images needs to be annotated individually with respect to the class the image belongs to. This helps the learning algorithm understand the relationship between input data and desired output, enabling them to learn patterns.

The training results of the dataset are shown in Figure 1. They indicate accuracy scores with respect to the epoch count during training.

Figure 2 represents the confusion matrix of the best model trained. The model has perfect accuracy for detecting 9 out of 10 products trained.

Figures 4 and 5 depict the working model in a web browser. Figure 4 represents the homepage users are greeted to and Fig. 5 is the final invoice generated for each unique order and user.

The development of an invoicing system has the potential to change the retail industry [12, 16] by enhancing customer experience. By utilizing technologies such as OpenCV and the YOLOv8 algorithm which can generate a custom trained model. With accurate object detection and with the help of a web application made using ReactJS [17, 18], Flask, and MongoDB, invoice generation can be achieved. The elimination of traditional barcode scanning and RFID tag scanning reduces checkout time and simplifies the overall shopping experience for customers.

To further enhance the invoicing system, several areas can be explored -

Integration with mobile applications: Developing a mobile application that allows customers to scan items and generate invoices using their smartphones can add convenience and flexibility to the system.
Customer personalization: Implementing machine learning techniques to analyze customer preferences and purchase history can enable the system to provide personalized recommendations and discounts, creating a more tailored shopping experience.
Integration with inventory management: Linking the invoicing system with inventory management software can ensure real-time tracking of product availability, preventing overstocking or out-of-stock situations.
Payment integration: Incorporating secure payment options, such as mobile wallets and contactless payments, can further expedite the checkout process and enhance customer satisfaction
. Analytics and Reporting: Generating insightful analytics and reports on sales, popular products, and customer behavior can provide valuable business insights for retailers, aiding in strategic decision-making and improving overall operations.

Acknowledgements

We thank our college and corresponding author and faculty Dr. Dharmapuri Siri for sponsoring our paper publishing endeavours. This paper would not have been possible without their involvement.

"DevDocs " Available at: https://devdocs.io/react/
"Welcome to Flask - Flask Documentation (2.3.x)," Available at: https://flask.palletsprojects.com/en/2.3.x/
Wang H et al (2016) "Machine Learn Basics " Deep Learn, pp. 98–164
Singh Y, Bhatia PK, Sangwan O (2007) A Review of Studies on Machine Learning Techniques. Int J Comput Sci Secur 1(1):70–84
Balachander B et al (2022) "The Prospects of Using Artificial Intelligence in Retail Billing in India," in AIP Conference Proceedings, AIP Publishing, 10.1063/5.0094273
"Faster SR (2015) R-CNN: Towards Real-Time Object Detection with Region Proposal Networks." Advances in Neural Information Processing Systems (NIPS),
Agrawal M (2017) Nathi Ram Chauhan 3D Object Recognition for Automated billing in a Supermarket using Hybrid PCA-SIFT-FREAK Algorithm., IJAEEIE,
Amine ME, Seddik et al (2018) "A Review on Deep Learning Techniques for Image and Video Object Detection." 10th International Conference on Modelling, Identification, and Control (ICMIC),
OpenCV : Open Source Computer Vision Library. Available at: https://opencv.org/
Redmon J et al (2016) "You Only Look Once: Unified, Real-Time Object Detection." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
YOLOv8 : You Only Look Once version 8. Available at: https://docs.ultralytics.com/
Arpita Vats DC, Anastasiu. (2023) “Enhancing Retail Checkout Through Video Inpainting, YOLOv8 Detection, and DeepSort Tracking”
Tianjiao L, Hong B, "An Optimized YOLO Method for Object Detection," in (2020) 16th International Conference on Computational Intelligence and Security (CIS), Guangxi, China, 2020, pp. 30–34. 10.1109/CIS52066.2020.00015
Lou H et al (2023) "DC-YOLOv8: Small-Size Object Detection Algorithm Based on Camera Sensor," in Electronics, vol. 12, no. 10, pp. 2323
Roboflow Used to manage datasets. Available at https://roboflow.com/
Nayyar K et al (2020) "Automated Checkout System Using Deep Learning." IEEE International Conference on Computational Intelligence and Computing Research (ICCICR),
Finch D, Santos R, Potes A, Ruelas R (2007) "REACT: An Object-Oriented Control Engine for Rapidly Building Control Systems," in 2007 2nd Electronics, Robotics, and Automotive Mechanics Conference, Morelos, pp. 123–128
Holmes E, Fedosejev A, Bray T, Hayward J, Vice R, Horton A, Prusty N (2016) "React: Building Modern Web Applications," Packt Publishing,

No competing interests reported.

Download PDF

Editor assigned by journal
08 Nov, 2023
Submission checks completed at journal
08 Nov, 2023
First submitted to journal
08 Nov, 2023

You are reading this latest preprint version

YOLOInvoiceWizard: Revolutionizing Invoice Generation through Object Detection

Status:

Version 1

Abstract

Figures

I. Introduction

II. Methodology

III. Results

IV. Conclusion

V. Future scope

Declarations

Acknowledgements

References

Additional Declarations

Status:

Version 1