Even though there are some pioneering methods to combine physical store shopping with AI-based e-commerce features, such as Amazon Go cashierless convenience stores, these self-checkout stations are expensive investments for local stores since they require renovating (remodeling) store layouts or paying monthly fees to cloud services. Furthermore, local businesses may need to construct data sets for niche market products since AI algorithms require broad data sets for methods based on computer vision, sensor fusion, and deep learning.

After scrutinizing the recent research papers and the granted patents on e-commerce methods, I noticed there are nearly no appliances focusing on merging physical store shopping with AI-based solutions without altering existing conditions for local businesses. Therefore, I decided to build a budget-friendly and accessible AIoT shopping assistant utilizing the most common and recognizable product identification method — barcodes — to bring AI-based e-commerce features to physical store shopping.

Since I wanted to build a shopping assistant providing a wholesome physical shopping experience with the most prominent e-commerce features, I decided to develop a full-fledged e-commerce web application from scratch in PHP, HTML, JavaScript, CSS, and MySQL. Since I developed this complementing web application to bridge the gap between physical store shopping and e-commerce, it is capable of communicating with the shopping assistant (device), obtaining product information from barcodes, and generating AI-based recommendations consecutively.

The complementing e-commerce web application allows the customer to:

• Create a user account identified with the unique 12-digit token
• Obtain the generated account verification QR code via an HTML email, including the unique user token
• Add or remove product information via barcodes scanned by the shopping assistant
• Display the current product list in the cart and the total cart price
• Inspect ChatGPT-powered recommendations for each product in the cart
• Place an order by making a payment via credit/debit card
• Get the payment confirmation QR code via an HTML email
• Check purchased items from previous orders

As suitable for local businesses, I decided to install an Apache HTTP Server (XAMPP) on LattePanda 3 Delta, which also has a MariaDB database, to host my web application. Since I focused on creating a low-budget but state-of-the-art AIoT shopping assistant, I decided to employ the OpenAI API to generate ChatGPT-powered recommendations related to the obtained product information by barcodes via the Open Food Facts JSON API. I also employed Brevo's Email API to send HTML emails directly from localhost to inform the customer of the generated unique verification QR codes.

After developing my e-commerce web application successfully, I started to work on building the shopping assistant (device). Since Wi-Fi and Bluetooth transmissions may not be suitable options for a shopping assistant requiring as minimal latency as possible to communicate with the e-commerce application, I decided to utilize WIZnet's W5300 TOE SHIELD providing high-speed bus communication and reliable Ethernet transmission speeds. Since W5300 TOE SHIELD cannot perform without an STM32 Nucleo-144 board, I decided to utilize the NUCLEO-F439ZI development board and program it in MicroPython.

Since I needed the shopping assistant to scan barcodes and QR codes simultaneously to recognize customers, activate/deactivate carts, and send the scanned product barcode with the selected command (add/remove) to the e-commerce web application, I connected a GM77 barcode and QR code scanner to NUCLEO-F439ZI. Also, I connected an SSD1309 transparent display and an RGB LED so as to inform the customer of the device status. Furthermore, I added a tiny (embedded) thermal printer to directly notify the customer when the customer requests a device status report or scans the payment confirmation QR code. In that regard, I wanted...