Close
0%
0%

FIMble

Concept of a plug-and-play IoT food monitoring system. It aims to reduce food wastage at consumer and distributor level

Similar projects worth following
FIMble (Food IoT Monitoring) system is an innovative and scalable food management solution for food inventories such as home fridges or retail food storage. It helps users in managing food stock levels and wastage reduction, JIT re-orders and freshness monitoring packaged together with powerful analytics transforming collated data into intelligence for insightful and incisive decision making.

For consumers, FIMble serves as full time personal food manager attending to your food nourishment needs. For companies, FIMble is key to optimizing business operations and strategic long-term decisions. There is an aim for zero wastage reduction to be realised, through responsive food management.

What do I want to build?

  •  FIMble (Cypress PSoC 6 WiFi-BT Pioneer Kit )

My aim is to build a simple device to reduce wastage due to improper organization of the foods, and improve efficiency in determining soon-to-expire items. Ideally, it should be easily implemented, with minimal setup and learning curve required. Additionally, it should be modular and portable, so it can be used on today's existing storage spaces and fridges. 

Existing prototypes of smart fridges, using barcodes or RFID technologies to update the inventory, require the user to scan or move the item close to the sensor (like tapping your card on the bus) every single time you place or remove something. This is very clunky and not user-­friendly.

By utilising long range RFID, FIMble is made possible. I will explain how it works below...


What is my inspiration?

In stores, often times there will be workers assigned to do inventory checks regularly. With a large inventory, keeping track of expiry dates becomes a pain.

At home, it is a routine to clean out the fridge at the end of the year, to get rid of expired food. These food items are usually buried behind other items, hence forgotten and left until it expires, amounting to quite some waste.

Technical details of the system

In certain food products industries, RFID tags are already being used for labelling. RFID tags are getting extremely cheap and the cost is negligible compared to the potential wastage of food.

Using FIMble, every time a food item is placed inside the fridge, an RFID scaner detects the proximity of the item and updates the inventory. When food is removed, the reader can no longer detect the item within the range and it’s removed from the inventory. This is done automatically and there is no need to manual intervention 

A phone app will connects to the reader via WiFi. Bluetooth location tracking is useful for larger scale retail storage. It makes it easier to find on which shelve or aisle a specific item is located at.

For example, a list of expiry dates can be easily generated and sorted by food type. For consumers, food nutrition and diet recommendations can be made available for monitoring the quality of food. They can also be alerted if any specific item is running low and needs to be topped up.

Prototyping Plans

For prototyping, I plan to use the Cypress PSoC 6 WiFi-BT Pioneer Kit (CY8CKIT-062-WIFI-BT). It has the features and peripherials I need such as:

  • WiFi - This will allow data to be sent to the cloud, where data analytics can occur
  • Bluetooth - This will allow multiple FIMble devices to exchange location data, such as shelve number and fridge number.
  • TFT LCD Display - User display interface to be shown in the fridge.
  • Touch buttons - User can interact and specify settings
  • Arduino Headers - An RFID shield can be attached such as the SparkFun Simultaneous RFID Reader which can do long range RFID.

Cloud applications and IoT data analytics

These are some examples of how using FIMble can improve the user's standard of living by collecting data related to food storage.

  • For foods with a shorter shelf life, which requires a more controlled storage environment. Examples include expensive seafood (like oysters and lobsters), and dairy products. The users need to be notified if it is being kept for too long; they can keep track of declining freshness of the products.
  • Sensors can be used to recommend the optimal environment for storage of certain food types.
  • Since the inventory is relayed to the phone app, users will also be able to check it when grocery shopping. They need not struggle to remember if they need to buy a certain item, preventing them from unnecessarily buying what they already have.
  • Users can search for where a certain item is via the app. This improves efficiencies within a storage unit.

My learning experience for this project

I look forward to learning more about IoT application especially with regards...

Read more »

presentation - 317.46 kB - 07/18/2020 at 12:41

Download

  • 1 × Cypress PSoC® 6 WiFi-BT Pioneer Kit (CY8CKIT-062-WIFI-BT)
  • 1 × DIY Paper Craft Enclosure
  • 1 × RFID Reader MFRC522 Arduino Module
  • 1 × Amazon Web Services Services used: IoT Core, Lambda, API Gateway, DynamoDB
  • 1 × Flutter App (Hosted on AWS S3) A mobile app to show the inventory list along with analytics

View all 6 components

  • AWS S3 For Flutter Web App

    zst12307/30/2020 at 08:39 0 comments

    When I created the Flutter Android App, I thought of cross-platform as one of my reasons for trying out Flutter. 

    I read that Flutter Web is still in early support, however, it will be good to try it out. I want to see how to host it on Amazon Web Services itself.  If I can host my app on AWS, it will turn my project into a fully serverless and cloud-based concept.

    Adding Web support to my Flutter App


    To add web support, first I ran these commands in my flutter app directory. 

    I followed this guide: https://flutter.dev/docs/get-started/web

    Run the commands:

    $ flutter channel beta
    $ flutter upgrade
    $ flutter config --enable-web
    $ flutter devices
    $ flutter run -d web-server --release
    

    Hosting on AWS S3


    In the end, I chose AWS S3 as a suitable service because I realised that I can generate the Flutter app as a static webpage.

    I followed this guide closely to set up the S3 bucket, permissions and properties:

    Go to S3 console and create a new bucket

    Under properties, I enabled static web hosting. I went with all the default settings

    Here, we can see that it is enabled afterwards

    We need to allow public access, so I went to permissions to edit the bucket policy.

    I modified to have this bucket policy for my use-case:

    {
      "Id": "PolicyForPublicWebsiteContent",
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "PublicReadGetObject",
          "Action": [
            "s3:GetObject"
          ],
          "Effect": "Allow",
          "Resource": "arn:aws:s3:::cypress-psoc/*",
          "Principal": "*"
        }
      ]
    }

    I then uploaded all the files from the build directory: ./flutter_fimble/build/web/

    And as simple as that, my app is now hosted online live! It was way easier than I thought!

  • Flutter Android App for Smart Inventory

    zst12307/22/2020 at 13:09 0 comments

    As of now, most part of my project is done. For the hardware side,  the only missing piece is the RFID reader which I have yet to receive due to issues with shipping delays. All the code is working perfectly and interfaced with AWS cloud so I will say it is 90% done for the hardware.

    My next plan was to make an Android app which will keep track of all the inventory items in the system. I have had experience coding Android apps before. But for this I decided to try something new and use Flutter which is a new framework which is cross-platform for Android, iOS and web apps. 

    Today, I finally completed my app and it is looking great. I will describe the features below.

    Run-through of the app

    This is the run-through of each page of the app. There are 3 major features which I will propose for my project.

    Inventory Page

    The first page will show all the items currently inside the inventory (eg. items inside your smart fridge). When the user clicks on an item, there will be a description of the item and also nutritional values of the food item.

    Statistics Page

    This page will display the settings of temperature control of the storage area. It is useful especially for sensitive food items which need to be stored at a certain temperature.

    A breakdown of the category of the items can also be derived via analytics from all the data gathered through this smart inventory system.

    Overview Page

    Lastly, the overview page is useful for large inventories especially in warehouses. By providing visual icon representation of the storage area, the workers in a warehouse can quickly navigate and find the item they are looking for.

    Working principles behind the app

    The app makes use of AWS API Gateway which serves a HTTP page to retrieve the items from the database dynamically. A lookup table is implemented to provide the item image and description for the user. In practical use, there are also online databases with thousands of food nutritional labels which can be made use by the app.

    A possible further improvement will be to host the app on the AWS platform itself, and make it truly a cloud solution. As I mentioned earlier, I used Flutter which allows making Web Apps. Using AWS EC2 or S3 may be a possible service to host it on. But I will focus on getting the final bits of my project working first, and try it out only if I have additional time.

    As usual, the source code is posted in my Github repo, feel free to take a look and discuss with me.

  • Paper Craft Enclosure Case Design

    zst12307/18/2020 at 12:15 0 comments

    Out of curiosity, I took a look at what other people were doing for this contest. I was inspired by this project log in the Nanodrone project. In the link below, the author designed a case for 3D printing. 

    I wanted to do something similar but I didn't have a 3D printer. And also all the Makerspace facilities near my home are closed due to the coronavirus situation. I decided to create a paper craft version of it. I can print it on colored stock paper.


    Final Design of Paper Craft Enclosure

    Thus, here is my design for the paper craft enclosure for the Cypress PSoC® 6 WiFi-BT Pioneer Kit (CY8CKIT-062-WiFi-BT).

    The original files can be found open-source on my github repository. Feel free to download it and make one yourself!

    Initial prototype using paper

    Final prototype using card stock

  • Temperature/Humidity using DHT11

    zst12307/18/2020 at 02:36 0 comments

    This is just a quick post on adding DHT11 wiring to my board and my plan is to connect it to AWS.

    In my Android app, I thought it will be good to know the analytics of temperature/humidity of my smart inventory fridge.

    First, let's look at the DHT11 pinout -- it needs 3 wires and a pull up resistor.

    I wired it on a breadboard as follows.

    And then I looked at the PSoC pinout. 

    With the LCD in place, I chose the bottom left corner pin, P13_6, which is unused by the LCD. Many other pins are used up by the LCD.

    And for VCC and GND, I used the Pmod connector VCC and GND.

    Final pinout.

    I will then add the code and upload the data on to AWS. 

    In my next post, I will finish the basic outline of my Android app.

  • emWin Bitmaps using Python script (cross platform)

    zst12307/10/2020 at 09:06 0 comments

    I wanted a nice splash screen for my project. I wanted to test the bitmap capabilities of the emWin driver, because I can see that it allows bitmaps to be displayed.

    There is a Segger emWin Bitmap Converter, but there are some issue with it.

    (1) It is Windows-only, and I am running my setup on Linux.

    (2) It is a demo version, which shows a demo watermark.

    As a hobbyist, I cannot afford the full version of the Bitmap Converter, hence I tried to derive the bitmap format myself and create an open-source cross platform using a Python script.

    I referred to these sources which show how emWin bitmaps C files are generated.

    From here, I can understand that I can have different formats of color encoding (for example, ARGB888, RGB565). The emWin LCD driver is very powerful too as it supports color palettes to compress bitmaps to a smaller size.

    I implemented these into my python script. https://github.com/zst123/Project_Fimble/tree/master/emWin-Converter/

    Here's the result from my work today.

  • AWS API Gateway to DynamoDB

    zst12307/04/2020 at 06:44 0 comments

    Since my app is of a food tracking system, I eventually need to implement an Android app to show an overview of what is inside the fridge inventory.

    I am planning for a HTTP API for (perhaps) an Android App to access. The request flow will be like this.

    • Android app → HTTP Request → AWS API Gateway → AWS Lambda → AWS DynamoDB

    I will try out AWS API Gateway today to retrieve contents in the AWS DynamoDB database.

    (1) Create Lambda to be invoked by API Gateway.

    Go to Lambda console

    • Function > Create function
    • Function name: CypressPsocApiLambda

    After creating the lambda go to IAM console

    • Access management > Roles > Permissions > (Click on the function name) 
    • Attach policy: AmazonDynamoDBFullAccess

    (2) Create API Gateway and integrate it to Lambda

    Navigate to the API Gateway service console, and choose Create API. In API name, type CommentsApi and type a short description. Finally, choose Create API.

    • HTTP API > Build
    • API name: CypressPsocApi
    • Stage name (leave as default)

    Create a route

    • GET request: /retrieve

    Create and attach and integration

    • Integration with: Lambda function > Choose “CypressPsocApiLambda”

    (3) Edit Lambda to parse DynamoDB items

    I used the document client to scan for the table name. This retrieves all the items without any filters. Perhaps I'll add a filter parameter in if I need it in the future part of this project.

    And now, I can go to the /retrieve page and I see all the test entry in my database.

    For next time, I want to do up the android app and also the RFID portion of the PSoC device.

  • AWS DynamoDB interfaced with IoT Core

    zst12306/27/2020 at 13:17 0 comments

    Before we start, I did some research on Amazon FreeRTOS and how Amazon IoT/MQTT works

    I previously confirmed that MQTT messages are sent successfully with the demo code. From the AWS website, we can monitor it too:


    Since my application is to do inventory tracking, I will need to use my IoT device to update a database.

    In AWS, I see that I can do this from IoT core -> Lambda -> DynamoDB. I followed this guide.

    Connect IoT core to Lambda using IoT Rules.

    Go to Act > Rule > Create a rule.

    I used this query string SELECT * FROM '#'

    Go to Add Action > Send a message to a Lambda function > Create a new Lambda function > Select your function > Create rule.


    In order for AWS IoT to call a Lambda function, you must configure a policy granting the lambda:InvokeFunction permission to AWS IoT.

    • aws lambda add-permission --function-name my-function --statement-id iot-events --action "lambda:InvokeFunction" --principal iotevents.amazonaws.com

    But when I run this command I get an error

    • An error occurred (AccessDeniedException) when calling the AddPermission operation: User: cypress-psoc is not authorized to perform: lambda:AddPermission on resource:

    To solve this, find your user in the IAM management console > Permissions > Add inline policies. Use these settings:

    • Service: Lambda
    • Actions: Permission management
    • Resource: all resources

    Now when I run the command again, it works.


    From lambda, when I used the “Test” button in Lambda, I got another “AccessDeniedException”. 

    From lambda to dynamodb, we also need to add permissions. I solved this by adding “AmazonDynamoDBFullAccess” permission.

    Go to your IAM console > Access management > Roles

    Find the role for your lambda. When I created mine, I left it as the the default name so it is Handler-role-0cwysi5m.

    Go to Permissions > Attach policies, and find  “AmazonDynamoDBFullAccess”.

    After this, I can successfully write to DynamoDB

  • Application Code, TFT display, CapSense

    zst12306/21/2020 at 03:35 0 comments

    (1) Finding out how Cypress FreeRTOS Demo Code runs

    I wanted to modify as little of the original files as possible. So I put my code around the demo code. This is how the code runs from the main.c until it reaches the demo code.

    In main.c (aws_demos/application_code/main.c)

    • Startup task is vApplicationDaemonTaskStartupHook()
    • It calls DEMO_RUNNER_RunDemos().

    In iot_demo_runner.c,

    • DEMO_RUNNER_RunDemos() will call DEMO_entryFUNCTION().

    In iot_demo_runner.h

    • DEMO_entryFUNCTION is defined as RunMqttDemo if CONFIG_MQTT_DEMO_ENABLED is defined.

    In aws_demo_config.h,

    • CONFIG_MQTT_DEMO_ENABLED is defined here.

    (2) Modify to run my own code

    I created my own app.c and app.h files with the startup function called RunMyApplication().

    In aws_demo_config.h,

    • I removed #define CONFIG_MQTT_DEMO_ENABLED
    • And I added #define CONFIG_MY_APPLICATION

    In iot_demo_runner.h,

    • I pointed to my own function RunMyApplication() by creating a new definition as follows...

    (3) How to add TFT display library

    I used reference from this blog post:

    Right click on the project inside Project Explorer > ModusToolbox > Library Manager

    Turn on these Libraries

    • Board Utils > CY8CKIT-028-TFT
    • PSoC 6 Middleware > emwin

    In the Makefile, add this to COMPONENTS. To indicate we are using an OS and it is Non-TouchScreen.

    • COMPONENTS=EMWIN_OSNTS

    In my RunMyApplication() function, I added these lines. To test the LED and the TFT display.

    cyhal_gpio_init(CYBSP_USER_LED5, CYHAL_GPIO_DIR_OUTPUT, CYHAL_GPIO_DRIVE_STRONG, CYBSP_LED_STATE_ON);
    GUI_Init();
    GUI_SetFont(GUI_FONT_32B_1);
    GUI_DispString("Hello World");

    And now it looks like this

    (4) How to add CapSense library

    Do not add CapSense library. Leave capsense unchecked in the Library Manager. Because Cypress FreeRTOS already includes the CapSense library.

    I created a new RTOS task and then initialised the CapSense using code from these examples.

    One problem I faced is I cannot get the slider gesture to work (flick left / flick right). But I can get the slider position (value of 0 to 300) working perfectly.


    This is a video of my progress for today. It shows the testing of CapSense, TFT display and WiFi working.

  • Testing Amazon FreeRTOS

    zst12306/20/2020 at 12:53 0 comments

    After doing some research, I think the best experience will be to use Cypress’ Amazon FreeRTOS. I tried looking into mbed OS but the documentation for this board is not clear and it seems it doesn’t have an easy way to import into ModusToolbox.

    As a start, I read the Cypress Official Start Guide Getting Started with Amazon FreeRTOS and PSoC62 + 43xxx.

    (1) AWS IAM Account

    Firstly, I created an AWS account and then logged in as root.

    After this, we need to create an IAM user account which is used to assign permissions for our IoT device to access later.

    I followed this blog post where he shows step by step photos on how to setup

    After that, followed this guide to add FreeRTOS permissions.

    From the guide above, make sure you download all the account files. We will use them in the next part. I ended up with these files:

    1. cypress-psoc_accessKeys.csv
    2. cypress-psoc_certificate.pem
    3. cypress-psoc_codecommit_credentials.csv
    4. cypress-psoc_privatekey.pem

    (2) AWS IoT Core

    To configure later, need to install AWC CLI tool. Follow this guide.

    mkdir aws_cli
    cd aws_cli
    curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
    unzip awscliv2.zip
    sudo ./aws/install

     Then we need to configure AWS

    Now, from the account files, use cypress-psoc_accessKeys.csv here.

    $ aws configure
    AWS Access Key ID [None]:     <<Use cypress-psoc_accessKeys.csv>>
    AWS Secret Access Key [None]: <<Use cypress-psoc_accessKeys.csv>>
    Default region name [None]: ap-southeast-1 <<Choose the server closest to me>>
    Default output format [None]:  <<I left this blank>>

    And after configuring check the endpoint. 

    $ aws iot describe-endpoint
    {
        "endpointAddress": "a2cgpzvh8vmm38.iot.ap-southeast-1.amazonaws.com"
    }

    Next, we need to register the board

    After done, we can finally see the board in the console (Manage > Things)


    (3) Cypress AWS FreeRTOS

    Clone the Cypress AWS FreeRTOS repo into your workspace

    In ModusToolbox IDE.

    • Import the demo projects: File > Import > General > Existing Projects into Workspace
    • Choose the demo for your board: ~/ModusToolbox/workspace_rtos/amazon-freertos/projects/cypress/CY8CKIT_062_WIFI_BT/mtb/aws_demos
    • Build and check no errors: Project > Build All

    Configure MQTT and your WiFi settings:

    Program the board and the connection should be shown on the activity page.

    I faced a problem where a lot of sematic problems appear but it compiles successfully.

    We actually need to disable some settings in Code Analysis (according to ModusToolbox user guide):

    Note
    While debugging, you may see various errors in your code (main.cpp file), such as "function ‘printf’ could not be resolved." These are likely not real errors. They are caused by the import of the make file into Eclipse. 
    
    To turn these errors off, go to Project > Properties > C/C++ > Code Analysis, and disable "Potential Programming Problems" and "Syntax and Semantic Errors.

    In my next post, I will modify the demo program to do my own application.

  • Hello, PSoC 6 WiFi-BT Pioneer Kit

    zst12306/19/2020 at 17:02 0 comments

    I just received my Cypress PSoC 6 WiFi-BT Pioneer Kit (CY8CKIT-062-WiFi-BT) in the mail. Thank you Hackaday, Cypress and Digikey for choosing my project.

    The main board is called the PSoC 6 WiFi-BT Pioneer Kit (CY8CKIT-062-WiFi-BT). The microcontroller on it is the CY8C6247BZI-D54 MCU.

    It comes with a shield called the CY8CKIT-028-TFT Display Shield.


    This is my first time working with Cypress microcontrollers. 

    First I downloaded the IDE from the official website. I will be using it on Ubuntu 20.04 LTS.

    I extracted the folder ModusToolbox from the zip file into my home folder ~/

    According to the readme file, I ran the installation commands as follows

    ~/ModusToolbox/tools_2.1/openocd/udev_rules/install_rules.sh 
    ~/ModusToolbox/tools_2.1/driver_media/install_rules.sh 
    ~/ModusToolbox/tools_2.1/fw-loader/udev_rules/install_rules.sh 
    ~/ModusToolbox/tools_2.1/modus-shell/postinstall
    sudo apt-get install libusb-0.1-4

    And then I started ModusToolbox by double clicking on this file

    ~/ModusToolbox/ide_2.1/eclipse/ModusToolbox


    This is the screen I see. Oops, right now I’m using the default ubuntu dark theme. But the light theme of the IDE is not usable.

    Go to Window > Preferences > Appearances > Theme and change to Dark theme. 

    Restart the IDE and it looks much better.

    From Quick Panel, choose New Application.

    In Project Creator, I faced an issue with retrieving the devices -- Failed to access remote server.

    The problem occurs because Ubuntu 20.04 does not include the libssl1.0.0 package. I must download it from the ubuntu 18.04 repo.

    Install the deb from the ubuntu repo.

    wget http://security.ubuntu.com/ubuntu/pool/main/o/openssl1.0/libssl1.0.0_1.0.2n-1ubuntu5.3_amd64.deb
    sudo apt install ./libssl1.0.0_1.0.2n-1ubuntu5.3_amd64.deb

    Click on Retry and everything works.

    I got the PSoC® 6 WiFi-BT Pioneer Kit so I chose CY8CKIT-062-WiFi-BT.

    And then I created a project from the hello world template.


    The board comes with a debugger, and the firmware is KitProg2. 

    $ lsusb
    Bus 006 Device 006: ID 04b4:f148 Cypress Semiconductor Corp. Cypress KitProg2 (CMSIS-DAP, Mass Storage)

    The ModusToolbox IDE needs us to upgrade it to KitProg3.

    Press and hold down SW3 to set it into bootloader mode and connect the board. LED2 should flash.

    Then go to terminal run the command 

    $ cd ~/ModusToolbox/tools_2.1/fw-loader/bin
    $ ./fw-loader --update-kp3
    

     Now check the device again

    $ lsusb
    Bus 006 Device 011: ID 04b4:f154 Cypress Semiconductor Corp. KitProg3 CMSIS-DAP
    

    Finally, go back to ModusToolbox. Build the project and program the board.


    Flashing LED successful.

View all 10 project logs

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates