Gary the multibot!

One bot many uses!

Similar projects worth following
In my perception a multibot would be like a Swiss army knife, different functionality in a single package. In Gary’s case, Gary has a main loop programmed in Python. Within that loop, Gary can perform the role of telepresence through the use of Flask streaming and OpenCV. Since Flask uses a template webpage, buttons on the webpage are passed to the Python routine to control Gary’s motors. Audio can also be streamed through the webpage. In reverse, Flask streaming can also send the operator’s camera image to Gary to be displayed on a small screen. Gary also has an autonomous mode. In the autonomous mode, Gary has a stereo vision range finding program to avoid running into things. Tensorflow lite allows Gary to identify objects. Gary has a text to speech converter to allow text files to become bedtime stories. Gary is based around a Raspberry Pi but will soon have a Pi and a NVIDIA Jetson Nano.

Gary the Multibot!

When I started this project, I wanted to build on tech developed from my past projects. Since these earlier projects, tech has advanced and I now have a 3D printer. In my excitement of starting a new robot, I was going to call it the “Terminator T.001”. My daughter saw the bot and calmly said “Let’s call it Gary.” Long story short, here is Gary the Multibot.

One bot, many uses. Gary is a bot built for telepresence for meetings, elderly care and a bot built for autonomous operations such as home security, games and a combination of both for child care, entertainment and education.

As a multibot, Gary will incorporate a variety of technologies allowing the bot to perform many different tasks. Areas that are already incorporated are stereo vision object avoidance, live streaming for telepresence, text to voice for telling stories, and tensorflow lite for object identification.

With a world changed by COVID and isolation, it is time for a robot to not just be a novelty but an assistant to help its owner and an extension of the person that owns it.

Here is Gary to introduce himself!

Within the last year, a lot has changed. With COVID 19 a reality and the risk of future outbreaks, working from home is likely to continue.  With working from home a new reality, the question is how to aid the individuals working from home to be more comfortable and more productive. In any work environment, stress created by both surroundings and family situations can cause anxiety and mental stress affecting the individual’s performance. One area that can affect the mental well being of a home worker is occupying the children in a way that is entertaining and educational since the child may also be home schooled, not only in the present but in the future, with the risk of future outbreak a very real possibility. And to do this while also performing one’s required job duties. 

The concept for a “multibot” is a robot that has multiple capabilities that can be called from the robot’s main program. With text to speech, the robot can tell stories to both entertain and educate while occupying the child. A WiFi access can allow the robot to interact and answer questions by using Google’s API in the same way SIRI and Alexa  answer questions. OpenCV face detection and tenser-flow lite will allow the robot to determine if the child is still present. While at the same time, a telepresence  capability that can easily be added through the use of a streaming framework such as “flask,” will allow the parent to monitor the child from a computer while also performing their job,  relieving stress while also allowing time to perform the necessary job duties.

 Another area of stress that can be reduced by a multibot is being able to monitor and interact with an elderly parent that may live alone. The same functionality that allows a robot to entertain, educate and monitor a child can also aid in remotely taking care of an elderly parent. Telepresence that allows the worker and the elderly parent to interact will be a comfort to both of them. A robot that can read text from a website or play music from a favorite playlist can occupy and entertain the elderly parent while also allowing the person working the peace of mind in knowing if the parent is okay or needs assistance. With a wide range of constantly growing possible functions that can be added, a multibot can be a valuable aid to anyone working from a home office. 

With a WIFI access and a main program in the multibot that will allow access to any of the robots functions, everything from watching the children, checking on an elderly parent or driving the multibot to check on anything in the robots vicinity can be accomplished from the comfort of the home work station.

*License GPLv3 unless stated otherwise for all software. License details available in the GitHub repository.*

... Read more »

  • 2 × USB Camera Webcam HD 720p Web Camera, USB with Microphone.
  • 1 × Raspberry PI 4
  • 1 × NVIDIA Jetson NANO

  • Gary the multibot’s new eyes.

    Dennis07/14/2021 at 15:16 0 comments

    Gary got an eye upgrade from cheap cameras I had laying around to autofocusing 720p HD cameras. The new cameras work much better for the stereo object detector/avoidance function and the telepresence. This was actually a couple of months ago.

  • Image illustrating how the project might be used.

    Dennis07/13/2021 at 21:38 0 comments

    Balancing home life/ family with working from home can be a challenge. When a child or elderly parent is involved, the challenge is even greater. Gary, with its text to speech function, can read a text document about a far off galaxy or an enchanting princess story to a child while streaming images and allowing interaction with a child from the parent’s work station. The same functionality can also allow interaction with an elderly parent either in the same home or at a different location via internet. Both scenarios can allow the working parent to multitask home life and work duties from the comfort of their work station.

     Below image illustrating how the project might be used.

    Image below:  Gary streaming images to a work station while a work document is also being worked on.

  • TensorFlow Lite

    Dennis07/11/2021 at 13:46 0 comments

    Another valuable feature added to the bot. First test were slow due to using the read method in openCV which once called blocks the main program from continuing until the read method has completed. Moving the read method to a different thread speeded up the process considerable. 

  • Stereo Vision

    Dennis07/11/2021 at 01:40 0 comments

    One of the biggest challenges in building an autonomous robot is giving the robot the ability to detect and understand its surroundings. Gary is using two cameras, python code and OpenCV to detect objects around the robot. Once an object has been detected in one camera’s image, the python code with OpenCV searches for the same object in the second camera’s image. The disparity between the two images of the object will tell the robot how to avoid the object.  

    Here’s Gary’s first test avoiding objects with stereo vision.

View all 4 project logs

Enjoy this project?



Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates