World’s first camera stabilized self balance robot

Using your phone’s camera and IMU to position stabilize a self balance robot – no additional sensors needed

Similar projects worth following
In this project your phone will be used to fully stabilize a self balance robot. For controlling the position of the robot you need information about the bot’s movement. This information is normally aquired by having motors which are equipped with encoders or using stepper-motors. In this project your phone‘s camera is providing this information via visual odometry – something also used by the mars rover(s).
The performance is strongly depending on the sensors used in your phone, some just delivier better results than others. Addtionally the enviorment needs to be structured, so small movements can be detected between successive images.

  • Position Sensor standalone

    SerialSensor07/15/2023 at 10:53 0 comments

    Here a video where you can see the position output while driving around

  • Tested phones

    SerialSensor04/10/2022 at 12:36 0 comments

    Overview over the phones which have been tested. Controller parameters must be fine tuned for each device.

    PhoneAndroid Version
    Samsung Galaxy S3
    Android 4.4
    No, processing power is not suffiecnt
    Google Nexus 4
    Android 5
    No, has no USB OTG functionality (and Bluetooth's latency is too high for this application)
    Blackberry PrivAndroid 6Yes
    Samsung Galaxy S6
    Android 7
    LG VS30
    Android 8
    Redmi 6
    Android 9
    No, has no Gyroscope
    Huawei P10
    Android 9
    Samsung Galaxy S20
    Android 10

View all 2 project logs

Enjoy this project?



SerialSensor wrote 07/15/2023 at 10:22 point

For specifically this application latency is everything. You already have a ton of delays until you get the latest data from the internal sensors. Espcially the camera of various phones I've seen 40-100ms latency. So you are already on the edge, since the system is unstable. Therefore you cannot allow additonally high latency, which I would expect the data transmission via screen blinking would introduce.
Do you have some numbers here?

  Are you sure? yes | no

James Newton wrote 07/15/2023 at 04:15 point

If this could be done in Javascript on a web page in the phone (access to the camera is possible, I'm just not sure it can process the data fast enough) then it would work with this system for interfacing with any smartphone:

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates