Autonomous rover with vision

Similar projects worth following
This project is our contribution as a team to a university laboratory competition, numbering six teams, each with around six members each. In the course of a single quarter, eleven weeks from the very beginning to the very end, each team is tasked with building an autonomous robot capable of competing in four of five total events. The events are held on the very last day of the quarter.

Shown below is our schematic. The STM32 is responsible for driving the motors, reading their encoder feedback, querying the GPS and IMU, driving the speaker (signal), listening for the go switch, and communicating with the PIC18F4321 which is responsible for processing the sonar data. All power is 5V and VCC is 3V. We originally planned to implement a Raspberry Pi 2 as our computer and use the OpenCV libraries for vision processing, however, due to time constraints we had to abandon task. Our system is entirely based on IMU and sonar for sensing, and the motor control/encoders for driving and feedback.

System Overview:

Our system design, as shown below consists of 4 essential processes. The first is a routine which queries all sensors, sonar, microphone, IMU (inertial measurement unit), and GPS. Using the following data, and based on the specified state mode, the second route, the robots guidance state machine, decides an action and executes it. If an obstacle or disturbance is introduced into the system, the robots halts the process and decides the appropriate contingency plan to execute. The third routine is the time sensitive control systems code which is responsible for odometry and keeping the motors at a target commanded value. The final routine is responsible for any low priority (yet important) tasks such as "mooing" the speaker.

Software Overview:

Having purchased appropriate DC motors, wheel hubs, tires, and a chassis, the next step in building our system was to develop an intelligent way of controlling the motors. From a high level point of view, it would be nice if we could simply think (programmatically, as robots do) about going straight at a certain speed or turning about a certain radius and having it happen. Our ability to reliably and efficiently carry out these "low level" tasks is pivotal; the entirety of our task-completing ability depends on this movement. It cannot be over-stressed how important a solid control system overlaying the DC-motor-controlled wheels is.

The first step here was to read our encoder values. This is done directly on the STM32, where we use timers and interrupts to determine the four frequencies. We only have one encoder returning from each wheel, which gives us the speed of the wheel; direction is determined from the current settings of our motor driver, which determines direction. Then we were able to advance to open- and closed-loop control of the wheels. Our command for each wheel is given as frequency, the desired frequency of the encoder feedback. Below is a simple block diagram of our closed-loop control.

Next we added our PID controller. Tuning each wheel became necessary, as no set behaved similarly enough to tune together. The good news, however, is that once PID tuning is completed, high-level programs can utilize the PID controller simply and easily. The tuning process was methodic and repeatable, which was good because we needed to re-tune a couple of times over the course of lab. The process is such: Having basic PID control code initialized, we set the integral and derivative terms to zero while tweaking our proportional term (KP). A good value is found by feel more than anything else; we could hear the motors respond and see (via live debugging) the current encoder frequencies. Next, keeping KP the same, we do the same with the integral term (KI), and finally with the derivative term (KD). Some slight changes to KP are made along the way to ensure a more ideal response, also. Each wheel set came out with its own gain values, with which our vehicle responded well.

We must note here that our PID tuning takes place on a stand, where the wheels are free to spin unloaded by the weight of the vehicle and with the friction of the ground. For the most part, the difference between unloaded and loaded operation did not seem to affect our tuned system. However, our back-left wheel did react noticeably slower when loaded. It did not take much more guess-and-test tuning to mitigate this...

Read more »

  • 1 × Pololu Motors with Encoders Motors
  • 1 × USB Hub Usb Hub
  • 1 × 11.1v 5000mAh LiPo Battery LiPo Battery
  • 1 × MPU9150 9-DoF IMU
  • 1 × Ublox MAX7 GPS GPS

View all 8 components

  • RPI2 running OpenCV/Haar Cascade libraries for object detection

    arko02/24/2015 at 20:15 0 comments

    I wasn't very confident that the RPI 1 could do vision processing, however, with lots of head bashing, cinder block pictures, and lots of training, I made it work.. but it was slow:

    The demo above uses the OpenCV and haar cascade libraries to detect cinder blocks (one of the obstacles in the game). It averaged 6-8fps depending on how many cinder blocks it was able to detect. A week after I got this demo working the RPI 2 was announced.. but quickly sold out. Thankfully my good friend Upu had a spare and donated one half way around the world to our team. After hours of compiling OpenCV and figuring out the mess of dependencies again I was able to run the same code, this time compiled for the ARMv7. The result was stunning. Without any objects in the frame, I was averaging 30fps. Once I introduced a cinder block or two the fps dropped to about 20-25fps (note this is all single threaded, the RPI 2 hovered at 25% cpu use).

    There's still a lot of work to do, optimization, better training data (100 positive and negative images isn't enough). Even so, I'm beginning to realize that perhaps haar cascades aren't the best way to detect objects in the field. The game field is going to be on grass, so it maybe a better idea to just look for blobs that aren't green...

    More updates to come..

View project log

Enjoy this project?



Hamza Abbasi wrote 09/15/2017 at 17:59 point

can you please provide me your data,report, material etc i need it for my final year pprojec purpose

please email:

  Are you sure? yes | no

Adam Fabio wrote 03/04/2015 at 06:07 point

OpenCV and an autonomous robot - awesome! I love projects like this - and I've been hoping someone would get the Raspi2 on a mobile platform.  Keep up the good work!

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates