07/22/2017 at 06:13 •
In this post, I try to explain how the robot tracks the ball using image processing algorithms.
You first need to install OpenCV in your Raspberry Pi which you can find the tutorials in following websites:
sudo apt-get install python-wxgtk2.8 python-matplotlib python-opencv python-pip python-numpy sudo apt-get guvcview
You can use both cv and cv2 libraries but I used cv2 for this project.The petdog.py code tracks a tennis ball using color detection methods and calculates center of mass of the tennis.
To capture a video, you need to create a VideoCapture object. Its argument can be either the device index or the name of a video file. Device index is just the number to specify which camera is used and in this case it's zero because there is only one camera module.
cap = cv2.VideoCapture(0)
Next you can use Property identifier to modify the height and width of VideoCapture object.
3- Width of the frames in the video stream.
4- Height of the frames in the video stream.
The VideoCapture object needs to be read frame by frame to perform object detection which is done using .read method:
_, frame = cap.read()
Now you need to convert image frames from one BGR color-space to HSV color-space.
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
Next you need to use upper and lower color thresholds to define range of yellow color in HSV which is used to find tennis ball. The values needs to be changed based on light settings.
lower_yellow = np.array([30, 255, 135], dtype=np.uint8) upper_yellow = np.array([40, 255, 185], dtype=np.uint8)
An image mask can now be created to get only yellow colors. The image mask is used to calculate image moment of tennis ball which is a certain particular weighted average.
mask = cv2.inRange(hsv, lower_yellow, upper_yellow) moments = cv2.moments(mask)
The image moment can also be used to calculate area and center of mass of the tennis ball.
area = moments['m00']
Now we need to check the area of the object before calculating the x and y centroids.If the object is too small then there is a high chance that it's not a tennis ball.
if(area > 100000)
Calculate x and y centroids:
x = moments['m10'] / area y = moments['m01'] / area
And finally we can implement the function that calculates robot movement based on changes in x and y centroids values.
def move_pet(x,y,area): if x > 330: pivot_right((x-320)/(320*12)) elif x < 310: pivot_left((320-x)/(320*12)) elif ((310 <= x <= 330) and area<1500000): forward(0.05) elif ((310 <= x <= 330) and area>4000000): reverse(0.05)
We know the center of each image frame is 320 because the total width of each image frame is 640. The robot moves after each movement so the tennis ball stays at the center of image frame.
The total time for pivoting or turning the robot in each direction can simply be calculated using:
You can also use the calculated area to move the robot back and forward as well as turning which you can see in the video.
For example you know the tennis ball is too close to the robot and the robot needs to move back in the following case:
elif ((310 <= x <= 330) and area>4000000): reverse(0.05)
Or move forward the robot petdog if tennis ball is too far:
elif ((310 <= x <= 330) and area<1500000): forward(0.05)
Now you can use the following commands to start the robot:
sudo modprobe bcm2835-v4l2 sudo python petdog.py
This includes the overview of object tracking code and you can go ahead and create your own raspberryPetDog robot. I hope you create your own raspberryPetDog robots with even fancier features.
07/21/2017 at 03:48 •
In this post, I will go over details to drive the robot manually using keyboard and command line.
I won't go over the details of setting up raspberry pi Raspbian OS as there are many good tutorials and it's not the main point of this project.
Once you login automatically then you can go ahead and connect to your wi-fi network.Once you are connected to internet then you can enable SSH and camera module.
You can enable SSH and camera module using Raspberry Pi Configuration if you use GUI and following command if you use command line:
Next update your system's package list by entering the following command:
sudo apt-get update
And upgrade all your installed packages to their latest versions with the command:
sudo apt-get dist-upgrade
Next find the IP-address of your pi using following command:
As you can see my IP-address of raspberry pi is 192.168.1.85.
Now go ahead and download robot_cmd.py from github repository of this project in your computer.
This code lets you drive the robot using keyboard keys through SSH.
I use PuTTY as a SSH client because I use a windows machine which you can find a complete PuTTY tutorial on it on raspberry pi website. You can use fileZilla to transfer downloaded code from github repository in your computer to your raspberry pi as it's shown below.
You can now run robot_cmd.py using following commands:
sudo python robot_cmd.py
Now using following table as controller guide to drive the robot:
Character Action w Move forward s Move reverse a Turn Left d Turn Right q Pivot Left e Pivot Right x Exit
Double check that your robot behave according to the guide otherwise you may need to switch wires of your DC motors connected to L298N dual H-bridge module.
Finally, I go over the code and explain how it works:
def init(): gpio.setmode(gpio.BOARD ) gpio.setup(7, gpio.OUT) gpio.setup(11, gpio.OUT) gpio.setup(13, gpio.OUT) gpio.setup(15, gpio.OUT) gpio.setup(16, gpio.OUT) gpio.setup(18, gpio.OUT) gpio.output(16, True)
You first need to set gpio mode to the Board to use GPIO pin numbers using following:
Now you setup GPIO pins as output and enable pin #16 to enable DC motors.
gpio.setup(7, gpio.OUT) gpio.setup(11, gpio.OUT) gpio.setup(13, gpio.OUT) gpio.setup(15, gpio.OUT) gpio.setup(16, gpio.OUT) gpio.setup(18, gpio.OUT) gpio.output(16, True)
The GPIO pins needs to cleared and reset at the end of each action by setting up status of GPIO pins to false:
def clean_up(): gpio.output(7, False) gpio.output(11, False) gpio.output(13, False) gpio.output(15, False) gpio.output(16, False) gpio.cleanup()
Now I explain the functions to move forward and reverse:
def forward(tf): gpio.output(7, False) gpio.output(11, True) gpio.output(13, True) gpio.output(15, False) time.sleep(tf) clean_up()
GPIO pin numbers 11 and 13 are set to True as they are responsible for moving DC motors forward.The pin numbers 7 and 15 are set to False as they are responsible for moving DC motors backward or reverse.
The input parameter tf defines how long the motors will turn.
time.sleep(time) function defines how long the motors have to turn before stopping using following clean_up function.
The function to move the robot backward works the same way except you set GPIO pins 7 and 15 to True and GPIO pins 11 and 13 to False.
def reverse(tf): gpio.output(7, True) gpio.output(11, False) gpio.output(13, False) gpio.output(15, True) time.sleep(tf) clean_up()
Now how does the turn_left function work? Well,That's easy! You just enable GPIO pin #13 as it turns forward both DC motors on the left side of robot.
def turn_left(tf): gpio.output(7, False) gpio.output(11, False) gpio.output(13, True) gpio.output(15, False) time.sleep(tf) clean_up()
The pivot left function pivots the robot or turn the robot around z-axis counter clockwise.To achieve this,the motors on right side need to turn forward while the motor on left side turn in reverse direction or backwards. The output status of GPIO pin number 11 and 15 are set to True to achieve this as it's shown below in pivot_left function.
def pivot_left(tf): gpio.output(7, False) gpio.output(11, True) gpio.output(13, False) gpio.output(15, True) time.sleep(tf) clean_up()
The following code enables the program to perform robot movement actions and keep listening to keyboard strokes without exiting the program.So it calls getch() to perform action based on character entered and move to next line and listen for next character.
def main(): while True: print("\nKey: '" + getch() + "'\n")
The initialize function needs to be called every time you enter a character to make sure GPIO pins are initialized and setup.You need to reset the and initialize GPIO pins after each action otherwise raspberry pi throws errors that the pins are being used.
def getch(): import sys, tty, termios init() sleep_time = 0.050
termios provides an interface to the POSIX calls for tty I/O control.
The sleep_time is set to 0.05 seconds to achieve the best possible performance but it can be changed to customize response time.
The following line makes sure only one character is read at a time.
ch = sys.stdin.read(1)
The conditional statements are used to perform actions based on entered character.
if (ch == 'w'): forward(sleep_time) elif ch == 's': reverse(sleep_time) elif ch == 'a': turn_left(sleep_time) elif ch == 'd': turn_right(sleep_time) elif ch == 'q': pivot_left(sleep_time) elif ch == 'e': pivot_right(sleep_time) elif ch == 'x': clean_up() sys.exit() else: clean_up() pass init()
This includes the explanation of manual control code of the robot and in next post, I will go over the petdog code which enables the robot to follow a tennis ball.
07/20/2017 at 04:54 •
In this post, I will go over instructions to setup and put together the Robot Platform.
First let’s start with components list:
- 1 x Raspberry pi 2 or 3
- 1 x Raspberry pi camera module
- 1 x L298N dual H-bridge module
- 1 x SainSmart 4WD Chassis Aluminum Mobile Robot Platform
- 1 x TriBorg GPIO replicator board
- 1 x Adafruit Mini Pan-Tilt Kit
- 2 x Micro servos
- 1 x Raspberry Pi USB Wifi adapter dongle
- 1 x Toggle switch
- 20 x Jumper wires
- 1 x 12 V rechargeable Battery or 8 AAA batteries
- 1 x Charging socket
The first step would be mounting 4 DC Motors to 2 side plates and then connecting the lower plate to side plates. Once that is done then you can go ahead and mount the L298N dual H-bridge module to the middle of lower plate.
The next step would be connecting the DC Motors to the L298N dual H-bridge module as it’s showed in following picture:
Each DC motor has two wires for rotating in each direction (One wire for turning the motor clockwise and one wire for rotation in counter clockwise direction). The forward wires of motors on the left side needs to be connected to out1 port of L298N dual H-bridge and the reverse wires are connected to out2 pin. The same goes for the right side of robot and forward wires are connected to out3 port of L298N dual H-bridge and the reverse wires are connected to out4 pin.
Next the rechargeable battery should be mounted at the back of lower plate as it’s showed above. Once the battery secured then you can go ahead and connect the +12v and GND wires of the battery to the VCC and GND inputs of L298N dual H-bridge module. A power switch and a charging connection needs to be used as it’s shown below in case you would like to use a reachable battery.
The GND and +5V of L298N dual H-bridge module need to be connected to pin #6 (GND) and pin #2 (+5V) of raspberry pi. I decided to use a TriBorg GPIO replicator board as it gives me more power pins and it replicates GPIOs.
The next step would be connecting the input pins of L298N dual H-bridge module to the GPIO’s of raspberry pi in following order:
- IN1 to GPIO pin # 7 ( Orange)
- IN2 to GPIO pin # 11 ( Yellow)
- IN3 to GPIO pin # 13 ( Green )
- IN4 to GPIO pin # 15 ( Blue )
- ENA to GPIO pin # 16 ( Purple )
- ENB to GPIO pin # 16 ( White )
You will need to look at raspberry pi pinout map to make sure pins are connected correctly.
ENA and ENB pins are used to enable and disable Outputs of L298N dual H-bridge.
Now we can go ahead and mount the top plate, front plate and back plate of Robot Platform.
This covers the setup of Robot Platform and you can visit Sainsmart website for more details on setting up the Robot Platform.
In next post, I will go over the driver code and explain how to move the robot using a wireless keyboard.
04/16/2017 at 00:07 •
The robot can track tennis ball at the moment.