Close

Anna the robot calling OpenCV functions face detect/tracking and canny frame from plain text commands with voice response.

A project log for Anna The Multibot!

Multifunctional robot to help bring the dream of a personal robot into reality.

dennisDennis 07/29/2016 at 23:570 Comments

Here’s two automated functions added to Anna. Face detect and canny frame. Both are called by plain text commands and both have a voice response. Below the video is a breakdown of the steps.

Now with a way to call different automated processes from the last log, it’s time to add some processes. I thought I would start off with some OpenCV functions. Luckily with a wrapper like EMGU, adding OpenCV is fairly easy. To automate the process, we need a process to call, a timer, points to check to see if the process is being requested and a way to make the point active.
I added a new class to contain the OpenCV functions called "OpenCV_functions". Within the class, I added routines for face detection and canny frame (more to be added later).
Two Boolean points are also added to the main page. One is called “face” and one is called “canny_frame”.
Public canny_frame As Boolean = False
Public face As Boolean = False
Both are Public to allow them to be set/cleared from other classes and they are both initialized as false.
In the “command” class under the “text_decode” sub I added the following to lines of code:
If command_input.Contains("show") Then show()
If command_input.Contains("cancel") Then cancel()
Also in the command class I added the following subs:


Public Sub show()
If command_input.Contains("canny") And command_input.Contains("frame") Then
Machine_vision.canny_frame = True
Dim Thread2 As New Thread(Sub() Speech.speak_text("Canny frame is now active, by your command"))
Machine_vision.Command_text.Text = Nothing
Thread2.Start()
End If
If command_input.Contains("face") And command_input.Contains("detect") Then
Machine_vision.face = True
Dim Thread2 As New Thread(Sub() Speech.speak_text("face detect is now active, by your command"))
Machine_vision.Command_text.Text = Nothing
Thread2.Start()
End If
End Sub
Public Sub cancel()
If command_input.Contains("canny") And command_input.Contains("frame") Then
Machine_vision.canny_frame = False
Dim Thread2 As New Thread(Sub() Speech.speak_text("Canny frame is now inactive, by your command"))
Machine_vision.Command_text.Text = Nothing
Thread2.Start()
End If
If command_input.Contains("face") And command_input.Contains("detect") Then
Machine_vision.face = False
Dim Thread2 As New Thread(Sub() Speech.speak_text("face detect is now inactive, by your command"))
Machine_vision.Command_text.Text = Nothing
Thread2.Start()
End If
End Sub


Last I added a timer to the main page:
Private Sub Timer1_Tick(sender As Object, e As EventArgs) Handles Timer1.Tick
load_jpg()
PictureBox1.Image = picture
If canny_frame = True Then Opencv_functions.canny()
If face = True Then Opencv_functions.face_detect()
End Sub
The load() loads the picture from the camera to proccess and if the Boolean point is set the timer will call it. The Boolean points are set and cleared in the command class based on text commands.
Below is the OpenCV functions:


Public Sub face_detect()
faces = Nothing
face_count = 1
Try

Dim ColordImage As Image(Of Bgr, [Byte]) = New Image(Of Bgr, Byte)(picture)
'Load the object detector
Dim faceDetector As New CascadeClassifier("haarcascade_frontalface_default.xml")
'Convert the image to Grayscale
Dim grayImage As Image(Of Gray, [Byte]) = ColordImage.Convert(Of Gray, [Byte])()
For Each face As Rectangle In faceDetector.DetectMultiScale(
grayImage,
1.1,
10,
New Size(25, 20),
Size.Empty)
CvInvoke.Rectangle(ColordImage, face, New MCvScalar(0, 0, 255))
Dim p As PointF = New PointF
p = face.Location
Dim size As String = face.Size.ToString
faces = faces + ((" face" & face_count & vbLf + (" X: " _
+ (p.X.ToString + (" Y: " + p.Y.ToString))) + " " + size + ";"))
face_count = face_count + 1
Next
Dim MyImage As Image = ColordImage.ToBitmap()
Machine_vision.PictureBox1.Image = MyImage
Catch
End Try
End Sub


Public Sub canny()
Try
Dim cannyFrame As New Mat()
Dim cannyFrame2 As New Mat()
Dim ColordImage As Image(Of Bgr, [Byte]) = New Image(Of Bgr, Byte)(picture)
Dim grayImage As Image(Of Gray, [Byte]) = ColordImage.Convert(Of Gray, [Byte])()
Dim pyrDown As New UMat()
CvInvoke.PyrDown(grayImage, pyrDown)
CvInvoke.PyrUp(pyrDown, grayImage)
CvInvoke.Canny(grayImage, cannyFrame, 100, 60)
Dim img As Image(Of Bgr, [Byte]) = cannyFrame.ToImage(Of Bgr, [Byte])()
Dim MyImage As Image = img.ToBitmap()
Machine_vision.PictureBox1.Image = MyImage
Catch
End Try
End Sub

Discussions