This is my 3rd robot project. This bot is 90% about software. I have put years of effort into developing a "self learning" verbal operating system and robot. This bot can learn new words on its own, learn things to say using a combination of wikipedia, RDF sources, Wolfram, etc. This bot builds topic specific knowledge graphs and uses them to answer questions.
For vision, this bot is using a combination of a 3D depth cam for vision and spacial awareness and a 3D tracking cam (with integrated IMU) for localization. The images are handed off to an Intel Neural Compute Stick for processing the frames through YOLOv3 and other models to for object detection and other purposes.