Say cheese!

A project log for Stack-chan - JavaScript driven super-kawaii robot

An easy-to-build and companion robot for everyone

Shinya IshikawaShinya Ishikawa 10/01/2021 at 15:120 Comments

Smiling Stack-chan, synchronized with my face!

It can follow my face orientation, eye/mouth opening and smiling.

I used Google MediaPipe for face tracking. MediaPipe extracts facial contours and meshes with very high accuracy. I used those meshes to calculate the orientation of the face and the opening and closing of the eyes and mouth.

I used WebSocket to send information to Stack-chan. The Moddable framework includes a WebSocket module. It's easy to use and fast enough, about 20 fps in the case of the above video.

Face synchronization adds a powerful capability to Stack-chan for many use-cases such as teleexistence avatar robot.