Close
0%
0%

HMPUS — Himani Multimodal Perception & Understandi

Modular ROS2 perception stack for a humanoid robot — camera, YOLO detection, custom object tracking, evolving toward a full world model.

Similar projects worth following
0 followers
  • Description
  • Details
  • Files 0
  • Components 0
  • Logs 0
  • Instructions 0
  • Discussion 0
Himani Multimodal Perception & Understanding System (HMPUS) is the perception "brain" being built incrementally for a humanoid robot, starting from raw camera input and growing toward full multimodal scene understanding. The system is designed bottom-up as a set of independent, single-responsibility ROS2 nodes rather than one monolithic AI pipeline: Camera Node — streams frames from an ESP32-CAM over Wi-Fi into ROS2 as sensor_msgs/Image, with no knowledge of what's in the frame. Perception Node — runs YOLOv8 (GPU-accelerated) on each frame to detect objects, publishing structured vision_msgs/Detection2DArray messages. Tracking Node — a custom-built IoU-based multi-object tracker (implemented from scratch, not just a library) that assigns persistent track_ids across frames, with full lifecycle management (NEW → TRACKING → LOST → REACQUIRED/REMOVED) so the robot understands the same object over time, not just isolated per-frame detections.

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates