• Ground Zero

    Rodolfo05/13/2019 at 06:13 0 comments

    The very first conversation about this project happened at my office back in 2018. A couple of students reached out to me asking if I could support a robotics club. We had since then have been working on different concepts, running activities to include others in our small world and trying to explore where to go from there. 

    This project page at Hackaday is our internal space to reflect about the findings and possibly about interesting experiments we have been carrying out. We started our journey into what became the Swarm Robot Research Team by doing literature review and exploring the different technologies available. 

    Before even starting with the development, we did an extensive literature review on what other universities have been working on. Regardless that our research group is within an undergrad program , it was good to learn what other people have been developing. The most noteworthy examples of robot swarms have been:

    RICE's R-One 

    Harvard's Kilobot

    Stuttgart's Jasmine Swarm

    Platforms we tested:

    - Raspberry Pi with OpenCV

    - arUco Markers transmitted by radio to micro:bit

    - ESP-32 with IDF as dev environment

    - ESP-32 with Arduino as dev environment

    - ESP-32 with micropython

    Even setting up the Raspberry Pi to work with NYU Shanghai wifi was a struggle. We were able to configure it thanks to Jack B. Du's tech documentation. Nevertheless for the Raspberry Pi 3 and for certain WiFi dongles, the drivers were also a challenge to set up.

    The experience with IDF for ESP32 was not as smooth as we wished. The errors were different in the diverse OS we had and installing all dependencies seemed to be a challenge itself. After a whole day of debugging, we finally were able to blink an LED on each ESP32. This proved to be a clear obstacle for our development, where we would be forced to first become familiar with the environment. 

    On the contrary, we already had expertise using Arduino IDE. Setting up ESP32 with the board manager was just a matter of minutes and we quickly started coding a webserver and a set of RGB smart LEDs with the FastLED library.

    It was also interesting to find that there were already WiFi mesh implementations ( easyMesh and painlessMesh). That would be great to satisfy with the requirement of scalability that WiFi alone could not satisfy if we decide to use ESP32.

    Next steps with ESP32 is to add it to a mobile platform. We are in the process of wiring WSP32 to Plobot because it has many of the sensors we need for the project.

    Besides of the communication between the different robots being scalable, the relative position from each other is critical for the project. We replicated the technique of Kilobot of reflecting infrared to calculate distance. 

    We also explored using computer vision as an exploration to detect positions and from there calculating the distances. This worked well when using a PC (mac mini, Windows, macbook pro). The only issue was that it required to calibrate the algorithm for the specific camera and we had to handle logistics by hanging a webcam on the roof plus getting an extra long (5m) USB cable.

    In contrast to the x86 architecture setup, it was an outstretch when we tried to set it up on a Raspberry Pi. The two reasons were that managing dependencies when moving to ARM architecture. When using RPi, we had to compile the OpenCV with the specific modules and only after that see that we had a poor frame-rate. More work should be developed 

  • Log 2: More Exploration

    AlisonYao05/10/2019 at 16:37 0 comments

    We did more exploration tonight. We tried out how to use Arduino to program ESP32 and control a LED matrix. Also, we improved the IR testing by reversing the wiring on the IR receiver. In the end, we tried again with the wifi connection of Raspberry Pi. 

    Several Things We Did:

    1. Using Arduino to program ESP32 and control a LED matrix

    Our professor, Rudi, is teaching Benny and James how to program ESP32 using Arduino.

    The work of James and Alison.

    2. Improving IR testing

    Thomas and Sheldon  reversed the wiring on the IR receiver and made the signal caption stronger. And they also tried using conductive tape to strengthen the reflection of the IR signal.

    A close-up of the transmitter.

    3. Wifi connection of RaspberryPi.

    Zander has tried to connect to the wifi of NYU, but it did not seem to be working. So this time, we tried to connect to a personal hotspot to see if anything different happens. Sheldon connect the RaspberryPi to his own hotspot, but for some reason, it did not work as well. 


    Today is mostly about the hardware part.


    1. Making good use of the Arduino Library can help one explore more about how to program ESP32.

    2. ESP32 can set Wifi so that the control menu will be easier to operate.

    IR testing:

    1. Accidentally  reversing the wiring on the IR receiver can bring a pleasant surprise! With the help of another professor, we understood why rewiring worked.

    2. The reflection of IR signal requires better material then just conductive tape, but right now, we are able to catch the signal from pretty far away.


    1. Continuing perfecting IR testing 

    2. Figure out why the wifi connection of RaspberryPi is not working.

  • Log 1: A Great Start

    Zander Mao05/03/2019 at 16:18 0 comments

    Tonight has been a very productive night. We set up our working space, gathered the materials we need, started tinkering with ESP32 and set up the test circuit for IR trasmitters/receivers.  Although there's still a long way to go to achieve our goal, at least we are making progress.

    Several things we did:

    1. Set up our storage room

    We cleaned up a room, move some of Rudi's robotics material and some other useful stuff (capacitors, resistors, and even an old computer!) to the storage room.

    Thomas moving the shelf
    Rudi and Sheldon check what's available

    Our discoveries

    2. Set up ESP-IDF

    It took us a long time to set things up, but eventually Sheldon, Alison and James all can upload examples to ESP-IDF. That's a great start.

    ESP32 running hello_world example

    3. Set up IR testing

    We set up the circuit to test how IR receiver/trasmitter work. We read the analog output of the IR receiver using an Arduino and used ESP32 as a power (no special reason, just because it's there). For now it doesn't work very well, but we'll try to improve that later.

    Prototyping IR test

    Circuit Provided by Rudi

    Preliminary Result, We read something, but not sensitive enough

    Test Plaform. IR receivers/trasmitters are glued to the plastic pieces.


    Basic Linux Command:

    ls: list all the file under current directory

    cd /children/path: change directory to childern path

    cd: go home

    cd .. : go back to the parent folder

    cp -r /file/you/copy /where/you/want/to/put : -r is a flag indicating recursive, meaning recursively copy every file and every subfolder in the directory you copied.

    chmod (*number) /directory: change users' permission to modify a file. Usually there are three numbers, which stands for the permission for the file owner, group and others. Translated in binary, each of the numbers corresponds to the status for three different kinds of permission: read(r), write(w), execute(x). For example, if I set some folder by typing chmod 777 /folder/path, it will be for file owner, group and others, the permission will be 111 (7's binary representation), which means they will be able to read, write and execute the file. If I set them to 444, all of them will only have the permision of 100(4's binary reprpesentation), which means they can only read.

    export IDF_PATH=/your/idf/path: set IDF_PATH

    mkdir: make directory

    cat: peek into file

    ESP-IDF configuration:

    1. clone from https://github.com/espressif/esp-idf.git

    2. set IDF_PATH to where you cloned the project : export IDF_PATH=your/path/to/esp/esp-idf

    3. add tool chain path to the PATH variable: export PATH=your/path/to/esp/xtensa-esp32-elf/bin:$PATH

    4. Go to project folder, use make menuconfig to set up ports to upload. By default it's ttyUSB0, but on Mac it might be ttyUSBtoUART (something like that). The chip uses a silcon lab CP2102 chip to handle serial communication, so probably a driver for that is needed.

    One more thing about ESP32:

    It has analog output (PWM) when using Arduino IDE. For simplicity, we will  explore that way of programming.


    1. Improve IR distance test

    2. Look into VSCode plugin "PlatformIO IDE" and try to simplify the steps to connect.