Prepare one ESP32-CAM / one FTDI USB to TTL Serial Converter / one mini USB to USB converter / 4 wires
Connect your ESP32-CAM to Serial Converter with the following picture:
Use Arudino IDE to program your ESP32-CAM
The code can be found here on SayHOLO Github >>> IMG_To_Google_Drive
* To program your ESP32-CAM, make your to use jumper or wires to connect IO0 & GND. Remove it when you finish compiling.
Make sure you update the following feature and replace with your own:
const char* ssid = "xxxx"; //your network SSID const char* password = "xxxx"; //your network password String myScript = "/macros/s/xxxx/exec"; //Create your Google Apps Script and replace the "myScript" path.
For "myScript", open Google APP Script and create a new project
Place your code with "Google_APP_Script.gs" found on Github and deploy your script to obtain the token ID.
*Please remember to make the script public such that your ESP32-CAM can transmit to Google Drive properly.
Once everything has been setup, you should be able to see your images taken by ESP32-CAM in your Google Drive.
3Generate 3D Human Model
Here comes to the exciting parts - 3D image formation
* Since I am using MacBook Pro to run the algorithm, it does not support certain library that require NVIDIA GPU. For alternative choose, Google Colaboratory can solve the issue.
Open SayHOLO Google Colab ( SayHOLO ) & save it to our own google account
First of all, run first line to connect Google Colab with your Google account.
Then run the rest of the program to generate 3D human model. Lets see whats going on with the Python script:
- The first algorithm the image run through is called U-2-Net. Basically, it removes the background and leaving the image of the person only. This step is helpful for generating more accurate 3D model.
- The second algorithm is call PIFuHD developed by people at University of Southern California, Facebook Reality Labs, and Facebook AI Research. After the 2D image has run through PIFuHD, it will generate 3D object file.
- After the object file has successfully generated, it will be modified and transmitted to AWS S3. For simplicity, I have created my own cloud storage on AWS S3 but feel free to change it to your if you prefer.
Tips: The background removal algorithm deal with clear background. If your background is too complicated, the algorithm might not be able to generate clear images. PIFuHD generate better 3D model if the image contains the whole body picture; half of your body's picture usually leads to disastrous results.
4View Your Result
I have design a custom ReactJS website to display the result generate from PIFuHD algorithm. The sample website can be view here >>> SayHOLO
Open the website with your tablet ( I use iPad for demonstration) and place the hologram projector on top of the tablet. The 3D model will be display on your hologram device.
5Lets View The Complete Steps