Lip sync​

A project log for TAST-E, the robot with a sense of taste and smell

An animatronic robot head with an artificial tongue and nose

m-bindhammerM. Bindhammer 12/05/2023 at 15:470 Comments

The following code lets the EMIC-2 speak what you enter into the serial monitor:

#include <SoftwareSerial.h>

#define rxPin   10  // Serial input (connects to Emic 2's SOUT pin)
#define txPin   11  // Serial output (connects to Emic 2's SIN pin)

// Set up a new SoftwareSerial port
SoftwareSerial emicSerial =  SoftwareSerial(rxPin, txPin);

void setup()  
  // Define pin modes
  pinMode(rxPin, INPUT);
  pinMode(txPin, OUTPUT);
  // Set the data rate for the SoftwareSerial port
    When the Emic 2 powers on, it takes about 3 seconds for it to successfully
    initialize. It then sends a ":" character to indicate it's ready to accept
    commands. If the Emic 2 is already initialized, a CR will also cause it
    to send a ":"
  emicSerial.println("P1");           // Select parser, P0: DECtalk, P1: Epson
  emicSerial.println("N4");           // Nx, select voice:  x = 0 to 8
  emicSerial.println("V5");           // Vx, set the audio output volume in dB from x = -48 (softest) to x = 18 (loudest).
  emicSerial.println("W200");         // Wx, set the speaking rate in words per minute from x = 75 (slowest) to x = 600 (fastest).
  emicSerial.print('\n');             // Send a CR in case the system is already up
  while ( != ':');   // When the Emic 2 has initialized and is ready, it will send a single ':' character, so wait here until we receive it
  delay(10);                          // Short delay
  emicSerial.flush();                 // Flush the receive buffer

void loop()  
  while (Serial.available() == 0); // Wait for data available
  String userInput = Serial.readString(); // Read until timeout
  userInput.trim(); // Remove any whitespace at the end of the String
    Wait here until the Emic 2 responds with a ":"
    indicating it's ready to accept the next command
  while ( != ':'); 

The problem with lip-synchronization is that there is no way on the software side to determine exactly when the EMIC-2 has finished the speech. The line

while ( != ':'); 

is completely useless for that. It merely shows when the EMIC-2 is ready for new commands, which will come true long before the end of the speech. Others have already had this problem and I will solve it in the same way. I will use the audio output of the EMIC-2 to control the four servos that are responsible for the movement of the mouth. This requires a so-called envelop follower, which reproduces the volume envelope of an applied waveform.

Fig. 1 A signal and its envelope marked with red

Fig 2. shows such an envelope detector circuit.

Fig. 2

Apart from the operating voltage, it is only important that the opamp has a rail-to-rail output swing. The LMV358 can also be used, for example.