Close
0%
0%

Atomic Synchronator => TicTacSync

Beyond '67 SMPTE timecode: YaLTC is a new GPS based AUX syncing track format for dual-system sound based on affordable hardware

Public Chat
Similar projects worth following
This is a cheap hardware dongle to aid in automatically syncing audio and video you're shooting. Eh oui, yet another LTC... hence the name: YaLTC.

It should interest scientists who need to timestamp data recordings within sub-millisecond precision and multicam /dual-system sound videographers
• who don't want to fork 200 CAD for a syncing program, neither 270 CAD for each recording device to sync (cams, audio recorders)
• who have encountered some limitations of their editing software "waveform analysis syncing".


Maybe I'll eventually distribute pre-flashed kits on tindie (essentially off the shelves components). A GUI application is considered for non-dev users (a lot of videographers DONT have a running python installation). Wrapping synced material under OpenTimelineIO is explored.


Project status: working, two continuous tone audio files have been synced within a 83 μs offset, see this log.


Project Stage: alpha

I've implemented an audio sync track generator using the 1 PPS from a GPS receiver. Between each seconds the time of day is BFSK modulated into an audio signal and parsed in post-production by  tictacsync.py, merging sound clips with their corresponding video tracks (thanks to FFmpeg) and this, before importing into your NLE of choice.

Side Note: Someone more competent than me should have tackled this! I've NO formal compsci education except two programming introduction courses 40 years ago! (Pascal and Fortran) But here we go...

Pros and cons:

  • no jam sync required
  • no settings. Plug'n go, it is frame rate agnostic (DF, NDF? meh)
  • no drift in GNSS mode (it's atomic)
  • subframe precision (tests show files are synced within 83 μs, that's 0.003 frame!)
  • low cost (20-30 USD per device)
  • can be used with cameras lacking mic input (acoustic coupling was a pain in the ass to R&D, though)
  • more resilient than NLE waveform analysis
  • it's FOSS

but

  • some assembly required
  • not standard (it's a feature, not a bug)
  • adequate GPS signal strength needed for wireless mode  (not a problem for most residential buildings)
  • CLI only, for now
  • one guy in his basement project (hopefully not for long)
  • crappy code (but I'm working on it)

All three hard/soft components have been implemented and tested:

  • two hardware dongles (SAMD21 based)
  • firmware
  • post-production analysis desktop software (proof-of-concept version)

Here's the analysis results for two test files:

Watch here the first complete automated audio/video merge: Syncing in the kitchen. For multicam sync, I'll use Olive capacity to read OpenTimelinIO files.

  • 1 × a SAMD21 board eg Adafruit Feather M0 proto, 20 USD
  • 1 × a GPS module with 1PPS output eg Adafruit Feather Ultimate GPS 40 USD
  • 1 × a single supply opamp for mid-level reference aka virtual ground I'm using a LM358
  • 1 × Arduino IDE, python and FFmpeg installed

  • This is getting serious

    Raymond Lutz03/10/2022 at 00:25 0 comments

    The project has it own domain name.

  • Dedrifting

    Raymond Lutz02/19/2022 at 03:35 0 comments

    Before merging any audio to a video (or to a reference audio, called main sound), any track will be dedrifted using pysox tempo transform if the detected clock drifts result in a delay longer than 5 ms for sound tracks merged together or 10 ms for merged audio and video.

  • Code rewrite

    Raymond Lutz02/03/2022 at 20:48 0 comments

    Pushing last updates to Source Hut

  • Cabling

    Raymond Lutz01/22/2022 at 19:42 0 comments

    Using Adafruit Feather boards.

  • It is now called the Tic Tac Sync!

    Raymond Lutz12/24/2021 at 23:05 0 comments

  • Do you have any reference? I don't want to block any DC

    Raymond Lutz12/20/2021 at 13:04 0 comments

    The prototyping area of the Feather M0 is handy for building a voltage reference (aka virtual ground) around a LM358:

    I know there is a DC blocking capacitor built in the recorder audio input stage (or yet a DC-servo circuit to track the signal mean over time): I did feed the 0-3V3 directly into it (hence a signal with a 1.65V DC offset) but it resulted in ringing and more importantly in a lag of almost 500 μs ! I'll try the same measurements with an onboard cap to see if it makes any difference. I'll post data latter.

  • Packaging crappy code

    Raymond Lutz11/07/2021 at 02:13 0 comments

    There's even a pypi package for my dualsoundsync script !

    Repositories are here:


    How bad is my code?

    • It is not commented yet
    • test coverage is null, for now

    at least

    • it doesn't crash
    • it works (syncing dual system sound recordings)
    • it doesn't use print() for logging
      to do (software side, for dualsoundsync ):
    • comments and tests
    • optional logging
    • implement syncing of videos with intermittent audio recordings
    • implement syncing of audio with intermittent video recordings

    in the long run, build a standalone GUI program (with Qt?)

  • Syncing in the kitchen

    Raymond Lutz10/14/2021 at 13:17 0 comments

    For now enjoy this dual-system sound demo. Stay tuned for a multicam demo with Olive and OpenTimelinIO

    Here is the camera recording: image + sync signal (fingers sound is absent, remember: this is a dual system sound set-up):

    This is the ZOOM H4n Pro recorder tracks. Note the thin spikes in the right channel (the yellow ovals): they are the sync pulses from the GPS, straight on the UTC second (the left channel are the fingers snipping). Those sync spikes are present in the movie too (by construction)

    Postprocessing with dualsoundsync.py, aligning the spikes in the audio recorder file with those in the camera file:
    $ dualsoundsync /media/shooting_files_rep
    
    scanning rushes_folder
    splitting audio from video...
    splitting audio R and L channels
    trying to decode YaLTC in audio files
    1 of 4: nope
    2 of 4: ZOOM_Rchan.wav started at 2021-10-13 01:45:08.263145+00:00
    3 of 4: nope
    4 of 4: fuji_a_Lchan.wav started at 2021-10-13 01:45:14.361967+00:00
    looking for time overlaps forming pairs...
    joining back synced audio to video: dualsoundsync/synced/fuji.mp4


    Tadaam, fuji.mp4 automagically synced!

  • Synced !

    Raymond Lutz09/11/2021 at 14:17 0 comments

    As a test for assessing system performance, the same pilot tone (a continuous sinus wave of 220 Hz) has been recorded simultaneously by two devices: a ZOOM H4n Pro and a FUJIFILM X-E1) alongside their atomicsynchronator dongle (recordings where started one after the other so they were off sync by approx. 3 seconds)


    Each devices recorded two channels: the common 220Hz CSW and the YaLTC signal from their respective synchronator (an Adafruit Feather based for the H4n and a Trinket M0 one for the FUJI). The camera was used as a stereo sound recorder: the movie itself being discarded.

    The two tone tracks have been synchronized using the timestamp deduced from their YaLTC track and mixed into a single stereo file (the earlier recording is simply trimmed of the exact samples number so they start at the same time). Below are the decoding program output; we see the H4n (ZM-YaLTC.wav) was started 39.247354 - 36.062314 seconds earlier. Take note: 7 out of  8 of those yummy digits are significant!

    The precision of the sync process is then given by how much the left and right 220 Hz recordings are out of phase:


    They're out of sync by a mere 4 samples: a 83 μs offset! (for reference, typical video frames are spaced 33000 μs apart). Cool.

  • But it's only parts per million!

    Raymond Lutz08/05/2021 at 01:04 0 comments

    The atomicsynchronator encoding is precise enough to evaluate each device *effective* sample rate: my Nikon D300S sound is spec'd at 44100 Hz but is really 44099.688 Hz (all digits are significant), a discrepancy more simply expressed in ppm (here 7 ppm). My Fuji X-E1 shows an error of 33 ppm. Why is this consequential? Because when syncing sound and video from 2 distinct devices at the clip start, the tracks will drift apart slowly and for long enough takes (interviews, live shows) it will be noticeable.

    How long, you ask? Take the maximum acceptable offset between video and sound to be 15 milliseconds (cf link); this small interval would be 33 ppm of the whole recording, hence T = (1e6/33)*15e-3 = 454 seconds! After about 8 minutes the drift is out of spec... The solution? Stretch (or compact) one of the sound track to match the reference track frequency using sox prior to merging sound.

View all 23 project logs

View all instructions

Enjoy this project?

Share

Discussions

Sean McVeigh wrote 01/03/2021 at 22:25 point

Considering this is an indoor device, why not use a chip-scale atomic clock?
EDIT: never mind, thought the price had come down, although you could sell as pro product :D

  Are you sure? yes | no

Raymond Lutz wrote 01/03/2021 at 22:36 point

ah ah! You got me on that one! And I thought you were joking until I looked it up!!! Those devices DO exist!!!  But for now prohibitively onerous... https://www.mouser.ca/ProductDetail/Microchip-Technology/090-03240-001/?qs=w%2Fv1CP2dgqpPJJa0jwVC6w%3D%3D

Anyway, you would have the hassle to jam sync them before use... Also this GPS chipset works OK indoor, even in a modern building with a steel framing (but in a basement reception in spotty).

  Are you sure? yes | no

Sean McVeigh wrote 01/03/2021 at 22:47 point

Yeah I worked on one in grad school (modded as magnetometer): https://tf.nist.gov/general/pdf/2703.pdf

  Are you sure? yes | no

Sean McVeigh wrote 01/03/2021 at 22:51 point

Wait, I assumed you only need one clock source? Use Ethernet for sync to capture devices: https://en.wikipedia.org/wiki/Precision_Time_Protocol

This is the technique used for multi-AP clock sync for triangulation in indoor location, often using GPS as CLK source. Didn't look too carefully if the recent updates meet your timing requirements.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates