Close
0%
0%

Atomic Synchronator => TicTacSync

Beyond '67 SMPTE timecode: YaLTC is a new GPS based AUX syncing track format for dual-system sound based on affordable hardware

Public Chat
Similar projects worth following
This is a cheap hardware dongle to aid in automatically syncing audio and video you're shooting. Eh oui, yet another LTC... hence the name: YaLTC.

It should interest scientists who need to timestamp data recordings within sub-millisecond precision and multicam /dual-system sound videographers
• who don't want to fork 200 CAD for a syncing program, neither 270 CAD for each recording device to sync (cams, audio recorders)
• who have encountered some limitations of their editing software "waveform analysis syncing".


Maybe I'll eventually distribute pre-flashed kits on tindie (essentially off the shelves components). A GUI application is considered for non-dev users (a lot of videographers DONT have a running python installation). Wrapping synced material under OpenTimelineIO is explored.

See the project New home


Project status: working. Audio clips are merged and synced to corresponding video, see this log.


Project Stage: alpha

I've implemented an audio sync track generator using the 1 PPS from a GPS receiver. Between each seconds the time of day is BFSK modulated into an audio signal and parsed in post-production by  tictacsync.py, merging sound clips with their corresponding video tracks (thanks to FFmpeg) and this, before importing into your NLE of choice.

Side Note: Someone more competent than me should have tackled this! I've NO formal compsci education except two programming introduction courses 40 years ago! (Pascal and Fortran) But here we go...

Pros and cons:

  • no jam sync required
  • no settings. Plug'n go, it is frame rate agnostic (DF, NDF? meh)
  • no drift in GNSS mode (it's atomic)
  • subframe precision (tests show files are synced within 83 μs, that's 0.003 frame!)
  • low cost (20-30 USD per device)
  • can be used with cameras lacking mic input (acoustic coupling was a pain in the ass to R&D, though)
  • more resilient than NLE waveform analysis
  • it's FOSS

but

  • some assembly required
  • not standard (it's a feature, not a bug)
  • adequate GPS signal strength needed for wireless mode  (not a problem for most residential buildings)
  • CLI only, for now
  • one guy in his basement project (hopefully not for long)
  • crappy code (but I'm working on it)

All three hard/soft components have been implemented and tested:

  • two hardware dongles (SAMD21 based)
  • firmware
  • post-production analysis desktop software (proof-of-concept version)

Here's the analysis results for two test files:

Watch here the first complete automated audio/video merge: Syncing in the kitchen. For multicam sync, I'll use Olive capacity to read OpenTimelinIO files.

  • 1 × a SAMD21 board eg Adafruit Feather M0 proto, 20 USD
  • 1 × a GPS module with 1PPS output eg Adafruit Feather Ultimate GPS 40 USD
  • 1 × a single supply opamp for mid-level reference aka virtual ground I'm using a LM358
  • 1 × Arduino IDE, python and FFmpeg installed

  • On masto

    Raymond Lutz02/02/2023 at 00:20 0 comments

    For those interested, I'm posting more frequent updates on my mastodon account (than here). They are tagged:

    https://mamot.fr/tags/atomicsynchronator

  • Timeline object completed

    Raymond Lutz07/25/2022 at 14:45 0 comments

    It's been a while!

    The Timeline object detects audio recordings time coincidences and sync-merged  them  to corresponding videos:

    Next step: processing multi-files recordings (there are recognized in but not merged yet).

  • This is getting serious

    Raymond Lutz03/10/2022 at 00:25 0 comments

    The project has it own domain name.

  • Dedrifting

    Raymond Lutz02/19/2022 at 03:35 0 comments

    Before merging any audio to a video (or to a reference audio, called main sound), any track will be dedrifted using pysox tempo transform if the detected clock drifts result in a delay longer than 5 ms for sound tracks merged together or 10 ms for merged audio and video.

  • Code rewrite

    Raymond Lutz02/03/2022 at 20:48 0 comments

    Pushing last updates to Source Hut

  • Cabling

    Raymond Lutz01/22/2022 at 19:42 0 comments

    Using Adafruit Feather boards.

  • It is now called the Tic Tac Sync!

    Raymond Lutz12/24/2021 at 23:05 0 comments

  • Do you have any reference? I don't want to block any DC

    Raymond Lutz12/20/2021 at 13:04 0 comments

    The prototyping area of the Feather M0 is handy for building a voltage reference (aka virtual ground) around a LM358:

    I know there is a DC blocking capacitor built in the recorder audio input stage (or yet a DC-servo circuit to track the signal mean over time): I did feed the 0-3V3 directly into it (hence a signal with a 1.65V DC offset) but it resulted in ringing and more importantly in a lag of almost 500 μs ! I'll try the same measurements with an onboard cap to see if it makes any difference. I'll post data latter.

  • Packaging crappy code

    Raymond Lutz11/07/2021 at 02:13 0 comments

    There's even a pypi package for my dualsoundsync script !

    Repositories are here:


    How bad is my code?

    • It is not commented yet
    • test coverage is null, for now

    at least

    • it doesn't crash
    • it works (syncing dual system sound recordings)
    • it doesn't use print() for logging
      to do (software side, for dualsoundsync ):
    • comments and tests
    • optional logging
    • implement syncing of videos with intermittent audio recordings
    • implement syncing of audio with intermittent video recordings

    in the long run, build a standalone GUI program (with Qt?)

  • Syncing in the kitchen

    Raymond Lutz10/14/2021 at 13:17 0 comments

    For now enjoy this dual-system sound demo. Stay tuned for a multicam demo with Olive and OpenTimelinIO

    Here is the camera recording: image + sync signal (fingers sound is absent, remember: this is a dual system sound set-up):

    This is the ZOOM H4n Pro recorder tracks. Note the thin spikes in the right channel (the yellow ovals): they are the sync pulses from the GPS, straight on the UTC second (the left channel are the fingers snipping). Those sync spikes are present in the movie too (by construction)

    Postprocessing with dualsoundsync.py, aligning the spikes in the audio recorder file with those in the camera file:
    $ dualsoundsync /media/shooting_files_rep
    
    scanning rushes_folder
    splitting audio from video...
    splitting audio R and L channels
    trying to decode YaLTC in audio files
    1 of 4: nope
    2 of 4: ZOOM_Rchan.wav started at 2021-10-13 01:45:08.263145+00:00
    3 of 4: nope
    4 of 4: fuji_a_Lchan.wav started at 2021-10-13 01:45:14.361967+00:00
    looking for time overlaps forming pairs...
    joining back synced audio to video: dualsoundsync/synced/fuji.mp4


    Tadaam, fuji.mp4 automagically synced!

View all 25 project logs

View all instructions

Enjoy this project?

Share

Discussions

zk0415004 wrote 12/13/2023 at 15:30 point

My cooperation with the project has been really beneficial to me. I'd want to share it with the CapCut apk mod team so they can read it and apply something new.

  Are you sure? yes | no

winfredp.rodriguez wrote 08/07/2023 at 06:17 point

Amazing analysis. I truly love it. I also want to use it for my website related to video enhancement. Here is the link: https://alightmoto.com/

  Are you sure? yes | no

alvipurvorex wrote 07/30/2023 at 16:00 point

My cooperation with the project has been really beneficial to me. I'd want to share it with the CapCut Pro team so they can read it and apply something new.

  Are you sure? yes | no

Sean McVeigh wrote 01/03/2021 at 22:25 point

Considering this is an indoor device, why not use a chip-scale atomic clock?
EDIT: never mind, thought the price had come down, although you could sell as pro product :D

  Are you sure? yes | no

Raymond Lutz wrote 01/03/2021 at 22:36 point

ah ah! You got me on that one! And I thought you were joking until I looked it up!!! Those devices DO exist!!!  But for now prohibitively onerous... https://www.mouser.ca/ProductDetail/Microchip-Technology/090-03240-001/?qs=w%2Fv1CP2dgqpPJJa0jwVC6w%3D%3D

Anyway, you would have the hassle to jam sync them before use... Also this GPS chipset works OK indoor, even in a modern building with a steel framing (but in a basement reception in spotty).

  Are you sure? yes | no

Sean McVeigh wrote 01/03/2021 at 22:47 point

Yeah I worked on one in grad school (modded as magnetometer): https://tf.nist.gov/general/pdf/2703.pdf

  Are you sure? yes | no

Sean McVeigh wrote 01/03/2021 at 22:51 point

Wait, I assumed you only need one clock source? Use Ethernet for sync to capture devices: https://en.wikipedia.org/wiki/Precision_Time_Protocol

This is the technique used for multi-AP clock sync for triangulation in indoor location, often using GPS as CLK source. Didn't look too carefully if the recent updates meet your timing requirements.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates