Close

Hack-an-Hour: audio QR codes

helgehelge wrote 09/11/2018 at 14:44 • 7 min read • Like

TL;DR: this is the result - a QR code you can read from an arbitrary spectrum analyzer app. It's a proof-of-principle endeavor, but maybe you can make use of the idea :)

The Project

--------------------

it's 1am and I feel like I have one hour of juice left in me... let's go. We'll be cooking up a bit of python code for fun and joy, so hang in there :) you'll see where we're going....


"audio qr code" brings up things quite different from what I'm up to. It's a bit like the Transformer movies made it next to impossible to find technical info on strange types of magnetics once they hit the market.

1) find a way to produce uncompressed RIFF wave files

https://docs.python.org/3/library/wave.html
pip3 install wave

2) we're building a new wave file. The lib is nice to avoid the gory format details. Done that once, no burning thirst to do it many more times.

https://web.archive.org/web/20140221054954/http://home.roadrunner.com/~jgglatt/tech/wave.htm

... and I got sidetracked with named tuples... so far:

#!python3

import wave

# create wave_write object
w = wave.open('code.wav', 'wb')

# create output
w.setparams((
    1, # nchannels.
    2, # sampwidth: in bytes  - 1 channel, 16 bit 
    48000, # framerate: 48kHz sampling rate? 
    1024, # nframes: will be changed as frames are written 
    'NONE', # comptype: only NONE is supported right now
    'NONE', # compname: only NONE is supported right now
    ))
    
# finish
w.close()

Bloody notepad++ broke the hexEditor plugin. Wow, such hate, complemented by

The bloody thing just died on me to make a point. Also claims there are no plugins installed and none available.
the .wav looks like this, which is nice. No hex editor :(
RIFF$   WAVEfmt....

 25 minutes in..

nframes is pretty much being ignored. We want some nice 10 seconds of annoyance, so let's aim for that, see if we can produce silence to start with.


3) QR code

let's use a picture for now, deal with code generation later.

https://www.qrcode-generator.de/

 we can do this!

turns out,

pip3 install Pillow

44 minutes in...

import PIL.Image

# load QR code picture
img = PIL.Image.open('code.png', 'r')
img_width, img_height = img.size

learning as a timed challenge. Who came up with this? Not sure whether I like it.

now we should have image size and data. Quick test?

print(img.getpixel((1,1))) yields (255, 255, 255)

we're good. another test at (1000,1): out of range... so we need to do range testing ourselves. Wonderful :) Floating point coordinates are accepted but I haven't read what they do. We'll find out soon enough.

Scale and range limit the first channel of the pixel

getpixelcoef = lambda x, y: img.getpixel((x, y))[0]/255 if (0 <= x) and (x < img_width) and (0 <= y) and (y < img_height) else 0

4) time's up :(

Here we are right now:

#!python3

import wave
import PIL.Image
import math
import numpy as np

# load QR code picture
img = PIL.Image.open('code.png', 'r')
img_width, img_height = img.size


# create wave_write object
w = wave.open('code.wav', 'wb')

# create output
w.setparams((
    1, # nchannels.
    2, # sampwidth: in bytes  - 1 channel, 16 bit 
    48000, # framerate: 48kHz sampling rate? 
    0, # nframes: will be changed as frames are written 
    'NONE', # comptype: only NONE is supported right now
    'NONE', # compname: only NONE is supported right now
    ))

    
dt = 1.0/w.getframerate()
dt_pixel = 0.1 # 100ms/px
total_time = 10.0
fstart = 17000
fend   = 20000
fsteps = 100
freqs  = np.arange(fstart, fend, (fend-fstart)*1.0/fsteps)

getpixelcoef = lambda x, y : img.getpixel((x, y))[0]/255 if (0 <= x) and (x < img_width) and (0 <= y) and (y < img_height) else 0

for n in range(0, int(w.getframerate()*total_time)):
    for f in freqs:
        pass
    
# finish
w.close()

We've started spitting out a wave file, managed to load a QR code image and after pip3 install numpy we're good to go with a list of frequencies to process every sample.

ROUND 2

Tomorrow will not be such a fine day ;-) What with waking up too late and so forth. We cannot stop now though. We want to hear the darn QR code.

howto 16bit signed value?

https://www.devdungeon.com/content/working-binary-data-python

1h30min: this is the slowest thing I've ever implemented. Wow. It's really super slow. Better try FFT next time...

#!python3

import wave
import PIL.Image
import math
import numpy as np


# load QR code picture
img = PIL.Image.open('code.png', 'r')
img_width, img_height = img.size

# create wave_write object
w = wave.open('code.wav', 'wb')
total_time = 10.0
framerate = 48000
# create output
w.setparams((
    1, # nchannels.
    2, # sampwidth: in bytes  - 1 channel, 16 bit 
    framerate, # framerate: 48kHz sampling rate? 
    int(framerate*total_time), # nframes: will be changed as frames are written 
    'NONE', # comptype: only NONE is supported right now
    'NONE', # compname: only NONE is supported right now
    ))

dt = 1.0/w.getframerate()
dt_pixel = 0.1 # 100ms/px
offset_time = 1.0
duration = dt_pixel * img_width 
fstart = 17000
fend   = 20000
fsteps = 100
freq_step = (fend-fstart)*1.0/fsteps
freqs  = np.arange(0, fend-fstart, freq_step)

getpixelcoef = lambda x, y : img.getpixel((x, y))[0]/255 if (0 <= x) and (x < img_width) and (0 <= y) and (y < img_height) else 0

for n in range(0, int(w.getframerate()*total_time)):
    t = n*dt - offset_time
    for f in freqs:
        frame = 0 
        if (t >= 0) and (t < duration):
            frame = int(100 * getpixelcoef(t/dt_pixel, f/freq_step) * math.sin(2 * math.pi * (f+fstart) * t))
        w.writeframesraw(frame.to_bytes(2, byteorder='little', signed=True))
    
# finish
w.close()

3 minutes in... ok this is really incredibly slow, what with using float coordinates on an image and then massaging every sample with trigonometric functions.

[many months later]

ps. it seems like I never showed the silly results around. When it comes to proper detection, a lot comes down to getting FFTs of sufficient resolution - encoded using 250 tones over 4 to 16 kHz and recovered using 4096 sample window FFT in spectroid.

detection from the FFT also works, if you haven't tried the code above yourself already:


Final demo code (please don't take it as a reference, it's just one way to get there at all, not an efficient implementation that should be used for educational purposes).

#!python3

import wave
import PIL.Image, PIL.ImageFilter, PIL.ImageEnhance
import math
import numpy as np
import random


# load QR code picture
img_raw = PIL.Image.open('code.png', 'r')
flt1 = PIL.ImageFilter.BoxBlur(1)
img = img_raw.filter(flt1)
img = PIL.ImageEnhance.Brightness(img).enhance(0.5)
img = PIL.ImageEnhance.Contrast(img).enhance(5.0)
img.save('code.processed.png')

img_width, img_height = img.size

# create wave_write object
w = wave.open('code.wav', 'wb')
total_time = 5.0
framerate = 44100
# create output
w.setparams((
    1, # nchannels.
    1*2, # sampwidth: in bytes  - 1 channel, 4*ch bytes = 32 bit IEEE754 float?
    framerate, # framerate: 48kHz sampling rate? 
    int(framerate * total_time), # nframes: will be changed as frames are written 
    'NONE', # comptype: only NONE is supported right now
    'NONE', # compname: only NONE is supported right now
    ))

dt = 1.0/w.getframerate()
dt_pixel = 0.015 # s/px
offset_time = 0.2
duration = dt_pixel * img_width 
fstart = 4000
fend   = 16000
fsteps = 250
freq_range = (fend-fstart)
freq_step = freq_range / fsteps
freqs  = np.arange(0, fend-fstart, freq_step)
for i in range(0,len(freqs)):
    freqs[i] = freqs[i] + 20* random.random()

getpixelcoef = lambda x, y : img.getpixel((x, y))[0]*0.00392156862 if (0 <= x) and (x < img_width) and (0 <= y) and (y < img_height) else 0

cnt = 0
for n in range(0, int(framerate * total_time)):
    t = n*dt - offset_time
    if cnt < int(t*10):
        print(t)
        cnt = int(t*10)
    frame = 0 
    if (t >= 0) and (t < duration):
        for f in freqs:
            frame = frame + (50+f/100) * getpixelcoef(t/dt_pixel, img_height*(1-f/freq_range)) * math.sin(2 * math.pi * (f+fstart) * t)
    w.writeframesraw(int(frame).to_bytes(2,byteorder='little', signed=True))
    
# finish
w.close()


Like

Discussions