The past few days I have been learning the basics of some useful telecommunications principles. I started with a search for a line code that would eliminate long runs of 0s and 1s in the pulses recorded to tape. The tests from the previous log show how long runs of like-pulses cause a DC drift. After some reading I learned about disparity and DC-balanced codes that exist to please channels where a DC component is problematic. I decided to look for a code that would satisfy both requirements of being run length limited and DC balanced.
First I came across 8b10b. The efficiency of the coding looked promising (10bits to represent an octet), but the complexity of implementation was a bit of a turnoff. Each code-word is unbalanced, meaning running DC imbalance has to be calculated and countered during the encoding process. This combined with the need to craft a large word table means 8b10b would be tricky to implement correctly and may not work as well as a code that has DC balanced code words.
After more searching I came across something much simpler that satisfied both requirements: The line code from Slice TS-FO-02 of IEEE 1355. This line code is intended to be used in optical cable at a rate of 200 megabits/sec. A stream on input octets (called data characters in Wiki's spec summary) are split into 2 4-bit symbols (bits 0..3 in the first, 4..7 in the second) that are each mapped to a DC-balanced 6-bit word. The code is DC balanced and limited to a run of 4 like bits using the table provided by the specification summary. Because there are more 6 bit words than 4 bit symbols, there are reserved words that are used in link control sequences. Below is a table of the code word mapping from the specification modified to explicitly assign fixed control characters. Words in the right column are transmitted leftmost bit first.
It is possible to construct bit sequences that cannot exist in or across the boundaries of encoded data words using combinations of control words. These combinations can be used as syncwords to frame segments of data in a bit-stream. I would like to explore frame synchronization as a mechanism for minimizing data loss due to corruption of part of the payload and as a way to enable seeking though data streams using the tape transport. The use of unique syncwords would also enable detection and correction of phase-reversal that can be caused by different record and playback hardware.
I tested this code out by recording and playing a long run 6-bit words at 2400 bits/sec. I chose this speed because it appears to be near the minimum stable speed my tape deck will reproduce without serious DC drift of runs of like bits. The test bit stream was assembled to include the longest runs of like bits possible using the table above. The test stream was CTRL 1, CTRL 2, 9, 7, C, 4, CTRL1 (101010010101011001100011110100001110001101101010):
The recovered waveform drifted a small amount in repeated runs of bits, but does not drift very far. The silent spot in the middle of the above screenshot is a gap between repeated bit patterns in the WAV file generated by my script. I included the spaces as a visual marker in the wave pattern; they will not be included in future tests. I could not get my USB sound card to play a very short bit pattern so I repeated it several times and recorded the longer pattern. In the test the bit pattern was repeated 8 times, but the playback only captured 2 (exactly 2) repetitions of the bit pattern. I'm not sure what caused this, but more experimentation is needed with other audio hardware and software to pinpoint what is to blame. Now that I have a line code to experiment with I can start working on the C code that encodes and decodes raw bit streams from audio streams.
In my previous test I tried encoding information using cusps to no avail. The tape player rounded off sharp changes at higher frequencies, but did a decent job at preserving the general shape of the waveform. The only major distortions that appeared were at zero-crossings near bit changes and during long runs of 1s or 0s.
I simplified the x^2 waveform to a triangle wave to avoid slope changes near zero when bit flips occur. I tested this waveform with the speeds I have tried in other tests: 300, 1200, 2400, 4800 bits/sec. I used a test pattern of "010100110000111100000000..." to see how the DC offset issue affects long runs of repeating bits.
At higher bit rates the DC centering seems to occur less, but is still an issue for long runs of the same bit. I decided to tests at 9600 bits/sec to see if even higher speeds would help more:
It appears that a simple triangle pulse shape might be a feasible start to high data rate tape storage. Because of the DC centering issues I'll be limited to fairly high data rates (for tape) of 2400 bits/sec and above. My script has some issues with creating smooth transitions at 9600 bits/sec, but I'm not really worried about that because soon I'll be starting encoder and decoder development in C. My next task will be finding a balanced line code to try and combat DC offset issues.
After completing some more experiments with pulse shaping I've learned a little more about my particular tape deck. I tweaked my bit generator script a bit more and tested out some waveforms with intentionally-introduced cusps. The idea was to encode bits as a modified sine wave (or similar) and introduce a cusp in the direction of the encoded bit.
I chose to use x^n as the function for generating the cusp-encoded data. The generator script was modified to encode a string of bits as individual pulses consisting of x^n mirrored over the length of the pulse. I chose several different values of n (2,3,4) and several different data rates (300, 1200, 2400, 4800) to test. What I found is that cusps are preserved well at lower data rates, but are rounded off at higher ones.
x^2 at 300 bits/sec:
x^4 at 300 bits/sec:
x^2 at 4800 bits/sec:
x^4 at 4800 bits/sec:
I am a bit disappointed that these tests didn't work out as well as I hoped. I did notice that at higher frequencies (higher data rates) the issue of the signal "self-aligning" around 0v is minimized. Even though my goal is to find an encoding that is entirely self-clocked (and therefore timing-independent), I may end up targeting a minimum data rate for future tests. Perhaps the increased frequency of the encoded data will stabilize the DC offset issues.
I may also try some form of modulation to create waveforms the deck likes, but most modulation schemes I've looked at expect somewhat precise frequencies or clock synchronization between systems. The varying speed of tape playback may mean I have to fight frequency and clock errors with a reference tone in the second stereo channel. I'd like to avoid anything that requires both channels to be sampled since all of my computers (and many other modern consumer models) only have mono audio input.
One other possibility I've been thinking about is to encode data onto both stereo channels and mixing them with a stereo to mono additive mixer before sampling with the computer. Mixing in hardware has the drawback of requiring custom cable, but such a cable would be very easy to make with an AUX cable and a couple of resistors.
Today I had some time to run some pulse shape tests. I revised the bit generation function of my script several times to generate 3 waveforms: triangle, saw, and half-period sine. I tested each of these waveforms at 300 bits/sec. The top waveform was computer generated, the bottom was recorded to tape and replayed to the computer.
I noticed the deck changed the test waveforms with regards to DC offset. The way I'm encoding information generates a waveform that "rides" one side of the ground reference voltage for more than one half-period of the wave. This waveform is ambiguous on tape since information is encoded using changes in flux (rather than around a fixed reference). The resulting waveform "straddles" 0v where the source waveform is only on one side.
My fundamental misunderstanding of how tape works has been a good opportunity to learn the hard way. I can currently think of a couple ways around the limitations I've found. The first is to use a carrier tone to cancel out DC offset and somehow cancel out this carrier (perhaps using both stereo channels). The second is to modify the encoding scheme not rely on pulses on either side of 0v, but on cusps like the ones seen in the saw test.
Today I ran some tests recording different rates of pulses on tape. I wrote a small python utility that accepts a string of bits (11011000100110...) and a bitrate and generates WAV file containing the bits as return-to-zero coded square wave pulses. I generated 4 files of different bit rates and recorded them all to tape using my Technics RS-T230. I played the tape back into the computer and recorded the input. All playback and recording was done using Audacity with a sample rate of 48000 samples/sec.
I tested the bit rates 300, 1200, 2400, and 4800. What I noticed is that the tape recorder does not like square waves much at all. The tests look somewhat promising, but the waveform does not hold value at the maximum value of the square wave.
In higher bit-rate tests distortion is less of an issue, but weirdness still exists. My next experiments will be with pulse-shaping. I think finding a waveform closer to a sine wave will help reduce the amount of distortion from the the tape deck.
Storing data on cassette tape is not a new idea. After Googling around for a bit, I found several old formats to store data on cassette:
I also ran into a few non-standard formats that hobbyists played with. This article has some useful information about CUTS as well as non-standard formats and their performance.
These systems have some drawbacks:
I realized in the Tape Artchive project that cassette players can be painful mechanical beasts. Much of the aging technology is hard to get working, and not the most reliable once it is working. I also realized that consumer-level equipment wasn't designed to give perfect playback speed or even a steady playback speed.
I would like to develop a storage format that accounts for the issues encountered on tape. I like some features of the formats I looked at, especially HITS. HITS is entirely self-clocked, meaning there is no fixed data-rate so small variations in speed should be well tolerated. I have access to more modern equipment that should (testing needed) be more friendly to square pulses than the equipment targeted by HITS, so using single pulses might be possible.
I also noticed all of the systems treat tape as a 2-state medium. I see tape as a ternary medium:
Given that tape has three states, a bipolar return to zero line coding can be used. Each bit can be encoded using a + or - pulse that returns to zero. This type of signal is entirely self-clocked, and uses a single pulse to store a bit
This type of coding has a disadvantage: not all tape equipment will have the same phase. Recording and playback equipment may not have a matching phase causing the bit stream to be received inverted. This can be fixed by using a differential encoding, or by framing the data. I'll worry more about this later; for now I'll be testing different waveforms and seeing how my tape players distort them.