I have been going through a number of sources covering topics from Bayesian statistics, genetic algorithms, machine learning, binomial regression, Bayesian binomial regression, gradient descent, and on and on and on. Right now I am thinking it is going to take me years to wade through the muddy waters of machine learning, understand each branch of mathematics, reread the material, apply it, and I would probably have better luck getting this thing to listen to enlgish and speak in french than I would interpreting my data.

However, I have had a book sitting on my shelf for a couple of years 'Computation: Finite and Infinite Machines' by Marvin L. Minsky. After finally picking up the book and reading the first two chapters I began to understand the first part of the problem I am having. I completely over looked the 'machine' part of machine learning, and the reality of this device. I have been looking at this circuit as some abstract of an idea, where in reality it is a finite machine.

Just as a mechanical adding machine has a finite number of parts and a finite number of outputs based on inputs, so too is the nature of this device.

So from section 2.2 'For a given machine M at a given time t, we can imagine an infinite variety of possible histories. The one that has actually occurred will determine the machine's response to the next stimulus. Now it may be that some events from the very remote past may contribute to determining this response function. If this is the case, one can say that the machine shows some "trace", or "memory", of those remote events. If every ancient event left a separate, independent trace, the machine would need to have infinite capacity, in some sense, to store them.'

I had gotten lost in the infinity because I smoothly skipped over the fundamentals of 'what is a machine'. Now since infinite storage capacity is not practical, nor necessarily useful I hope that I can break the problem down and focus on the finite number of histories that need to be considered. And, hopefully the rest of the book continues to be as helpful as the beginning (and hopefully I am not just fooling myself into thinking that I am understanding).