|Prev: Deconstruction||Up: ^ Stereo Mainpage ^||Next: Fusion and Vergence|
Coherence-Detection by Integrate-And-Fire Neurons
The neural hardware of cognitive systems does an amazing job: stable perceptions are created out of unreliable, inconsistent data, transmitted and analyzed by unprecise and noisy hardware.
The crucial postulate of coherence-based stereo is that one can, and has to, detect the coherence between independently operating disparity units in a fast and reliable way, by neural hardware.
Coherence-Detection is a highly non-linear, selecting operation, since only the data available within the coherent cluster is fed forward to later processing stages. The operation is therefore extremely robust: most of the available data is thrown away, and only the tiny coherent portion of the neural information streams are transmitted for further analysis.
There are several possible neural realizations of this process, but the simplest one uses the synchronization properties of pools of neural oscillators.
As an example, Fig. 1 displays spiketrains of integrate-and-fire neurons, driven by actual data from a single disparity-stack. In the simulation of this figure, the neurons operated independently from each other, and as expected, no obvious structure shows up in the spiketrains of these neurons. Due to the high amount of noise signals, it would be very hard for subsequent processing layers to make sense of this data.
However, there is of course structure in this data, simply because the data is created by the outside world. The object at which the disparity stack is looking at has a certain disparity, and the whole stack is split by this into two classes: one which is coding coherent values within each other, and the rest of the neurons.
Now: by simply introducing a fixed, but weak global coupling between the neurons, the dynamical behaviour of the pool of neurons changes dramatically, as displayed in Fig. 2. The coherently coding cluster is synchronizing fast; it becomes clearly marked by synchrony within the spiketrains of the neurons belonging to the coherent cluster!
Of course, the synchronized pool of neurons is easily detectable by further processing layers, and in doing so, one obtains Fig. 3, a disparity map calculated out of real image data by a network of simple spiking integrate-and-fire neurons:
The neural synchronization process described here is actually computing the disparity map, by performing a dynamical coherence detection between signals in a very noisy environment.
A simple average or weigthed linear combination of the data would not lead to a useable disparity map, due to the high noise level within the disparity pools. Click here for a comparision, with and without the synchronizing links.
In Fig. 4, the output currents of two identical pool of neurons are displayed. The pool without weak-coupling (brown trace) shows only small modulations of its total output current. The pool where the weak-coupling is switched on at time=20 (blue trace) develops very fast an oscillatory component indicating the development of a coherence cluster.
Actually, the modulation depth of this output current reflects the number of coherent neural units, and is thus a measure of the validity of the signal. This can be seen in Fig. 5, which shows the modulation depth for the example image of Fig. 3. The dark areas in the coherence map clearly mark image areas where disparity estimates are in doubt, for example around the borders of the dragon head, where occlusion effects prevent correct estimation.
More information about the topic covered in this page can be found in technical reports, available at this page. You can run a neural network simulation using coherence detection on your own stereo images, by clicking here.
Check also the "Mus silicium" by John Hopfield and Carlos Brody, which uses the coherence detection scheme described here as computational tool in word recognition experiments.