Mainpage
  Stereovision
  Segmentation
  Publications
  Online-Processing

Links
Contact
StereoVision
Prev: Natural Scenes Up: ^ Stereo Mainpage ^ Next: Random-Dots

Difficult data

Given two stereo images, it is not always possible to calculate disparities reliably. For example, occlusion might cause an image area visible in the left view to be missing in the right, or vice versa. Matching can be impossible in areas with minor image detail is corrupted by sensor noise.

It is of course of importance to know the areas where disparity estimation is unreliable - whatever the cause is.

The coherence-based stereo algorithm is able to detect areas with potentially bad disparity estimate because a verification count is available within the network structure. It is simply the amount of coherence detected within each disparity stack.

Here's an example, where the original stereo images have large areas with only minor image detail, but quite large sensor noise. As expected, reliable disparity estimates are only obtained at the object borders; however, they clearly show up as bright areas in the coherence map.

Left View
left picture
Right View
right picture
Disparity Map
calculated disparity
coherence map
coherence map
verified data
verified data
(threshold=0.2)
Setting a high threshold for verified data, coherence-based stereo mimics a classical feature-based algorithm when confronted with difficult data. However, feature-based stereo algorithms have to preselect the image features they are using (i.e. edges, zero-crossings or what ever). Only in image regions where these preselected features are found a disparity estimate can be obtained.

If image detail is present, but not of the chosen feature type, no estimates will be obtained by classical feature-based approaches. This has lead to the idea of a "bag of tricks", many parallel processes utilizing different features in image data (cmp. J.P. Frisby's & S.B. Pollard's article "Computational Issues in Solving the Stereo Correspondence Problem", p. 331 in "Computational Models of Visual Processing", eds. M.S. Landy & J.A. Movshon).

This riddle of feature-based stereo algorithms can simply be explained by coherence-based stereo: since it is a feature-less algorithm; disparity is obtained in all image regions with sufficient detail, independent of the type of detail present. The "bag of tricks" turns out to be routed in a simple mechanism, only appearing as a combination of many different algorithms.

Here's another example, showing how aware coherence-based stereo is of image areas with problematic disparity estimates:

Left View
left picture
Disparity Map
disparity map
Coherence Map
coherence map
Verified D-Map
verified data
This stereo pair shows specular highlights, transparency, repetitive structures and areas with only minor image detail, all reducing the quality of the disparity estimates. However, all these areas are clearly marked by low coherence-values.

Occlusions

In natural scenes, texture is usually abundant. Here, the main reason for missing disparity estimates are occlusion effects: image areas visible in the left view are occluded in the right view by nearer objects. Coherence drops in these areas, making it possible to detect candidates for occlusion boundaries:
Left View
left picture
Disparity Map
disparity map
Coherence Map
coherence map
Verified D-Map
verified data

Filling-In

In sparse random-dot stereograms, filling-in is known to occur in human stereovision. Filling-in interpolates missing disparity values based on disparity estimates of neighboring image regions.

Filling-in is usually realized with cooperative algorithms refining iteratively an initial estimate of the disparity map. This process can last several hundred iterations, and slows down these types of algorithms.

With the coherence-based stereo, filling-in occurs from coarser spatial channels whenever the verification count of the fine resolution channels is not sufficient for a valid disparity estimate. This is a fast, non-iterative process which is simply a by-product coherence-detection scheme.

As an example for filling-in, the following figure displays a random-dot stereogram with a pixel density of only 3%, together with its calculated and its true disparity map.

Left Image
left picture
Right Image
right picture
calculated disparity
calculated disparity
true disparity
true disparity

© 1994-2003 - all rights reserved.