CogStereo: Neural Stereo Matching with Implicit Spatial Cognition Embedding
Lihuang Fang, Xiao HU, Yuchen Zou, Hong Zhang
AI summary
Problem
Current deep stereo networks rely on local geometric correspondence and dataset-specific tuning, leading to poor zero-shot generalization and frequent failures in ill-posed regions like occlusions and weak textures.
Approach
CogStereo integrates implicit spatial cognition from a frozen monocular depth foundation model into a dual-condition refinement process that combines pixel-wise uncertainty estimates with cognitive features to globally correct disparities while preventing metric drift.
Key results
- State-of-the-art zero-shot generalization across synthetic and real-world benchmarks
- Significant error reduction in occluded and textureless regions via uncertainty-guided attention
- Novel KNN-based scale-and-shift alignment prevents metric drift in reliable areas
- Robust cross-domain performance without dataset-specific fine-tuning
Why it matters
Provides a foundation-model-inspired paradigm for reliable depth estimation in autonomous driving and robotics, eliminating the need for domain-specific tuning.
Abstract
Deep stereo matching has advanced significantly on benchmark datasets through fine-tuning but falls short of the zero-shot generalization seen in foundation models in other vi- sion tasks. We introduce CogStereo, a novel framework that ad- dresses challenging regions, such as occlusions or weak textures, without relying on dataset-specific priors. CogStereo embeds implicit spatial cognition into the refinement process by using monocular depth features as priors, capturing holistic scene understanding beyond local correspondences. This approach ensures structurally coherent disparity estimation, even in areas where geometry alone is inadequate. CogStereo employs a dual- conditional refinement mechanism that combines pixel-wise uncertainty with cognition-guided features for consistent global correction of mismatches. Extensive experiments on Scene Flow, KITTI, Middlebury, ETH3D, EuRoc, and real-world demonstrate that CogStereo not only achieves state-of-the-art results but also excels in cross-domain generalization, shifting stereo vision towards a cognition-driven approach. More de- tails are available at https://github.com/lhfang228/ CogStereo.