Foveated imaging with optical folding

Jinwen Wei and Liangcai Cao, "Compact Neural Pancake Camera for High-Perceptual-Quality Foveated Imaging," ACS Photonics (2026).

Link: https://doi.org/10.1021/acsphotonics.6c00691 

Pancake catadioptric optics utilize optical folding to effectively reduce the optical-path thickness in virtual-reality display systems. However, limited optical throughput and optical degradation make image reconstruction for pancake cameras severely ill-posed, hindering broader imaging applications. In this article, we propose a neural pancake camera with adaptive-prior deconvolution, achieving compact, high-perceptual-quality imaging. By introducing latent-space projection, adaptive prior deconvolution alleviates the trade-off between pixel fidelity and perceptual quality and addresses the excessive smoothing inherent in conventional pixelwise optimization. The proposed neural pancake camera reduces the ratio of axial length to physical aperture diameter by 3.2 times compared with other flat cameras with high imaging quality. Experiments and ablation studies substantiate that the proposed adaptive prior deconvolution improves perceptual quality by 70%, as measured by CLIP-IQA, while also outperforming the state-of-the-art deep learning models on pixel-level fidelity. As a representative application of the proposed neural Pancake camera, this work further showcases bioinspired foveated imaging, highlighting its potential for bandwidth-efficient imaging in next-generation edge and portable devices. 

 

Figure 1. Pipeline of the proposed neural Pancake camera. (a) Conceptual illustration of the proposed compact Pancake camera scheme. (b) The inherent trade-off between perceptual quality and pixel-level quality of computational imaging. This work proposes adaptive prior deconvolution to promote the perceptual imaging quality of Pancake cameras while improving pixel fidelity. (c) Overview of the training process of the end-to-end adaptive prior deconvolution. The framework operates by optimizing the learnable deconvolution network while leveraging a latent natural-image manifold prior, anchoring the restoration output to the natural image manifold to reconcile the perceptual-pixel trade-off.

 

Figure 3. Quantitative evaluation and visualization of the adaptive-prior deconvolution. (a) Schematic illustration of the inference process based on the adaptive-manifold prior. (b) Quantitative performance profiles of averaged MUSIQ, SSIM, and PP-IQA across inference steps. (c) Visual comparisons demonstrating the effects of the low prior, the adaptively selected proper prior, and the over-prior.

 

 

Figure 4. Demonstration of neural Pancake camera-based foveated imaging. (a) Schematic illustration of foveated imaging in the human visual system. (b) Profiles of high-frequency content proportion and relative acuity versus field of view. (c) Comparison of reconstructed and raw-captured images across different fields of view.