"If a picture tells a thousand words, then we might ask ourselves how many photons does it take to form a picture? In terms of the transmission of the picture information, then the multiple degrees of freedom (e.g., wavelength, polarization, and spatial mode) of the photon mean that high amounts of information can be encoded such that the many pixel values of an image can, in principle, be communicated by a single photon. However, the number of photons required to transmit the image information is not necessarily, at least technically, the same as the number of photons required to image an object. Therefore, another equally important question is how many photons does it take to measure an unknown image?
For intensity images, it seems that one detected photon per image pixel is a realistic guide, but this may be reduced by making further assumptions on the sparsity of an image in a chosen basis, such as spatial frequency. In this last respect, the advent of machine learning, knowledge-based reconstruction, and similar techniques alleviates the need for a user to explicitly define the sparse basis, but rather the prior is determined from a library of previously recorded images of a similar type. This machine learnt prior can then potentially be designed into the optimum measurement strategy. It seems likely therefore that future imaging systems will combine state-of-the-art single photon detectors with knowledge-based processing both in the design of the system itself and in the processing of the collected data to yield images or decisions based on these data on the basis of extremely low numbers of photons, potentially well below one photon per image pixel."
Once we are at single-photon imaging, the International SPAD Sensor Workshop (ISSW 2020) held as an on-line event in June published a nice SPAD photon-accumulation video of the city of Edinburgh:
Given the latest trends in generative AI, I think the answer might be zero! See this: https://image-sensors-world.blogspot.com/2023/06/a-lens-less-and-sensor-less-camera.html
ReplyDeleteThis is cool. But I think the assumption of 1 photon per pixel is not complete story.
ReplyDeleteThe wonderful book by Al Rose of RCA labs (A. Rose, Vision: Human and Electronic. Plenum Press, New York, 1973) has a fascinating discussion of many aspects and fundamental limits of shot-noise limited imaging. The book is unavailable now but I inherited Carver's copy of it and have periodically reread parts of it and scanned part of the book to be able to share this beautiful discussion: https://drive.google.com/file/d/0BzvXOhBHjRhebW0ya0JTbEtKaFk/view?usp=sharing&resourcekey=0-DXpVX9Y_9ED_V2TEvPhwEw
Check out the discussion if you are interested in this subject (it is not a matter of reading only 5 minutes...) and please comment if you see holes in the argument based on subsequent 50 years.
I found a complete version of the book in Ethiopia...
ReplyDeletehttp://ndl.ethernet.edu.et/handle/123456789/17393
Rose was, for sure, a big pioneer in the days of non-solid-state imaging devices (tubes).
Creating good images from low-photon-count images has been demonstrated using computational imaging techniques over the past 5-10 years by several groups, including Stanley Chan at Purdue and Mohit Gupta at Wisconsin. Of course such computed images are always subject to minor hallucination but then again, so are humans in the dark. Rose was focused (rightly so) on high QE and high SNR. More recently, low read noise has changed that focus a bit, especially now that 3D stacked solid-state image sensors can have deep sub-electron read noise and high QE or PDE.
For example, see J. Ma, S. Chan and E. R. Fossum, "Review of Quanta Image Sensors for Ultralow-Light Imaging," in IEEE Transactions on Electron Devices, vol. 69, no. 6, pp. 2824-2839, June 2022, doi: 10.1109/TED.2022.3166716 which discusses CMOS-QIS and SPAD-QIS devices. Each has advantages and disadvantages at their current state of development.