AIT Austrian Institute of Technology, ETH Zurich, Western Sydney University, and University of Illinois at Urbana-Champaign publish a pre-print paper "Bio-inspired Polarization Event Camera" by Germain Haessig, Damien Joubert, Justin Haque, Yingkai Chen, Moritz Milde, Tobi Delbruck, and Viktor Gruev
"The stomatopod (mantis shrimp) visual system has recently provided a blueprint for the design of paradigm-shifting polarization and multispectral imaging sensors, enabling solutions to challenging medical and remote sensing problems. However, these bioinspired sensors lack the high dynamic range (HDR) and asynchronous polarization vision capabilities of the stomatopod visual system, limiting temporal resolution to ~12 ms and dynamic range to ~ 72 dB. Here we present a novel stomatopod-inspired polarization camera which mimics the sustained and transient biological visual pathways to save power and sample data beyond the maximum Nyquist frame rate. This bio-inspired sensor simultaneously captures both synchronous intensity frames and asynchronous polarization brightness change information with sub-millisecond latencies over a million-fold range of illumination. Our PDAVIS camera is comprised of 346x260 pixels, organized in 2-by-2 macropixels, which filter the incoming light with four linear polarization filters offset by 45 degrees. Polarization information is reconstructed using both low cost and latency event-based algorithms and more accurate but slower deep neural networks. Our sensor is used to image HDR polarization scenes which vary at high speeds and to observe dynamical properties of single collagen fibers in bovine tendon under rapid cyclical loads."
In common words, what may this be used for?
ReplyDeleteRobot car
DeleteI'm no expert on polarization vision, but it is really interesting area because across many species animals use it for hunting, foraging, communication, and navigation. I was hoping we could get a tank of those wonderful cuttlefish (https://youtu.be/rbDzVzBsbGM), which apparently use a lot of polarization in their patterns they display, but they are hard to find in aquariums. Interestingly, they seem to lack polarizaiton vision, in direct contrast to the other supreme predators the mantis shrimps (https://www.youtube.com/watch?v=eGuZifKr0h4).
ReplyDeleteFor practical purposes, polarization seems useful to see better in fog, to see through glass, and to create contrast out of strain. Surgeons start to use it to see stained cancer tissues (https://www.youtube.com/watch?v=Qts264cpvfs), and underwater vehicles can use it for a crude kind of GPS localization. Interesting area that has been held back by lack of knowledge and sensors, starting to come into its own. This paper is a step in the direction of making such sensor that is more like what biology has evolved.
There are two areas of particular interest for this technology: biomedical and remote sensing. For biomedical application, this technology can record rapid changes in polarization signatures, such as evaluating stress in tendons at the single fiber level or label-free neural activity. On the remote sensing side, celestial polarization patterns can provide GPS-like information. This would be useful in GPS-denied areas.
ReplyDelete