"ASP Vision: Optically Computing the First Layer of Convolutional Neural Networks using Angle Sensitive Pixels" by Huaijin Chen, Suren Jayasuriya, Jiyue Yang, Judy Stephen, Sriram Sivaramakrishnan, Ashok Veeraraghavan, Alyosha Molnar.
Abstract: Deep learning using convolutional neural networks (CNNs) is quickly becoming the state-of-the-art for challenging computer vision applications. However, deep learning's power consumption and bandwidth requirements currently limit its application in embedded and mobile systems with tight energy budgets. In this paper, we explore the energy savings of optically computing the first layer of CNNs. To do so, we utilize bio-inspired Angle Sensitive Pixels (ASPs), custom CMOS diffractive image sensors which act similar to Gabor filter banks in the V1 layer of the human visual cortex. ASPs replace both image sensing and the first layer of a conventional CNN by directly performing optical edge filtering, saving sensing energy, data bandwidth, and CNN FLOPS to compute. Our experimental results (both on synthetic data and a hardware prototype) for a variety of vision tasks such as digit recognition, object recognition, and face identification demonstrate 97% reduction in image sensor power consumption and 90% reduction in data bandwidth from sensor to CPU, while achieving similar performance compared to traditional deep learning pipelines.
I assume you were meaning to be funny with the headline - I thought I was about to read an article on how imaging made it onto a blurb on Cable News Network.
ReplyDeleteYes, it's supposed to sound funny.
DeleteI am not sure why the authors chose to normalize by area. But the Sony sensor pixel has 50x less area than the authors' pixel (1.43 vs. 10 um). So if we take the area normalization out, in fact the Sony sensor is lower energy per pixel than the purported low power sensor. Generally I wouldn't care too much, but in this case, the paper is all about power and bandwidth savings.
ReplyDeleteI am all for focal-plane image processing and I think this is an interesting paper, but it is too bad the authors tend to make a big deal out of the wrong things instead of focusing on their true (and good) contributions.
Response from the authors: The authors wanted to thank Dr. Eric Fossum for pointing out a correction to our energy comparison. We should not have normalized by area, but we also defined an ASP "pixel" to be the size of the tile, not the individual pixels themselves. Therefore, the actual savings for the 384 x 384 pixel ASP image sensor will be 33pJ/frame/pixel, which yields a savings of 90%. We acknowledge that this comparison is approximate and does not take into account process technology and other factors.
DeleteWe will update our ArXiv preprint and website to reflect this change, and thank Dr. Fossum again for his correction.
please refer to this Nobel Price work :
ReplyDeletehttp://hubel.med.harvard.edu
-yang ni
Why do you suggest this?
DeleteSince all the computation model has been proposed by David Hubel. I start my research work by inspiring from this book. He called "Functional Segregation" in his book.
Delete-yang ni