New Imaging Technology introduces a stereoscopic camera reference design based upon its global shutter HDR sensor NSC1001 with DR of more than 140dB. The reference design is intended for robotic and automotive applications.
Youtube video shows the camera capabilities:
Is that white shadow-like thing around the person an occlusion artifact? If so it is a good illustration of the limits of stereo depth images.
ReplyDeleteHas nothing to do with NIT's sensor of course, just shows that there is no universal solution for range finding thus far. TOF has its own limitations as well.
Eric, I think you're right. When he moves the mug closer to the camera, its white shadow gets bigger. This could be an improper inter-ocular distance for this scene.
ReplyDeleteMy concern is the region of no hits at the end of the desk - low-contrast?
I wonder if the stereo algorithm operated on the high bit-depth image or the tone-mapped image.
~AndyU
The white shadow is indeed due to occlusion - that is nothing to do depth sensing but how binocular vision works and we human suffer that too.
ReplyDeleteThe closer an object, the bigger the occlusion.
If they add another camera, the occlusion can be significantly reduced or even eliminated.
Humans do not "suffer" from occlusion. Our brains do a pretty good job of maintaining depth perception even for areas occluded from one eye. Just close one eye and see if your sense of depth disappears. Gets worse perhaps, but not so bad.
ReplyDeleteEric, humans do suffer from occlusion. We are not always conscient about our visual deficiency, the best example is the existance of a blind spot on our retina but nobody awares it.
ReplyDeleteOur 3D vision is very subjective. There was (is and will be) debat around the 3D perception from recognized objects or directly from image pixels (please see "Vision" of David Marr). His random stereogram doesn't fully resolve this issue.
Seeing is not always believing !
-yang ni
You made my point Yang Ni...we aren't suffering. We have adapted additional processing strategies. Again, close one eye and walk around. Not so bad.
ReplyDeleteBut I cannot say that I walk better with one eye closed than 2 eyes openned :)
ReplyDelete-yang ni
Let me clarify further on human vision.
ReplyDeleteHuman vision for objects within an arm length is the classical quantitative binocular vision - one can determine the exact distant for grasping.
However, for objects beyond an arm's length (what's the need for depth when you can't grasp it), it is qualitative based on contextual inference and one can no longer infer the exact depth.
Hence occlusion happens within an arm's length for human vision, and when it happens, the brain gets confused for seeing two different scenes in each eye; in computer vision terminology, there is no correspondence.
I just tried closing one eye and grabbing my coffee cup. It worked ok. Then I tried a pen on my desk, and my mouse. Also ok. So again, stereo or binocular vision is a cue to depth perception but definitely not the only mechanism. Also, when we were swinging thru the trees, we probably needed to get the distance right to the next branch, well beyond our immediate grasp. So I think depth beyond arm's length is/was pretty important. Lastly, you say all occlusion happens within an arm's length and the brain gets confused. This is also a broad and faulty statement, and goes against your comment that binocular vision evolved for exactly that same range. Occlusion does get worse inversely (?) with distance, but it is a continuous problem that goes well beyond arms length, and other "computation" in the brain is used to stay unconfused in most instances. Otherwise the system would function the worst in the range region where you say it is needed the most.
ReplyDeleteDear AndyU,
ReplyDeleteThe camera works on USB 2.0. The stereo matching is applied directly on the images coming from the sensor with an 8 bits pixel depth. Both sensors are NIT WDR logarithmic sensors that deliver WDR images without further processing. There is no tone mapping step.
Christian
Eric, please do the following try:
ReplyDeletein front of an uniform background, put a pencil in vertical pointing up. Then you try to point another pencil to touch the point of the vertical pencil ... you can understand that biocular perception is sometimes a MUST :)
-yang ni
Yang Ni - I tried your experiment but found that I could consistently tell where the pencils were and touch them together. There are three ways we determine depth - stereo eyes, knowledge of where our eyes are focused, and relative size. Close one eye and you still have two of them. See http://image-sensors-world.blogspot.com/2012/02/nit-presents-stereo-wdr-camera.html?showComment=1329235510740#c7163338439594370972. Note this is a problem with stereo TV - you have two of the three - stereo eyes and relative size, but are focused on the TV surface.
ReplyDelete~AndyU
There is no need to repeat all these experiments.
DeleteThe conclusion is already made from the benefits of stereoscopic endoscope over mono endoscope in a scene of semi-familiar nature and within an arm's length.
yes, one can use mono endoscope to perform and to tie knots, but at much slower speed and make more errors due to the lack of depth information. Go to check the rate of incontinency of prostate surgeries with mono endoscope.
Thanks a lot for this information !
DeleteI didn't know this before.
-yang ni
Sorry, meant http://kb2.adobe.com/cps/898/cpsid_89868.html.
ReplyDeleteAndyU