Ben Gurion University, Israel, research by Ben Nassi, Dudi Nassi, Raz Ben Netanel, Yisroel Mirsky, Green, and Yuval Elovici "Phantom Attacks Against Advanced Driving Assistance Systems" shows how projected images can change behavior of camera-based ADAS systems, such as Tesla X Autopilot or Mobileye 630 Pro:
May this justify the utility of LIDAR ?
ReplyDeleteYang, It is really an issue in the philosophy of safety - are the sensors used in "AND" mode or "OR" mode? Any shift from the current "OR" mode ("better safe than sorry" in the original article) towards the "AND" mode has implications in liability assessment. All this really demonstrates is that machines are going to be nearly as bad as people in making decisions in complex environments because deciding what to ignore is very hard. Even tagging all real objects does not help much because the tags could also be faked.
DeleteThe implication is that good decision-making relies on experience with real environments which then implies that autonomous vehicles need to learn as they go.
Probably this makes the cars smarter than the drivers, who will be annoyed by witnessing decisions they don't understand. Maybe cars will need semi-opaque domes like those in "Sleeper" to keep the drivers from going nuts.
Well the key benefice of an artificial intelligence is to remove some subjectivity in our sensing and jugement. Personally I was scared in night driving when I mistook my car reflected from an advertissement kiosque as a car coming from side ... All the vision sensor can hardly do better than human in complexe environment. From this point of view, LIDAR has its unique advantage here.
DeleteHonestly, I think most of this is a "non-issue". E.g. if someone places a card-board sign there instead of a projection, also most humans will be fooled. Of course, some plausibility checks would make sense, e.g. a 120 sign inside a city will likely be not correct. But still, if a speed sign is faked, not much will happen as long as the vehicle still drives in a "safe way". The car owner will only risk a speeding ticket.
ReplyDeleteSimilar in the case with the projected person on the road, the "better safe than sorry" approach is OK in my opinion.
More interesting is the project lane example, mainly because I have seen similar situations already in real life (Roadworks on German Autobahn, 3 "new" lanes with yellow lines, put over 3 "old" lanes with white lanes, at night, wet road, lanes partly damaged, approaching with 100 km/h -> even a human has a very hard time to make sense of the situation and where to drive. And also simply stopping on the Autobahn is no option...)
But I think it is good that people think about such "attack" scenarios so that the ADAS software can consider these situations.