Gregor Lenz (roboticist, and cofounder of Open Neuromorphic and Neurobus) has written a two-part blogpost that readers of ISW might find enlightening:
https://lenzgregor.com/posts/event-cameras-2025-part1/
https://lenzgregor.com/posts/event-cameras-2025-part2/
Gregor goes into various application domains where event cameras have been tried, but faced challenges, technical and otherwise.
Wide adoption will depend less and less on technical merit and more on how well the new sensor modality will fit into existing pipelines for X where X can be supply chain, hardware, software, manufacturing, assembly, testing, ... pick your favorite!
Very interesting and original take on event cameras!
ReplyDelete"A workable solution will likely follow a two-stage architecture: a lightweight, streaming tokenizer that aggregates local spatiotemporal activity into short-lived micro-features, followed by a stateful temporal model that reasons over these features efficiently. Such representations preserve sparsity, maintain temporal fidelity, and scale naturally under variable scene activity."
This reminds me of this product intro from iniVation https://www.youtube.com/watch?v=tv-GqKg4Mak
I'm not sure I agree with some of the points though. For example, is this claim in Part 1 of the blog really true that "current systems already perform well enough in most of" the safety critical edge cases like collision avoidance?
ReplyDeletePerhaps it is better to say that the higher-rate temporal data from event cameras offers no benefit vs. traditional CMOS cameras for cases like collision avoidance. For a robotic system, image capture is just a small part of the control loop including perception, planning, and mechanical actuation. These control loops operate at 10's of milliseconds, and the temporal benefit of an event camera does not provide a significant improvement to this overall time constant, and it (typically) comes at the expense of dramatically lower spatial resolution, computational complexity, worse low-light performance, and added algorithm developer time.
DeleteDespite the earlier comments, it seems that people are building quite impressive robotic system products using event sensors - both in terms of speed and spatial resolution. 😉
Deletehttps://www.summerrobotics.ai/
https://www.summerrobotics.ai/technology