Videos du jour [Aug 23, 2023]

 

Snappy Wide 8K wide aspect ratio CMOS image sensor

Snappy Wide is Teledyne e2v's new 8K wide aspect ratio CMOS image sensor designed specifically for logistics applications where larger conveyor belts are becoming increasingly common. A single Snappy Wide sensor can cover this large field of view successfully, replacing multiple sensors for a more efficient and cost-effective solution.

Recurrent Vision Transformers for Object Detection with Event Cameras (CVPR 2023)

We present Recurrent Vision Transformers (RVTs), a novel backbone for object detection with event cameras. Event cameras provide visual information with sub-millisecond latency at a high-dynamic range and with strong robustness against motion blur. These unique properties offer great potential for low-latency object detection and tracking in time-critical scenarios. Prior work in event-based vision has achieved outstanding detection performance but at the cost of substantial inference time, typically beyond 40 milliseconds. By revisiting the high-level design of recurrent vision backbones, we reduce inference time by a factor of 6 while retaining similar performance. To achieve this, we explore a multi-stage design that utilizes three key concepts in each stage: First, a convolutional prior that can be regarded as a conditional positional embedding. Second, local and dilated global self-attention for spatial feature interaction. Third, recurrent temporal feature aggregation to minimize latency while retaining temporal information. RVTs can be trained from scratch to reach state-of-the-art performance on event-based object detection - achieving an mAP of 47.2% on the Gen1 automotive dataset. At the same time, RVTs offer fast inference (less than 12 ms on a T4 GPU) and favorable parameter efficiency (5 times fewer than prior art). Our study brings new insights into effective design choices that can be fruitful for research beyond event-based vision.
Reference:
M. Gehrig, D. Scaramuzza
"Recurrent Vision Transformers for Object Detection with Event Cameras"
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, 2023
PDF: https://arxiv.org/abs/2212.05598
Code: https://github.com/uzh-rpg/RVT