Microsoft Surface Hub 2 Camera

The Verge has a new article titled "How Microsoft built its smart Surface camera" about the new Surface Hub 2 camera. It required huge effort on the entire computational photography pipeline: the optics, the image sensor and on-board machine learning-based processing algorithms. The camera is expected to have a $800 price tag.

 

Source: https://www.theverge.com/2022/3/22/22990673/microsoft-surface-hub-2-smart-camera-interview-report

 

Some excerpts below.

“From day one of Surface Hub 2, we knew we were going to make our cameras smart,” explains Steven Bathiche, who oversees all hardware innovation for Microsoft devices, in an interview with The Verge. Microsoft’s surprise $799.99 Surface Hub 2 Smart Camera debuted last week, offering automatic reframing without the warping and distortions you might typically see on other conference room cameras.

 

It can detect faces and bodies, in an effort to make sure everyone in a room is visible during meetings whether they’re close to the camera or up to eight meters away. The Surface Hub 2 Smart Camera is able to pretty much see an entire conference room thanks to its 136-degree field of view, which keeps the people at the front in focus alongside those in the back.

 

Bathiche and his team have created Microsoft’s own optics, AI model, and edge computer to go into the Surface Hub 2 Smart Camera and power its computational photography. “It has onboard compute, 1 teraflops of compute that essentially houses a really large AI model that we’ve built,” says Bathiche. “It includes the autoframing application, it resides in the camera, so what comes out is just a 4K image so it literally looks like a webcam to the Surface Hub.”

“We designed an 11 element, completely glass lens with super sharp focus and basically close to the refraction limits,” explains Bathiche. Behind the lens is a 12-megapixel sensor (4000 x 3000) with an f/1.8 aperture that all generates the 4K cropped image. “The actual lens is a 184-degree field of view, so the camera can look behind itself.”

Computer vision on the edge is a theme we have seen before in this blog. For example, Sony and Microsoft had started partnering on smart image sensor solutions back in 2019. Sony had presented a smart CMOS image sensor with embedded AI processing in 2020.