PetaPixel article on an 18K (316MP) HDR sensor

Link: https://petapixel.com/2023/06/12/sphere-studios-big-sky-cinema-camera-features-an-insane-18k-sensor/

Sphere Studios’ Big Sky Cinema Camera Features an Insane 18K Sensor

Sphere Studios has developed a brand new type of cinema camera called The Big Sky. It features a single 316-megapixel HDR image sensor that the company says is a 40x resolution increase over existing 4K cameras and PetaPixel was given an exclusive look at the incredible technology.

 

 

Those who have visited Las Vegas in the last few years may have noticed the construction of a giant sphere building near the Venetian Hotel. Set to open in the fall of 2023, the Sphere Entertainment Co has boasted that this new facility will provide “immersive experiences at an unparalleled scale” featuring a 580,000 square-foot LED display and the largest LED screen on Earth.

As PetaPixel covered last fall, the venue will house the world’s highest resolution LED screen: a 160,000 square-foot display plane that will wrap up, over, and behind the audience at a resolution over 80 times that of a high-definition television with approximately 17,500 seats and a scalable capacity up to 20,000 guests. While the facility for viewing these immersive experiences sounds impressive on its own, it leaves one wondering what kind of cameras and equipment are needed to capture the content that gets played there.

The company has said “an innovative new camera system developed internally that sets a new bar for Image fidelity, eclipsing all current cinematic cameras with unparalleled edge-to-edge sharpness” — a very bold claim. While on paper it doesn’t seem much different from any other camera manufactures claims about their next-gen system, spending time with the new system in person and seeing what it is capable of paints an entirely different picture that honestly has to be seen to be believed.

“Sphere Studios is not only creating content, but also technology that is truly transformative,” says David Dibble, Chief Executive Officer of MSG Ventures, a division of Sphere Entertainment focused on developing advanced technologies for live entertainment.

“Sphere in Las Vegas is an experiential medium featuring an LED display, sound system and 4D technologies that require a completely new and innovative approach to filmmaking. We created Big Sky – the most advanced camera system in the world – not only because we could, but out of innovative necessity. This was the only way we could bring to life the vision of our filmmakers, artists, and collaborators for Sphere.”

According to the company, the new Big Sky camera system “is a groundbreaking ultra-high-resolution camera system and custom content creation tool that was developed in-house at Sphere Studios to capture stunning video for the world’s highest resolution screen at Sphere. Every aspect of Big Sky represents a significant advancement on current state-of-the-art cinema camera systems, including the largest single sensor in commercial use capable of capturing incredibly detailed, large-format images.”

The Big Sky features an “18K by 18K” (or 18K Square Format) custom image sensor which absolutely dwarfs current full frame and large format systems. When paired with the Big Sky’s single-lens system –which the company boasts is the world’s sharpest cinematic lens — it can achieve the extreme optical requirements necessary to match Sphere’s 16K by 16K immersive display plane from edge to edge.

Currently the camera has two primary lens designs: a 150-degree field of view which is true to the view of the sphere where the content will be projected, and a 165-degree field of view which is designed for “overshoot and stabilization” particularly useful in filming situations where the camera is in rapid motion or on an aircraft with a lot of vibrations (ie a helicopter).

The Big Sky features a single 316-megapixel, 3-inch by 3-inch HDR image sensor that the company says is a 40x resolution increase over existing 4K cameras and 160x over HD cameras. In addition to its massive sensor size, the camera is capable of capturing 10-bit footage at 120 frames per second (FPS) in the 18K square format as well as 60 FPS at 12-bit.

“With underwater and other lenses currently in development, as well as the ability to use existing medium format lenses, Sphere Studios is giving immersive content creators all the tools necessary to create extraordinary content for Sphere,” the company says.

Since the media captured by the Big Sky camera is massive, it requires some substantial processing power as well as some objectively obscene amounts of storage solutions. As such, just like the lenses, housings (including underwater and aerial gimbals), and camera, the entire media recorder infrastructure was designed and built entirely in-house to precisely meet the company’s needs.

According to the engineering team at Sphere, “the Big Sky camera creates a 500 gigabit per second pipe off the camera with 400 gigabit of fiber between the camera head and the media recorder. The media recorder itself is currently capable of recording 30 gigabytes of data per second (sustained) with each media magazine containing 32 terabytes and holds approximately 17 minutes of footage.”
The company says the media recorder is capable of handling 600 gigabits per second of network connectivity, as well as built-in media duplication, to accelerate and simplify on-set and post-production workflows. This allows their creative team to swap out drives and continue shooting for as long as they need.

Basically, as long as they have power and extra media magazines, they can run the camera pretty much all day without any issues. I did ask the team about overheating and heat dissipation of the massive system, and they went into great detail about how the entire system has been designed with a sort of internal “chimney” that maintained airflow through the camera ensuring it would not overheat and can keep running even in some of the craziest weather scenarios ranging from being completely underwater to surrounded by dust storms without incident.

What’s even more impressive is the camera can run completely separate from this recording technology as long as it is connected through its cable system, this includes distances of up to a reported mile away.

Since the entire system was built in-house, the team at Sphere Studios had to build their own image processing software specifically for Big Sky that utilizes GPU-accelerated RAW processing to make the workflows of capturing and delivering the content to the Sphere screen practical and efficient. Through the use of proxy editing, a standard laptop can be used, connected to the custom media decks to view and edit the footage with practically zero lag.

Why Is This A Big Deal?
While the specs on paper are unarguably mind-boggling, it’s practically impossible to express just how impressive the footage and experience is to see it captured and presented on the sphere screens it was meant for.

The good news is that PetaPixel was invited to the Los Angeles division for a private tour and demonstration of the groundbreaking technology so we could see it all firsthand and not just go off of the press release. I wasn’t able to take photos or video myself — the images and video in this write-up were provided by the Sphere Studios team — but I can confirm that this technology is wildly impressive and will definitely change the filmmaking industry in the coming years.

When showing me the initial concepts and design mock-ups, the team didn’t think of the content they deliver as simply footage, but rather “experiential storytelling” and after having experienced it for myself, I wholeheartedly agree.

During my tour of the facility, I got to see the camera first hand, look at live footage and rendering in real-time, as well as see some test images and video footage, including some scenes that may make it into “Postcard from Earth” which is the first experience being revealed at the Sphere in Las Vegas this fall that has footage captured from all over the planet that should give viewers a truly unique perspective of what the planet and this new camera system has to offer.

On top of the absolutely massive camera, the system they have developed to “experience” the footage includes haptic seating, true personal-headphone level sound without the headphones from any seat, as well as a revolutionary “environmental” system that can help viewers truly feel the environment they are watching with changing temperatures, familiar scents, and even a cool breeze.
“Sphere Studios is not only creating content, but also technology that is truly transformative,” says Dibble.

“Sphere in Las Vegas is an experiential medium featuring an LED display, sound system and 4D technologies that require a completely new and innovative approach to filmmaking. We created Big Sky – the most advanced camera system in the world – not only because we could, but out of innovative necessity. This was the only way we could bring to life the vision of our filmmakers, artists, and collaborators for Sphere.”

Something worth noting is all of this came to life effectively in just a few short years. The camera started out as an “array” of existing 8K cameras mounted in a massive custom housing. This created an entirely new series of challenges when processing and rendering the massive visuals, which lead to the development of the Big Sky single-lens camera itself, which is currently in its version 2.5 stage of development.

Each generation has made the system more compact and efficient also. The original system was over 100 pounds with the current (v2) weighing a little over 60 pounds, with the next generation lens being developed bringing the system under 30 pounds.

Equally impressive was the amount of noise the camera made, which is to say it was practically silent in operation. Even with the cooling system running it was as quiet or even quieter than most existing 8K systems in the cinematic world — comparing it to an IMAX wouldn’t even be fair… to the IMAX.

The Big Sky cameras are not up for sale (yet) but they are meeting with film companies and filmmakers to find ways to bring the technology to the home-entertainment world. A discussion we had on-site revolved around gimbals mounted on helicopters, airplanes, and automobiles and how those systems, even “the best” still experience some jitter/vibration which is often stabilized which causes the footage to be cropped in.

The technology built for Big Sky helps eliminate a massive percentage of this vibration, and even without it, the sheer amount of resolution the camera offers can provide a ton of space for post-production stabilization. This alone could be a game changer for Hollywood when capturing aerial and “chase scene” footage from vehicles allowing for even more detail than ever before.

Big Sky’s premiere experience at Sphere in Las Vegas is set to open on September 29 with the first of 25 concerts by U2, as well as many other film and live event projects that will be announced soon.