Virtual Tours, 360 degrees videos, virtual reality

Filming in Virtual Reality with Manifold: what is the volumetric video

Filming in Virtual Reality with Manifold: what is the volumetric video

manifold and the volumetric video

Facebook announces Manifold during Oculus Connect 5. A 360-degree 3D volumetric video camera dedicated to creatives and designed for storytelling.

Filming in virtual reality is the goal that Manifold aims to achieve; a VR volumetric video camera based on Surround 360 technology.

A project born from Facebook and RED, which aims to represent the world’s first end-to-end solution for 3D and 360-degree video capture.

Mainfold was presented to the world during the Oculus Connect 5, as a technology designed to connect creative professionals with the public in a completely new way.

Filming in Virtual Reality with Manifold: what is the volumetric video

Filming in Virtual Reality with Manifold: what is the volumetric video

Volumetric video is the process of acquiring moving images of the real world, of people and objects that can be subsequently viewed from any angle at any time in time (known as six degrees of freedom).

The ability to move from left to right, up and down and forward-backward (as well as being able to rotate around x, y axes).

The volumetric video market is expected to grow from $ 578.3 million in 2018 to $ 2.78 billion by 2023, according to a recent market research report.

Most of the media attention surrounding volumetric cameras such as the Manifold has focused on its potential to create more engaging and compelling virtual reality content. When paired with positional location systems within most VR headsets, such as Facebook’s Oculus Rift and Quest, volumetric video allows viewers to navigate alone in new worlds.

In essence, the viewer becomes the director, choosing every single “shot”. Not to mention the technical potential that will be made available to creatives and filmmakers in the creation of video content.

 

Create a volumetric video with Manifold

Manifold 3D VR Camera is, according to Facebook, built for one purpose: «to offer new generation stories». In fact, it is configured as a device designed for storytelling, acquiring multiple camera angles simultaneously from a given volume and allowing you to generate infinite perspectives from any direction within a field of view.

In this way, thanks to the sensors for RED cameras and the subsequent image processing, a higher image quality is obtained already during the shooting phase; think for example of its use during LIVE performances or high budget shows.

Manifold is the first professional camera capable of completely capturing a series of spherical images to accurately recreate entire scenes. The unique combination of camera layout and set-ready features makes it an ideal capture tool for professional filmmakers.

 

How Manifold works: technical aspects

As for the technical specifications Manifold 3D VR Camera includes 16 RED (R) Helium 8K (R) sensors designed to allow 360 6DoF capture that record RAW from 16 cameras and simultaneously perform 8K at 60fps.

Equipped with 8mm Schneider fish eye optics, 180 degree custom F4.0, a single SMPTE 304M cable for power, control and data, SDI outputs for monitoring or processing of third-party points, SDK included for post- video processing and a web app-based control interface. The camera control unit and storage device can be positioned up to 100 meters away.

Filming in Virtual Reality with Manifold: what is the volumetric video

Filming in Virtual Reality with Manifold: what is the volumetric video

Filming in Virtual Reality: Manifold will be the future of the video industry

Built in collaboration with RED Digital Cinema, Manifold seems to be a VR camera, primarily dedicated to the world of professionals and therefore, a particularly promising tool for the future of the traditional video industry.

“It would be foolish of me to say that Manifold is not about virtual reality, “said director of photography Andrew Shulkind, who moderated a panel on the new Oculus Connect 5 camera.” It absolutely is, but it concerns more than that.”

The prospect of “more than this” is what most attracted Red Cinema, Facebook’s partner in the Mainfold project.

Rather than volumetric video applications for 360 and VR video, RED’s CTO, Uday Mathur, said he was very excited by the potential of the system for VFX capture, which is virtual green screen image capture. A subject shot against a background of a particular shade of green, which can be removed and replaced in post-production with any scenario.

The ability to use the camera to capture engaging content is secondary to its potential for traditional film projects, according to Mathur.

“For the director of traditional photography, it’s a new type of tool they can use, allowing for a new kind of experience as well.”

The volumetric video captures the scene from multiple points of view and then the information is used to create a depth map. Once a volumetric video is made, it could potentially be used far beyond its original purpose: to create additional resources for the same project but also as a virtual set for an unlimited number of other projects.

Partnership with software and post-production companies

Volumetric video will also present new challenges for the workflow. Facebook has partnered with post production software companies, including Adobe, CaraVR, Nuke, Foundry, Otoy and others, to work on the implementation of tools and processes.

The development of Mainfold and its hypothetical use in real contexts will require intense collaboration between the pre and post-production teams. Simulations, in which the production teams are likely to be more deeply involved in the development of a project, to ensure that the post-production teams with greater control remain true to the original vision.

“Some positions in the film crew could even be eliminated.” said Facebook technical director Brian Cabral. But new positions would be created. For example, since the perfect continuity of every single object is fundamental, Cabral estimates that one day there will be a job in which a team member makes sure that nobody moves anything on the scene.

Cabral understands that these applications are far away. But, as he said, the Manifold represents the culmination of Facebook’s last five years of research and development work on interactive videos, as well as the starting point for the next five years of projects.

Compare the Manifold to his work at Nvidia, developing the first GPU GP (generic processing on graphic processing units). “Now (the GPU GP) is de rigueur…” he said. “We will start to see people making interesting shots – not just for virtual reality … And it will only be de rigueur.”

Cabral then continued:

“We will capture everything and create VR resources, we will create AR resources, we will obtain 2D resources, but we will capture them simply because everything on the set is expensive and we will find out later. This will free up a lot of creativity.”

 

New protagonists and new approaches to volumetric video

Of course, Facebook isn’t the only company that invests in volumetric videos. And a system like the Collector isn’t the only way to do it.

For example, in January of this year, Intel opened a volumetric video studio in Los Angeles.

His approach to volumetric video comes from the outside. Rather than a single system that captures the scene from the inside out, like Manifold, Intel Studios surrounds the 10,000 square foot capture area with dozens and dozens of cameras.

The technology is already in use in several football stadiums across the country to shoot sporting events from all angles and recreate 360 ​​degree highlights. Intel is also already working with Paramount to develop a cinema-based application.

However, the implementation of 38 5K cameras positioned specifically connected to a server that process up to 1 TB of data every 15 seconds (as in the case of those stages) would still be more prohibitive than the reverse volumetric camera (or any reverse volumetric camera ) of Facebook, regardless of how much the collector can cost.

In addition, with an external volumetric shooting you can capture high quality images of people and things, only in a limited space in a highly controlled environment, while with a volumetric video on the reverse side you have more flexibility and you can shoot much more spaces and places wide.

Manifold is not just a VR camera

“The beauty of (Manifold) is that it is the versatile tool par excellence,” Shulkind told OC5. He plans to use the camera to create results not only for VR headphones, but also specific content for visual experiences, whether the audience is watching on a smartphone or TV, from an airplane or in a theater.

“Manifold isn’t just about virtual reality or AR or the future of immersive entertainment, it’s writing code for the future of content,” said Shulkind. “I can see a cascade of results that are all relevant, that make me feel like I never miss the scene, each scene in a different perspective.” All captured by a camera.[/vc_column_text][/vc_column][/vc_row]

Manifold market launch and future developments

The launch date and the Manifold price have not yet been announced. The camera in its current form still has many limitations. We are talking about a technology that could potentially extend wildly throughout the audiovisual and cinematographic sector, radically changing its approach.

Like all ‘extreme’ innovations, it is premature and almost impossible to foresee their developments and possible fields of use, but undoubtedly it is a tool intended to enter the video sector by law, whether it be virtual reality or video shooting. traditional.

We just have to wait anxiously and anxiously for the release of Manifold on the market.

 

Filming in Virtual Reality with Manifold: what is the volumetric video

Excerpts from this article are taken from content originally published in Redshark News.

Leave a Comment

Your email address will not be published.

*