LIBRARY: Tutorials Reviews Interviews Editorials Features Business Authors RSS Feed

VR Filmwork: Immersive Begins to Emerge ... Slowly

COW Library : : Andy Marken : VR Filmwork: Immersive Begins to Emerge ... Slowly
CreativeCOW presents VR Filmwork: Immersive Begins to Emerge ... Slowly --  Editorial

Andy MarkenAndy Marken
Andy Marken. All rights reserved.

Some people think we like 360/VR just because it devours so much storage capacity. Ok that's a good reason but still the good VR stuff is really really good. Not just scare your pants off good but good to experience, enjoy, be immersively entertained, informed. Despite what Musk says we may never experience travel to the outer reaches of space or the depth of the Mariana Trench or climb Mt. Everest but 360/VR can let us experience, really experience these ventures and more. We know because some friends took us there and back. VR is good and it's getting better all of the time.

"I need to have something in the foreground to give it some scale."
Hooper, "Jaws," Zanuck/Brown Productions, 1975

Want to see a storage person's heart skip a beat ... or two? Flutter even?

Just mention VR and BAM!, they get real excited.

Creating good (if not great) VR is getting slightly easier, but it still poses serious challenges for the professionals working in this emerging medium.

We've seen some of the stuff Andrew Shulkind and Andrew Cochrane have been up to (scripted and unscripted), and we know it's good.

We were all fired up to hear Shulkind at CineGear and then later presenting with Cochrane at the Creative Storage Conference to get their insight into the progress that has been made with the technology and when we'll be able to see VR as "regular fare."

Before this tag team took the stage at Creative Storage, Tom Coughlin gave his annual overview on the state of storage in the M&E field.

With 4K content consuming about 6,880GB/h and 8K gulping roughly 91,729GB/h, we're starting to talk some serious storage capacity.

360° and 180° immersive video and AR/VR must be even more data-hungry.

It's Great – Most of the people who discount VR film viewing either haven't experienced it or watched some really bad VR immersion (it is out there). Once you experience good/great VR storytelling, you're ready for more.

After all, darn near everyone is ready to immerse themselves in great content.

Taking a conservative approach to forecasting VR storage needs, Coughlin projected that stereo 360° video would require about 115 GB/s of data or roughly 414 TB/h due to its 16,000 x 16,000-pixel resolution, 10+ bits per channel and a frame rate of up to 120fps.

Both Andrews thought these requirements were "on the modest side;" and they should know, they're busy doing projects around the globe.

For almost two decades, Shulkind has been at the leading edge of where technology meets entertainment. He's become the "go-to" expert on the future of entertainment; bridging immersive content, advertising, cinematic quality, gaming and mixed reality.

He is a rainmaker for both Hollywood and tech giants making Hollywood-style steps.

He's done projects all over the globe and even works with foreign governments to help foster local production economies but seems most proud of his advisory work for SpaceCraft VR, a NASA-sponsored interactive infrastructure for space exploration.

Beyond Gravity – Shulkind's work with the Spacecraft VR project, sponsored by NASA, is a worldwide collaborative effort that excites you whether you're young or old, male or female. It is addictive to see how people come together to expand our understanding of the universe.

Cochrane's pedigree is just as impressive, including running digital and interactive projects for over six years at Guillermo del Toro's Mirada, and now managing projects as a consultant and independent creative. Recently, he directed five CG-animated 360° films for the Vegas: Alter Your Reality experience and has created immersive content for USA Today, Disney, Intel and Visa, with multiple projects currently in production.

Before the session, we sat down with Cochrane and Shulkind to get an update on the progress that has been made with VR over the past few years.

Sound Growth – Everything is getting better about VR film work, including the experience of the people at the forefront of immersive content. They're learning how to help viewers tell their story. The only shortfall has been the wild projections of early analysts. It will take time.

Shulkind said the immersive content business is the latest example of how storytellers have always used the best tools available to engage and thrill the biggest audiences in the most immersive way possible.

From stories painted around a cave, to amphitheaters and coliseums, to 19th century panoramic paintings and haunted houses, to Cinemascope and Cinerama, to surround sound and gaming chairs, entertainment has always striven to be engaging and immersive.

Cochrane agreed that the application and purpose for placing a person directly in the movie or experience is only the beginning, and the future of this medium is when users can interact with the content.

"We've only just begun to tap the potential of immersive media. From 360° and 180° video to realtime experiences, there's a growing library of interesting, innovative content that people really want to experience," he added.

It's Alive – OK, "The Strain" was spooky enough on FX but Cochrane helped people get up close and personal with the evil strain of vampirism that ravaged New York with VR segments. If you aren't scared when you take off the HMD ... you're one of them.

At the beginning of their presentation, Cochrane told the audience that VR is a massively exciting new medium that has ignited the imaginations of technicians, creatives and businesses.

"We have only scratched the surface as to what can be done with and how people can experience and interact with virtual environments," he noted. "But what gets lost in the excitement is that it isn't going to be an instant success. We still have a long way to go and obstacles to overcome."

Both filmmakers noted that the challenge is being able to efficiently and effectively handle all of the data required to create this content.

Poor Math – VR film production done right means that 1+1 doesn't equal 2, it equals 4 – content for everywhere you look, might look, couldn't possibly look. As resolution quality increases, so does the data.

Shulkind explained that 1080p per eye can be readily handled within traditional production pipelines. But when you scale up to 3D 4K VR, the requirements grow exponentially. "At the moment, the weakest link is not the capture but the display devices," he stated. "Outside of a couple of early devices that still have very low adoption (like StarVR 5K and Pimax 8K), every headset is still below the 4K display threshold.

"Even if you wanted to deliver in 4K, you couldn't see it," he noted. "The current displays have such low pixel density that when viewed through the cheap plastic magnifiers in the headsets, the image is dramatically softer than what we shot. Newer displays will improve upon this quickly; but in the meantime, you have to take these factors into consideration when selecting your pipeline and how you spend money."

Robust - Filmmakers who understand it well know that the content is more than just HD, 4K or even 8K. VR adds a new dimension of workflow and workload for the production/post teams.

"4K is not twice 2K, it is four times 2K!" quipped Cochrane. "If we want to match the abilities of our eyes, we'll have to reach 64k resolution, at 72 frames per second or higher with at least 11bits per channel, per eye; so, we've got quite a way left to go.

"The production, distribution and viewing issues all have to be addressed with a project, and it isn't simple," he added.

More Glass, More Storage – VR cameras and rigs are being improved much more rapidly than the headgear people wear to experience the content. Rigs like the Headgear VR unit at the right also use a lot of SD cards to capture 360 degree and immersive content.

Shulkind stated that VR cameras and rigs are still in their evolutionary stage with new and better solutions shown every year.

The one thing they all have in common is that they capture a lot of data – usually on SD cards – which needs to quickly and reliably copied to the computer and larger, more easily managed storage solutions.

Shulkind noted that rendering of the detailed images requires robust CPUs and GPUs because a detailed 4K image can require more than 5.6ms per frame.

"If rendering requires 700ms of every second, the system has to move more 2.988GB in 300ms or 9.9GB/s," he explained.

Heavy Lifting – High-performance CPUs and GPUs are needed to process immersive RAW data during the stitching and rendering processes. And storage has to be fast enough to keep the pipeline full.

"A pivotal point for VR filmmakers and producers, is our ability to move RAW and final data quickly and efficiently which means high performance GPUs and ultra-high-speed storage solutions," Shulkind continued. "We always have someone on set with multiple laptops churning through rushes and focusing on what is being done so we can more quickly and easily complete the fine stitching after the edit."

He noted that the fastest consumer SSD available today tops out at 3.5GB/s. As a result, VR filmmakers have either striped several SSDs to handle 4K VR or do what he and several friends have done, moved to the OWC ThunderBlade. The unit can transfer up to 2800MB/s and two units provide speeds of 3800MB/s to handle the high bandwidth content stream requirements.

Technology such as volumetric and light field capture, photogrammetry and spatialized audio will continue to generate extremely large, complex master files that will dwarf what is being done today in terms of both size and required functions.

Virtual is Real – Shulkind emphasized to the audience that VR proponents tend to be overly optimistic on when the film experience will be mature; and naysayers tend to point to how growth isn't keeping pace with industry projections. The truth is somewhere in between, and both progress and acceptance are taking place.

While VR has its optimists and pessimists, both Andrews feel significant progress has been made in the past few years. The new cameras, software, computing assets and storage are paving the way for exciting opportunities for people to immerse themselves in a wide range of content – documentary and entertainment.

"We're working on new experiences and examining new ways the audience is responding," Shulkind said. "At the end of the day, when we've done our best work, it's the reaction/interaction of the viewer that keeps us going."

"We've moved into an exciting new space," Cochrane added. "We're not just visual storytellers anymore, we are really enabling and encouraging people to be story-livers. VR is quickly becoming dynamic, personal, social, and even transformational.

"The technology, software, talent and user behaviors are still in the early stages," he continued. "We haven't seen the extent of what VR can deliver. We don't want to oversell it, and we don't want people to sell it short; it's still in its infancy."

More than just filmmakers; Shulkind, Cochrane and other active VR professionals are integrating the polish of immersive interactivity and data optimization, setting guidelines, focusing on standards and using AI to turn skeptics and non-believers into VR fans.

For the Creative Storage audience, they had one final message ... the volume of data and content they are creating and working with is growing rapidly as they create more interesting, more exciting and more immersive video experiences.

Or, as Brody said, "You're gonna need a bigger boat."

# # #


Related Articles / Tutorials:
3 Ways to Improve Video Engagement

3 Ways to Improve Video Engagement

Engagement is my favorite metric to track, as it represents the overall performance and health of your video. Unlike vanity metrics such as Views (which simply shows the reach of your video), metrics like Audience Retention tell you the percentage of the video the audience watched (engaged with) and exactly what parts were watched. Analyzing this metric will help guide you on what your audience really wants to see so you can do more of it in the future.

Andy Ford
Fix Reverb Issues with DeVerberate 2

Fix Reverb Issues with DeVerberate 2

Reverb is an issue we all have encountered. Sometimes you just can’t help recording in an enclosed area. You may not have dealt with the issue because it seemed difficult to fix, but life is now easier with the DeVerberate 2 plugin from Acon Digital

Andy Ford
Bringing Homeland Home: Color Grading The Final Season

Bringing Homeland Home: Color Grading The Final Season

Keith Shaw of Keep Me Posted (a FotoKem company) has been the colorist for Showtime's Emmy-winning drama Homeland since its pilot episode in 2011. The show's eighth and final season brings the most extreme looks yet to a show shot in locations around the world, and where a scene that takes days to shoot unfolds in only minutes on screen. The challenge to create unified, consistent looks has never been steeper, but a rock-solid workflow from a global team is bringing the popular series home to a strong finish.

Karen Moltenbrey
Blackmagic Design Announced WHAT?

Blackmagic Design Announced WHAT?

As if this wasn't already the hottest summer on record, Blackmagic Design's Grant Petty rolled out some blazing news across a wide range of the company's product lines in cameras, production hardware, and software.

Creative COW
Makin’ Planets! Adobe After Effects Tutorials by Graham Quince

Makin’ Planets! Adobe After Effects Tutorials by Graham Quince

Here's a great new Adobe After Effects tutorial series from Creative COW leader Graham Quince that takes advantage of the free Orb plug-in from our friend Andrew Kramer at Video Copilot. Graham supplies project files, tips and tricks, and good humor on this tour through building every planet in the solar system, with customization options galore!

Creative COW
Rising Sun Pictures with SPIDER-MAN: FAR FROM HOME

Rising Sun Pictures with SPIDER-MAN: FAR FROM HOME

In its first contribution to the Spider-Man franchise, studio produces hologram that reveals the origin of the Elementals

Editorial, Feature, People / Interview, Business, Project
Creative Cow
Blackmagic Design's Epic Hardware Releases for NAB 2019

Blackmagic Design's Epic Hardware Releases for NAB 2019

Blackmagic once again delivers a jaw-dropping lineup of advanced hardware -- ATEM Constellation 8K with built-in Fairlight mixing in the industry's largest live production audio mixer, plus 16 hardware keyers, 24 8K outputs, and more; the New Teranex Mini SDI to HDMI 8K with dual on screen scope overlays, HDR, 33 point 3D LUTs and monitor calibration; HyperDeck Extreme 8K broadcast deck with advanced H.265 recording, touch screen interface, internal cache, built in scopes, HDR, and traditional deck control; and the URSA Mini Pro G2 camera featuring new electronics, Super 35 4.6K HDR image sensor, 15 stops of dynamic range, 300 fps high frame rate shooting, and Blackmagic RAW -- for prices even lower than how low your jaw just dropped reading them.

Creative COW
Making The Forest Leaves Dance for Ridley Scott's Hennessy Short, "The Seven Worlds"

Making The Forest Leaves Dance for Ridley Scott's Hennessy Short, "The Seven Worlds"

"The Seven Worlds" is the latest advertising epic short directed by Ridley Scott, a marvelous odyssey spanning surreal scenes inspired by the seven tasting notes of Hennessy premium X.O. cognacac. Realizing Scott's unparalleled vision required a masterful team and state-of-the-art technology to assist, that's where London's The Mocap Studio team came in. The mobile motion capture specialists lent their expertise to the project, using Xsens MVN inertial technology to help capture a human performer on-location in the heart of a Czech forest.

Creative COW
How Cinesite and Allegorithmic Substance Recreated a Legendary Medieval City for Lionsgate’s ‘Robin Hood’

How Cinesite and Allegorithmic Substance Recreated a Legendary Medieval City for Lionsgate’s ‘Robin Hood’

The city of Nottingham perpetually exists in two states: the metropolitan center that it is today, and the fictional home of one of the world’s most famous outlaws. So when the filmmakers behind 2018’s Robin Hood looked to recreate the fictional Nottingham, it needed to build it from scratch with help from London’s Cinesite Studio.

Creative COW
Perfecting Audio: Professional Audio Apps for Your iPad

Perfecting Audio: Professional Audio Apps for Your iPad

There's huge development going on for professional audio on iPad, whether for recording, hardware control for digital audio workstations, or music creation. Longtime audio engineer Keith Alexander looks at some specific options that audio pros will find indispensible.

Adorama TV
© 2020 All Rights Reserved