Santa Monica California USA
©2012 CreativeCOW.net. All rights reserved.
When director James Cameron took a solo dive to the deepest spot on earth, he turned to Sydney-based Digital Pictures to deliver a range of post production services. In turn, Digital Pictures relied on StorageDNA to help them create a cost-effective LTO-based conform system that handled massive amounts of 4K data.
explorer-in-residence and director James Cameron launched the DEEPSEA CHALLENGE
project -- a historic solo dive to the Mariana Trench
in the western Pacific Ocean -- not just for the adventure but to document it for an upcoming feature film. The untitled 3D documentary, which is due out in 2014, will comprise footage from nine dives to various depths and showcase the lightweight compact 3D 4K cameras and rigs from Cameron Pace Group
The RANGER Data Cart, a streamlined content management system designed to manage the data demands of a production in a mobile or remote location.
Behind the scenes, Cameron and his team partnered with Australian film/TV post production company Digital Pictures
to deliver a range of post services. Given the massive amounts of 4K data that the shoot would produce, Digital Pictures aimed to create a digital post production pipeline that consisted of a high performance archive and retrieval workflow solution that was both efficient and cost effective. The solution needed to decrease time spent in post production, be able ingest/digitize film/tapes to DPX, reduce costs for stock media, SAN storage and nearline disks, minimize form factor and maximize performance, and ensure format flexibility.
"The existing post production workflow involved time-consuming manual processes with high stock media and SAN storage costs," says Digital Pictures Technical Director Nic Smith. "The producers wanted the ability to quickly back-up data, transcode to dailies and view everything right out of a camera negative while protecting the digital assets." The production also needed a solution for its RANGER Data Cart, a streamlined content management system designed to manage the data demands of a production in a mobile or remote location; the RANGER Data Cart would be on the ship from which the underwater dives would be launched. "This was a very challenging and complicated project because it was a multi-camera shoot with every kind of file format you can think of," adds Smith.
At the heart of the project was the effort of Digital Pictures, partnered with workflow/archive company StorageDNA
, to turn LTO from a strictly archival medium into an online conforming medium. The results, thought Smith, would give Digital Pictures exactly what they needed to offer cost effective and efficient solutions to archive, restore and conform content in a tapeless workflow at very high speeds.
The solution -- and the relationship -- began, as these stories often do, at NAB. "Nic Smith approached me at NAB
2010," says StorageDNA CEO Tridib Chakravarthy. "They wanted to deploy file-based pipelines. There are a number of vendors out there that focus on the LTO archiving space, but he wanted to do something different."
StorageDNA CEO and President Tridib Chakravarthy
"Nic asked if it was possible to turn the LTO into a medium that you can do conforms from," he continues. "We scratched our heads and thought, this could be possible." Smith worked with StorageDNA on specifying the conform process until, two years ago, Chakravarthy was able to give him a beta version of what he was looking for. "We've turned LTO into online storage," he says.
After using the product on several projects, says Chakravarthy, Smith came back with a bigger project -- the DEEPSEA CHALLENGE -- that was switching from tape to tapeless workflow. "Everyone wants to go tapeless," says Chakravarthy. "Everyone puts material on LTO. But, after you put your material on LTO, what is the most creative way of getting your content back?"
"Normally, they would have to keep the high resolution media live on their online storage," he says. "With our solution, an EDL of the low-res sequence finds the frames or sequences across all the LTO tapes -- no matter how many -- and brings only those back to for conform. It's a huge cost savings. You reduce your storage by one-third no having to use online media, and no one has to manually find the frames or sequences."
Digital Pictures customized the RANGER cart with StorageDNA's DNA Evolution
for the archive workflow solution. "The DNA Evolution LTO-5 archive workflow has allowed us to save tremendous time and costs for our clients," says Smith, who notes that Digital Pictures has used DNA Evolution for several clients' productions since July 2011. "Since we're now working in a digital pipeline, the need to ingest has been eliminated and the conform process has been fully automated."
DNA Evolution™ user interface: Conform and restore options for Avid, FCP, P2, ProRes. The editorial team edits in low-res and then feeds us the EDL. Using reel and shot number metadata, we find the right tapes and automatically pull out the correct media. Digital Pictures had 24 1.5 terabyte tapes online, so it functioned like near-line storage.
"The reason we chose this LTFS archive solution is we needed maximum performance and reliability since we often deploy in the field for our clients," he adds. "DNA Evolution met all of our rigid requirements."
The pipeline is applicable to any file-based acquisition, from RED
Epic to GoPro
cameras. The workflow for DEEPSEA CHALLENGE project presented very challenging parameters. "As raw footage came onto the ship, the first thing they'd do is off-put it to LTO tape and also make an editorial low-res copy to DNx145 or ProRes HQ that went to editorial," says Chakravarthy. "The editorial team edits in low-res and then feeds us the EDL. Using reel and shot number metadata, we find the right tapes and automatically pull out the correct media. Digital Pictures had 24 1.5 terabyte tapes online, so it functioned like near-line storage."
On the ship, the team actually made two LTO copies, one for conform and the second for its more traditional use -- disaster recovery or archive. Transfer between DNx145 and LTO tape was also fast: under nine minutes for an hour's worth of footage, says Chakravarthy. "We are trying to put all our innovation and transform something that ordinarily sits on the shelf for 50 years into something that's more active and near-line. The performance and price benefits are pretty amazing."
The system also scales. "There is no limit," he says. "I could have 5,000 LTO tapes sitting in a warehouse and you can give us an EDL and we can tell you what tapes you need, plug them into the auto robot and we can restore them to you. You can get 100 TB of LTO tape for under $4,000 so it's infinitely scalable and cost effective."
Chakravarthy says they're getting ready to deploy this LTO-based pipeline for reality TV and episodic TV, for different reasons than what made it ideal for the DEEPSEA CHALLENGE project. "It's not that they have a lot of content because of 4K cameras," he says. "It's because they deploy Panasonic P2
and Sony HDCAM
and record a lot of material."
DNA Evolution™: X400-48 product with HP DL120/320 server and 48 slot, 2 or 4 drive autoloader. Digital Pictures thought out of the box to provide Cameron with a solution to meet his extreme shooting conditions. In the process, they nudged StorageDNA to reinvent the use of LTO tape.
As content creation moves increasingly into a tapeless world, Chakravarthy adds, the issue of how to handle massive amounts of data will become even more crucial. "With tape, producers had storage media they could put on the shelf," he says. "Now they're making LTO copies, but think of it as archive. Nobody has stepped up to transform it into near-line. The pipeline is pretty messy."
"Anyone can use LTO for an archive," he adds. "But who can do intelligent restores and conforms? That's where we focused. If you're mandated to shoot tapeless, you still have a time crunch and a budget and vendors haven't come up with a solution. You only need one hard drive to fail to be convinced that keeping drives on a shelf is not a great idea."
Digital Pictures thought out of the box to provide Cameron with a solution to meet his extreme shooting conditions. In the process, they nudged StorageDNA to reinvent the use of LTO tape, and come up with one solution for handling data and metadata in today's often confusing tapeless workflows.
If post workflows are snowflakes, then StorageDNA won't be the only company to come up with a solution to challenges specific to today's TV/film productions. But they're a good example of how ingenuity paired with technology is capable of inventing new ways of working. Chakravarthy says Storage DNA is already planning their next line-up of innovations. You can be sure we'll cover those -- and other companies' innovations -- as they become available.