3net's Best Practices for 3D TV
COW Library : Stereoscopic 3D : Debra Kaufman : 3net's Best Practices for 3D TV
The mid-adventures of poorly planned 3D productions are legion. One production used a jerry-rigged 3D rig that couldn't change interaxial distance. Another didn't synch the cameras, and yet another had only one of the two cameras in focus. Then there was the production with a giant smudge on one of the camera lenses, because the production only monitored the output of one of the cameras. Another production swapped the left and right eyes in post.
"We've seen everything from people trying to fabricate crazy rigs to put two cameras together, to those not investing in good editing tools to manipulate stereo images," says Discovery Communications' Director of Technology and Standards Bert Collins. "We've seen every shortcut you can imagine and none of those things turn out really well. You might have gotten away with some of those problems in an HD production but, in a 3D production, some problems -- such as geometric misalignment of the cameras -- are impossible to fix in post without serious miracle work."
3net, the 3D TV channel launched in February 2011 that is the joint venture of Discovery, Sony and IMAX, had every reason to want its 3D TV programmers to avoid the pitfalls involved in stereoscopic production and post. According to Collins, the impetus for the 3net 3D production guide -- which is available for free on the 3net site -- came originally from 3net President/CEO Tom Cosgrove.
That's where Collins and fellow Discovery Communications Director of Technology and Standards Josh Derby came in. In their roles, the two have been looking at 3D for a long time, before 3net was even launched. "It's our role at Discovery to scout and look for future technologies and investigate how those technologies apply to Discovery," explains Derby. "Four or five years ago, our management saw the beginnings of 3D content in digital cinema, and asked us to start researching what we would need to make this work for documentary and real-life TV."
These two were joined by co-authors Bruce Dobrin, Don Eklund, Buzz Hays, Jim Houston, George Joblove and Spencer Stephens to create the 49-page 3net 3D production guide. "The important thing to note is that these documents are a work in progress," says Hays. "But we tried to be as complete as humanly possible and answer most of the questions that a production would face."
Sony Pictures had been engaged in holding 3-day workshops for filmmakers interested in learning the basics of 3D production. Those workshops focused on the creative issues of storytelling in 3D, says Hays.
The 3net 3D Production Guide deals specifically with all the technical issues required to produce a successful 3D stereoscopic TV show. "We wanted to make sure that the companies doing 3D production were as up to speed as possible," Hays says. "There are a lot of naive expectations regarding 3D filmmaking. My mantra is that it's easy to make 3D, but it's hard to make it good."
Collins and Derby also gained a lot of information on what first-time 3D filmmakers needed by sitting in on phone calls with every production launched by 3net. "We refined a lot about what our expectations were and what the things were that people were most likely not to understand," he says.
One of those challenges was the fact that many of these filmmakers were producing Discovery-style documentaries. "One of the biggest things that people struggled with was choosing cameras and rigs," says Collins. "That was new to a lot of production companies, even if they'd been doing work for Digital Cinema. The gear that worked on a movie set didn't always work in a more documentary setting."
At the same time, however, the authors of the guide state very clearly that they do not recommend specific cameras, rigs or editing systems. "Our goal here isn't to restrict peoples' choices in production or post," says Collins. "First, because the tools are always changing. And, second, because we don't think we have the monopoly on good ideas. There are a lot of people in the industry who are innovative, and this guide is to help them think through their choices to make sure it comes together to make successful TV programs." Sony's Hays agrees. "The tool has to be designed for the job, which is why we don't recommend specific tools," he adds.
The guide is also not a primer for aesthetic considerations of stereoscopic TV programs. "3D isn't a new art," says Derby. "There's plenty of material out there on stereography, but there wasn't any information about planning a 3D production workflow. That's what the guide attempts to help people with, and specifically for TV."
PLANNING, PLANNING AND MORE PLANNING
Pre-planning and the use of metadata take up more than half of the guide. "Almost from the very beginning, we found that producing 3D programs requires quite a bit more planning and testing," says Derby. As they say in the guide, "producing 3D programs is a complicated business that requires far more planning than producing programs in 2D. There are new crew members and responsibilities, new and complicated equipment, and new post processes to understand. A production's progress can be brought to a standstill at any point by an unwelcome surprise."
As Hays puts it, the core message is a film school adage: "Plan your shoot and shoot your plan." Collins points out that the producers facing their first 3D production may know how to operate cameras and move a production through its workflow. "But now they have to keep track of twice the amount of assets," he notes. "We are both advocates of advance planning and use of tools like metadata in every aspect of production, be it 2D or 3D. But it's even more important to use all those tools in 3D. We're trying very hard to stress that you can get away with a little 'off-the-cuff' production in HD, but you can make mistakes in 3D that you can't recover from."
From the Sony 3D Workshop. Photo by Beth Dubber. Shot with a Canon EOS 5D.
Hays notes the ways that data management can be a real pain point in 3D productions. "A lot of these productions just go out there and shoot 3D," he says. "They need to start planning. They're dealing with massive amounts of digital data and people don't factor in to the schedule or the budget the time it takes to move files from one place to another."
Hays recounts that when he produced Monster House, they spent four or five hours writing information to the hard drives, driving them to the post house, which would then spend another four to five hours to offload it. "It takes time to move all the data around and you have to factor that in," he says. "It's one of the biggest problems we have."
It's not simply that the producer will be faced with twice the number of assets, adds Derby, but that the production will also have to keep track of the information related to the two cameras, from geometric alignment to color and synch. "In 3D, there are a lot more things that come into play, such as understanding interaxial distance," he says. "In a file-based workflow, you can have a lot of rich metadata, and that metadata becomes extremely valuable later on in post."
Underlining the case for compiling rich metadata for 3D productions, the 3net 3D Production Guide quotes one producer who describes how his production benefited from metadata. "It was amazing how much of a difference it made when we returned from the shoot and started logging and importing footage," enthused the producer. "Our editing system was able to sort media into bins based on clips with metadata about the scene and the characters we were able to import and organize three days of shoot footage in four hours. The time we saved allowed us to spend more time in the offline edit, honing our story."
Collins also notes that the producers need to follow all the steps in planning. "If you miss one, you'll break the chain," he warns. Among the steps, he emphasizes that it is important to understand how imagery will look on the target screen. "In HD, whether you look at something on a cell phone screen or a 50-foot screen doesn't particularly matter, other than resolution," he says. "But in 3D, you have to understand what size picture the end viewer will be looking at. We see the results of people who didn't plan the camera geometry very well, so the resulting images don't play well with what they are trying to accomplish."
TESTING THE WORKFLOW
An important part of planning is testing the workflow end-to-end, say the authors. "One of the things we try to impress on producers is the importance of getting things right as you move through the process," says Derby. "One thing we found early on is that there are a number of mistakes you can make in the acquisition process that are nearly impossible to fix later. With HD, people got used to 'Oh, I can fix it in post'. There's very little you can do in HD acquisition that makes the footage totally unusable later. But something as simple as certain kinds of camera or rig alignment is impossible to fix in 3D."
The ability to move the footage from the camera through dailies and the entire post process out to distribution sounds like a no-brainer, but in 3D, working out the details in advance prevents unpleasant surprises. "With all the different cameras and rigs out there, it's important to make sure that the camera works with the editing software," says Derby. "It's more important [to ascertain this] in 3D because the formats in 3D cameras are newer and the editing tools aren't mature."
3D POST PRODUCTION
There are plenty of potential pain points for 3D post production. "But if everything has been done right in acquisition, post will be a lot less painful," says Collins. "There will be a lot less opportunity for error." But there is a big caveat. Producers who have not fully tested their workflow or who have made serious errors in acquisition can face tremendous challenges in post. "
One potential issue of not testing the entire workflow can occur if the production chooses an editing system that doesn't work with the camera output. The "solution" is to transcode the footage more than once, resulting in degraded quality of the image...sometimes to the point where the footage is unusable.
"IF every post tool worked with every file format natively, that would be ideal," says Derby. "But that's not the case, so you can end up with very complicated workflows. You have to pick compatible tools."
Hays notes that the post production tools have changed for the better in the last six months. "When we were writing this document, we didn't have the editing tools," he says. "You could get Cineform for Final Cut Pro, but you had to figure out the workflow. You had to design around it and make sure you have the right playback equipment."
Although the guide does not recommend specific tools, Hays notes that Avid has since come out with Media Composer 6, which he calls "a very usable solution" to editing 3D footage. Noting his position as an employee with Sony, he also points out that there is a 3D stereoscopic workflow with Sony Vegas Pro 10.
2D TO 3D CONVERSIONS
3net will not take 2D-to-3D conversions, and a quick conversation with the authors makes it abundantly clear why the network has made this decision. "Our philosophy is if you're going to do justice to a story in 3D, you need to shoot it in 3D, watch dailies in 3D, and post it in 3D," Hays says. "If you shoot in 2D, by the time you see the final version in 3D, you realize you would have made different choices."
Collins notes that 3net president/CEO Tom Cosgrove "felt it was only worth doing if it were visibly imperceptible to viewers." "Ultimately he didn't feel it got to that point in a model that worked in the TV environment," he adds.
In fact, the tighter deadlines and more limited budgets of TV programming make 2D-to-3D conversions unfeasible. Hays points out that a conversion can cost $20,000 per minute, a sizeable chunk of an ordinary TV episode's entire budget, as well as take weeks to accomplish. "It's impossible for a TV show," he says of the conversion process. "It's the greatest tool in the world for certain things but impractical for TV."
But this doesn't mean that 2D-to-3D conversions will never be accepted by 3net. "It just means, don't expect you can sell us a conversion product because that's not what we're looking for now," says Collins. "The idea is to build a great library of natively stereographically produced material, and the expectation that when you turn on 3net, that's what you'll be seeing."
From the Sony 3D Workshop. Photo by Beth Dubber. Shot with a Canon EOS 5D.
THE UPSIDE OF PRODUCING 3D FOR TV
Producing 3D content may require different gear and new skillsets, but the authors of the 3net 3D Production Guide point out the ways that TV producers already have a good starting point. One advantage is the size of the screens on set and location. "People get used to using monitors on set for the big screen, and the way images translate to the big screen is a leap of faith," Hays says. "With TV, what you see on set is what you're going to get on TV. In a lot of ways, if you're monitoring properly, you're seeing right away what the end user will see."
"Personally, I highly regard the notion of working in 3D for television," continues Hays, who has worked on plenty of 3D movies. "Production cycles are quicker. In a film like The Amazing Spider-Man, we had to make choices two years ago for a film that will come out this year. In TV, we can constantly update the workflow. The budgets are lower, so people have to be more resourceful."
This guide is intended to help TV producers and creatives learn just that: how to be confident in their knowledge, capable of finding the additional information they need, and resourceful enough to make killer 3D TV programs on budget and on time.