LIBRARY: Tutorials Reviews Interviews Editorials Features Business Authors RSS Feed

Heavy Iron

CreativeCOW presents Heavy Iron -- Business & Marketing Editorial


DigitalFilm Tree
Hollywood California USA
CreativeCOW.net. All rights reserved.




Cheap software that promises the world is nice. Software like Final Cut Pro has made our business possible. We are proud to have been an important part of posting the first major feature film ("Cold Mountain") and primetime network series ("Scrubs") to use FCP.

But as you start to move higher up the production food chain, things expand really fast - even if the building blocks are lots of cheap workstations, storage, and IO. It becomes apparent very quickly that no real production facility, no matter how much it leverages the current crop of great software, can survive without serious attention to hardware.

When you hear the word "facility," you might think that we are bigger than we are, especially when you hear that we are currently posting three TV series, as well as a number of shorts, at the same time. There are in fact nine of us.

Yet, we have been able to triple our workload in the last two years, keeping roughly the same (relatively) sane hours - thanks to carefully selecting hardware to help with the heavy lifting.


DESIGNING A FACILITY

In 2007, we designed a new facility to meet the opportunities we saw coming. We had already been working with Sony HDCAM SR and the Genesis Viper, and saw the potential of other emerging, high data rate cameras, so we made a simple assumption: 4K.

The 4K data rate is insanely high (1.35GB of data per second, over 80GB/min.) Playback of such high res data is fraught with peril, even with local storage. I will go into more detail about this later, but shared storage was a critical part of our vision - and sharing 4K is impossible with less than a fibre network. Our entire facility is wired with fibre for this purpose, with switches capable of scaling from 2Gb all the way up to 8Gb data rates.

Rather than a bunch of suites, we wanted to design an editing facility. All of our Mac Pros are fully tricked out with Final Cut Pro or Avid Media Composer - and all of them are stored in a central machine room. All we have in each edit room is monitors, keyboards and mice, with a single Cat-6 Ethernet cable linking each computer via a KVM (keyboard, video, mouse) network.

Control Room

KVM products are simple and inexpensive, and work great with even just a couple of computers. The KVM system has created a huge ripple effect for us. Any computer can be called up from any bay with a couple of keystrokes. No one bay is more powerful or more sought after than any other.

We also now have the ability to work on multiple edit stations at the same time: Our editors or post operators often click to one computer, start captures, switch to another, start renders on that one, switch to a third one and continue to QC the show, while the previous two computers capture or render away.

When our operators need to leave one edit location, and say, move to the machine room to lay off the program, they can call up the computer they had just been working on from the new location.

And if we need another bay, all we need to do is connect an Ethernet cable, a keyboard, a mouse and a monitor. We could add one in a closet if we needed to.


A FINISHING HOUSE

When Final Cut Pro and its related hardware allowed anyone to create "online" quality video, being an "online facility" became meaningless. Instead, we positioned ourselves as a "finishing house," handling all of the broadcast and technical requirements needed to finalize and deliver a show.

While it is true that inexpensive cameras and laptops can do great things, they are not suited to heavy lifting for delivering air masters for broadcast. The broadcast specs from PBS, for example, run over 80 pages. There are details about which line should the ancillary data be encoded to, the precise specs for Closed Captioning, and much more.

We set up a facility-wide Tektronix-based scope system that allows us to output and monitor broadcast quality color and luminance. The same system also performs automated quality control (QC) analysis of all outputs, and creates detailed logs of any errors. We also created a sophisticated hardware and workflow system to output numerous deliverables for our clients, each of them properly captioned.

Which brings us to the key ingredient for a successful tapeless workflow:


TAPE

Shooting and post workflows are becoming increasingly tapeless, but a tremendous amount of work is still being shot on tape.

We found this ourselves as we worked on the film, "The Forbidden Kingdom," starring Jet Li and Jackie Chan. The production spent a great deal of time on location in China, far from any cities. They were way out in the jungle, and in mountains and deserts. Even the primary soundstage was in a rural village, a five-hour drive from Beijing.

We used the Panavision Genesis, which has proven to be an extremely reliable and rugged camera. Even though integrated solid state storage is available for it, the production relied on tape. Computer problems are far more likely than deck problems, and if something had broken down this far away from anywhere, the production would have been completely screwed.

Control Room

Shooting to tape was also more efficient: no need to offload data before the next round of shooting, no concerns for storage other than a box to keep the tapes in.

The format of choice for digital cameras recording to tape is Sony HDCAM SR, one of the few formats that can handle the high data rates associated with 4:4:4 RGB recording.

Even if recording is to disk or solid state media, especially when you get to broadcast, delivery is exclusively on tape. Nobody that we work with in either Los Angeles or New York is going to air from digital masters on disk. Broadcast facilities simply can't reconfigure to disk-based infrastructures overnight - and from what we have seen so far with the ones who are starting the transitions, the last part of the system to change will be playout to air.

This is because every part of the process leading to air is tape-based, starting with QC systems. These are automated so that nobody has to watch every second to ensure standards compliance, and they are built for tape.

Our workhorse tape formats are Sony HDCAM SR and Panasonic D5. We get plenty of both, on a regular basis.

We also have just about every other format represented, including HDCAM, HDV, DV, DVCAM, Digital Betacam, Betacam SP, all the way down to ¾" U-matic (You'd be surprised how much work those ¾" U-matic decks get to this very day.)

Not that digital workflows aren't increasing. ABC has added a requirement for IMX dailies, MXF files which they use for their in-house asset database. Any of their executives can log on with a web browser and review the production in process.

The idea is that this prejudice against tape in this "modern, digital age" may be quite misguided.

One of the best uses of tape is actually for backing up data recorded on drives and solid state! To this very day I am hard pressed to find a more efficient, lowercost way of creating a digital backup for a one hour show, than by simply outputting it via a digital path to a robust digital tape format. Blank tape is ridiculously cheap, the "backup" happens in real time, and tape is a proven and reliable way of storing media.

Backing up digital material to robust tape formats is actually the recommendation of the Science and Technology committee of the Academy of Motion Picture Arts and Sciences, as presented in their report, "The Digital Dilemma."

While the paper notes that film has the longest reliability by far, the research they present convincingly points to a much longer life cycle for tape than for computer media.

In any case, even when shooting RAW files to disk, and an entirely tapeless post workflow, everything for broadcast ends up on tape, for both master layoff and air master delivery.


PLATFORM AGNOSTICISM

Even though we have always been known for our work with Apple software and hardware, we were not willing to let platform allegiance stand in the way of our ability to succeed.

That did not just mean adding Windows to our environment. That meant adding everything: Linux, IRIX, the whole shebang. Each of them runs unique applications that are essential to our business.

We knew that we needed to run all of these applications, and work with all of the media associated with them, on a single, shared-storage network, also known as a SAN.

We had been SAN-savvy from the very get-go, which now seems another lifetime ago. Our network design for "Cold Mountain" for example, was based on OS 9.

Because we had been working with Apple for so long, we looked closely at Apple Xsan. We learned that the foundation code for Xsan is licensed from a product called StorNext, now part of Quantum. Xsan was simply a subset of StorNext, with a sexy Apple GUI (Graphic User Interface) on top.

One of the advantages that we found in going directly to StorNext was the ability to create a single network shared by all of our platforms.

I have already mentioned our Video Village, and the tape formats we work with. Here is how the rest of our facility lays out:

Xsan Metadata Controllers require their own, dedicated Ethernet network to send all of their commands, while the fibre pipeline is devoted to data transfer only. We also have a backup server so that we can provide uninterrupted service if the other fails - a key idea in any SAN environment.

We've amassed a few of the 128 port, 4Gbps Qlogic 9000 Fibre switches, where all of the fibre connections reside. We have found the Qlogic switches to be one of those miracle pieces of electronics that just keep on ticking. Not a single issue we've ever had to resolve in our SAN fabric has ever been traced to the fibre switch. The Qlogic switch is a perfect piece of machinery, if there is such a thing.

Our Xsan storage is striped as RAID 5 for maximum redundancy and protection. If a drive module fails (and once in a while it does), we can replace the module and rebuild the RAID, without any loss of data or time.

We have more storage based on the HP Storage- Works Scalable File Share for Linux.

Our combined storage space varies, and with typically anywhere from 50 TB to 100 TB on line. It can easily go higher.

Mac OS X is one of the many platforms we run in our facility. We have a boatload of Mac Pros, all running Final Cut Pro and using either AJA Kona 3 or Blackmagic Design Decklink video cards for capture and output. Other key pieces of software we run on OS X are Apple Color and Motion, Adobe After Effects, Maya, Avid Media Composer, and many others.

We use Windows XP-based render nodes, and Linux-based tracking software.

We have several SGI CPUs running the IRIX platform, for running our Autodesk Flint and Flame applications.

Many people thought that After Effects would kill off hardware-based solutions such as Flint and Flame. Almost all daily effects work today at DigitalFilm Tree involves AE. However, when you need heavy lifting, there is no substitute for heavy iron.

For example, we did almost all effects work in Flint and Flame for the film "Crank," starring Jason Statham. There is simply no substitute for the quality and speed those hardware-based solutions offer. They were also handling 2K and 4K files long before apps like AE and FCP could come close. When it comes to high-end special effects work done under the gun, very little can match them.


IN PRACTICE

Before we were even halfway through our facility upgrade, we started to realize the fruits of our labor.

Our editorial, effects, color and titling could all proceed simultaneously, as opposed to in a linear order. The colorist could start roughing out the show before the editors or the effects artists were through. Editors could make changes and the rest of the staff working on the show could catch up without any lengthy emails or details going around. We realized huge savings in storage space, because we were not doubling up media.

Right away, we were able to take on a second TV show without any additional staff. And then a third one, with the same number of people. We let the hardware take care of the heavy lifting.

One of the very first shows to take advantage of this with us was "Everybody Hates Chris." One of their producers, Erik Whitmeyer, has said that their show realized a savings of $11,000 per week, due to this new way of working. He also noted that other benefits included working sane hours and less time spent making changes.

Now underway: work on three series ("Cougartown," "The Sarah Silverman Show," and "NCIS Los Angeles"), while also working on other projects, including coloring a film for Apple, featuring Walter Murch's digital workflows on the set of Francis Ford Coppola's next picture.


HOW TO BEGIN

Buying all of this was expensive - but we didn't buy it all at once and sit in our shop waiting for work to materialize. When we began, we survived by being an Apple Authorized Training Center. We trained people to edit in FCP and FilmLogic by day. At night, when there were no classes, we used the training computers to do our work.

Then we bought our first DV deck. That was a monumental event!

We got friendly with ebay and very slowly bought the basics. We scoured the internet for post places going out of business and bought wiring and other essentials. Sometimes we found those essentials in a large cardboard box that they had put in a dumpster.

It is so very important to say that we were not rich, nor did we have any bank loans or investors. We survived by paying ourselves next to nothing, and putting the money back into our company. Some of us slept in the office to save on rent.

Once in a while, DigitalFilm Tree founder Ramy Katrib's Mom or brother would drop by with a basket of food that would last us a few weeks. It was about as home-spun an operation as one can imagine.

We definitely never overstepped our bounds. We never purchased a piece of equipment until we had the job firmly in hand, often with some of the money paid up front, so that we could run out and buy the equipment to help us do the job. We were very open and transparent with our clients about this.

As you consider getting started in your own business, and then prepare to grow, watch and control your payroll. That is key. By the time health insurance and all the Federal and State requirements are satisfied, payroll expenses become massive. Measure the increase in business that you may gain from adding another member to the team, against how much it will cost to pay them.

We do not have any specific secrets to share beyond these, Our view is that, in times like these, you just have to try harder. It takes more effort to find and get the work. But it can be done. TV shows are still airing, movies are still being made, and commercials are still being played. All of it needs posting.

Even if everybody needs to take a pay cut, now is not the time to stop investing in your future. While we certainly wish everyone success, we have seen in the past that failure in one place can lead to opportunity in another.


WHAT COMES NEXT

I do not mean to suggest that big investments are the only ingredient for big success. We have also built our business on embracing disruptive, low-cost technologies.

We see the emergence of these technologies going through the five phases of grief: denial, anger, argument, depression and acceptance.

First, denial. "This can't be happening!" Anger comes next: "This is hurting my business and my way of life."

Then comes the argument phase, where people turn academic on you. "Well, I offer THIS, and you only offer THAT."

Next, depression, often accompanied by frustration and bitterness. And finally, for the survivors, comes acceptance.

For example, even though we have been working successfully with it in mainstream, large-scale Hollywood productions since 2002, we think that FCP is finally arriving at the acceptance stage.

Over and over again, we have see this cycle repeat. What is interesting is just how Darwinian the marketplace can be. If it works for people, they will go for it. No amount of Jesuit distinctions or "‘Good enough' is not good enough" logic can stop the marketplace from adopting what works.

The first step is to watch for coming changes. Instead of fighting to hold them off, embrace them quickly. After you have settled into the mindset of adaptability, the next step is to wisely invest in the hardware that will ensure your differentiation from the cheapest options, and ultimately, your success.

One thing is for sure: the future of post technology has never been brighter. There have been hard times before, and there will be hard times again. Business opportunity abounds for those who look forward.


Zed Saeed Zed Saeed
Hollywood, California USA


Zed is the Senior Post Production Consultant with DigitalFilm Tree. The workflows that Zed and DFT designed for "Cold Mountain" were covered in the book, "Behind the Seen." They also worked on "Weeds," "Crank," "Full Frontal," and "Sky Captain and the World of Tomorrow," among many others, and authored the books "Advanced Color Correction and Effects," and "Advanced Editing and Finishing Techniques" in the Apple Pro Training Series.



Find more great Creative COW Magazine articles by signing up for the complimentary Creative COW Magazine.







Comments

Brilliant and useful
by Larry S. Evans II
As someone whose "dayjob" is in systems integration, I applaud your technical staff for designing something based on what it's required to do, and with the obvious input of those who need to use it.

We see all too often companies that buy off on "shiny" and "new" that gets poorly integrated (if at all). This rarely works and is quickly discarded for the next shiny new thing that will make everything perfect forever.

Throwing money and people at a problem is never as good (and almost always more expensive) an answer as figuring out how to do it right. Bravo to your staff for not only adopting it, but for passing on that wisdom to those who would otherwise have to find it through the long and painful path of error.
Tape is dead..long live tape
by Mike Cohen
I think I have used that subject line on a half dozen forum posts recently, but it is worth repeating, as Zed has elucidated nicely.

Lots of money has been spent by the major camera manufacturers touting the awesomeness of tapeless workflow. It is cool, within limits of course. But you still need to back stuff up, and if using hard drives for backup, you need a backup of the backup.

While I'd love to be shooting XDCAM EX, recording 10 hours per camera x 3 cameras per day without time to offload files would take a lot of non-tape media.

I know, prices are dropping.

Mike Cohen

As ever, you are indeed The Man, Zed
by Ron Lindeboom
A great article, Zed. As ever, well worth the read.


Related Articles / Tutorials:
Business & Marketing
Seven Ways to Make Your Own Luck in The Film Industry

Seven Ways to Make Your Own Luck in The Film Industry

HBO Director of Workflow, post-house founder, owner of the first two RED cameras, founder and developer of Endcrawl, technologist, futurist, educator, and more: John 'Pliny' Eremic, is regularly asked for career advice in the field of filmmaking. Step One, he says, is to consider a new job. There's much more of course, delivered with Pliny's peerless wit, directness, and insight. Whether you're just starting in the business, or looking to break through to the next level, you won't want to miss this guide to making your own luck.

Feature
John 'Pliny' Eremic
Business & Marketing
Creative COW Turns 15! A Celebration of Being Uncool

Creative COW Turns 15! A Celebration of Being Uncool

15 years is a long time on the internet! Travel back with us to the days before YouTube, social media, digital cinema, smartphones, iPods and all the rest, back to the founding of Creative COW by Ronald & Kathlyn Lindeboom in April 2001. Join us for an insider's look at the earliest days of the professional digital video revolution, all the way through the events that have shaped the world's largest community of media professionals, right here at Creative COW.

Editorial, Feature
Tim Wilson
Business & Marketing
7 Customer Service Rules for Better Post Production

7 Customer Service Rules for Better Post Production

Dealing with people is industry-agnostic, but in an industry as competitive as ours, repeat business is everything -- and your clients are more likely to return to you if your customer service skills are great. Here are some customer service tips that are especially helpful for post production.

Feature
Kylee Peña
Business & Marketing
An Odd Delight: A Corporate Editor's Leap Into Broadcast

An Odd Delight: A Corporate Editor's Leap Into Broadcast

Creative COW Contributing Editor Kylee Wall moved from Indianapolis to Atlanta as part of a move from corporate video into broadcast. Sure, some things stayed the same, but so much more was so different -- a new place to live, a new kind of workspace, new kinds of projects, AND TAPE -- that transitioning to a new NLE in Adobe Premiere Creative Cloud was the least of it. Certainly nothing compared to a fever of 104 that took her out for most of her first week. It's a remarkable tale that Kylee tells as only she can.

Editorial, Feature, People / Interview
Kylee Peña
Business & Marketing
Women In Post Join Forces

Women In Post Join Forces

Women in Post is a new HPA (Hollywood Post Alliance) committee formed by and for a decidedly minority group in the world of film/TV high technology. After three meetings -- two of them successful round table discussions, featuring accomplished women in the industry -- the group is expanding its plans to offer networking, mentoring and camaraderie and more.

Editorial, Feature, People / Interview
Debra Kaufman
Business & Marketing
Suck It Up, Buttercup

Suck It Up, Buttercup

The Best Excuses from New or Underemployed Filmmakers and Freelancers: "You've really really got to stop being lazy and making excuses for not getting what you want," says Creative COW Contributing Editor Kylee Wall. "Seriously. It's sad and it makes me sad for you. So sad in fact, that I've created this BEST OF compilation of stupid excuses. It's perhaps a little more brash than my usual fare. Don't mistake this for arrogance. I'm young and stupid too, but I'm still allowed to almost rant. Pseudo-rant. Pretend I made you cookies and you're eating them as you read this."

Editorial, Feature
Kylee Peña
Business & Marketing
Get Hired! Be Professional and Pay Attention to Detail

Get Hired! Be Professional and Pay Attention to Detail

"As technology gets better, communications skills seem to get worse." So says Creative COW leader and Contributing Editor Walter Biscardi, one of the industry's most respected business owners. "The same talented people who can create amazing things on screen have absolutely no idea how to represent themselves via a resume or online demo. Most of what I'm about to say seems to be common sense, but apparently it isn't." You definitely don't want to miss this potentially career-changing advice from an industry luminary!

Editorial, Feature
Walter Biscardi
Business & Marketing
The Heroism of Joyful Creativity

The Heroism of Joyful Creativity

I'm constantly inspired by the pleasure and the pride that the people in Creative COW Magazine take in doing the right things the right way, in always trying to improve, and always keeping their eyes peeled for new possibilities. I aspire to do my own work as creatively and joyfully as they do.

Editorial
Tim Wilson
Business & Marketing
Promote Your Company Worldwide, FREE.

Promote Your Company Worldwide, FREE.

Creative COW's Services Offered Directory is a Powerful Free Resource for Your Business.

Editorial
Ronald Lindeboom
Business & Marketing
The Back Forty: The Real Truth Of How Creative Cow Magazine Did Not Win An Award

The Back Forty: The Real Truth Of How Creative Cow Magazine Did Not Win An Award

Tim Wilson takes over the Back Forty for this issue to say, "Allow me to interrupt our regularly scheduled Back Forty to tell you what happened when our publisher Ronald Lindeboom called me one morning in April, saying, 'You're not going to believe this, Creative COW Magazine finally won an award!'"

Editorial
Tim Wilson
MORE
© 2016 CreativeCOW.net All Rights Reserved
[TOP]