In this expanded version of the article from Creative COW Magazine, VFX Supervisor and 2nd Unit Director Rob Legato shares some of the secrets of a tribute to the love of making and watching movies, and insights into two filmmaking masters. All of us at Creative COW congratulate Rob on his Academy Award nomination for his visual effects work on Hugo.
Hugo had perhaps a greater mission to fulfill than many VFX-themed movies because it was celebrating the life and work of the very first visual effects artist, Georges Méliès.
He was also the first multi-talented auteur who wrote the movie, painted the sets, acted, and was his own editor and VFX supervisor. He did everything. When you study the work, you see what a genius and forward thinker he was, all the way back to his first films in 1896. There was no such thing as movie trickery before him.
The first meeting I had with with Hugo Director Martin Scorsese, we talked about the scene where Hugo fixes a mechanical toy mouse that he presents to Méliès, having made it work better than originally designed. Marty said, "What if we did this stop motion?" My response was, "Well, it'll look like stop motion. We don't need to do it that way unless you want it to specifically look that way." Then he said, "That's exactly what I want it to look like."
Director Martin Scorsese with Ben Kingsley.
Asa Butterfield as Hugo with Ben Kingsley as Georges Méliès, with the mechanical mouse in Méliès' toy shop
in the train station.
Now all bets were off. It immediately raised the question: suppose you throw something into the middle of an action picture that's pretty obviously not state of the art? If you do it once, it looks like a mistake. But if you do it throughout the movie, it emphasizes and embraces the state of the art of cinema at the turn of the last century, and now it works on another level.
When the Station Master is snagged by the moving train, the train doesn't actually move. I moved the platform he was standing on, which also held the camera, so it appears that the train is moving. This is an old technique, and when done well, it is still totally invisible to the audience. We amplified the gag by actually photographing the ground he's on. Usually, you cut this kind of effect off at the knees, but in this case, we went the extra mile and built 60 feet of photographic floor on dolly track. I loved doing this because it felt more like an optical magic trick dating back to Méliès' time than a modern visual effect.
Even on the set, it was really magic. Your eye tells you one thing, and your brain tells you something else. The whole movie becomes a continuous homage to tricks. It was a unifying theme for everyone on the movie, to do everything we could in-camera.
Once you incorporate 3D as a storytelling tool, the one thing that becomes clear is that you have to see it as you're shooting.
If you were producing a color film but shooting in black and white, adding color later, color wouldn't be artistically or emotionally considered at every step of the way. It wouldn't be a building block to incorporate and fold in as the movie progresses. It is the same with 3D design: you need to artistically consider how depth affects the emotional portion of the scene while you're shooting it.
The Station Master played by Sacha Baron Cohen
Here's an example of an artful discovery that can only happen while you're shooting: the Station Master, played by Sacha Baron Cohen, starts leaning in, and the intraocular puller has to chase him. At a certain point as he leans forward, he extrudes through the screen. You're looking at that as a cinematographer, and thinking that the action really works much better in depth. The ever-changing proximity to the screen and the other person is what makes it work, and you use that as a storytelling device.
You can even use convergence as a tool where the shot or the actor is locked off, but the person seems to come closer and closer to you in depth, past the proscenium and into the audience, without ever moving. If we do it over a long enough number of frames, the audience doesn't know it's being done, but they have a psychological response to it. It's like a new sort of dolly move.
Part of how that works is the daily learning process that everyone experiences from Day One. Everybody gets to sit in dailies and see what works in 3D and what doesn't. Because of that, every new set, every new sequence and every new design incorporates what 3D can give to you in a very subtle but palpable way.
Martin Scorsese's cameo in Hugo
This is quite different from shooting a 2D movie and then converting it to 3D. Not that that process itself isn't a good one, but it loses the day-to-day artistic accumulation that performances build on. It's a little bit like what I did for Avatar creating the virtual camera. [Ed. note: the virtual camera feeds previz environments into the camera, so that actors and camera moves are seen in the same context that they will be seen in the final, composited shot.] So many decisions are made when you look through the camera.
The previous way of doing motion capture is to stage a play, and add the camera much later. Now that I've added a camera while I'm staging, I can see that I should have blocked it much differently. You can't easily post-emphasize the drama by upstaging and downstaging the actors to the camera if it wasn't originally captured that way. Your artistic opportunities are fewer.
We tried everything I could think of to keep the theme of in-camera trickery alive.
One of the major sequences in Hugo was inspired by a train crashing through the Montparnasse station in Paris in 1895. Rob and the team at New Deal carefully emulated the physics of that crash to very effective result.
In the scene where the train flies off the track and lands in a heap, we exactly recreated the physics behind the famous Gare Montparnasse 1895 derailment photograph. To achieve that, we shot miniatures, which were done for us by New Deal Studios. I had worked with them on Aviator and Shutter Island, and I shoot a lot of post production second units with them. They provide a full service, including building sets for me to shoot live non-VFX inserts.
The 1895 train wreck at Montparnasse
For the movie's digital shots, Pixomondo did the heavy lifting, using all of their facilities around the world. Matte World did five or six matte painting shots and artistic supervision. Lola VFX did thirty-five or forty shots focused on de-aging, and did a beautiful job. Uncharted Territory created about twenty or so exterior Paris environments. And ILM did the opening shot where it looks like streak photography, and another scene where Hugo points out that the world is a machine.
Generally speaking, though, I'm loath to farm a film's visual effects out to too many different companies. At that point, it becomes something for a producer to manage, rather than an artistic solution for a visual effects supervisor to manage. The way Marty works, it's also hard to do. If I give eight shots to one company, and those shots are dropped in favor of six others being done elsewhere, that first company still has to charge me for the work they've done that's not actually being used in the film.
Pixomondo did a beautiful job under hard circumstances. We did Hugo for $16 million, which is unbelievable. If this were structured in the normal VFX model -- turning in sequences by certain hard dates, and not changing those dates without a traditional change order -- it would have cost $25 million, easy.
A Scorsese picture doesn't operate that way. The film is continually improved and honed, and is a very free-flowing organic process. The normal model is too restrictive and creatively punitive for this type of work to flourish.
To begin, we didn't generally storyboard sequences. I did previs to see how we would shoot them, and then used it as a temp while we edited. It was at best a loose concept and a placeholder for a better idea. Remarkably, it all came out equitably in the end, without the extra stifling drama during the process.
I cautioned Pixomondo before we began that this is simply the way it was going to have to be, continually robbing Peter to pay Paul, swapping dropped shots for new ones, so that we were able to do that caliber of work for $16 million with little or no waste. The work is as good as anything I've ever done before.
I'm a fan of this way of working: to figure out what the movie is, come up with the amount it can cost (even if the exactness is always in flux), and that's your playpen. The accounting process can't become the tail that wags the dog relative to the creative process.
Before and after shots featuring Asa Butterfield as Hugo Cabret. Please click on individual images above for larger view.
I am a strong believer in not stopping to color time while you're shooting. If you spend ten minutes a setup on color correcting, that adds up to a couple hours a day, which is three or four more shots or takes that you can't have. You're only shooting part of a scene, and you're diminishing your creative output by adding another function that's not that well-informed -- and you don't get that day back.
We lit the scene, and used the light meter, and looked at the monitor, but we weren't going into a blackened tent with calibrated monitor to do that. We used only one standard Log to Lin LUT, used a LCD 3D monitor to view it on the set, and eliminated the rest. Instead, we created a live DI throughout shooting and the post process that we could change at will.
Rob Legato. Hugo was shot with ARRI Alexa cameras in Cameron Pace 3D rigs.
We also kept everything as two discrete streams -- right eye and left eye -- so that we'd have flexibility to change the convergence and adjust stereo after the fact. We created a pretty simple workflow to support that: we shot everything to HDCAM SR tape, which then went over to the lab that processed the material to get it color corrected and into the proper file format for editorial.
That process was identical to shooting on film and creating HD dailies. I conformed the movie with an EDL, recapturing the material and building a conform in the FilmLight Baselight. It was sort of like a tape-based workflow that became tapeless: editorial operated exactly the same, with the addition of another EDL track for the other eye.
The pipeline was 3D all the way through. Editor Thelma Schoonmaker would make a cut in 2D, then switch the same monitor over to 3D on her Lightworks station, and make any adjustments needed based on seeing it in depth. She would eventually screen scenes on the projector to see how the big screen alters the perception of the cut and the pace. It was done that way throughout.
800 VFX SHOTS TIMES TWO
Working on one of the train station's miniatures.
Below, Paris from the opening sequence of Hugo.
One other part of workflow: when I recaptured VFX material (which I did in the basement of my house), I'd send raw DPX frames of the right and left eye to Pixomondo, and they'd send it back to me comped. I'd take the same non-color-corrected material, go into the Baselight and find the day the plate was color corrected. The identical power windows and other color corrections then got applied to the material. After that, the edits didn't bump because they were identical to the rest of the dailies.
We had 800 shots to cut in, so 1,600 unique pieces of media for the two eyes, and they all had to be organized perfectly. Every time I got a new shot or version back, I'd have them stamp the DPX files with new, unique tape names and timecode. The metadata includes auxiliary tape name and timecode that matches back to the original dailies, but the new, unique metadata guarantees that every version can be automatically found using an ordinary EDL. The file-based system includes the tape name in the folder structure and the timecode in the header information, so as long as the new shot was cut into the working copy of the film, the rest was automatic.
On Shutter Island, I struggled to find a better, faster way to color correct the visual effects. Previously, we would have to do it in a DI facility and pay a lot of money. I found that the easiest way to do it for Hugo was to color correct it myself, day or night, and send it to New York City using the Aspera high-speed data transfer system. Within 20 minutes of getting the shot, editorial would have the newest iteration, properly labeled with all the metadata, and an identical color correction to the original dailies.
Tom Overton at Laser Pacific (which is now Technicolor) networked the machines so that I could go directly from my Avid Media Composer and their Baselight to screen VFX shots in context the moment they came in. I was then able to look at them under the same stack of the newly revised color corrections. I'd hit play, immediately see our shots in the edit, and without transcoding, instantly see how they needed to be addressed for the finals.
When you have a huge time crunch, all of those saved minutes get translated into doing a better job, instead of wasting your time with transcoding plumbing.
HOW EDITS INFORMED THE ONGOING SHOOT
I shot about 60 days of second unit as a director/cameraman concurrent with production.
When we left the stage, I'd go shoot at New Deal Studios for inserts. Because it was digital, I'd shoot it there, take the tapes back that evening, bring them into the Baselight, and cut the footage into the scenes. Then I'd reapply and modify color correction, and send dailies out to New York editorial.
The more you do this, the more it informs and improves other areas of the production. Most cinematographers just see the results, not the steps along the way that the lab does. The lab may have fixed the footage for you, masking your mistakes, and along with them, the knowledge of how to improve your work. But if I photographed it, I'd see what the color correction actually did, which would inform me how to shoot the matching footage much better.
Above, Rob Legato directing on the set of Hugo.
I used that model to become more proficient with green screen. Most people who shoot it don't see all the work it takes to get it to where it needs to go. If you shoot and go directly to comp the shot, only then do you begin realize what you should have done better. Next time you shoot, you now know the ramifications of your choices. It's the same thing with color correction: I see the ways I need to fix the shooting process to make the color turn out better.
The more extreme the post fixes are, the lesser quality the final result generally becomes. The more you shoot as you wish to ultimately see it -- and do only modest corrections -- the more it yields the highest quality work. One of the greatest things with Hugo is that we had the opportunity to do that. We'd shoot it, and see how it translated in dailies, sometimes in the same day.
Because I was photographing my unit, I also had to try to have the same intent and understanding of what Cinematographer Bob Richardson designed for the scene. I had to find a way to emulate what Bob might have done with the same shot. It was a tall order, as Bob is such a master cameraman. Being able to see under the hood, so to speak, gave me a fighting chance to match his incredible work. If I only got in the same ballpark, I was more than delighted.
THE JOY OF FILMMAKING
Working on Hugo was satisfying in so many ways. Everyone who worked on it loved this wonderful, unifying theme and message of the movie. It became so satisfying to be able to participate on such a high order with the other incredible collaborators under the direction of the grand master himself, Martin Scorsese. The subject matter alone felt like it was well worthwhile to put your very best effort into.
Left, Georges Méliès in his train station confectionery and toy shop, and right, the set reproduction in Hugo.
Above, the Méliès studio.
Below, Georges directing Le Barbe-Bleu (1901)
What's heartening is that we put the work of Georges Méliès back on screen. People who have never seen his films, or even heard of him before, are fascinated by his creative genius, every bit as much as those first audiences were.
In some cases, we didn't alter the images from Méliès' films at all; we just put them on screen, exactly as he created them. Some of his films we presented in 3D, expecting that it was something he would have done himself given the opportunity. The Lumiere Brothers experimented with 3D at the time, and you know Méliès would have loved it and used it if he could have.
It was also a huge thrill to recreate the studio where he actually created those fantastical moments. There's a kind of amazing sensation you get -- and the whole crew got -- in that glass house, recreating his studio as faithfully as we did. I had the same sort of feeling with Titanic, getting the same exact point of view as someone who was actually on the ship.
We're seeing exactly what Méliès saw in 1896. There was a tremendous sense of awe that you were on sacred ground, the birthplace of motion pictures. It was thrilling. There was magic, pixie dust, in the making of the movie that hopefully gets seen and felt when you watch it.
The entire crew's own joy of filmmaking is part of the fabric of the film. We're all so inspired by this man who adored what he was doing. His work becomes relevant again, and can still entertain us as it did more than 100 years ago.
The original Le Royaume des Fées here was beautifully recreated for Hugo.
In approximately 550 films that Georges Méliès made between 1896 and 1913, he pioneered a wide variety of visual effects and storytelling techniques that continue today.
For example, he was the first to use split screens for actors to act opposite themselves, and one of the first to use multiple exposures and timelapse photography, dissolves, and color film via hand-tinting prints. Business failures drove him from making films to selling candy and toys at the Montparnasse train station in Paris, which is where we meet him in the film Hugo.
As described in the movie, many of his film prints were confiscated by the French army to be melted for war material in the First World War, primarily boot heels -- especially ironic since he was once the manager of the highly successful boot factory that his parents had founded.
Others of his films were lost to the processes of decay typical for many celluloid films up through the 1950s. Many of his approximately 200 films that survive did so because they are preserved as paper prints in the US Library of Congress.
For clips from some of the films of Georges Méliès, see below.
At top, Méliès is seen starring in 1902's L'homme a la Tete de Caoutchouc ("The Man with the India Rubber Head"). Also from 1902: the iconic Voyage dans la Lune ("Voyage to the Moon"), which was originally released both in black and white and hand-tinted color. Le Royaume des Fees ("Fairyland: Kindgom of the Fairies"), one of 25 movies he made in 1903, is considered one of his materpieces.
More from Méliès
In addition to writing, directing, starring in, creating effects for, and building the sets for his films, Méliès also handled art direction, starting with production drawings such as the one seen here.
L'homme Orchestre (One Man Band), 1900.
Clips from Méliès' hand-tinted print of Voyage dans la Lune (Voyage to the Moon), 1902.
L'homme a la tête en caoutchouc (The Man with the Rubber Head), 1901.
Sure, some of the biggest movies are still being shot on film, but in a world where film cameras haven't been in production for years, where only one company still makes motion picture film, and by the end of the year, there may be no more film prints for theatrical distribution, it's no wonder that the number of businesses who can sustain themselves by processing film has plummeted. In this latest dispatch from the film BUSINESS, Creative COW Contributing Editor Debra Kaufman finds industry leaders asking themselves, will 2014 be the year we see the last film lab?
After collaborating on the 1985 pilot for the highly successful "Power Rangers" franchise, DP and Digital Cinema Society President/Co-Founder James Mathers recently worked on a new pilot with the same producers. This offered him the perfect opportunity to put several new products through their paces on his RED Epic-based 4K production: including greenscreen virtual sets and on-set VFX previz tools, a wide variety of lighting options, the Aja Ki Pro Quad, and much more.
What were the big technology trends in media and entertainment over the past year? What's going to be significant in the coming year? Those are questions that many of us are asking, so we went to some of our savviest Creative COW contributors to ask their opinions of where we've been and where we're going. Here in Part 1, we offer an overview of their perspectives. (See Part 2 for their additional insights.)
What were the big technology trends in media and entertainment over the past year? What's going to be significant in the coming year? Those are questions that many of us are asking, so we went to some of our savviest Creative COW contributors to ask their opinions of where we've been and where we're going. Here in Part 2, our conrtibutors offer in-depth insights. (See Part 1 for a broader industry overview.)
The digital revolution was supposed to save us all from endless streams of paper but instead has spawned an even deeper addiction to the printed page. Here's how one filmmaker decided to take the road-less-cluttered and literally enter the ring of paperless production for Lionsgate and World Wrestling Entertainment.
Award-winning cinematographer David Franco sat down with Ben Consoli, host of the Go Creative Show for a revealing interview that covers his entire career and focuses on his breathtaking visual direction for the HBO hit show "Boardwalk Empire".
The UP series, directed by Michael Apted, began in 1964 with Seven UP, which interviewed 14 children from diverse backgrounds from all over England. Every seven years, Apted has returned to these same children to ask about the progression of their lives. The result, the UP Series, is according to critic Roger Ebert, "an inspired, almost noble use of the film medium. Apted penetrates to the central mystery of life." Cinematographer George Jesse Turner has shot all the UP Series films since 21 UP in 1977.
In August 2008, eleven climbers died on K2, the Himalayan peak nicknamed the Savage Mountain. In The Summit, filmmaker Nick Ryan delves into the mystery of what actually happened that day, in a masterful collage of footage taken by the climbers, stunning aerial shots of K2, interviews and re-enactments. Take a look behind the lens with Nick Ryan as he relates how he respectfully re-captured those tragic moments.