The Invisible Art: Matte Painting/Set Extension Tips
COW Library : Autodesk 3ds Max : Andrew Manzella : The Invisible Art: Matte Painting/Set Extension Tips
Matte painting is becoming more and more popular with advances in technology and the availability of software like Photoshop, 3ds Max, and After Effects (just to name a few). What was once called, 'The Invisible Art,' is now in the mainstream of the VFX world. Most often when people think of matte paintings they think of epic establishing shots of places that don't exist, or don't exist yet, or haven't existed in a very long time. The truth is, matte painting doesn't just live in grand shots of big budget movies anymore. In fact, 'The Invisible Art,' has found its place in productions big and small and for a variety of reasons.
Several years ago, I was part of a small team that utilized matte painting techniques on a local commercial promoting a summer food drive. While we didn't need a vast establishing shot for the commercial, we did need to show a large warehouse full of boxes containing food. We had no warehouse, so we had to rely on matte painting techniques (or digital set replacement) to create this world to convey the story we had hoped would resonate with viewers.
This project began with a small team including Rossana Sietter (Producer), Randy Fulp (Lighting Director), Josh King (Producer/Video Geek) and myself (Producer/Video Geek). Rossana had an idea for WSOC's annual Food Drive spot involving a boy pulling out cans of food that were stored in a warehouse and spelling out the words, 'I am Full of Hope.' We had several discussions about the spot and the idea was solid, but we had no warehouse to shoot in...and no budget to rent one. So, instead, we decided to shoot in WSOC's prop shed and use matte paintings to extend the scene.
We shot with a Panasonic 2700 Varicam at 29.97 frames a second and a 1920x1080 frame. We knew our final output would be smaller, but wanted to capture as much information as possible.
In order to be able to read the cans once they had been laid out on the floor, we needed a high vantage point to shoot from. The prop shed had a loft on one side. We set the camera and tripod on the loft, established our shot and locked the camera down.
Next, we staged the set with a handful of boxes full of cans. Most importantly, we placed boxes in the frame. These would later become a reference objects for us in 3ds Max in efforts to re-create the scene.
Prior to the shoot, we had discussed in detail the project with our lighting director, Randy Fulp. Randy lit the set in a way to mimic what a warehouse would look like. He gave style and accents, but kept the scene realistic. We also had wanted to give the illusion of a large skylight in the roof that is common in warehouses. Randy made some custom cookies to accomplish this effect.
With the set staged and lit, we broke out the notebook and tape measures. We measured and recorded everything we could think we might need. We recorded the focal length of the camera lens. We measured the distance from the camera to the center point in frame. We measured how high the camera was off the ground.
We also measured the dimensions of the different cardboard boxes that we would build in 3ds Max.
With all of the measurements recorded we started shooting.
After the shoot, we took a handful of the cardboard boxes, and with a Canon 5d took pictures of each side of the boxes. These would later be the textures we would add in Photoshop. We also took photos of the boxes full of cans.
Now that production was complete, we moved into the digital world. First, we exported a still from the footage to be used as a plate.
We brought that plate into 3ds Max and set it as the viewport background to use as a template. We also changed the aspect ratio of the render frame to 1920x1080 to match the footage.
When we were recording measurements, we also recorded how far the floor was from the camera. Referencing all of our measurements, we re-created the camera in 3ds Max. We used boxes as reference measurements to make sure the distance between the camera and the floor was accurate and that the height of the camera was accurate.
We created a box and made it the exact size of the real box and placed it at the correct distance from the cg camera. We rotated the box to match the placement of the real box and finessed the camera orientation and field of view until it matched.
In total, we needed two matte paintings. So, with the digital world set up and the cameras recreated, we moved to the next phase.
As I began cleaning the plates in Photoshop, Josh built the boxes in 3ds Max and populated the scene. All of the boxes were created with the measurements of the real boxes we had used earlier on the shoot. For variation, some boxes were closed and some open. In addition, Josh built a cart and handcart for interest and variety.
At this point we were ready to begin work on the final matte paintings. Instead of UV mapping and texturing in 3ds Max, we decided to render out the scene in grey. One reason we decided to work this way was because of speed. We didn't want to spend the time UV mapping. More importantly, we chose this method so that we could quickly add more variation into the models and avoid a sterile and cloned look. We cut a matte out of the plate and added the grey cg models into the scenes.
The first thing I like to do with projects like this is to place a Hue/Saturation adjustment layer over everything in the scene and set the later -100 Saturation, making the image grey scale only. I also make a layer that has a black evolution on it – it will have progressively darker shades of grey on it until it goes from white to black. Together, I use those elements to continually to make certain that my levels remain true to the original image. One of the easiest give-a-ways that an image is cg is that there is too much contrast. Generally, elements that are farther away from the camera have less contrast and less saturation. A Hue/Saturation adjustment layer set to -100 Saturation over the entire painting removes the distraction of color and let's you really focus on the levels.
It was time to begin painting and texturing. We used our library of cardboard boxes and cans that we shot to begin bringing the scene to life. Using the free transform tool we began laying the appropriate pieces over top of the grey models. Since we had all of the lighting information on the grey render, we switched the transfer mode of the texture layers to 'Overlay.' This let the cg lighting information bleed through the texture and match the scene.
This was the process we used to texture and paint the entire scene.
We also used a Curves adjustment layer over everything to paint in shadows. We pulled the curves down to match the original shadows and made a mask for the Curves adjustment layer and filled it with black. Then, using soft brushes, we painted in the shadows by painting white into the mask.
Here is the original plate with the 3d models added in.
And here is the final matte painting.
There was still some additional work to be done in After Effects. We needed some rotoscoping and grading. After review, there was a fear that the cans were too hard to read in the shot. For a solution, we decided to use particles to make a light trail fly through the scene and highlight the cans.
This was a great project to work on, especially since all of us are friends. Knowing and trusting each other so well made it really easy to hand off pieces and parts of the project while knowing that in the end it would all come together.
In 2012, this project was nominated for a Mid-South Emmy Award for Technical Achievement. I was skeptical of our chances (I don't think I was the only one) because, in theory, if we had done a good job with the matte paintings, then no one should have noticed them. Not even the judges. We were not allowed to show a before and after video. However, Josh did write up a detailed explanation of the project and had also included some still images that we were allowed to submit. Needless to say, we were all thrilled and extremely appreciative when the announcers called our project as the winner. It always feels good to have something you've worked hard on be recognized. Winning was great, but winning with a team made up of friends was the best part.