LIBRARY: Tutorials Reviews Interviews Editorials Features Business Authors RSS Feed

Procedural Matte Creation

Procedural Matte Creation

by Barend Onneweer, Gouda/Rotterdam, The Netherlands
©2001 by Barend Onneweer. All rights are reserved. Used at CreativeCow.net by kind permission of the author.


Barend Onneweer

ARTICLE FOCUS:
Barend Onneweer explains Procedural matte creation by combining multiple keying types into a single matte for extensive control over the final matte. This is essential for pulling those difficult mattes from badly lit greenscreens.



INTRODUCTION

An always recurring problem in digital video compositing is the creation of a matte from blue or green screen footage. Most of these problems are due to lack of experience in this field (it definitely is not as easy as it would seem, I learned the hard way…). Also, the nature of DV as a compressed video tends to be problematic for this kind of work. The DV compression produces tiny artefacts in the image, especially around the edges, which is exactly where the problems will arise when trying to pull a decent matte from the footage.

Most of the quality of the final matte and composite is actually established during the shoot. There are a couple of basic guidelines to consider when shooting the footage that will make life a lot easier in post.

That said, this article focuses on the approach after the harm has been done. This approach is actually widely used at high-end composite studios but it's essential to pulling a 'difficult matte'.

The following is a step-by-step walkthrough for what's called procedural matte creation. The examples were made using Adobe After Effects 4.1 Production Bundle. The same approach could also be used using other compositing software of course. I've tried to use only the standard plug-ins that come with the Production Bundle, the results might even be a little bit better using 3rd party keying filters like Ultimatte or Pinnacle's Primatte.

Usually you're not going to be able to produce a decent matte from simply applying one keying filter or effect to one piece of footage. Basically what we're going to do is using different versions of the same clip and different type keying filters to produce the final matte. I have de-interlaced my footage and rendered it out as full frames before I imported them back into this project. I find it easier to create mattes from de-interlaced footage then from interlaced footage.

STEP 1: Setting up

After importing the piece of footage (which I called Original.tif for this tutorial) into an After Effects project, I create two compositions, make sure all the composition settings are identical for these. The first one is named Final and the other is called Matte.

I drag the footage into the Final comp. Then I copy this layer from the Final comp into the Matte comp. This is important, because in other situations where you have a more complex comp, you have to make sure the copy of the clip in the Matte comp is placed at the same in- and out-points, so that the matte remains synchronized to the original clip.

STEP 2: The matte

In the Matte comp, I'm going to create several mattes that will be combined to create the complete matte. This comp has produce a high contrast black and white image to be used as the matte source for the original clip. I put a black solid at the bottom of the comp, because of possible interference of alpha channels.

The basic approach uses a Core Matte and an Edge Matte. The Edge Matte will look nice around the edges, a lot of semi-transparent tones and detail. I often use the After Effects' Color Difference Key filter for this, it's in the Production Bundle. I like the detailed edges this filter produces and it doesn't show a lot of the DV artefacts.

I select the layer of footage in the Matte Comp, and add the Color Difference Key.

It's important to set the view option in the Effect Control window to [matte corrected]. This way the effect produces a black and white matte.

As you can see the Edge Matte has some nice features around the edges, especially along the left side of the girl and the barrel of the gun. This happens a lot with edge mattes, that only one side has a really sharp edge.

CLICK ON THE IMAGES TO SEE IT AT THE ORIGINAL SIZE
EDGE MATTE
CORE MATTE
COMBINED MATTE
EDGE MATTE (detail)
CORE MATTE (detail)
COMBINED MATTE (detail)

So another Matte is needed to fill the holes left in the Edge Matte. This is accomplished by duplicating the layer we have been working on in the Matte comp, removing the effects in the duplicate and setting the transfer mode of the top layer to [screen], so that the results of both layers are combined. It's best to turn off the visibility of the Edge Matte while working on this Core Matte.

I produced the Core Matte using the Color Range keyer, found in the keying filters menu. This filter keys out a range of colors or shades of the same color, the way you specify them. You start by clicking the color picker in the small preview window in the Effects Control window. With the add-color picker (with the little +) you can add points in the preview window until the whole screen is keyed out, but the foreground is still intact. I had the fuzziness set at 20 for this effect.

As I pointed out earlier, the Matte Comp is supposed to be black and white, so I'll add the Levels filter to this layer, found in the Adjust filters. In this effect I set the Output Black to 255, effectively making this whole layer white. Now there's a white silhouette in this layer, but the edges are quite rough as you can see in the details below. So that is why I'll add a last effect to the Core Matte, the Matte Choker, found in the Matte Tools in the effect list. This effect basically smoothes the outlines of the matte and gives the option to shrink and soften it.

THE DIFFERENT STAGES OF THE CORE MATTE
COLOR RANGE KEYER
ADJUST LEVELS
MATTE CHOKER

The Matte Comp would now look something like this:

Creating the matte this way gives a lot of control over the combined matte. Sometimes it is impossible to get a satisfactory matte by just using keying filters. I have on occasion animated small circles on top of these layers to fill certain holes that I couldn't fix with the Core Matte.

Another possibility is the addition of what is called a Garbage Matte. If for instance the camera is static, and there are static objects in the foreground, you could create a black and white image in Photoshop to mask out these objects. This would give you more room to fiddle around with the other layers, since you wouldn't have to worry about those objects.

WARNING: especially with choked mattes, 'fluttering' of the edges can occur, so it's wise to test-render or RAM-preview some sequences instead of only relying on the still.

If you want to see my effect settings for these two layers, click here.

STEP 3: Applying the matte

Now that we've finished the Matte, we'll have it applied to the original footage. I drag the Matte comp from the Project Window into the Final Comp. Again make sure that it starts at frame 0, so that it is in sync with the original footage. The visibility of this layer can be turned off, it will only serve as a matte layer in this comp.

Select the Original footage in the Final Comp, and add the effect Set Matte, found under the Channel effects. This is where I apply the Matte comp as a source for the transparency of the Original footage.

The important settings here are the two roll-down menu's. The upper one defines which layer is set as a source for the matte, the lower defines from which parameter the transparency is derived.

Now the Final comp should look something like this:

As you can see I have added a background layer, to check the matte. If there is still a serious aura of green (or blue, depending on the colour of the screen) surrounding your objects, you can go back and choke the Core Matte a little more, or adjust the settings of the Edge Matte. There are also some other tools available to tweak the edges of a matte, which I'll discuss in the last part of this article.

Step 4: Fine-tuning the final image

To make this effect convincing there are a couple of important things to consider. First of all, the colours and brightness and contrast of the foreground have to match the background. You can adjust these by applying Levels or Color Balance effects to the background or foreground, or both.

Then there are some possibilities of removing what's called green or blue 'spill', the tiny green or blue edges around the foreground. The most basic way to remove these is After Effects own Spill Suppressor, found in the Keying effects. This filter will let you choose a color, which it will then remove from the layer it's applied to. You tend to lose a lot of color in the whole layer though, while you actually only want to remove the spill from the edges.

There are some 3rd party filters that succesfully blend in the colors of a specified background layer around the edges of the foreground layer. This way it looks as if some of the color or light from the background reflects around the edges of the forground. I use the Light Wrap filter for this, which is included in Puffin Designs Composite Wizard, but there are others. These filters also succesfully mask any residual screen that comes through, like it can be seen above in the lower left corner of my Combined Matte.

CLICK ON THE IMAGE TO OPEN THE ORIGINAL SIZE IMAGE
ORIGINAL FOOTAGE
BACKGROUND FOOTAGE
FINAL COMPOSITE

 

The images are taken from my (at the moment almost finished) graduation project, and ultra low-budget science-fiction short, completely shot on DV and edited on off-the-shelf desktop equipment. Check it out at www.raamw3rk.net.

(c) 2001 Barend Onneweer

 

Barend Onneweer is a leader in the Adobe After Effects Creative COW. Drop by and discuss this or other effects. Like to see who Barend is? Click here.

Barend has a new article in the library called 'Shooting for Chromakeying'. In it he has put together a list of things to consider when shooting for chromakeying. Would you like to see that article? Click here.

The Adobe After Effects Creative COW is a part of CreativeCOW.net -- online Creative Communities of the World. This tutorial is for the use of Creative COW members and visitors and may not be reproduced without permission by the author and CreativeCOW.net.

 


 

Visit Creative Cow's website and forums if you got here by direct link to this article...



 

Comments

...edge matte!
by Anders Hattne
Darn it!
After two days of fixing dirty green screen footage, and even clearing a green clad man from a green background, I read this which makes me think I should have re-done all the keys!!

I'll give the edge and core matte technique a try so I remember it for next time!!
Thanks



Related Articles / Tutorials:
Adobe After Effects
Compositing Secrets Everyone Can Use: Pt. 3 - Alpha Channels

Compositing Secrets Everyone Can Use: Pt. 3 - Alpha Channels

When most people hear the words "alpha channels", they think "transparency", but that's not exactly accurate. The truth is more complex, and a quite bit more interesting! Join longtime VFX artist, editor, software developer, and business owner Simon Ubsdell for Part 3 of the best look behind the technology of compositing that you've ever seen, packed with practical advice for applying the secrets of alpha channels that's simply not possible before understanding these underlying principles. No matter which applications you're using for editing, compositing, or visual effects, this one is a must-see!

Tutorial
Simon Ubsdell
Adobe After Effects
Compositing Secrets Everyone Can Use 2: Advanced Blend Modes

Compositing Secrets Everyone Can Use 2: Advanced Blend Modes

Whether you're a full-time compositor and VFX artist, an editor working in one of the many NLEs that supports Blend Modes (including Adobe Premiere, Apple FCP and FCPX, Avid Media Composer, DaVinci Rsolve, and VEGAS Pro), or whether you just want the additional graphics power that comes with insight into the secret world of pixels, this is the tutorial for you! Join longtime VFX artist, editor, software developer, and business owner Simon Ubsdell for Part 2 of the best look behind the technology of blend modes that you've ever seen, with some practical steps you can start taking today to make your work look better than ever.

Tutorial
Simon Ubsdell
Adobe After Effects
Compositing Secrets Everyone Can Use: Pt. 1 - Blend Modes

Compositing Secrets Everyone Can Use: Pt. 1 - Blend Modes

Join longtime VFX artist, editor, software developer, and business owner Simon Ubsdell for the best look behind the technology of blend modes that you've ever seen. This isn't just for graphics and VFX, but for video editors too -- anyone who puts anything together, and wants to learn more about HOW images combine at the most basic level, in a way that applies to every application you might use, whether Adobe Premiere Pro, Apple FCPX, Avid Media Composer, DaVinci Resolve, VEGAS Pro, and yes, graphics and VFX programs like After Effects, Motion, Fusion, Nuke, Scratch, and many more.

Tutorial
Simon Ubsdell
Adobe After Effects
Disintegration VFX in Adobe After Effects

Disintegration VFX in Adobe After Effects

Join VFX guru Tobias Gleissenberger of Surfaced Studio for a classically energetic Adobe After Effects tutorial combining difference mattes, displacement maps, edge glows, and compositing tricks to dissolve (or explode!) one element in your frame while leaving the rest alone.

Tutorial
Tobias Gleissenberger
Adobe After Effects
Adobe After Effects: Common QT & Export Problems & Solutions

Adobe After Effects: Common QT & Export Problems & Solutions

Problems with Adobe After Effects? Can't import or export QuickTime videos? Exports too big? Missing codecs? AE guru and VFX whiz Tobias Gleissenberger of Surfaced Studio has the answers you're looking for to make every project come to completion more smoothly, including a comparison of exported file sizes for common codecs.

Tutorial
Tobias Gleissenberger
Adobe After Effects
Fancy Title Animations In Adobe After Effects -- FAST!

Fancy Title Animations In Adobe After Effects -- FAST!

Follow along as VFX guru Tobias Gleissenberger shows you how how to quickly create professional-looking animated text elements using the free presets that are included with Adobe After Effects, customizing them to suit your needs. Tobias will also show you how to apply masks to your text layers to have them appear from behind other elements.

Tutorial
Tobias Gleissenberger
Adobe After Effects
Remove People from Moving Video in After Effects with Mocha

Remove People from Moving Video in After Effects with Mocha

Learn how to remove a person from a moving video using the Remove module of the Mocha Pro plug-in for Adobe After Effects! In one of his most epic tutorial's yet, Surfaced Studio's Tobias Gleissenberger then adds a bonus Beam Up effect using some of the great filters from the Boris Continuum and Sapphire collections.

Tutorial
Tobias Gleissenberger
Adobe After Effects
Basic Clone Effect Interaction - After Effects Tutorial

Basic Clone Effect Interaction - After Effects Tutorial

Want to take your CLONE EFFECT to the next level? Want to know how to make your clones INTERACT? Tobias Gleissenberger shows you how.

Tutorial
Tobias Gleissenberger
Adobe After Effects
Adobe After Effects: Glitch Effects Without Using Plug-ins!

Adobe After Effects: Glitch Effects Without Using Plug-ins!

Join VFX guru Tobias Gleissenberger for a fun, high-energy look at how to create all sorts of digital noise, glitch, and other disturbing video effects with any version of Adobe After Effects, without using any plug-ins!

Tutorial
Tobias Gleissenberger
Adobe After Effects
Adobe After Effects Energy Ball

Adobe After Effects Energy Ball

In his latest high-energy Adobe After Effects tutorial, VFX guru Tobias Gleissenberger of Surfaced Studio combines a variety of effects to create the pulsating energy ball, composited with motion tracking, optical flares, and more.

Tutorial
Tobias Gleissenberger
MORE
© 2018 CreativeCOW.net All Rights Reserved
[TOP]