LIBRARY: Tutorials Reviews Interviews Editorials Features Business Authors RSS Feed

Turning Paint Into Masks in After Effects

Turning Paint Into Masks in After Effects
A Creative COW After Effects Tutorial




Timothy Kurkoski Timothy Kurkoski
Beaverton, Oregon, USA

©2003 Timothy Kurkoski and Creativecow.net. All rights reserved.

Article Focus:
Paint strokes and masks in After Effects are both forms of vector paths, but converting one into the other isn't always straightfoward. This tutorial shows how to take strokes from the Vector Paint effect, as well as AE 6's new Paint tools, and turn them into masks
.


Project File .zip Project File .sit

Paint strokes and masks are both versatile tools in After Effects, but sometimes you may find you've used one when you should have used the other. Fortunately, both features are based on vector paths, although going from the Vector Paint effect to a mask isn't the most straightforward procedure. In this tutorial we'll walk through that process, and we'll also show how much easier it is to do with AE6's new Paint tools. Finally, we'll look at an alternate method of getting a mask from your paint strokes by using AE6's Auto-Trace feature. It's a little rougher, but the potential time savings are enormous.

Note: The project file download includes both AE 5.5 and 6.0 version files. For the first part of the tutorial, you must have AE 5.5 Production Bundle or AE 6.0 Professional. For the second section, you need AE 6.0 Standard or Pro. For the third part, AE 6.0 Pro is required.


Part 1: Converting Vector Paint strokes into masks

Start by opening on the Paint_to_Mask.aep project file. Look at the Bessy Roto - Start comp. Here we have Ol' Bessy in front of her barn. We want to isolate just Bessy's head without the background or the text of the logo. I've started rotoscoping her out using the eraser brush in Vector Paint, but have very quickly realized that using a mask will be faster. Rather than start the mask from scratch, what we're going to do is take that paint stroke and convert it into a mask. Start by opening the effects for the Bessy layer and selecting the Vector Paint effect. The Vector Paint tools will appear on left hand side of the comp window.

  1. In the Vector Paint tools, choose the selection tool (the arrow).
  2. Select the paint stroke. It will become highlighted by showing a thin line inside of it.


  3. Copy the paint stroke by using the keyboard shortcut (CTRL+C on Windows, Command+C on Mac). It is important to use the keyboard shortcut here instead of using the Edit > Copy menu command. Using the menu command would copy the whole Vector Paint effect instead of just the paint stroke.
  4. Create an adjustment layer: Layer > New > Adjustment Layer
  5. Select the adjustment layer and choose Edit > Paste, or CTRL/CMD+V (it's safe to use the keyboard shortcuts now that we're out of Vector Paint). Press U, and the Position and Scale properties of the adjustment layer will show the new keyframes.


  6. Deselect the Position and Scale properties (F2 or click away from the layers).
  7. Select the Position property. All of the position keyframes will become highlighted.
  8. Edit > Copy or CMD/CTRL+C to copy the position keyframes.
  9. Go back to the Bessy layer and draw a new mask. It can be any shape or size.
  10. Press M on the keyboard, which will show the mask values for the Bessy layer. Select the Mask Shape property.
  11. Edit > Paste or CTRL/CMD+V to paste the position keyframes into the mask.

    We should now see our mask, although because there are so many handles it kind of looks like a double mask. Zooming in gives you a better view.

  12. Zooming in also reveals another problem: the mask is open. Go to the Layer menu > Mask > Closed (or context-click on the mask and choose Closed). Make sure to zoom back out to see the result: the background of the logo has been masked out.


We've accomplished our goal. The paint stroke has been transformed into a mask, and now Bessy is rotoscoped out of her logo. A little cleanup will make Bessy look prettier, though. Delete the adjustment layer and turn off the Vector Paint effect. I tweaked the mask feather and expansion properties to give Bessy a smoother edge. Then I added a drop shadow, animated the position and rotation properties, and gave Bessy a little dialogue. Preview the Bessy Roto - Final comp to see the results. Ta da!


If your project involves a lot of Vector Paint work, it's clear that converting all of the paint strokes to masks could be a time consuming process. AE only allows you to copy one paint stroke at a time. After Effects 6.0 has two new tools that can make the process a lot easier: the new Paint effect and the Auto-Trace command.


Part 2: Converting After Effects 6.0 Paint effect strokes into masks

Turning paint strokes into masks is far less complicated when you start with AE6's new Paint effect. It's just a matter of copying the paint stroke shape to the mask shape. Note that I said "start with"- there's no easy way to convert Vector Paint strokes into the new Paint strokes, other than turning them into masks and copying those into Paint. So plan ahead. The new Paint tools are worth the time to learn.

Open the Paint Roto - Start comp. This is identical to the Bessy Roto comp, except that the roto painting has been done using the eraser tool of the new After Effects 6.0 Paint effect instead of Vector Paint.

  1. Twirl down the Effects for the Bessy layer in the timeline window, then twirl down the Paint effect and open the Eraser 1 stroke.
  2. Select the Shape property and choose Edit > Copy or CTRL/CMD+C.


  3. Draw a new mask on the Bessy layer, any shape or size will do.
  4. With the mask still selected, choose Edit > Paste or CTRL/CMD+V.

    Just like with Vector Paint, the mask appears doubled, and it also needs to be closed.

  5. Go to the Layer menu and choose Mask > Closed (or context-click on the mask and choose Closed).

Just like with the Vector Paint instructions, a little cleanup work is desirable. Turn off the visibility of the Paint effect (or just delete it), and adjust the mask feather and expansion to clean up the edges. I also added some text and a couple of drop shadows. Preview the Paint Roto - Final comp for the result.

It's worth mentioning that not only can Paint strokes be copied into masks, but masks can be copied into Paint strokes (something that can't be done with Vector Paint). To do so, just reverse the process: copy the Mask Shape, draw a paint stroke, and paste into the stroke's Shape property. If you want to use the mask to roto-paint, make sure you draw your stroke with the eraser tool. (TIP: You can also convert any existing paint stroke to an eraser stroke by changing the color to 100% black and the Channels property to Alpha.)


Part 3: Using Auto-trace to turn paint strokes into masks

Converting paint strokes into masks can be tiresome if you have a lot of strokes. What if you don't have the time or patience? There is a quick and dirty alternative to have AE6 make a mask out of paint strokes for you: the Auto-trace feature. The difference is that instead of using the vector path of the paint stroke, Auto-trace creates the mask from of the outline of your paint strokes.

Open the Speed Bessy - Start comp and preview it. Here's Bessy, happily zooming around the barnyard. I've added a Vector Paint stroke as a cartoon speed trail. It's kind of cheesy looking. With the help of Auto-Trace, let's turn it into a smoke trail instead.

  1. Open the effects for the Bessy layer, and double-click on the Vector Paint effect. In the Effects Control window, change the Composite Paint property to "Only". This will isolate the paint strokes by themselves.


  2. Go to the Layer menu and choose Auto-trace...
  3. In the Auto-trace dialog, under Time Span choose "Work Area", choose "Alpha" for the Channel, and check the "Apply to new layer" option. Click OK.


  4. A new white solid layer will be created, with a mask that follows the outline of our vector paint stroke. Rename the layer to "Smoke" and move it two layers down, so it is directly below the Bessy layer.


  5. On the Bessy layer, turn off the visibility of the Vector Paint effect.
  6. Select the Smoke layer and apply the Fractal Noise effect (Effects > Noise > Fractal Noise).
  7. Open the controls for Mask 1 on the Smoke layer, and increase the Mask Feather property to give the smoke effect a softer edge. I animated the value with hold keyframes: 9 pixels on Bessy's first pass, 12 on the second, and 20 for Bessy's closest flyby.

Presto! Bessy is now burning rubber, er, pixels across the countryside. Preview the Speed Bessy - Final comp to see the results.

In this example, I started with the Vector Paint effect. If you were to start with AE6's new Paint effect instead, the process requires only a single adaptation: In step 1, open the Paint effect settings and switch the Paint on Transparent property to On.

Practical application of using Auto-trace with your own comps may vary in difficulty. Most likely you'll need to play with the tolerance and threshold values in the Auto-trace dialog to get an accurate mask, especially when dealing with airbrushed or feathered paint strokes.


Summary

We've covered a lot of detail about paint strokes and masks. Let's run through the important points one more time:

  • To turn Vector Paint stroke into a mask: Copy the stroke and paste it onto another layer. Then copy the position keyframes of that layer into the Mask Shape property of a mask.
  • To turn a stroke form AE6's Paint effect into a mask: Copy the paint stroke's Shape property and paste it into the Mask Shape property of a mask.
  • To create a mask from paint strokes using the Auto-trace feature: Set the paint effect to "Only" or "Paint on Transparent", then choose Layer > Auto-trace.


Questions? Comments? Discuss this tutorial in the After Effects forum at CreativeCow.net.


If you found this page from a direct link, please visit our forums or read other articles at CreativeCOW.net

Comments

Reply to Timk reply to Harry
by Harry Putnam
> Reply to Harry
> by Timothy Kurkoski on Nov 30, 2009
> Wow. I haven't looked at this tutorial in ages.

Thanks so much for the reply. And you hit it dead right about how I goofed it up. I skated right over that open path part thinking (and this is pretty dumb) that since I knew I had entered a closed path, it didn't apply to me. But of course when I entered the position data.... that made it an open path, and I'm still thinking about the path I entered... hoboy. So sorry to have missed the boat there.
Reply to Harry
by Timothy Kurkoski
Wow. I haven't looked at this tutorial in ages.

Harry, it sounds like you've missed step 12. When you paste the motion path (the position keyframes) onto the mask shape, it is an open path and can't exclude pixels. You need to close the mask in order to make the background disappear. In After Effects CS4, the name of the menu command has changed slightly; it's now Layer menu > Mask And Shape Path > Closed.

For what it's worth, Vector Paint is very outdated now and I'd concentrate on Part 2 of this tutorial for practical uses. But Part 1 is still useful in that it demonstrates that Position keyframes (and other X/Y dimension properties) can be turned into vector masks.
Backgroud did not disappear
by Harry Putnam
I hope I haven't just completely missed boat on this brief
Tutorial

Authored by Timothy Kurkoski

Just the first section kind of left things hanging far as I can tell.

The tutor says `ta da' as if something has happened but I don't see any difference other than my path is outlined in a heavy yellow line.

I still see the logo background... but apparently it is supposed to have disappeared. Can anyone tell me what is supposed to have happened there?

The tutorial then jumps to the finished product, where the logo background is now gone, but never really says how that happens.

=================================

In point 12 where the author says:

"We've accomplished our goal. The paint stroke has been transformed into a mask, and now Bessy is rotoscoped out of her logo. [...] "

I still see what is shown in point 11.

That is, the logo background is still there. Author says it is rototscoped out so I'm guessing I should not be seeing it?

I have followed all the points. I do have the mask... so when or how does the rotoscoping happen and logo background disappear?


Related Articles / Tutorials:
Adobe After Effects
Imagineer mocha Pro 5 Plug-In for Adobe: An In Depth Review

Imagineer mocha Pro 5 Plug-In for Adobe: An In Depth Review

Imagineer mocha Pro 5 Plug-in for Adobe brings all the amazing features of the professional version of the mocha Planar Tracker directly into After Effects and Premiere Pro in the form of a plugin. In this in-depth review, After Effects tutorial guru Tobias Gleissenberger of Surfaced Studio will show you what you can do with this new plug-in, and discuss what he likes and doesn't like about the new update.

Tutorial
Tobias Gleissenberger
Adobe After Effects
After Effects 2015.3 - My Favorite Features

After Effects 2015.3 - My Favorite Features

Learn why you should upgrade to After Effects CC 2015.3 - 13.8.1 - a close and detailed look at the latest release of After Effects (August 2016). Roei Tzoref will be focusing on his favorite features that set this release apart from previous versions: Performance, Queue in AME, Lumetri Color new features, and more.

Tutorial
Roei Tzoref
Adobe After Effects
Advanced Masking in Adobe After Effects

Advanced Masking in Adobe After Effects

Some of the coolest stuff you can do inside of Adobe After Effects is only possible once you unlock the power of masks. Join After Effects whiz Tobias Gleissenberger of Surfaced Studio to learn about mask animation and interpolation, using the variable width feathering tool, managing mask modes and ordering, and more.

Tutorial
Tobias Gleissenberger
Adobe After Effects
Fixing Common After Effects Problems and Mistakes

Fixing Common After Effects Problems and Mistakes

Got problems using Adobe After Effects? Exported files too large, Expressions not working, mixing shapes and makes, modes/switches, selecting previews for layers vs. comps - Surfaced Studio's Tobias Gleissenberger shows you fast fixes for these and more!

Tutorial
Tobias Gleissenberger
Adobe After Effects
mocha AE Planar Tracker for Absolute Beginners

mocha AE Planar Tracker for Absolute Beginners

Want to learn how to create advanced visual effects? Learn how to use mocha AE to track your shot and add advanced visual effects to live action footage inside Adobe After Effects. mocha can help you track shots that would be hard to track using traditional 2D point or feature trackers because it is a PLANAR TRACKER. A planar tracker uses planes and textures to track as opposed to points or groups of pixels. This allows the tracker to stay on track even if your shot contains motion blur or a very shallow depth of field. mocha AE comes included with Adobe After Effects and is available since CS3 and there is no reason for you not to use this awesome tool to make it easier for you to track your shot, replace screens or rotoscope!

Tutorial
Tobias Gleissenberger
Adobe After Effects
How to Make After Effects Faster with Proxies

How to Make After Effects Faster with Proxies

Learn how to make Adobe After Effects faster by using PROXIES! A proxy is a placeholder that stands in place for a very large video file or image sequence in your project. You can easily create proxies for the large source files that you are using and After Effects will automatically link them to the item in your project panel.

Tutorial
Tobias Gleissenberger
Adobe After Effects
Stabilize & Smooth: mocha 5 Plug-in for Adobe & Avid

Stabilize & Smooth: mocha 5 Plug-in for Adobe & Avid

Imagineer Systems and Boris FX product specialist Mary Poplin shows you how to stabilize with the new mocha Pro 5 plug-in inside of Avid Media Composer, Adobe Premiere Pro and After Effects. This tutorial covers artistic stabilization, such as smoothing out camera movements or stabilizing around moving objects, completely locking down shots, and automatically replacing edge fill on planar backgrounds.

Tutorial
Mary Poplin
Adobe After Effects
Adobe After Effects Puppet Tool

Adobe After Effects Puppet Tool

Become a puppet master by learning how to use the Puppet Tool in Adobe After Effects! This intermediate-level tutorial from After Effects guru Tobias will show you how the Puppet Tool allows you to add joints and animations to bring life to any static image!

Tutorial
Tobias Gleissenberger
Adobe After Effects
How to Spawn A Clone in Adobe After Effects

How to Spawn A Clone in Adobe After Effects

Want to learn how to create a cool clone spawn effect in Adobe After Effects? Follow along with After Effects whiz Tobias from Surfaced Studio in this exciting new visual effects tutorial that combines green screen using Keylight, CC Vector Blur, the Liquefy Effect, CC Particle World, and much more, delivered in Tobias' inimitable style!

Tutorial
Tobias Gleissenberger
Adobe After Effects
Creating A Flame on Your Finger with After Effects

Creating A Flame on Your Finger with After Effects

It is easy to do some motion tracking and attach a basic stock footage element of fire onto your hands, but there is a little bit of work involved if you actually want to make it look good. In this intermediate tutorial by After Effects expert Tobias, you'll see how to use a fire stock footage element to set your thumb on fire! There are lots of useful tricks for null objects, expressions, and more in this tutorial that will help you create all sorts of other cool visual effects -- or set even more things on fire!

Tutorial
Tobias Gleissenberger
MORE
© 2016 CreativeCOW.net All Rights Reserved
[TOP]