Looking at traditional methods of time-remapping, frame-rate conversions and other "spatial continuum" methods, we'll see how the popular Twixtor plugin compares to the task at hand. The results are interesting with some surprising results. We'll also look at the ways Twixtor works well with its sister plugin, "ReelSmart FieldsKit". Those looking into film-look conversion techniques will especially benefit from this review.
It's safe to say that Pete Litwinowicz (pictured) and Pierre Jasmin, the creators of the ReelSmart plugins are a couple of pretty smart guys. They've developed a process in which their plugins look at pixel information and interpolates new data based on morphing algorithms as opposed to traditional frame-blending methods. And best of all, the new data that is created is formulated from areas of the image that need it, leaving the areas that don't need it untouched, thus preserving the integrity of the shot with as less interpolation "loss" as possible. Now that's "Reel Smart"!
Let's first look at the basics surrounding the Twixtor plugin. You'll be hard pressed to find an application that won't run Twixtor. Supported applications include Apple Final Cut Pro, Apple Shake, Adobe After Effects, Adobe Premiere, Discreet Combustion, Discreet Flame, Avid, Pinnacle Commotion, Quantel generationQ and Eyeon Digital Fusion. Supported operating systems include Mac OS 7.5 to OSX, Windows 95 to XP and Irix 6.5 and up. Now that's compatibility! For the full low-down on product support, visit the RE:vision compatibility page. The process of this review will be using Twixtor in Adobe After Effects 5.5 on OSX.
Twixtor comes in two flavors: a standard version ($329.00 USD) and a Pro version ($495.00 USD). Note that some applications (e.g., Shake and Avid) only offer a Pro version with slight price reconfiguration. Some applications also support Render Node support ($165.00 to 198.00 USD). There are also upgrade offers from the Standard version to the Pro version ($165.00) for applications that support the Standard version.
The Pro version adds 16-bit rendering support (for applications that support it, like Adobe After Effects), which to me is the biggest reason to go for the Pro version since I work with nothing but 10-bit uncompressed files in 16-bit After Effects projects in "Trillions mode". Also in the Pro version is the ability for Twixtor to assign the foreground/background setting with a matte so Twixtor knows which portion of the image comprise a foreground object(s) and which areas comprise a background object(s) that move in dissimilar directions (including the possibility of one object(s) staying static). This function can prove vital for those really tough clips with intricate motion circumstances.
The "prime cut" to this beefy plugin is it's ability to change the speed of a clip (either constant or keyframed) and to change frame-rate speeds. As noted from the RE:Vision Effects website:
ReelSmart Twixtor enables you to speed up, slow down or frame rate convert your image sequences with visually stunning results. In order to achieve its unparalleled image quality, Twixtor synthesizes unique new frames by warping and interpolating frames of the original sequence... employing RE:Vision's proprietary tracking technology that calculates motion for each individual pixel.
This is very important to understand because Twixtor doesn't merely frame-blend the motion together like traditional methods (although the option to do that is there).
Getting started with the plugin
The first thing you'll encounter when using Twixtor for the first time is its inherently high learning curve. Even seasoned compositing pros might find the plugin a little daunting at first. But rest assured that given the time and patience, you'll soon grasp the concepts, functions and power of Twixtor. It's really a matter of getting familiar with the plugin terms and function relationships/differences. Download all the Twixtor tutorials, study and fiddle around with them. I found this the best way to get accustomed to the plugin as my initial ideas of controlling the parameters were way off target. Don't waste time trying to figure it out on your own like I first did... download the tutorials, read the manual and you'll be much happier in the end!
The biggest thing to understand when using Twixtor is how to initially treat the footage you import in. For example, when you import a clip into After Effects, the default settings for the clip will most likely be set to either "Upper Field First" or "Lower Fields First". You must change the Separate Fields setting to "Off" for Twixtor so it can fully use both fields of information from the clip. My initial lack of desire to read the manual and play with the tutorials (but over ambitious desire and excitement to just start "playing with the plugin") transcended into results that I referred to as, "ewww... why does this look so bad?!!" In fact, failure in going through the manuals and tutorials may make you utter words like, "man, this plugin sucks!" In reality, proper education will result in words like, "man, this plugin rocks my world!"
Next on the "to-do" list of items is deinterlacing the footage. Even if you're bringing interlaced footage in and spit it back out to the same exact frame-rate with interlacing, Twixtor still deinterlaces the footage in order for it to properly work its magic. This can be done in two ways. The easy way is to have Twixtor deinterlace the footage using its own simple deinterlacing function. The best way (and the only way I do it) is to use RE:Vision Effects ReelSmart FieldsKit. This will add to the already long render time you'll encounter, but it will be worth your efforts as the quality is bar none.
As I mentioned earlier, deinterlacing your footage is required in order to work with the Twixtor plugin. This process occurs even if your input and output settings are set to Lower or Upper Field First in Twixtor's settings. If you're using Twixtor to simply alter the clip's speed, then you should match your current clip's attributes to the other footage based in your overall project. With NTSC footage, this means setting the Input and Output Frame Rate to 29.97 with Upper or Lower Field set appropriately on both. Changing the "Output: Fields" setting to None will output your effect with no interlacing (as shown in the image). Some people like this non-interlaced flicker effect as it reminds them of a film-look process (more on this later in the article).
For the most accurate deinterlacing methods, let's look at RE:Vision Effects' ReelSmart FieldsKit plugin set. FieldsKit is composed of three distinct plugins; Deinterlacer, Pulldown and Reinterlacer. We won't be using the Pulldown or Reinterlacer plugins in this review, but we will be using the Deinterlacer plugin in association with Twixtor from this point on in the article. The two plugins together are a perfect marriage for optimum quality and I personally wouldn't do it any other way. So let's get a closer look at Deinterlacer.
The screenshot of the FieldsKit Deinterlacer plugin shows the default settings. Here's our goal: we want to deinterlace the footage in a way that will maximize the available amount of information that is currently in our source clip's 29.97 interlaced footage without throwing out the little information we have while also not introducing unneeded interpolation. The best way to perform this is to take each even and odd field group and turn them into their own frames. In other words, we want the upper field to create a full new frame and the lower field to create a full new frame and so on. We can do this in two ways... doubling the clip's duration, or doubling the clip's frame rate. Opting for the latter method simplifies things so we don't have deal with duration conversions. So let's get to work.
Import your footage. Like the earlier Twixtor example, control-click or right click the clip in the AE Project Window, select "Interpret Footage", click "Main..." and change the Separate Fields setting to "Off". Create a new comp and make the Frame Rate of that comp 59.94 FPS. Place your imported clip in the AE Timeline. Apply the Deinterlacer plugin to the clip. Let the dog out to go pee. Oops! Thinking out loud here!
Now we can start tweaking with the Deinterlacer settings. Change the Timing Mode setting to "2X FPS, 1 Frame Per Field". Since we doubled the comp's frame rate to 59.94, we'll need to use the 2X FPS option. Select the proper Field Order in accordance with the type of clip you are using. For me, I'm using an AJA Kona SD for this review so my setting is Lower First. My Aurora Igniter system on the other hand uses the Upper First setting (note to self: call up all the manufacturers and beg for a common standard).
So now we've created a 60p comp (59.94 progressive). This is incredibly valuable when dealing with any kind of time remapping because we essentially have an overcranked "clip" (precomp) to work with. So for example we want to create a 400% duration increase, we'll have a smoother result based on the extra frames of the 60p sequence. If we worked from a 30p sequence, we'd be creating frames based on further interpolation values, which in turn will result in a more "interpolated look." Think of this 60p technique as a higher "bit-rate" for time expansion! There are times when going from 60i to 60p isn't totally necessary and going from 60i to 30p is good enough. You'll get a better grasp of this with experience.
The next ten settings, from Fill Method to Sharpen are what you the user tweak to work the magic of the Deinterlacer plugin. Each type of clip will require different settings altogether. There is no "general settings guide" because each setting does something different and has advantages/disadvantages against the other settings. A clip of a head shot will require different settings than a clip with two trucks crossing their paths with text on the sides of them. This is a whole other article on its own! But for the sake of demystification, I sometimes have the following settings:
Fill Method : Blend or Best 5 Neighbors (sometimes 3)
Detect Motion: 3 Frames Compare
Detect Method: Large Area
Motion Tolerance: .35 (I've never gone lower than .30 or higher than .40)
Mask Suppress: 0 or 1, typically
Mask Grow: 0 to 5
Mask Feather: 0 to 5
Smooth Vertically: 0 but if it's called for, between .5 and 1
Remember, these are general settings that get tweaked around on a clip-by-clip basis. Simply plugging these values into Deinterlacer on all your clips wont allow Deinterlacer to work at its full potential and may even make things worse. And USE the "View Motion Mask" option to see what areas of the image Deinterlacer is affecting. This is a very cool and powerful tool to use and lessens the guesswork in all of this. Play around with the settings until you get it right!
Join panelists Andrew Kramer of Video Copilot, Jayse Hansen, and Mary Poplin of Boris FX, along with moderator, Victoria Nece of Adobe, as they discuss the world of visual effects from an artist’s perspective.
How do you add SCARS, TATTOOS or DIGITAL MAKEUP to a person's face? Learn how to use Mocha Pro's planar tracker and the Mesh Warp tool to insert a flat image/video on a (non flat) human face! Join VFX guru Tobias G from Surfaced Studio for a closer look!
The first challenge to understanding the nature of brightness in compositing starts with remembering that we're not actually seeing color at all, but rather something of an illusion that appears to us as color! Join longtime VFX artist, editor, software developer, and business owner Simon Ubsdell for Part 5 of the best look behind the technology of compositing that you've ever seen, as he takes a look at the math behind brightness, and how to apply that to the compositing toolsets in your favorite editing, compositing, and color grading applications.
A Motion Graphics Template, referred to as a MOGRT, is an animated sequence that is self-contained and can be used in Adobe Premiere Pro, Adobe Premiere Rush and Adobe After Effects, combining graphics, text, audio and video files, as well as vector or still images (including logos), to create a still or animation that can then be customized by the MOGRT user. The result is a dynamic creative tool that provides design freedom and is consistent to its users across apps and devices. Reuse, share, and even sell them!
When most people hear the words "alpha channels", they think "transparency", but that's not exactly accurate. The truth is more complex, and a quite bit more interesting! Join longtime VFX artist, editor, software developer, and business owner Simon Ubsdell for Part 4 of the best look behind the technology of compositing that you've ever seen, packed with practical advice for applying the secrets of alpha channels that's simply not possible before understanding these underlying principles. No matter which applications you're using for editing, compositing, or visual effects, this one is a must-see!
When most people hear the words "alpha channels", they think "transparency", but that's not exactly accurate. The truth is more complex, and a quite bit more interesting! Join longtime VFX artist, editor, software developer, and business owner Simon Ubsdell for Part 3 of the best look behind the technology of compositing that you've ever seen, packed with practical advice for applying the secrets of alpha channels that's simply not possible before understanding these underlying principles. No matter which applications you're using for editing, compositing, or visual effects, this one is a must-see!
Whether you're a full-time compositor and VFX artist, an editor working in one of the many NLEs that supports Blend Modes (including Adobe Premiere, Apple FCP and FCPX, Avid Media Composer, DaVinci Rsolve, and VEGAS Pro), or whether you just want the additional graphics power that comes with insight into the secret world of pixels, this is the tutorial for you! Join longtime VFX artist, editor, software developer, and business owner Simon Ubsdell for Part 2 of the best look behind the technology of blend modes that you've ever seen, with some practical steps you can start taking today to make your work look better than ever.
Join longtime VFX artist, editor, software developer, and business owner Simon Ubsdell for the best look behind the technology of blend modes that you've ever seen. This isn't just for graphics and VFX, but for video editors too -- anyone who puts anything together, and wants to learn more about HOW images combine at the most basic level, in a way that applies to every application you might use, whether Adobe Premiere Pro, Apple FCPX, Avid Media Composer, DaVinci Resolve, VEGAS Pro, and yes, graphics and VFX programs like After Effects, Motion, Fusion, Nuke, Scratch, and many more.
Join VFX guru Tobias Gleissenberger of Surfaced Studio for a classically energetic Adobe After Effects tutorial combining difference mattes, displacement maps, edge glows, and compositing tricks to dissolve (or explode!) one element in your frame while leaving the rest alone.