Marco Solorio reviews: RE:Vision Effects, ReelSmart Twixtor with FieldsKit Deinterlacer

As I mentioned earlier, deinterlacing your footage is required in order to work with the Twixtor plugin. This process occurs even if your input and output settings are set to Lower or Upper Field First in Twixtor's settings. If you're using Twixtor to simply alter the clip's speed, then you should match your current clip's attributes to the other footage based in your overall project. With NTSC footage, this means setting the Input and Output Frame Rate to 29.97 with Upper or Lower Field set appropriately on both. Changing the "Output: Fields" setting to None will output your effect with no interlacing (as shown in the image). Some people like this non-interlaced flicker effect as it reminds them of a film-look process (more on this later in the article).

For the most accurate deinterlacing methods, let's look at RE:Vision Effects' ReelSmart FieldsKit plugin set. FieldsKit is composed of three distinct plugins; Deinterlacer, Pulldown and Reinterlacer. We won't be using the Pulldown or Reinterlacer plugins in this review, but we will be using the Deinterlacer plugin in association with Twixtor from this point on in the article. The two plugins together are a perfect marriage for optimum quality and I personally wouldn't do it any other way. So let's get a closer look at Deinterlacer.

The screenshot of the FieldsKit Deinterlacer plugin shows the default settings. Here's our goal: we want to deinterlace the footage in a way that will maximize the available amount of information that is currently in our source clip's 29.97 interlaced footage without throwing out the little information we have while also not introducing unneeded interpolation. The best way to perform this is to take each even and odd field group and turn them into their own frames. In other words, we want the upper field to create a full new frame and the lower field to create a full new frame and so on. We can do this in two ways... doubling the clip's duration, or doubling the clip's frame rate. Opting for the latter method simplifies things so we don't have deal with duration conversions. So let's get to work.

Import your footage. Like the earlier Twixtor example, control-click or right click the clip in the AE Project Window, select "Interpret Footage", click "Main..." and change the Separate Fields setting to "Off". Create a new comp and make the Frame Rate of that comp 59.94 FPS. Place your imported clip in the AE Timeline. Apply the Deinterlacer plugin to the clip. Let the dog out to go pee. Oops! Thinking out loud here!

Now we can start tweaking with the Deinterlacer settings. Change the Timing Mode setting to "2X FPS, 1 Frame Per Field". Since we doubled the comp's frame rate to 59.94, we'll need to use the 2X FPS option. Select the proper Field Order in accordance with the type of clip you are using. For me, I'm using an AJA Kona SD for this review so my setting is Lower First. My Aurora Igniter system on the other hand uses the Upper First setting (note to self: call up all the manufacturers and beg for a common standard).

So now we've created a 60p comp (59.94 progressive). This is incredibly valuable when dealing with any kind of time remapping because we essentially have an overcranked "clip" (precomp) to work with. So for example we want to create a 400% duration increase, we'll have a smoother result based on the extra frames of the 60p sequence. If we worked from a 30p sequence, we'd be creating frames based on further interpolation values, which in turn will result in a more "interpolated look." Think of this 60p technique as a higher "bit-rate" for time expansion! There are times when going from 60i to 60p isn't totally necessary and going from 60i to 30p is good enough. You'll get a better grasp of this with experience.

The next ten settings, from Fill Method to Sharpen are what you the user tweak to work the magic of the Deinterlacer plugin. Each type of clip will require different settings altogether. There is no "general settings guide" because each setting does something different and has advantages/disadvantages against the other settings. A clip of a head shot will require different settings than a clip with two trucks crossing their paths with text on the sides of them. This is a whole other article on its own! But for the sake of demystification, I sometimes have the following settings:

• Fill Method : Blend or Best 5 Neighbors (sometimes 3)
• Detect Motion: 3 Frames Compare
• Detect Method: Large Area
• Motion Tolerance: .35 (I've never gone lower than .30 or higher than .40)
• Mask Suppress: 0 or 1, typically
• Mask Grow: 0 to 5
• Mask Feather: 0 to 5
• Smooth Vertically: 0 but if it's called for, between .5 and 1
• Sharpen: 0

Remember, these are general settings that get tweaked around on a clip-by-clip basis. Simply plugging these values into Deinterlacer on all your clips wont allow Deinterlacer to work at its full potential and may even make things worse. And USE the "View Motion Mask" option to see what areas of the image Deinterlacer is affecting. This is a very cool and powerful tool to use and lessens the guesswork in all of this. Play around with the settings until you get it right!

So how does Deinterlacer compare to other deinterlacing methods? Glad you asked. Let's just say the other methods don't compare in the least! Most deinterlacing plugins simply blend the to fields together resulting in loss in vertical resolution (although Deinterlacer does have this option should you opt for it). Below is a dynamic image comparison example of different deinterlacing methods. This is an impractical image in real world applications, but what the test elements do is challenge the deinterlacing plugins to some impossible tasks. The most impossible task is preserving 1-pixel lines. Why? Because the deinterlacing method takes 1-pixel lines (thinking they're interlaced fields) and blends them together. Because of this phenomenon, detail in sharp edges and text can also get tossed out the window through traditional methods. Another problem with traditional deinterlacing tricks is that a single object can double in appearance due to frame blending. An example of this would be a moving, talking head and instead of two eyes looking at you, you get four eyes trying to blend together... scary!

You'll need JavaScript active to make the comparison function operate.

1. Source Image 2. Single Field 3. Single Field Deinterlace 4. AE Deinterlace Plugin 5. Twixtor Deinterlace 6. FieldsKit Deinterlacer 7. FieldsKit Multilayered

Image Notes:

1. Source Image - This is the original clip running at an interlaced 29.97 FPS (lower fields first).
2. Single Field - This process simply uses one field only. This is the result you would get when capturing footage with only one field (as some have done for trying to achieve a film strobe effect).
3. Single Field Deinterlace - Same as above but adding the standard AE Deinterlace plugin to it.
4. AE Deinterlace Plugin - This is the standard AE deinterlacing plugin set to "1". For the pros: this plugin produces the same exact results as using two layers of the same clip, separating the opposing field-order on each layer and blending the above layer to 50%. Performing a difference test reveals this.
5. Twixtor Deinterlace - This is the result when using Twixtor's standard deinterlacing feature. Note that Motion Blur Amount was set to zero to aid in the deinterlacing of this wicked image.
6. FieldsKit Deinterlacer - Now we can start seeing some resolution back in the image. Although it's not perfect (as compared to a native 30p clip), it's still showing us it can maintain vertical resolution in tough situations. The problem with this test image is that I've purposely implemented different tasks for the plugin to attack. With FieldsKit, you would typically look at the overall image and asses the best type of setting for that task. Ah, but there's even a solution for multiple tasks...
7. FieldsKit Multilayered - For the true compositing nerd such as myself, you could easily use FieldsKit on multiple layers of the same clip (and masking only the areas of concern). As seen here, the detail in the right-hand grid images are in full swing. The model's hair and face are more natural looking and ALL the text and logo retain full detail without blurry after tastes. Another example of this would be a waterfall in the background with a truck passing from right to left of the frame with a logo and text on the side of the truck. The organic nature of the water in the background will require different settings than those needed for the hard lines of the truck and its text and logo. Of course a happy medium can be assigned, but the ability is there to multi-layer for optimal results.

Barend Onneweer, one of our esteemed Creative Cow leaders has a deinterlacing article here at the Cow that covers the different methods of deinterlacing video with After Effects. He too concludes that FieldsKit Deinterlacer is the best way to go.

Okay, enough on deinterlacing. Let's start really playing with Twixtor!

	Frame rate conversions: 60i to 24p

One of Twixtor's powerful features is its ability to convert frame rates. The most popular method for this of course is converting standard 60i video footage to 24p. If you're new to the video world, the reason for doing this is to convert NTSC 29.97 FPS interlaced video to a 23.976 FPS progressive format, known as 24p. This 24p conversion process is part of the technique to make your video look like it was telecined from a film source since film is commonly shot at 24 FPS. The basic process of converting 60i to 24p is three-fold. Use FieldsKit Deinterlacer to convert 60i (29.97 FPS interlaced) to 60p or 30p. Precomp or sub-sequence the clip. Bring the precomp into a new comp and apply Twixtor to the precomp and change the settings to input 60p or 30p and output 23.976. Render your comp with 3:2 pulldown back to 60i (29.97 FPS interlaced). Those with 24p NLE support need not perform this step. So let's dissect this three-step process. Deinterlacing Use the techniques mentioned earlier in the article to deinterlace your footage. I personally prefer to always convert to 60p as opposed to 30p since this will be beneficial to other areas of concern. You now MUST precomp this layer. If not, Twixtor will merely look at the image information, minus the deinterlacing process. So precomp this layer and place it in a new comp. Apply Twixtor to the precomp layer. Add 2 cups of sugar and follow as directed. 60p/30p to 24p conversion This must be done correctly in order for you to achieve the best results. Remember how I mentioned the Deinterlacer plugin has different settings to achieve optimal results in different situations? Well Twixtor is the same way. You have different ways to perform interpolation, motion vectors, motion blur and motion sensitivity. You really need to tweak the settings to get the best results... no "general settings" here either. The one thing that will always be the same is the way in which you convert the frame rate. Looking at the image, we can see I told Twixtor the incoming source is progressive (marked as "None") with a frame-rate of 59.94 FPS. I then told Twixtor that I need to keep it progressive on output but at a new frame-rate of 23.976 (as displayed by 23.98). The thing that's tricky about Twixtor is avoiding a goopy mess when interpolating frames. Through serious tweaking of the settings, you can avoid this. Additionally, using the Pro version can really help avoid this by using a matte to specifically tell Twixtor which portions of the image need interpolating and which don't. Below is another dynamic image comparison example of the finished 60i to 60p/30p to 24p conversion process by means of different methods.

Frame Rate Naming Conventions 60i = 29.97 FPS, interlaced 60p = 59.94 FPS, progressive 50i = 25 FPS, interlaced (PAL) 50p = 50 FPS, progressive (PAL) 30p = 29.97 FPS, progressive 25p = 25 FPS, progressive (PAL) 24p = 24 or 23.976* FPS, progressive The number represents the amount of images (not frames) to be seen in one second. E.g., in 60i video, there are 2 images for every frame of video (vertical resolution cut in half for each image... one image for even fields and another image for odd fields), thus a total of 59.94 "half" images are viewed every second. The letter after the number says whether those images are interlaced (half resolution) or progressive (full resolution). Note that the Twixtor manual says "30i" to represent 29.97 interlaced frames and does this so new users don't confuse 60i with 60 FPS. In reality, 30i means 15 FPS, interlaced. *Note that this article uses 23.976 FPS for all 24p examples in accordance with NTSC compatibility standards.

1. Source Image 2. AE Deinterlace & Frame Blend 3. Twixtor (no FieldsKit) 4. FieldsKit & Twixtor

Image Notes:

1. Source Image - This is the original clip running at an interlaced 29.97 FPS (lower fields first).
2. AE Deinterlace & Frame Blend - After Effects blends the adjacent frames together. In most cases, this produces a doubling of the image with the addition of added softness and excessive motion blur. Although this might appear to look more aesthetically pleasing as a still image, the result is most unfavorable on an NTSC monitor upon playback. In other words... YUCK!
3. Twixtor (no FieldsKit) - Motion really starts to shine, but the deinterlacing could use a little help. Cue, example 4.
4. FieldsKit & Twixtor - Once again, all the text and 1-pixel lines are kept in tact. There's no excessive motion blur or added softness. Most importantly, vertical resolution is maintained throughout. Viewed on an NTSC monitor, this result is by far the most favorable.

Going back to 60i and adding 3:2 Pulldown
There are several instances when going back to 60i with 3:2 pulldown isn't needed. They include finishing off to progressive DVD, CGI production, PAL conforming to 25 FPS or printing to film. Otherwise you'll need to go back to 60i video and introduce 3:2 pulldown. And if you don't perform the 3:2 pulldown, you're motion will have an ugly fast-slow judder to it. Some systems (like my AJA Kona SD and Aurora Igniter Film) can fully edit in native 24p, so rendering 3:2 pulldown isn't necessary (this puts a VERY large smile on my face!). These 24p compatible systems add the 3:2 pulldown on the fly on output to your monitor and deck. Totally trick.

The image below shows part of the After Effects Render Settings window to add 3:2 pulldown to 60i from a 24p timeline.

The process in whole looks something like the image below. Keep in mind this is a very basic diagram of the process and that other methods can be introduced (like conforming instead of interpolating).

What a freakin' mess! Obviously if you have means to a 24p camera... use it!!! Note also that the 30p conversion step can also be a 60p conversion process (my preference ni most cases) but is shown as 30p for simplicity's sake.

A little off topic: Conforming
Obviously the best way to get a 24p look is to shoot with a 24p camera. But if that's not possible, then shoot with a PAL camera running at 25 FPS. Then convert that 50i footage to 25p with FieldsKit Deinterlacer and conform that 25p conversion to 24p. By conforming, you're keeping all the frames in tact and only decreasing the speed by 96%... hardly noticeable by any account.

You can also conform your 60i to 30p conversion to 24p, which will reduce the speed by 80%, which is noticeable. This is great when audio is not part of the footage and you want a slight overcranking effect. The process slightly slows down the footage enough to give it a smoother appearance in motion.

	Time remapping