Strictly speaking, photogrammetry is the process of measuring and extracting data from photos, but in the film and video world there is a more exciting twist to the idea - if you've seen Panic Room you'll know what I am talking about. Photogrammetry is used to create or replace real-world objects with CGI models - in the case of Panic Room, the camera was able to swoop through banister railings and through the handle of a coffeepot because these objects were created digitally. This gives the Director a huge amount of freedom, and the picture more chances of an effects Oscar :)
During a break from work, I grabbed my trusty PD100 DVCam and started experimenting with this technique. The idea was to create a shot where the camera approached the back door of my house, flew through the keyhole and out into the back yard. You can see the result in either Windows WMV and QuickTime formats -- (download from the green bar above.) You can also see I haven't tidied up the back porch.
Now the effect isn't perfect, but you'll get the idea - I wasn't planning to spend hours trying to get the shot to look perfect. There are ways I could have improved it which I'll discuss at the end of the tutorial.
I considered supplying all of the project files, footage and media for you to download, but the files are quite large, and require you have the same versions of Maya and Premiere that I do, and I felt that it would be better to give a more generic overview of what I learned that you could then use with the software packages you use.
Preparation
First, I shot a few seconds of video at keyhole level of my back door. I approached the door slowly, trying to keep the keyhole centred in frame so the outside world could be seen through the hole.
Then I opened the back door and shot a seconds of footage where I moved the camera in exactly the same way as before, except I continued out through the open door and into the back yard.
Initial Edit
In Premiere, I imported both clips and cut them together in such a way that the motion up to and past the door was as smooth as possible, so the door simply seemed to "disappear" from one shot to the next. I tried to end the clip of the camera approaching the keyhole as close to the door as possible, neatly lined up, and not in the middle of a camera shake - because the CGI part of the process would be a smooth camera movement, I didn't want to jar the viewer when transitioning from the handheld to the CGI footage. As you can see, I was only partly successful.
So the last frame of the first clip is where the CGI begins, supposedly continuing the camera movement through the keyhole. I exported this last frame as a bitmap to use in Maya - here's the frame:
It's not quite perfectly straight, and this caused me a problem later on as you'll see.
Maya
As I'm from the PAL side of the world, the image is 720*576. So in Maya, I created a simple camera and using the Render Globals and Camera Attributes, made sure I had my scene set up the same way, with the resolution outline displayed in the interface. I left the camera at the origin, and at -5 units on the Z-Axis, created a polygonal cube (NURBS and Sub-Ds are too scary right now) to represent the side of the door facing the camera. In the Camera view, I scaled it so it exactly fit the 720*576 resolution outline. I made it about 0.05 units thick, so it approximated a thin sheet of wood.
I then created a Lambert texture for the cube and used the exported frame from the video clip as the image map. Add an ambient light and voila! What should happen, if you render this scene, is you get an image identical to the original frame from Premiere.
To "cut out" the keyhole so the camera could fly through it, I used 2 cylinders and a cube scaled to fit exactly over the keyhole section of the door panel. You can see the front entrance to the keyhole has a rim of light at the top... I made sure the top cylinder wouldn't obscure this. Using the polygonal Boolean difference Operator I used the 2 cylinders and the cube to cut a keyhole-shaped section out of the door object.
Now, to make the flythrough a little more effective, I wanted to give some "depth" to the keyhole, so I decided to create another panel to represent the rear of the keyhole. So I took all the work I had done so far, dumped it onto a layer and hid the layer, then created another thin 720*576 cube in the camera view to represent the back of the keyhole, at -5.5 on the Z-Axis.
Now I struck a problem. As you can see from the still above, the rear exit of the keyhole does not appear to be the same shape as the front of the keyhole, and because the camera wasn't perfectly straight-on, it wasn't centred either (oh for motion-control!) So when I created the cylinders and cube to cut out the back of the keyhole, I needed to scale and orient them to fit over the opening exactly as in the still above. But, by the time my digital camera was passing through the keyhole, the rear exit of the keyhole needed to be the same size and shape as the front entrance, as in the still below:
So, I cheated. After some experimentation, I determined that 40 frames was roughly the right length for this animated flythrough to match the speed of the original footage. So at frame 0 in Maya, I had my cylinders and cube keyframed to match the original still frame, and at frame 40, I resized the objects to match the cylinders and cubes I used to cut the front entrance of the keyhole and keyed them again. This slowed Maya down a bit, and it had to re-Boolean the keyhole exit at every frame, but the effect is hardly noticeable in the finished version, the keyhole exit seems to grow naturally into the same shape as the keyhole entrance as the camera gets closer.
To create the interior of the keyhole, I was originally going to build some metal "nernies" (I.e. meaningless 3d doohickeys) for the interior, but in the end I didn't bother. For the dark texture of the keyhole exit, I just used the original still, and in Photoshop cut away everything except the metallic colours near the edge of the keyhole and made the rest black. I also created a "metal" texture from a section of the original still just below the keyhole to texture things like the inner edges of my Booleans, I never did quite get that right, but I am only a "boy racer" Maya user :)
Animation
I animated the fly through by simply setting keyframing the camera attributes at frame 0 and then setting up the camera at frame 40 so it had just gone through the keyhole, i.e. the edges of the exit of the keyhole were just out of frame. I had to add another keyframe around frame 30 (the image above) so the camera would be lined up neatly for its flythrough.
The frames were all exported as 720*576 TGA files and imported into Premiere 6.5 as single frame stills
The Final Composite
I placed the TGA stills on Layer 2, lined up with and above the second video clip (flying out the door). Because the first frame of animation was identical to the last frame of the first clip, I trimmed the first clip back one frame, otherwise there would have been a pause in the movement. Also, because the stills were rendered as frames and my video obviously fields, I deinterlaced the first clip (otherwise the shimmer of the interlaced footage would noticeably vanish once we hit the stills)
I used the Video Options->Transparency setting on each still to make the alpha transparent so we could see through the keyhole to the footage of the outside world. Note that I tried selecting all the clips and applying this transparency setting using the Clip menu in Premiere, and this crashed the program. So it was a frame-by-frame, right-click process.
My final problem was the discontinuity between what you see through the keyhole as the camera approaches it and what you see once the stills kick in (i.e. you see footage from the second shot) - there's a noticeable jump in backgrounds - see below
A dissolve helped, but if I'd spent more time I could have cleaned it up :)
What I could have done better
- The original camera footage should have been smoother and straighter (I should have used the GlideCam!)
- Rather than a straight fly-through with the CG camera, I could have keyframed some wobble to try and match the camera wobble
- The interior of the keyhole could have been more interesting
- More time could have been spent matching the colour/lighting of the two sequences.
- Motion blur added to the CG footage might have helped.
- I could have rendered the CG footage as frames to match the original video.
- If I was a better Mayan, I could have cut the keyhole out more accurately using some clever CV trimming work and then made smoother edges to make them seem more worn. As a side note, our expert Maya user has just read all this and tut-tutted, especially when he saw me keyframing Booleans. So, be warned - consult a professional Mayan first!
- To get rid of the jump between what you see through the keyhole before and during the CG, I could have rotoscoped/masked what is seen through the keyhole and replaced it with a clip that more subtly dissolves (or morphs) from scene to the other.
I hope this has helped you and perhaps given you some ideas for projects of your own. My aim was to see whether this sort of effect was possible with the tools I had. It's by no means perfect, and there are probably much cleverer ways to set this effect up, but now I know it can be done without having to visit ILM.
|
Reply
Like
