Orchestrating a Simple Wipe
COW Library : Live Events & Streaming : Mark Grossardt : Orchestrating a Simple Wipe
A simple wipe, just a wipe. That's how this whole thing started.
Five years ago, Clark Creative Group was approached by world-renowned artist Jun Kaneko about the sets for Puccini's Madama Butterfly, a production Jun had been commissioned to design by Opera Omaha. A painter and sculptor, Jun had worked with CCG in the past to design promotional print materials for some of his other artistic endeavors. He was vaguely aware of our agency's postproduction capabilities, and he had a small problem that he hoped we'd be able to solve.
Part of Jun's set design incorporated 35mm film slides projected onto a screen behind the performers. He wanted a simple wipe to transition two images, but couldn't figure out a good way to do it. Once we explained how easily a digital workflow could solve Jun's dilemma, the gears in his head began turning. Starting to understand the immense capabilities of computer generated animation, Jun quickly developed more animation designs. Though we had a very limited knowledge of opera, we happily agreed to help him out with them.
When it was all said and done, we ended up animating over 45 minutes of material for the show -- and we learned a lot along the way. We've since delivered animations for two other operas: Verdi's Aida, designed by Katherine Ferguson for Opera Omaha, and Beethoven's Fidelio, designed by Kaneko for the Philadelphia Opera.
Designing animation for live opera presents unique challenges. Now that we've been able to muddle through the workflow a few times, we've got a better idea of how to handle it.
Jun Kaneko's production of Madama Butterfly.
For most mograph jobs, it's pretty easy to open an After Effects template and start working in a ready-made template with predetermined dimensions, running times, and pixel aspect ratios. But AE doesn't have presets for opera productions -- we checked.
Each production began with extensive meetings with the artist designing the production. We toured the artists' studios to get a sense of their unique styles. Jun's large-scale geometric minimalism is far different in tone than Katherine's bold organic designs, and we needed to be able to mimic each artist's aesthetic in our animations.
We then met with the artists, directors, lighting directors, and projectionists to get a sense of how much animation we'd be producing, how many screens we'd be filling, and the dimensions of those screens.
For Aida, setup was relatively simple. Each animation we created had the same set of dimensions, though one of the screens was partially transparent so that the animation could play over the action going on behind it -- an important consideration when designing the scene.
Madama Butterfly and Fidelio both had multiple animations to be projected on different screens with different dimensions. Fidelio also had one animation to be projected onto a stylized, upsidedown human head. We had to coordinate with the lighting director and projectionist to come up with the best method of projecting onto the head without spilling light onto the surfaces behind it.
We could have used masks in AE, but ultimately decided that this problem was best solved by the projectionist masking out the shape. Once the basics of dimensions and durations were nailed down, we could begin the real fun.
SPEAKING THE ARTIST'S LANGUAGE
Since we work in a highly collaborative field, we're used to taking direction from other creatives. But we are accustomed to working with other digital artists who speak the same language of RGB colorspace and alpha channels.
Fine artists are a different breed, and we quickly realized how different the world of brush, oil, and clay is from pixels, color gamuts, and keyframes.
When Jun first explained the wipe transition problem, he brought us into his studio where he had set up two slide projectors, each projecting a separate image onto the same wall. He carefully moved his hands in front of the two projectors to simulate a wipe from one image to the other. He did this for about five minutes, all to convey the simple idea of a wipe.
When working with Katherine Ferguson early in the creative process for Aida, she once asked, very concerned, "I'm not sure I like that color there. Is that something I could change, and if so how long would it take you?" Two button clicks and two seconds later, a new color was selected.
Kevin Reiner (left) and Mark Grossardt of Clark Creative Group visit the studio of Katherine Ferguson to view her concept designs for Verdi's Aida for Opera Omaha.
These two examples illustrate how important it was to establish a design-shorthand to communicate effectively with fine artists. Clearly, one of our challenges was explaining the immense capabilities of our computers and software. And much to their credit, the artists we worked with had to step outside of their comfort zones to sit in our suites and entrust their visions to a couple of dudes pounding out commands on silly-looking colorcoded keyboards. Recognizing that trust helped us stay focused on delivering each artist's vision to the best of our abilities.
One of the methods we used to help the artists and us work on the same page was to have the artists create a few frames of animation using whatever medium they desired. We then replicated those frames in After Effects as precisely as we could. In a few cases, we even went as far as scanning the artists' drawings to get the actual feel and texture of a colored pencil stroke, rather than try to approximate it with AE filters.
We found it important to demonstrate to the artists the notion that our tools may be wildly different from theirs, but we can still accomplish their visions. Once the artists were confident we could reproduce their styles down to the most minute details, we were ready to attack the rest of the animation process.
With style established, we moved on to animating. Working with each artist was a very different experience.
Jun's approach to opera design was very ordered and very specific. He liked to create animations by hand, much like a cel animator. He then made time notations on each "cel" expressing exactly when each element should start and stop animating. As a result, the first drafts that we would show Jun often ended up being very close to the final product. In many respects, this kind of designer is heaven for a mograph artist: an art director who gives clear, concise directions on exactly what the final product should look like.
Above, hand-drawn storyboard from Jun Kaneko's production of Fidelio, along with notes for timing the animations to the music.
In another instance, Jun had a very specific scribble- on effect in mind for a portion of Madama Butterfly. Rather than try to recreate his strokes and pacing, we had Jun use a tablet to animate it himself. We could've used any number of automated effects, but this method gave us a more organic feel and had the added benefit of using the artist's actual strokes in the live performance. It's something that the rest of the audience would never know, but as an animator trying to help an artist realize his vision, it was gratifying to see on screen.
We later found out that this exercise was the first time Jun had ever used a computer. This was a shocking realization to us, but it further illustrates an important point: in all of our dealings with any client, we need to remember that while we might be very computer savvy and ultra-reliant on our machines, our clients often don't operate in the same world. It's our job to understand that and still be able to communicate effectively with them.
Working with Katherine on Aida was a different type of collaboration all together. She had specific thematic ideas, but was very open to input on interesting ways to achieve these ideas. She gave us the freedom to experiment and try different techniques. We went through many revisions to realize her designs, and the give and take resulted in pieces that captured Katherine's vision while keeping her style and vision intact.
The danger in having this kind of freedom was running too far after an idea that wasn't doing the production justice -- "chasing rabbits" as we often refer to it. And at times, we took Katherine's ideas and chased rabbits.
In one instance, we toyed around with a stylized, aggressive animation meant to express the awesome power of Ra, the sun god. We were rather proud to show off how we used the latest supercool AE plug to bang out a killer animation.
Above, animations for Aida projected on a partially transparent screen. The title graphic for this article is also from Aida.
Katherine was less than impressed (though far too polite to tell us what she really thought), and requested an entirely different tack. Her suggestions were more subtle and didn't tax our systems or AE chops -- and her direction was much more in line with the rest of the production design and didn't interfere with the music. It was definitely a better choice.
Ultimately, when working with both of these artists, we had to constantly remind ourselves that our animations were just one piece of the whole experience. And as much as we love making pretty pictures zoom around on screen, anything detracting from that experience would do the opera a huge disservice.
One of the most pressing technical issues we had to overcome was how to sync our animations to a live symphony.
In the initial design stages, we would create our animations timed to a track from the definitive recording of each opera. Once the artist was happy with our animations, we had to submit them to the orchestra for practice. The orchestra would then rehearse with our animations guided by the stage directors and projectionists.
In the overture of Aida for example, a visual cue had to sync with an aural cue. Fortunately, the conductor for Aida didn't seem bothered by being tied to animation playing behind his orchestra.
This technique worked fine for the shorter animations in Aida, but Madama Butterfly and Fidelio had many long passages of constant animation that covered parts of the performance lacking an orchestral component. Some of these pieces were close to an hour in length, so we needed a new strategy.
Jun Kaneko's interpretation of the tragic romance between American Navy lieutenant B.F. Pinkerton and a beautiful young Japanese woman, Cio-Cio San, plays against a backdrop of bold colour and delicate beauty.
We solved the problem by creating segmented chunks of animation with long, looped endings, each lasting about five minutes. We designed the animations so that the incoming animation's first few seconds would be virtually indistinguishable from the outgoing animation.
The stage director could then follow the live performance closely and cue the projectionist to start dissolving into the next animation at the appropriate time. In this manner, long passages of animation could be seamlessly synced to a live orchestra.
Once the artists approved all of the animations, we had to coordinate with the projectionist to review any problems we saw in rehearsals and devise solutions.
Maybe the colors in a certain passage didn't translate well. Maybe a blending mode that looked great on a small LCD monitor looked atrocious on a fifty-foot screen. The last thing we wanted was for the artist to see something on screen and think, "That's not what I had in mind!"
Often, the projectionist could make a simple adjustment to fix an issue, but there were occasions that required going back and adjusting entire animations.
It should probably go without saying, but these projects required storage and organization of massive amounts of data. Since that data represented months and months of work, we had to make sure it was secure. If a drive went down the night before during a final render, we wouldn't just be able to pull an allnighter to rebuild a lost animation.
We got into the habit of transferring critical files to external hard drives and taking them home overnight. When the project was finally locked, we copied essential files to optical storage and solid-state storage, as well as leaving project files on our in-house RAID array. Other opera companies around the country might purchase the productions in the future, so we had to make sure these animations were well protected and had a long shelf life.
For a shop that lives off of thirty-second spots, this kind of redundancy seemed like overkill, but it was something we felt we couldn't take chances on.
DIFFERENT, SCARY, EXHILARATING
Diving into opera was a completely different, scary, exhilarating experience. The long hours and huge amount of work were all worth it. Opera is a world we had never thought to get into, but we're glad we did.
Right now we're in the middle of animating Jun's third opera production, Mozart's The Magic Flute, for the San Francisco Opera. Set to open in 2012, it's by far the most ambitious production we've ever been involved with. But we're confident that our past experiences will help us contribute to another amazing production.
And to think, it all started with a simple wipe.