LIBRARY: Tutorials Reviews Interviews Editorials Features Business Authors RSS Feed

Orchestrating a Simple Wipe

CreativeCOW presents Orchestrating a Simple Wipe -- Live & Stage Events Feature

A simple wipe, just a wipe. That's how this whole thing started.

Five years ago, Clark Creative Group was approached by world-renowned artist Jun Kaneko about the sets for Puccini's Madama Butterfly, a production Jun had been commissioned to design by Opera Omaha. A painter and sculptor, Jun had worked with CCG in the past to design promotional print materials for some of his other artistic endeavors. He was vaguely aware of our agency's postproduction capabilities, and he had a small problem that he hoped we'd be able to solve.

Part of Jun's set design incorporated 35mm film slides projected onto a screen behind the performers. He wanted a simple wipe to transition two images, but couldn't figure out a good way to do it. Once we explained how easily a digital workflow could solve Jun's dilemma, the gears in his head began turning. Starting to understand the immense capabilities of computer generated animation, Jun quickly developed more animation designs. Though we had a very limited knowledge of opera, we happily agreed to help him out with them.

When it was all said and done, we ended up animating over 45 minutes of material for the show -- and we learned a lot along the way. We've since delivered animations for two other operas: Verdi's Aida, designed by Katherine Ferguson for Opera Omaha, and Beethoven's Fidelio, designed by Kaneko for the Philadelphia Opera.

Designing animation for live opera presents unique challenges. Now that we've been able to muddle through the workflow a few times, we've got a better idea of how to handle it.

Jun Kaneko's production of Madama Butterfly.

For most mograph jobs, it's pretty easy to open an After Effects template and start working in a ready-made template with predetermined dimensions, running times, and pixel aspect ratios. But AE doesn't have presets for opera productions -- we checked.

Each production began with extensive meetings with the artist designing the production. We toured the artists' studios to get a sense of their unique styles. Jun's large-scale geometric minimalism is far different in tone than Katherine's bold organic designs, and we needed to be able to mimic each artist's aesthetic in our animations.

We then met with the artists, directors, lighting directors, and projectionists to get a sense of how much animation we'd be producing, how many screens we'd be filling, and the dimensions of those screens.

For Aida, setup was relatively simple. Each animation we created had the same set of dimensions, though one of the screens was partially transparent so that the animation could play over the action going on behind it -- an important consideration when designing the scene.

Madama Butterfly and Fidelio both had multiple animations to be projected on different screens with different dimensions. Fidelio also had one animation to be projected onto a stylized, upsidedown human head. We had to coordinate with the lighting director and projectionist to come up with the best method of projecting onto the head without spilling light onto the surfaces behind it.

We could have used masks in AE, but ultimately decided that this problem was best solved by the projectionist masking out the shape. Once the basics of dimensions and durations were nailed down, we could begin the real fun.

Since we work in a highly collaborative field, we're used to taking direction from other creatives. But we are accustomed to working with other digital artists who speak the same language of RGB colorspace and alpha channels.

Fine artists are a different breed, and we quickly realized how different the world of brush, oil, and clay is from pixels, color gamuts, and keyframes.

When Jun first explained the wipe transition problem, he brought us into his studio where he had set up two slide projectors, each projecting a separate image onto the same wall. He carefully moved his hands in front of the two projectors to simulate a wipe from one image to the other. He did this for about five minutes, all to convey the simple idea of a wipe.

When working with Katherine Ferguson early in the creative process for Aida, she once asked, very concerned, "I'm not sure I like that color there. Is that something I could change, and if so how long would it take you?" Two button clicks and two seconds later, a new color was selected.

Kevin Reiner (left) and Mark Grossardt of Clark Creative Group visit the studio of Katherine Ferguson to view her concept designs for Verdi's Aida for Opera Omaha.

These two examples illustrate how important it was to establish a design-shorthand to communicate effectively with fine artists. Clearly, one of our challenges was explaining the immense capabilities of our computers and software. And much to their credit, the artists we worked with had to step outside of their comfort zones to sit in our suites and entrust their visions to a couple of dudes pounding out commands on silly-looking colorcoded keyboards. Recognizing that trust helped us stay focused on delivering each artist's vision to the best of our abilities.

One of the methods we used to help the artists and us work on the same page was to have the artists create a few frames of animation using whatever medium they desired. We then replicated those frames in After Effects as precisely as we could. In a few cases, we even went as far as scanning the artists' drawings to get the actual feel and texture of a colored pencil stroke, rather than try to approximate it with AE filters.

We found it important to demonstrate to the artists the notion that our tools may be wildly different from theirs, but we can still accomplish their visions. Once the artists were confident we could reproduce their styles down to the most minute details, we were ready to attack the rest of the animation process.

With style established, we moved on to animating. Working with each artist was a very different experience.

Jun's approach to opera design was very ordered and very specific. He liked to create animations by hand, much like a cel animator. He then made time notations on each "cel" expressing exactly when each element should start and stop animating. As a result, the first drafts that we would show Jun often ended up being very close to the final product. In many respects, this kind of designer is heaven for a mograph artist: an art director who gives clear, concise directions on exactly what the final product should look like.

Above, hand-drawn storyboard from Jun Kaneko's production of Fidelio, along with notes for timing the animations to the music.

In another instance, Jun had a very specific scribble- on effect in mind for a portion of Madama Butterfly. Rather than try to recreate his strokes and pacing, we had Jun use a tablet to animate it himself. We could've used any number of automated effects, but this method gave us a more organic feel and had the added benefit of using the artist's actual strokes in the live performance. It's something that the rest of the audience would never know, but as an animator trying to help an artist realize his vision, it was gratifying to see on screen.

We later found out that this exercise was the first time Jun had ever used a computer. This was a shocking realization to us, but it further illustrates an important point: in all of our dealings with any client, we need to remember that while we might be very computer savvy and ultra-reliant on our machines, our clients often don't operate in the same world. It's our job to understand that and still be able to communicate effectively with them.

Working with Katherine on Aida was a different type of collaboration all together. She had specific thematic ideas, but was very open to input on interesting ways to achieve these ideas. She gave us the freedom to experiment and try different techniques. We went through many revisions to realize her designs, and the give and take resulted in pieces that captured Katherine's vision while keeping her style and vision intact.

The danger in having this kind of freedom was running too far after an idea that wasn't doing the production justice -- "chasing rabbits" as we often refer to it. And at times, we took Katherine's ideas and chased rabbits.

In one instance, we toyed around with a stylized, aggressive animation meant to express the awesome power of Ra, the sun god. We were rather proud to show off how we used the latest supercool AE plug to bang out a killer animation.

Above, animations for Aida projected on a partially transparent screen. The title graphic for this article is also from Aida.

Katherine was less than impressed (though far too polite to tell us what she really thought), and requested an entirely different tack. Her suggestions were more subtle and didn't tax our systems or AE chops -- and her direction was much more in line with the rest of the production design and didn't interfere with the music. It was definitely a better choice.

Ultimately, when working with both of these artists, we had to constantly remind ourselves that our animations were just one piece of the whole experience. And as much as we love making pretty pictures zoom around on screen, anything detracting from that experience would do the opera a huge disservice.

One of the most pressing technical issues we had to overcome was how to sync our animations to a live symphony.

In the initial design stages, we would create our animations timed to a track from the definitive recording of each opera. Once the artist was happy with our animations, we had to submit them to the orchestra for practice. The orchestra would then rehearse with our animations guided by the stage directors and projectionists.

In the overture of Aida for example, a visual cue had to sync with an aural cue. Fortunately, the conductor for Aida didn't seem bothered by being tied to animation playing behind his orchestra.

This technique worked fine for the shorter animations in Aida, but Madama Butterfly and Fidelio had many long passages of constant animation that covered parts of the performance lacking an orchestral component. Some of these pieces were close to an hour in length, so we needed a new strategy.

Jun Kaneko's interpretation of the tragic romance between American Navy lieutenant B.F. Pinkerton and a beautiful young Japanese woman, Cio-Cio San, plays against a backdrop of bold colour and delicate beauty.

We solved the problem by creating segmented chunks of animation with long, looped endings, each lasting about five minutes. We designed the animations so that the incoming animation's first few seconds would be virtually indistinguishable from the outgoing animation.

The stage director could then follow the live performance closely and cue the projectionist to start dissolving into the next animation at the appropriate time. In this manner, long passages of animation could be seamlessly synced to a live orchestra.

Once the artists approved all of the animations, we had to coordinate with the projectionist to review any problems we saw in rehearsals and devise solutions.

Maybe the colors in a certain passage didn't translate well. Maybe a blending mode that looked great on a small LCD monitor looked atrocious on a fifty-foot screen. The last thing we wanted was for the artist to see something on screen and think, "That's not what I had in mind!"

Often, the projectionist could make a simple adjustment to fix an issue, but there were occasions that required going back and adjusting entire animations.

It should probably go without saying, but these projects required storage and organization of massive amounts of data. Since that data represented months and months of work, we had to make sure it was secure. If a drive went down the night before during a final render, we wouldn't just be able to pull an allnighter to rebuild a lost animation.

We got into the habit of transferring critical files to external hard drives and taking them home overnight. When the project was finally locked, we copied essential files to optical storage and solid-state storage, as well as leaving project files on our in-house RAID array. Other opera companies around the country might purchase the productions in the future, so we had to make sure these animations were well protected and had a long shelf life.

For a shop that lives off of thirty-second spots, this kind of redundancy seemed like overkill, but it was something we felt we couldn't take chances on.

Diving into opera was a completely different, scary, exhilarating experience. The long hours and huge amount of work were all worth it. Opera is a world we had never thought to get into, but we're glad we did.

Right now we're in the middle of animating Jun's third opera production, Mozart's The Magic Flute, for the San Francisco Opera. Set to open in 2012, it's by far the most ambitious production we've ever been involved with. But we're confident that our past experiences will help us contribute to another amazing production.

And to think, it all started with a simple wipe.


Re: Orchestrating a Simple Wipe
by Ragnar Visage Sigrunarson
Interesting article, looks impressing as well what you did.

I´m doing a simular project for a Barok / Opera in Cape Town, Animations done in AE, I wanted to ask, where I´m a bit clueless on that point, what software does the projectionist use to control / sync the animations at the live performances ?

Best regards

Related Articles / Tutorials:
Live Events & Streaming
Pharos by Design Studio 2n Sets Immersive Experiences

Pharos by Design Studio 2n Sets Immersive Experiences

Pharos took place outside Auckland, New Zealand in the world’s largest inflatable dome; hour-long set of stunning visuals created with NVIDIA® Quadro® GPUs powering Epic Games’ Unreal Engine using their new nDisplay cluster rendering technology

People / Interview
Creative Cow
Live & Stage Events
NAB 2012: Snell

NAB 2012: Snell

At NAB 2012, Snell unveiled several new products: Kahuna 360 Compact, a smaller-frame version of the company's Kahuna 360 video switcher that is targeted at the live production market; KudosPro, a format-flexible and cost-effective next-generation image-processing platform; Momentum, an asset management (MAM), workflow automation, and resource planning solution; and a 4-RU version of its Vega routing platform. The company also showcased enhancements to its ICE channel-in-a-box and Morpheus automation systems.

Debra Kaufman
Live & Stage Events
OK Go: This Too Shall Pass

OK Go: This Too Shall Pass

Capturing an unbroken 4-minute take of a Rube Goldberg machine occupying two floors of a 10,000 sq. ft. warehouse, while dodging flying hazards and hoping that the machine actually runs its course, might sound hard. Here's the story of just how hard it was.

Mic Waugh
Live & Stage Events
Multicam Madness!

Multicam Madness!

Madness because everyone wants to do it. Madness because if you start on the wrong foot, you'll drive yourself mad. Here are some of the things that industry leader and multicam expert Mitch Jacobson has learned from the high-pressure, high-stakes world of rock and roll multicam; some of his favorite tools for improving your own multicam experience; and some surprising stories from multicam's origins...with Desi Arnaz and I Love Lucy.

Feature, People / Interview
Mitch Jacobson
Live & Stage Events
Flypacking for the Dalai Lama

Flypacking for the Dalai Lama

Need a single, portable production package for image magnification, broadcast and webcast - all at the same time? Don't panic. Let Todd Gillespie's experience help lead you to inner peace.

Todd Gillespie
Live & Stage Events
Journey - Anyway They Want ItJourney - Anyway They Want It

Three weeks to build 90 minutes of world-class video and animation for a one-of-a-kind performance by one of the world's legendary rock bands.

People / Interview
Bob Bonniol
Recent Articles / Tutorials:
Adobe After Effects
Create an Arrow or Pointer preset for After Effects

Create an Arrow or Pointer preset for After Effects

In this video tutorial, Graham reveals a really quick way to use Shape Layers to make an arrow preset that you can use to point to anything in Adobe After Effects.

Graham Quince
Adobe After Effects
Realistic Water Movement with Fractal Noise

Realistic Water Movement with Fractal Noise

In this tutorial video, Andy Ford shows you how to use the Fractal Noise effect in After Effects to add movement to water in a still photo

Andy Ford
Business & Career Building
How To Succeed in the Business of Video Storytelling

How To Succeed in the Business of Video Storytelling

You may have great storytelling chops, but it doesn’t matter if you can't help your client tell theirs. Nobody knows this better than Rob Shore, who began his filmmaking career in 2005, honing his skills as a creative director with an in-house video team in Washington D.C. before establishing his own video production company, Picture This Productions in 2015. Adobe's Eric Philpott spoke to Rob about the challenges of storytelling when it’s someone else’s story.

Eric Philpott
Adobe After Effects
Adobe Creative Cloud September 2020 Update: Streamlined workflows that make storytelling easier

Adobe Creative Cloud September 2020 Update: Streamlined workflows that make storytelling easier

The current environment is forcing us to rethink and reimagine so much. Content creators, from broadcasters to streaming services to social video creators, are finding new ways of working that prove creativity and resourcefulness are inherent to the video industry. Adobe's Eric Philpott explores Adobe's developments in response to the ever evolving challenges we face today.

Eric Philpott
© 2020 All Rights Reserved