An animated show that completes an entire episode, from writing to air, in only six days? Delivered to the network with only minutes to spare? Meet the producers and editors of South Park, winner of 3 Emmys, a Peabody Award for Excellence and many more, as they take you inside the tools and technologies enabling one of the most pressure-packed workflows imaginable.
Each episode of "South Park" comes to life in 6 days, start to finish -- a crazy pace for an animated series
Even crazier: each episode arrives at Comedy Central in New York via uplink somewhere between 6:30 and 8:30 PM on Wednesday, to be shown that night at 10 PM.
Cutting it close? Even a production as relatively simple as a late-night talk show, where delivery might entail no more than walking across the same building it's taped in, leaves more like five hours to air than three.
And because work proceeds on each episode until the very last minute, an awful lot of things have to go right, in very short order, with virtually no margin for error.
It's not just that the script has to be completed in time for voice recording, creating and rendering animation (including lip sync), color correction, visual effects, scoring and audio post. It's that the script keeps changing to respond to the world's most current events, as well as the perfectionism of the show's creators. Which means that everything downstream from the script keeps changing too.
Their perfectionism is paying off. Now in its twelfth season, "South Park" has been nominated for 7 Emmy Awards for Best Animated Program, winning in 2005 and 2006 -- and just as this issue was wrapping, the three-parter "Imaginationland" was awarded the 2007 Emmy for "Outstanding Animated Program (For Programming One Hour Or More)." South Park has also won a GLAAD Award, an NAACP Image Award, a CableACE Award, and the prestigious Peabody Award among others.
Not bad for a cartoon show about four foul-mouthed boys intended to look like it was animated from construction paper cut-outs.
Workflow at South Park Studios has evolved with the sole purpose of giving creators and executive producers Matt Stone (below, left) and Trey Parker (below, right) the room to write, direct, add additional music to the work of South Park's composers, and if called for, to write songs, in such little time.
(Before attending the University of Colorado where he met Matt, Trey studied at Boston's Berklee College of Music. His song "Blame Canada," co-written with Marc Shaiman for the movie "South Park: Bigger, Longer, Uncut" was nominated for an Academy Award for Best Song.)
The two also provide voices for most of the show's male characters.
Supervising Producer Frank Agnone is the keeper of the workflow for the sixty people who put the show together. From the time that pages for the next episode arrive early in the morning after the previous one airs, he makes sure that a lot of things happen simultaneously.
The basics are traditional enough: those early pages are recorded, the dialog is cut up and passed to storyboard artists. Editors start building animatics by cutting together the storyboards and dialog to start shaping the scene.
Of course, even after the first pages arrive and scene construction begins, nobody necessarily knows where in the show it's going to end up. The script evolves as the week progresses: the final script is generally in place around 2 AM Wednesday, about 12 hours before the show is uploaded for air.
Until then, the work carries on, scene by scene. As editor David List (below) notes, "It doesn't matter so much to us whether a scene is at the beginning or end, as far as editorial is concerned. The challenge is more for Matt and Trey as they build the story structure. For us, it's basically cutting. We know from the beginning that there will be changes as we go, but we've been doing this for so long that we know how to keep moving."
At the same time that Frank is working with Trey to refine individual layouts, 3D modeling begins. Those shots move very rapidly through lip sync and into the hands of the animators.
"So it's an ever changing formula," says Frank, "but ultimately it's my responsibility to make sure that the team of sixty people is staying on schedule and we are hitting our deadlines for Trey, hitting our video deadlines so that color correction can happen on time, and then our audio deadlines so that we're making broadcasts on time."
"TWO DAYS BEFORE THE DAY AFTER TOMORROW"
Things get especially hairy on the last day: a 30-hour stretch that begins early Tuesday morning on the way to completion by Wednesday afternoon.
Picture lock comes in for a landing at 9 AM Wednesday morning, when the video is recorded to tape and sent out for final color correction. At the same time, the final clean-up and reconform from last-minute edits gets sent through audio post one last time.
"Then the scramble is on for those guys to make sure all of the proper dialog and ADR work is in place," says Frank. "Sound design for shots that are coming in until that 9 AM hour on Wednesday morning are also attended to, and then a mix begins."
The mix comes together just as the color graded picture comes back at 1 PM Pacific, when Frank pulls the plug on any additional work. "If I'm lucky, I'm out the door by 2 PM, sometimes 2:30, to our uplink facility to start fibering the episode to New York. On average the show arrives on the east coast between 6:30 and 7 PM [Eastern], and it's on the air at 10 PM. "We had a couple of shows in this last run where I was getting the show there as close as 8:30 for a 10 o'clock broadcast."
South Park Supervising Producer Frank Agnone
It's difficult enough to keep track of all this as it's being described that it raises the obvious question: how does Frank track all of this as it's actually happening? He walks the floor to stay in touch with his key people, but "to be honest with you," he says, "I probably do 80% of it by memory."
"GOOD TIMES WITH WEAPONS"
Part of Technology Supervisor J.J. Franzen's job is keep looking for technologies to enable ever-increasing production values in the fixed six-day cycle.
"We have a very fast blade-based network switch in place now, replacing a bunch of standard switches," J.J. says. "We also have a BlueArc Titan, a very robust, very specific, hardware-accelerated file serving system. All it does is serve files as bloody fast as possible." "Fast" means 2GB/sec., over 8 4Gb fibre ports and 2 10GbE clustering ports.
Those files come from a LOT of storage: 26 TB of production storage, 33 TB of nearline storage, and 15 TB of Avid Unity shared storage. Below, JJ with some of that storage.
They've recently upped the number of processors on the render farm from 120 to 320, housed in forty 8-core OS-X Apple Xserves. "Our work stations are Apple-based so we can farm out rendering, compositing, effects, compression. Basically anything that artists can do, they can now do on the farm."
The original "South Park" infrastructure was SGI IRIX. Even after transitioning to Windows on the desktop, the render farm ran under Linux. That's obviously changed with wall-to-wall Macs.
"There are pluses and minuses for having a homogeneous computing environment -- doing everything anywhere is definitely one of the pluses. Since I'm an old IRIX/Unix head, the fact that Macs now bring that level of functionality to me while also bringing one of the best end-user experiences for my artists made the switch a win-win scenario."
Increased processing power has "helped us up our game," J.J. says, including deeper textures on characters, fluid and particle effects -- some of which are also added with Apple Motion -- and more sophisticated scene lighting. "It helps the overall look of the show and also helps Trey. He is definitely thinking more cinematically now than he used to in the past, because he knows what he can pull off."
A small example of more sophisticated rendering, far beyond the look of construction-paper cutouts
One of J.J.'s next tasks is to find a new renderer for Maya. Current candidates include Renderman and Mental Ray. "Some of the renderers out there do an amazing job with photo realism, the kind of thing that makes you say 'wow.' But the thing is, we don't want photo realism. We want something that'll look exactly the same as we've already got, but give us flexibilities to go further."
And faster. Although Comedy Central won't be HD until January 2009, "South Park" episodes are already being produced in HD. An even bigger task is going back to re-render the first 174 episodes!
"We've always been digital pack rats, so we still have all the Maya (and before season 5, Alias PowerAnimator) scene files we used to create the show," says J.J. "Since HD has become a real possibility, we've started re-rendering all those old episodes at full 1080p, which also means re-framing all the shots from standard 4:3 to full 16:9.
"It's very labor intensive, but it also means that we'll be the only animated show from the pre-HD world that will have its entire catalog of episodes in full native 1080p. It's pretty sweet."
"FOLLOW THAT EGG!"
New ideas from Matt and Trey are constantly coming in, and the process of tightening and refining each scene goes on until the very last minute. "Basically every time Trey walks away from the Avid, we have another version of the show," says J.J.
Depending on the nature of those changes, they can ripple back through the entire pipeline: voices need to be re-recorded for new animation plus lip-sync, the new footage added back into the edit, and most critically because they get the new scenes last, new audio post.
With production deadlines so extremely tight, the audio post team doesn't have the luxury of waiting until an entire episode is complete before beginning their work. The disaster to avoid is having them work like demons to finish scenes that have substantially changed, or worse, are no longer in the show at all.
The additional challenge is, once a scene has been changed, getting it into audio post as quickly as possible. Shared storage would seem to be the easiest alternative -- but it's simply not an option: the Avid Media Composer and Digidesign Pro Tools systems they use don't work together on Avid Unity storage.
David describes the process this led to. "First, I asked our lead editor, Keef Bartkus, if he could stop what he was doing to allow me to have a few minutes to update the current sequence. After copying the sequence, I'd throw down a dummy track for audio to reference new edits. Then I would initiate the conform, basically consolidating media into an OMF and then copying the media (upwards of 8 or 9 GB) onto as many as 3 external FireWire drives."
With five to ten new cuts of an episode on Tuesday night alone, the copying process was sucking down 8 to 12 hours when the team could least afford it.
They've recently turned to StorageDNA 360, a software bridge for data distribution and synchronization across multiple storage systems. All the video media is constantly being pulled across from the Unity to local hard drives for the Pro Tools stations. When a cut is completed on the picture side, the media is on the drives for audio post, fully synced and ready to go in closer to 2 minutes than the previous 30-45 minutes for each cut.
J.J. says this means that "Matt and Trey can throw out a random idea and we can say 'Sure, we'll give it a shot,' because we feel relatively confident that we can turn anything around in a certain amount of time."
While distributing to local drives, the StorageDNA 360 is also passing everything to a nearline archive, freeing up space on the primary storage that would have been tied up by mirroring. They're also automatically preserving every version of the episode remotely.
"It gives us a disaster recovery scenario if the Unity were to die, allowing the editors to get back to work with a minimum of downtime," says J.J. "With the timeframes we work under, every safety net counts."
The Emmy Award-winning "Imaginationland" was an exception to the six-day pace: the idea came from the summer before it aired. "They called in the whole crew for about 6 weeks and we just worked on random stuff that the lads had come up with, basically developing concepts for episodes in advance of the run," says J.J. "'Imaginationland' was one of those concepts.
The shortest version of the concept is that a battle between all of the good and evil characters ever imagined spills over into the "real" world. Featured players run from Aslan to Zeus, and include Charlie Brown, both a Predator and an Alien, Al Gore, Gandalf, Luke Skywalker, the Blue Meanies, Michael Bay and Strawberry Shortcake.
And that's a really, really short summary. There's something in there about nuclear weapons, too.
"The number of elements that we had to design from scratch to produce these episodes was enormous," says Frank. "I think we approached about 120 hours a week, each of those three weeks, in order to get that show put on the air."hours a week, each of those three weeks, in order to get that show put on the air."
An uncensored director's cut of "Imaginationland," along with previously unseen footage is available on DVD and at SouthParkStudios.com, which includes the original episodes. You'll quickly see why the story was spread across three episodes, why it won an Emmy, and, frankly, why the episodes were originally aired with bleeps aplenty.
Another scene from "Imaginationland"
"YOU KNOW, I LEARNED SOMETHING TODAY"
There's plenty in every episode to be offended by beyond the language.
"South Park" takes aim at every religion, and atheists to boot. No politician or political view is exempt. The internet, rain forests, video games, AARP, aliens from outer space, racism, tolerance and head lice all get their turns in the cross-hairs too, as do Hollywood celebrities of every sort.
Not that "South Park" is so easily categorized as anti-everything. Touching moments sneak up on you, such as the obvious sympathy for Britney Spears' exploitation in "Britney's New Look" from Season 12, which also observes that the distinction between the world of tabloids and "real" news is barely worth talking about anymore.
Actually, theirs is satire at its very best: outrageousness that doesn't quite mask its humanity and its emotional commitment to these issues, demanding the same of its audience. Which is exactly why nobody gets off the hook.
"That's sort of the genius of Trey and Matt," says David. "It's just really smart, smart, smart humor, and it's a win-win situation for wherever you stand politically. I think that's the beauty of the show, and I think that's why it's been around so long."
It's clear speaking to everyone involved that they're proud to be part of a show that ultimately means something. Not that they have much time to enjoy it.
"It's gratifying on a Wednesday night, when emails and text messages and phone calls start pouring in. 'Oh my goodness, can't believe you got away with it, how do you guys do it, amazing,'" says Frank. "They start flooding in after a 33 hour day that begins on Tuesday morning, so it's gratifying for a minute and half, and then we move on to the next episode, just like that."
(Frank's "moving on" includes supervising syndication, international versioning, the HD transition -- including the back library -- and the exceptionally extensive South Park Studios website, a treasure-trove of clips, full episodes, behind the scenes footage, and much more.)
How can anybody keep up such an intense production cycle? The answer is, they can't.
Each season is broken up into two 7-episode parts, with the second half of the twelfth season just begun as you read this. With a month or so of work on either side, that's just under half the year off.
Still, as David says, the team enjoys the show so much that they're excited to get to work on the next episode as quickly as possible. "I think if you sat everybody down," says Frank, "they'd all say that we're lucky to be a part of such a wonderful production that has had not only success, but longevity, which is such a rarity in this industry."
It's crazy, but it's clearly working.
Tim Wilson is the Managing Editor of Creative COW Magazine. This article represents an extremely rare exception to the magazine's core trait. Rather than general-purpose authors writing about productions, most of the Cow's authors are people doing the production: nearly 50 different authors in the past year, all describing their own work.
At a time when many print publications are floundering, the COW Magazine has added well over 1,000 subscribers in the past week alone. "We believe that the key to keeping a print magazine going is making a magazine that people want to read," says Tim.
He also notes: ""I'd have finished this article a whole lot faster if I hadn't kept stopping to watch more episodes of 'South Park."
Exhibit A: the article title and each of the section headings is taken from the title of an episode of "South Park." Pathetic.