LIBRARY: Tutorials Reviews Interviews Editorials Features Business Authors RSS Feed

The ABC Player App for iPad: Start Here

COW Library : Telestream Episode : Robert Longwell : The ABC Player App for iPad: Start Here
CreativeCOW presents The ABC Player App for iPad: Start Here -- Telestream Episode Feature


Hollywood California USA
CreativeCOW.net. All rights reserved.


2004 might not seem like it was all that long ago, but in the digital video world, it's a really long time ago.

I was hired at ABC.com simply as an Avid editor, and my only job was to cut the occasional "behind the scenes" piece, promo clips or sneak peeks of what was coming. We had a VHS deck capturing Jimmy Kimmel at night, and I'd come in the next day hoping that the timer had worked correctly. We were even ingesting episodes of "Lost" from DVD. At most, we were making maybe 10 videos a week, transcoding them to 50-100 files, tops, and putting them up at ABC.com.

Fast-forward to today: we're creating hundreds of videos weekly made up of short-form and episodes, and creating 200,000 transcoded files a month, mostly from 1080p/23.98 sources -- and not just for ABC.com anymore. (Thankfully I had a few more skills up my sleeve beyond Avid editing.) Now we represent ABC Primetime, ABC Late Night, ABC Daytime, ABC Family, SOAPnet, Oscar. com and ABC episodes for the ABC Player iPad app, too. We also deliver content to Hulu, iTunes, Amazon, Xbox, Playstation, YouTube, AT&T, QuickPlay , Verizon and MobiTV, so that they can further transcode before posting the files on their own sites or devices. We are accountable for making sure that those several hundred thousand files successfully arrive, and are error free.

It's a far cry from 2004.

We built the architecture at the time to be scalable to grow with the business, but to be honest, we could never have anticipated everything that was coming. We didn't know that we were making groundbreaking decisions that would sustain us through today.


FINAL CUT PRO

The first big decision for us was getting off of Avid and going to Final Cut Pro. We outfitted two bays with Final Cut Pro, a couple of Macs and Apple Xsan storage at a relatively small cost. Not only was it a smart financial decision, but also less constrained in terms of video formats, frame dimensions and so on.

Honestly, I haven't looked at Avid since then. Media Composer can probably do all the same things now, but at the time, Final Cut made it far easier to adopt a digital workflow. Today we have more than 10 FCP edit bays.


TELESTREAM EPISODE

The second big decision was using software for Mac called Compression Master, a desktop transcoder that could handle every format in and out natively, made by a Swedish company called Popwire. Popwire also had a program called Compression Engine, which let us submit jobs to a cluster of Apple Xserves.

Telestream has since acquired Popwire and combined these software applications under the name "Episode."

We didn't have a big cluster at the time -- just three Apple G5 Xserves -- but now those two edit bays were dropping QuickTime files into watch folders, and the watch folders were kicking off jobs to the Xserves.

As the business has grown and we've added more jobs to do, we've simply added more servers. Today we have 40 nodes of Telestream Episode, all running on Intel-based Apple Xserves.

Our verification of the transcoded files before we deliver them has been and is still rather basic. We spot check a couple of them just to make sure that they don't have any glaring, obvious errors -- like no audio or no video! We have found that those are almost always user file creation errors or something that got corrupted in the workflow, rather than transcoding errors.


VERIZON MEDIA PLATFORM

In early 2006, we were still only feeding videos to ABC.com, and a handful of mobile partners. We used an Excel spreadsheet to manually track the entire workflow. "If you do a 'Lost' sneak peek, send it to ABC, GoTV, and Helio, but don't send it here, here or here. Start it on this date, end it on this one."

The spreadsheet also described transcoding rules. "If it goes here, it has to be Windows 9 at 320x240. If it goes there, it has to be 640x480 H.264," and so on. It also contained the rules for FTP, including who got emailed when the job was complete.

Seeing how much time we were spending managing this with just three or four partners and hand-writing much of the metadata, we quickly realized that we were going to have to hire a new person or two for every new partner we took on, in order to shepherd all the transcoded files through the processes outlined in the Excel spreadsheet, send the media, metadata, and alert the partners.

That's when we made our third big decision, to use a program made by Verizon Business called the Media Platform. It automates much of what we do, and tracks all of it. As part of delivering 200,000 files every month, we can have hundreds of different ways to deliver the program metadata ALONE! Not the files -- just the metadata. Once you add in the different video formats for each and every partner, you can have thousands of different combinations.

Media Platform shows us everything, using a simple software dashboard. There's a big green check that says, for example, "I know for a fact that I successfully delivered 5 JPEGs, 4 AIFF tracks and an MPEG 2 file, and I sent the XML. I'm good." We also see any failures. If someone's FTP servers go down and they can't accept our delivery, we get a red X telling us to check what's wrong.

It doesn't get much simpler than that.


Episodes for ABC's Wipeout as seen from the iPad app.
Episodes for ABC's Wipeout as seen from the iPad app.


METADATA

Metadata hasn't really changed much for us in the last several years. We don't embed metadata in the assets themselves. In fact, no one has ever asked us to do that...yet. Metadata is always passed in an XML file, and the Verizon Media Platform manages what we call the metadata superset. When the screen pops up and it's time to enter our metadata, there are approximately 60 fields to fill, including actors, genres, keywords, ratings for language or violence, start and end dates, and descriptions of the episode.

We have multiple versions of the descriptions that we deliver to various partners. One partner might want the descriptions at 150 characters, so they subscribe to that version. Another partner might only want 80 character descriptions, so they take that version, and don't bother subscribing to the longer one. Beyond that, everybody's metadata formatting is different. I call it the digital media wild west: you come to our town, you do it our way.

How we convert our metadata into their format is where the magic of one of my administrators, Andy Bauch, comes into play. He has written XSL templates in the Verizon Media Platform that say for example, "Map this field to that field. This field plus that field equals a new field. This field needs to be in all caps."

Then, each partner wants to handle the same information in different ways. For example, iTunes doesn't like "NR" ratings, so if a show has an NR rating in the superset, we turn it into a null value for them. NR stands for "no rating," so if it goes to Amazon, NR gets translated to the word "None."

There are some overarching rules -- like, that the metadata is passed in an XML. But what goes in the XML? It really is different for everyone. Thankfully, we have the metadata superset that we just transform.


Telestream Episode software control panel. Note in upper right: 2033 jobs finished, 54 jobs running, and 2502 jobs queued.
Telestream Episode software control panel. Note in upper right: 2033 jobs finished, 54 jobs running, and 2502 jobs queued.


HD

When iTunes started offering TV shows in October 2005, we were the first ones there. In May 2006, we were the first major network to offer full episodes online, for which we won an Emmy®.

We were still a standard-def shop then, working from DigiBeta. Trying to convince Final Cut to capture in 16x9 anamorphic, and inverse telecine to reject all the extraneous frames to get me back the progressive 24 frame so it could look good in (at the time) Flash VP6 -- it was a disaster.

After a few months of that, we finally said, you know what? Let's just bite the bullet. We can get a D5 deck for less than one third of what it cost a couple of years ago and forget about telecine, anamorphic and everything else. We can capture everything in 1080p/23.98 and just be done with it. Acquisition would be completely native and more or less future proof.

By August of 2006, we also jumped to HD for internal productions, and by the next fall, ABC was the first to stream full episodes in true 720p HD online.

Today, we get most of our programming on both D5 and HDCAM SR, although we also get ProRes HQ files. Both methods have their pros and cons, but tapes still seem to work better. There are only a few ways to make a bad tape, but a million ways to make a bad file. There are so many options for a file, so many places for something to go wrong. You can wind up with eight tracks of mono vs. four tracks of stereo, mixed up apertures between 1920x1080 and 1888x1062, all kinds of messed up metadata -- and it's hard to fix those mistakes once they're there in the file. With tape, there's signal on the tape, or there's NOT signal on the tape. It just works, and it seems to generally be more accurate.

Don't get me wrong -- I want to be tapeless as much as anybody. It's the Holy Grail, but I think we're also getting slightly ahead of ourselves. Until these files feel like I'm moving around a really small MP3 file, and it doesn't have any impact based on the speed of the network, and file size has no bearing because we're working in petabytes of storage, it feels a little premature for the moment to rely on it solely.

In the meantime, we're capturing from tape at 1080p/23.98, directly to ProRes HQ, with our AJA Kona 3 cards.


Verizon Media Platform
Verizon Media Platform


A small section of the Xserve Encoding Cluster
A small section of the Xserve Encoding Cluster


QUANTUM STORNEXT STORAGE MANAGER

Our online SAN today is 230 terabytes. Offline and near-line tape is another 800 terabytes or so at this point, and growing daily.

In the beginning, every time we created a file, that file just sat on a disk. And eventually that disk filled up. The choice was to either get the file off, or keep adding more disks. We kept adding disks until our racks started to fill up, and power and cooling requirements were getting a little crazy. We started moving files to data tapes -- you can get a lot of media on an LT0-4 tape -- but we still need to get to those files quickly.

Quantum StorNext Storage Manager balances our need for immediate access to current files, and easy access to files we don't currently need. It handles this essentially automatically for us. Once a file lands on a disk, Storage Manager says, "If you don't touch that file for the amount of time that you define, I'm just going to move it off the disk."

It only exists on tape, even though the file looks like it's still on the disk. As soon as you touch a file that is not really present, the Storage Manager sends a robot to retrieve the tape, load it, and then copy the files to online storage. After maybe a 10-minute lag, the file is fully active and ready to work with. So when we sign up a new partner, like Amazon, they might say, "We need your entire catalog of 'Lost' and 'Desperate' and 'Grey's,' and oh yeah, we also want 'Schoolhouse Rock'." Everything is easily available, so it's just right-click, transcode, send.


iPAD

From the beginning, our team's choices have focused on the kind of scalability and flexibility that would leave us able to face almost anything, even if there was no way to prepare specifically for any individual challenge in advance.

That's exactly what happened with the iPad. We found out about the iPad at the same time you did. ABC immediately set as the goal to be the first network to stream ad-supported full episodes on the iPad -- but from the time we decided to actually build an app for the iPad's launch was less than five weeks!

I had to re-encode 300-some-odd episodes from the current season of ABC right away, and if I didn't have all those files a right-click away in a near-line tape archive, we wouldn't have made it. There just wouldn't have been time to restore or re-ingest that much data. "It's got to be ready by April 3rd, and you have to have a full catalog of programming. Oh, and by the way, we don't know how to build an iPad app yet, or how to serve ads, and we don't have an iPad for you to look at or test anything with!" That's how it got started.

We had 12 in-house software engineers working on the app itself, and they didn't have an iPad either. We were all working in a software emulator. I had some video specs, and I was able to go by the experience I had transcoding for other Apple devices. After that, I hoped for the best. You can emulate as much as you can, but until you have the iPad in your hands, you don't know what you're going to get.

Launch day, April 3, was a Saturday. Some of us had pre-ordered our iPads, so we all got on the phone together early that morning. We'd hear, "Hey, the UPS truck just showed up," and listen as each of us got to crack open their iPad for the first time, and download the ABC Player app from iTunes.

We were on the West Coast, but wanted to be together when people on the East Coast were picking up their iPads from the stores. If we heard any reports of things not working, we wanted to get on those issues right away. I don't personally use Twitter, but that's what everyone else does, so we watched what people had to say. Almost immediately, they started tweeting. "Hey, this works," and a few others said, "This doesn't work."

When people would say, "Every time I do this or this, the ABC app crashes," we actually reached out to a few of them directly. We quickly discovered a memory leak, and by noon on that same Saturday, we had it fixed, and submitted to the iTunes apps store for approval. It was released by that night. In fact, we fixed most of the problems we'd seen on Twitter by the end of the first day.

Amazingly, the response we got was overwhelmingly positive, which is something we don't normally get in a typical feedback scenario. We have feedback buttons on ABC.com and the other websites, and in the iPad app itself, but most of that feedback we get is negative. People use those feedback mechanisms more to submit bugs than to offer praise. But following Twitter gave us a really great feeling. Ninety-eight percent of what we saw was, "It works! The video looks fantastic! I can't believe that they can do this on an app!"

And, "What's taking the other networks so long?"

All of this was on Saturday. We loved it!


ABC.com

One of several views of the "ABC News for iPad" app, choosing "Archives" from a drop-down menu. The globe can be spun by touching, and stories can be randomized by shaking. News categories and stories are also available in list view.
One of several views of the "ABC News for iPad" app, choosing "Archives" from a drop-down menu.
The globe can be spun by touching, and stories can be randomized by shaking.
News categories and stories are also available in list view.



ABC iPad app
ABC iPad app


ABC: DIY

I don't know much about how it works at other networks, but Disney/ABC really thrives on having a technology team in-house.

If you use external vendors for everything, you learn nothing. You learn how to write a check, but you haven't actually acquired any of the knowledge. As a result, you're not ready for the next challenge, and there's always another challenge.

Five weeks to build an app with that much content is a tight schedule, no matter how you slice it. But we built it on the foundation of what we have already built for ABC.com and every other partner. We leveraged everything we already know, and leveraged the content already in our archives. We just had to figure out the new video format, how to transform the metadata superset in a new way, and how to get it all out the door.


ABC tapedecks
ABC tapedecks


THE FUTURE

The thing is, I don't know what's coming. I don't know what the next device is. I don't know what the next set top box is. I know what you know.

Maybe less! I used to surf the web all the time. "Woo-hoo! There's this new toy coming out! It would sure be fun to get our stuff playing on that." But now we're so focused on making sure that the infrastructure is running, and that it's solid, and that it's ready for the next big thing, that I don't even know what the next big thing is anymore. As it is, I get the phone call telling me what the next thing is, and that I have to have it live in a few weeks!

Even if we were aiming for a single, specific "next big thing" further in the future, it might not actually be the next big thing. All you can do is make choices about flexibility, scalability, and accessibility -- and have talented staff that put yourself in the best possible position for whatever comes next.


 


 

Robert Longwell, Creative COW Magazine

Robert Longwell
Los Angeles, California USA


Robert Longwell is Director of Video Operations, Digital Media, for the Disney/ABC Television Group. He oversees all aspects of Digital Media's video workflow and has built the process from the ground up. This includes HD & SD video asset management, ingest, post-production, metadata creation, video transcoding, file delivery, video workflow tools and automated asset archival and restoration. Before joining ABC in 2004, Robert was a linear and Avid editor for 12 years. You can find him hosting in Creative COW's Telestream Episode forum.







Comments

Re: The ABC Player App for iPad: Start Here
by Robert Moore
Thank you for the Great article! I have worked with (many years as Producer/Editor & Engineer) and now am working with (Engineer) all the tech you mention (FCP, Avid, Xsan, Episode...), except VMP.

Verizon Media Platform sounds interesting. Did you look at Final Cut Server? (I know it's not too complete, but it is good, right?) I'll be looking into VMP.

I work at a company that provides a very wide range of services, from Production (ABC/Disney a specific daytime soap, HD Tapeless/Avid...), GSN, MGM HD and others) to media transcode and Cable Origination (many, many channels) and Post (FCP and Avid.... for the most part, we're all tapeless, but we do have every tape format on the planet here, to accommodate ABC/Disney (D5, D2, DigiBeta, Beta...), MGM HD (D5) and the others.

For future articles (to Creative Cow, maybe the Cow doesn't allow this?), is it possible to include a link to apps mentioned in the article? Like the ABC iPad app, it would be great to link to the Apple Store. For Verizon, it would be great to find a link in the article to "Verizon Media Platform", which is great information, I didn't know Verizon offered that (I'll be sending an email to the person you mention in a reply).

Is there ever an open house at your facility? I'd be interested in seeing first hand.

Thanks Again,
RMoore
Encompass Digital Media
Los Angeles, CA
Re: ABC.com: Start Here
by Sacha Thomas
Thanks for sharing your thoughts! I like that you brought it back to three principles-flexibility, scalability, and accessibility. Nicely done.
Re: ABC.com: Start Here
by David Komer
Hi Robert,

Great article!

I tried looking for Verizon Media Manager and couldn't find it on their website anywhere... any chance someone can steer me in the right direction?

Thanks!

http://omnivibe.com
Re: ABC.com: Start Here
by Robert Longwell
Your best bet is to contact George Richards at Verizon Business. He can be reached at george.richards@verizonbusiness.com. Mention the article, he'll know what you're talking about.

Robert Longwell
Re: ABC.com: Start Here
by David Komer
Thanks :)

http://omnivibe.com
Re: ABC.com: Start Here
by Mike Cohen
I was intrigued that Verizon has all these different business services having little to do with telephones!

I was relating your article to a co-worker this week, since we are talking about devleoping iPad apps and/or videos for this platform. iPads are becoming very popular in the medical world.

Mike Cohen
Re: ABC.com: Start Here
by Richard van den Boogaard
Great read! Thanks for providing this insight... it's great to see how you and your team really have grown with the flow of things!

Two question:
- for which purposes could you imagine integration of metadata into the media container itself instead of an accompanying XML? Mobile use, perhaps?
- Do you also actively use YouTube's partner program back-end for tracking/monetizing ABC copyrighted content?

Richard van den Boogaard
cameraman / editor / video marketing consultant

Branded Channels
W: http://www.brandedchannels.com
Re: ABC.com: Start Here
by Robert Longwell
- for which purposes could you imagine integration of metadata into the media container itself instead of an accompanying XML? Mobile use, perhaps?

Embedding the metadata would only be necessary if the device actually required it. For example cable set top boxes traditionally require the CC metadata embedded in the video stream as that is the only way for a TV to decode. So far all online partners are not in this scenario, they accept the metadata delivered separately. As more TVs get plugged-in, I suspect we'll need to find a way to embed at that point.

- Do you also actively use YouTube's partner program back-end for tracking/monetizing ABC copyrighted content?

Admittedly that's more of a business question that I'm not particularly involved in. I deliver to YouTube and that's where my involvement ends.

Robert Longwell
Re: ABC.com: Start Here
by Andrew Kimery
Great piece. Going from capturing VHS to kicking out hundreds of thousands of videos a month in span of about six years is amazing.


-Andrew

3.2GHz 8-core, FCP 6.0.4, 10.5.5
Blackmagic Multibridge Eclipse (6.8.1)

Re: Article: ABC.com: Start Here
by Mike Cohen
Robert - fascinating article. Good to know that ABC/Disney has had to figure stuff out as they go just like the rest of us.

Mike Cohen
Re: Article: ABC.com: Start Here
by Robert Longwell
What can I say, it's been a fun ride and it's far from over!

Robert Longwell
Re: Article: ABC.com: Start Here
by Bob Zelin
I agree - this article was amazing. You started as an AVID editor, and all of a sudden, you have to worry about digital distribution, countless formats, and become an expert in Telestream Episode.
Very inspirational article.

Bob Zelin

Re: Article: ABC.com: Start Here
by Robert Longwell
Yes I was hired solely to Avid edit. By heart, and by trade I suppose, I was more of a computer/video geek for lack of a better term (I will never say "expert" because I don't and never will know it all).

It was sheer fortune that ABC recognized I had far more skills than for what I was originally hired. We've both grown together.

Robert Longwell
Re: Article: ABC.com: Start Here
by Bob Zelin
none of us know it all. I was the original AVID tech in NY back in the stone age - I knew nothing about it then, and I never really was an expert, even though I fooled people into thinking I was all those years ! Today, I fool people into thinking that I know everything about shared storage.

Bob Zelin

Re: Article: ABC.com: Start Here
by Nathan Veer
Great article. Thanks for candidly sharing your experience. I think many of us are in similar situations if not on the same scale. We are currently looking at switching from Apple's Compressor as our house encoder over to Episode 6 with Engine. Your article rekindled my interest in Episode. Someday I would love to come see your facility and learn.


Related Articles / Tutorials:
Telestream Episode
Telestream Episode 6.2 Opens New Doors in Encoding

Telestream Episode 6.2 Opens New Doors in Encoding

Telestream recently announced the release of Episode 6.2, a new version of the multi-format video encoding software. Overall, the improvements maximize Episode system integration and increase responsiveness and stability, positively impacting high-volume video transcoding. Read on for more details.

Feature
Debra Kaufman
Telestream Episode
Telestream Episode Pro

Telestream Episode Pro

Recently, Creative COW leader and business owner Walter Biscardi needed a very flexible and fast compression tool that also provided extremely high quality end products for his clients. Oh, and it needed to be easy to use. Enter Telestream's Episode and Episode Pro. In this review, he's using Episode Pro 5.1.1 and he'll explain the differences between the two at the end of this article.

Review
Walter Biscardi
Recent Articles / Tutorials:
Art of the Edit
Unearthing Apollo 11 in Large Format: An Interview with Director Todd Douglas Miller

Unearthing Apollo 11 in Large Format: An Interview with Director Todd Douglas Miller

Director-Editor Todd Douglas Miller creates more than just another space documentary with his film Apollo 11. He has helped expand the horizons of what we thought we knew about one of humankind's signature achievements by unearthing nearly 300 reels of previously unseen large-format film, up to 65 and 70mm, digitized with a first-of-its-kind 8K scanner, along with 11,000 hours of previously unheard audio recordings to provide previously only-imagined perspectives on our first mission to the moon. Creative COW's Courtney Lewis reveals how he put it all together with NASA, the National Archives, post house Final Frame, and tools from Adobe.


Courtney Lewis
Apple Motion
Apple Motion 5: EPIC Title Tutorial by Simon Ubsdell

Apple Motion 5: EPIC Title Tutorial by Simon Ubsdell

Join longtime VFX artist, editor, software developer, and business owner Simon Ubsdell for some really useful and interesting techniques for creating a fake 3D title treatment with dramatic depth, texture, reflections, staining and other details that goes way beyond what you can achieve with the 3D Title Tool. Not only that, these techniques work well with any graphic object, for maximum motion graphics flexibility.


Simon Ubsdell
TV & Movie Appreciation
Designing Tarantino's Once Upon A Time in Hollywood

Designing Tarantino's Once Upon A Time in Hollywood

Barbara Ling, production designer for Once Upon a Time in Hollywood, discusses how she was able to turn back time and recreate 1960’s Hollywood. Barbara and Go Creative Show host Ben Consoli discuss how Quentin Tarantino kept the film’s script a secret, how she restored Hollywood to the 1960’s, not using green screen, sourcing vintage props from eBay, filming the real Playboy Mansion, the challenges of filming on Hollywood Boulevard, and much more.


Ben Consoli
Blackmagic Design Announced WHAT?

Blackmagic Design Announced WHAT?

As if this wasn't already the hottest summer on record, Blackmagic Design's Grant Petty rolled out some blazing news across a wide range of the company's product lines in cameras, production hardware, and software.


Creative COW
Apple Motion
Apple Motion 5: Light Art Tutorial

Apple Motion 5: Light Art Tutorial

Discover how to create this moody light art scene with Apple Motion 5! Join longtime VFX artist, editor, software developer, and business owner Simon Ubsdell for a tutorial that also includes number of useful tricks, including how to create an ambient occlusion effect and textured specular highlights.


Simon Ubsdell
Cinematography
The Lion King's Virtual Cinematography: Caleb Deschanel, ASC

The Lion King's Virtual Cinematography: Caleb Deschanel, ASC

Caleb Deschanel, cinematographer for Disney’s live-action The Lion King, shares how they used traditional cinematography to create the life-like virtual film. Caleb and Go Creative Show host, Ben Consoli, discuss modeling cameras and lenses for virtual filmmaking, how Caleb was able to move the sun around in virtual space to get the perfect lighting, using a real drone for the Circle of Life sequence, and more!


Ben Consoli
Adobe After Effects
Adobe After Effects Reverse Stabilization

Adobe After Effects Reverse Stabilization

You're going to be blown away by how you can power up your After Effects workflow with reverse stabilizing your footage! By separating your tracking from your compositing, you can focus on each step, and in addition, overcome the render order complexities when match moving elements and effects on a moving shot.


Roei Tzoref
Makin’ Planets! Adobe After Effects Tutorials by Graham Quince

Makin’ Planets! Adobe After Effects Tutorials by Graham Quince

Here's a great new Adobe After Effects tutorial series from Creative COW leader Graham Quince that takes advantage of the free Orb plug-in from our friend Andrew Kramer at Video Copilot. Graham supplies project files, tips and tricks, and good humor on this tour through building every planet in the solar system, with customization options galore!


Creative COW
MORE
© 2019 CreativeCOW.net All Rights Reserved
[TOP]