LIBRARY: Tutorials Reviews Interviews Editorials Features Business Authors RSS Feed

Avoiding Digital Delivery Disasters

COW Library : Broadcasting : Bob Zelin : Avoiding Digital Delivery Disasters
CreativeCOW presents Avoiding Digital Delivery Disasters -- Broadcasting Editorial


Orlando Florida USA
CreativeCOW.net. All rights reserved.


Editor's Note: We add new forums at CreativeCOW.net as our members tell us they need them. Among the most frequent requests of late, and our newest forum: Digital Delivery. The transition from sending tapes to sending files to clients and networks is rapidly accelerating. Needless to say, there's nothing even vaguely resembling standards for this. Rather than ask one person to provide a comprehensive overview, we have asked the irrepressible Bob Zelin, Contributing Editor, to introduce the lay of the land. To further illuminate, we offer a small slice of the many conversations already springing up in the new Digital Delivery forum.

Most of those posts have been gently edited from their original online appearance for length, and for reading outside their original threads.

Having dealt with videotape formats for all these years, I never thought about what happens to an HDCAM tape, or DigiBeta tape, once it leaves a facility. But I was recently asked to build a small TV station. It's just video equipment, I thought. It would be the same stuff as always. Boy, was I wrong! Now I had to start understanding how to get these tapes -- and the output of a video switcher that was creating a daily news show -- onto an on-air broadcast server.

All of a sudden, MPEG-2 became very important, as most professional on-air servers use this format. So instead of being concerned about getting the best image quality from a Final Cut Pro or an Avid system, all that matters now is, how the heck do I get anything -- and I mean anything -- into the specific flavor of MPEG-2 that this particular on-air server wants to see?

Faced with all of this new equipment, which included a Broadcast Pix video switcher, and several Grass Valley editing systems, the only end result that mattered was that stinking on-air playback server.

So, I started to ask around, what is everyone else doing? How are files being delivered?

It turned out that many of the "big boys" were using products like Telestream FlipFactory to take any sort of file format and convert it to MPEG-2. I spoke with my local cable company about their requirements for digital delivery, and they, too, used FlipFactory, along with another product from Telestream called AdManager to automate this process.

All of a sudden, I started to understand why people bought Sony IMX VTRs that were native MPEG-2 file format. All of a sudden, I understood why products from Telestream, Anystream or Digital Rapids were so important -- because there need to be quick, easy, and efficient ways to transcode from the input files to the required file formats.

File conversion was only half the battle. Speed is the other concern. Normal broadband Internet connections may seem fast when you are looking at YouTube, but are a joke when you are trying to send a 2 GB file to or from an FTP server on the web.

To send files from one place to another, I purchased a little product called Pogoplug, a very inexpensive FTP (File Transfer Protocol) appliance that you plug a USB2 drive into -- and now you have your own FTP site on the web. That's it. I did a little test with a text file, and it all seemed so easy.

Then we tried to do a real test: send a 200MB MPEG-2 file across the internet. All of a sudden, reality kicked in. It took about 30 minutes to accomplish this. We tried similar techniques, like Dropbox, which is a free service, and YouSendIt, free for files up to 100MB, and $9.99/month for files up to 2GB. But the reality was that if you had to send an entire show, like a two or three gig file, it was going to take a long time.

You soon see that Internet connection services like T1 are not that wonderful when you are sending or receiving massive files. We have a dedicated DS-3 line from the phone company that is used to receive a live TV network feed in H.264 from Miami to Orlando. DS-3 is 45Mb/sec -- perfect, but guess what -- it's $3000 a month. Not only is this not realistic for most small companies, it's not even realistic for this station for "secondary" feeds, like receiving completed shows or commercials.

For now, commercials and graphics will use a slow, cheap FTP internet service, but TV shows must still be delivered via Federal Express.

There ain't no free lunch. Not yet -- not to my knowledge. Until "the rest of us," without expensive MPEG VTRs, know how to deliver the exact file format that these stations and cable companies want, it's still going to be a long wait.

-- Bob Zelin


Photo courtesy of Kamyar Adl
Photo courtesy of Kamyar Adl


Re: Following up re: digital delivery Rich Rubasch 6:34:26 pm

Here are some options to consider.

Internal FTP site. We set up a Buffalo Technology TeraStation NAS server to hold about 1 TB of SD promos for BBC and American Public Television. It is local at our facility, so getting files to it is a snap. From the outside, we allow anonymous access via any web browser and set it up to be accessed via port 9000 or FTP port 21 with simple web redirects: they type in our web url with a directory name and it takes them to the file list.

So far it has been working swimmingly. We only had to increase our internet upload speeds at the shop to 4 Mb to meet the bandwidth requirements.


Station's FTP site. I have FTP folders on two network and one cable provider's sites. I upload files to their directories with files to their specs (HD or SD) and they retrieve them. This system is ok, but not bulletproof. I have to make sure they got the file and have to make sure they QC the darn thing.


DG Fastchannel. It's DG FastChannel for me.

This digital delivery service is actually much less work than using the direct via FTP method. Preparing four different timelines and then encoding four different files and then naming each one differently, keep track of them all and upload to four different FTP sites, then track with the station that they got them and that they play on their server -- bah, humbug!

What's nice about DG is that I only need to encode one file and send it to many stations. Using Telestream Episode to encode to their unique MPEG-2 specifications, I can get a spot from my editing timeline to 20 or more stations in less than 1/2 hour. I charge $30-40 for each station, so you do the math.

DG FastChannel gives me most peace of mind. Clients love getting the order from DG stating that everything has been delivered. It is like a FedEx tracking number.

The quality is fine and their service has improved over time. Big plus: stations love them.


One way of delivering a product.
One way of delivering a product.


Re: Following up re: digital delivery -- Chris Blair 5:29:21 pm

Last year we uploaded hundreds of media files to a client in the Middle East, including the contents of DVDs that were anywhere from 1-2GB. We tried several digital delivery services, and we found that the upload speed was almost wholly dependent on our connection speed. They were no faster (in our experience) than using straight FTP software and connecting to a server somewhere.

We use a professional level FTP software application (in our case CuteFTP Pro). We encode the spots to the specs of the particular station or cable system (usually MPEG-2 or QuickTime H.264, but also a few other formats/codecs). Then we log onto the station or cable system using FTP addresses, user names and passwords they provide. Next, we just drag and drop the file to the specific directory they've given us (which can be navigated to automatically by the FTP software).

Now if all that sounds simple, IT'S NOT. We end up going round and round with stations and cable systems about finding the files, even though they're uploaded to the SAME place time after time.

I will say that account execs LOVE this process because they literally don't have to do anything -- except call and email you looking for their files, even though they've been uploaded to the same directory, on the same server, for the 35th time this year.


Delivery questions to ponder -- Michael Klinger 9:21:14 am

Over those years, we've become steadily more ambitious with uploads and downloads. At first, it was just a few files from a voice record, then more and more and more, pushing our luck along the way. We are now routinely moving finished or reference QuickTime or other media around as large as 4GB.

Even so, many variables are out of our control, including maybe the most difficult: the client's perception of what is a realistic experience to deliver this way.

Case in point: A couple of nights ago, we uploaded a 4 gig file to London from Los Angeles. It had to be done in one of the short list of required high-quality file types (including uncompressed). Based on their list and relative quality consideration we chose the least of the evils, and went with ProRes HD 1080.

Like others here have experienced, it had to be loaded to their site and the company receiving the file wasn't willing to come and get it from ours. Even though we have a decent connection and the receiving end had a decent connection, we weren't getting even 1/10th of our potential upload speed. It ended up as a 7+ hour upload -- even though there was no other activity in our office at the time.

No harm done, I suppose in the middle of the night with no tight deadline, but of course, my client didn't anticipate this. Their only point of reference is what it is like to load a YouTube video and they want to know what is wrong with us, why would it take 7 hours?


Re: speed of free "FTP" sites -- Erik Freid 7:21:27 am

There are companies like Media Silo that host a UDP server on the cloud that send data faster than FTP, via a Java applet that opens in a browser.

First, understand the difference between UDP and FTP. FTP basically sends out a small packet of data, waits for verification from the server, then sends more. It is relatively slow to do, and there is a file size limitation of around 2GB -- unless the ftp server is running 64-bit, and then you can be pretty stable to 4GB before it fails.

UDP is faster because it sends out larger packets to the other side, and does not verify packet receipt. It just sends and sends. Companies like Aspera, FileCatalyst, etc., use a secondary TCP stream (from the hardware server to another server, or from the Java applet running in the browser) to verify delivery, which does not slow down the transfer.

All that said, you cannot go faster than your bandwidth. FTP maxes out at about 1.5 Mbit/s, regardless if you have more bandwidth. Depending on the controls set on the software and license purchased, UDP can go up to 50-100 Mbit/s.

I have seen a transfer freeze an entire network because it took up all the available bandwidth. Everyone complained that their internet was unusably slow, but they were getting 80 Mb/s on the UDP transfer: 1GB every two minutes.

Most of the new systems let you set a "speed limit" on the transfer to avoid this flooding of the gates, so to speak.


Delivery in the 'good old days'.
Delivery in the "good old days".


Re: Following up re: digital delivery -- Mark Grossardt 7:25:24 am

We've only just begun using Extreme Reach, and so far, I've been impressed. They were founded by the guys who started DG FastChannel, so I get the impression that ER is the new and improved version of DG. They take a high quality MPEG-2 or ProRes HQ master file, and encode it specifically for each station's playback server, so that automated re-encoding by the stations shouldn't be an issue.

(Incidentally, the ProRes HQ file is a much more accurate representation of the spot sitting on my NLE timeline than the MPEG-2 format for either ER or DG).

The list of stations using ER is large and growing, though not quite as ubiquitous as DG's. If a station isn't partnered with them yet, they can also dump the spot to Beta SP and have it delivered, much like DG also does.

Lest I start to sound like an ER honk, I'm really hoping that ER's HD delivery prices start coming down, as the price is still about 10x that of SD delivery. That's enough to make me want to actually call up each station to request their HD formatting specs/FTP address info -- and we all know how much fun that conversation can be.

I once had a guy tell me I could just email a flash file. Seriously.


Re: Automation -- Chris Blair 5:29:21 pm

The problem you're likely going to run into with stations and cable systems is that most of their systems are automated. People are not involved, and trying to convince a station to give your file special treatment will be difficult, if not impossible.

We can't even get stations to implement Telestream provided fixes for a field reversal issue, which is just a tiny, simple plug-in that Telestream will provide and help setup for the station.

The reason? It would mean setting up a special watch folder for these problem files, which the stations and cable systems refuse to do. And believe me, we've asked.

In fact, we got Telestream to figure out the problem by sending them several of the files stations were having problems with. Within an hour, they emailed back and said that there was nothing wrong with our encoded files, but that some of their presets for playout servers did indeed reverse the fields -- usually a 720x480 to 720x486 conversion issue or vice-versa.

They said it was an easy fix, but the station would need to set up that special watch folder, since the plug-in wouldn't need to run on "non-problem" files. The stations refused, saying they couldn't do that, because their engineers and IT people wouldn't allow changes to the system -- despite the fact that it fixed a glaring problem with the quality of their spot playback!


Re: Delivery questions to ponder -- Michael Klinger 12:41:13 pm

When you build a business model around providing services with scarce equipment, best to think ahead to its eventual obsolescence and give thought to what the replacement revenue streams will be.

For a long format hi-def project delivered to a network, we currently get about $500 for a 60 minute HDCAM-SR master. Our clients usually pay their own shipping. Subtract about 100 bucks for the stock, and that leaves $400 for the "labor" and profit per tape. We do these all day long.

There are other functions that add more value and a larger price/profit on some masters, such as Dolby E encoding, and closed caption authoring and encoding. Nevertheless, the pure operation of putting the deck into record bills out $200 to $300 per hour, an amount that will help to re-coup the $100,000 cost of the equipment in a reasonable time period.

At some point, I no longer have to buy $100k decks, but I also no longer have the revenue and profit of offering a service on a relatively scarce piece of equipment. So I contend that there is a lot to consider in order to adapt a business model in coming years, following the needs, trends, and profit centers: adapting from offering the services of scarce equipment, to adjusting it to a cheaper, better delivery model that represents a viable business.

I look forward to this evolution.


And that's just the beginning. In addition to the Digital Delivery forum, you can find these issues being discussed in forums for Telestream Episode, Compression Techniques, Broadcast Video, Business & Marketing, Adobe After Effects, Apple Final Cut Pro, and more. Just click the Search button on any of the terms in this article, and you may be amazed by what you find!


Comments

Re: Avoiding Digital Delivery Disasters
by Garrett Evans
I've found that Hardware is still trump over Software for video digital delivery. Get a external 800, USB or the the new fangled Thunderbolt drive, load it up and let FedEx do the delivery.

In some cases where my clients are bandwidth challenged, FedEx Priority Overnight is faster than up/download.

MacPro OSX 10.6.6 2x2.26 QCIX 10GB 1066MHz DDR3 FCP7.0.3/FCS3
FCSvr1.5 Xsan2.2 DotHill 24TB QlogicSANbox 8GB ATTO 8GB


Related Articles / Tutorials:
Broadcasting
Westdoc Online interview with National Geographic Channel's VP of Development, Charlie Parsons

Westdoc Online interview with National Geographic Channel's VP of Development, Charlie Parsons

Charlie Parsons is Vice President of Development for the National Geographic Channel, responsible for developing new projects, discovering new talent and serving as one of the network’s main points of contact for the global production community. Since joining the Channel in 2010, Parsons has developed popular series “Doomsday Preppers”, “Inside Combat Rescue”, “Mars” and “One Strange Rock”, as well as the global natural history event “Earth Live”. He developed the Emmy Award winning Special “Space Dive”,as well as the Emmy Award winning feature doc “LA 92”. He also served as an Executive Producer on the network’s most watched program in history, the Emmy nominated three-hour movie event “Killing Jesus”, as well as the second and third most watched programs, “Killing Kennedy”??"also Emmy nominated for Outstanding Television Movie??"and “Killing Lincoln”, respectively.

Editorial, Feature, People / Interview
Chuck Braverman
Broadcasting
NAB 2015, ZELINIZED: HD IS OVER

NAB 2015, ZELINIZED: HD IS OVER

Bob Zelin applies decades of engineering and facility design experience to finding products from the 2015 NAB Show that will make a real-world difference for you on a daily basis. Here's Bob's take on everything from thumb drives to 40GigE.

Editorial, Feature, People / Interview
Bob Zelin
Broadcasting
The Drive to Create: The Drive: PAC-12 Football

The Drive to Create: The Drive: PAC-12 Football

The 25-year career of 16-time Emmy winner Jim Jorden began with NFL Films, and included 7 years creating programs for NASCAR before returning to football. Along the way, he has led the production of 69 new series on 20 networks, winning Emmys for his work as a cameraman, an editor, and an executive producer. He brings every bit of that experience to bear on "The Drive: PAC-12 Football," now in its second season. It's the first show to cover an entire season, week by week, combining the classically cinematic approach Jim took to his work for NFL Films, and the all-access approach he pioneered with his work on Hard Knocks. You won't want to miss this remarkable look at full-contact TV production.

Editorial, Feature, People / Interview
Tim Wilson
Broadcasting
Quantel Unveils 8K 60p Pablo Rio Workflow

Quantel Unveils 8K 60p Pablo Rio Workflow

Quantel has unveiled a new Pablo Rio working in real time at 8K 60p, supported with hardware by NVIDIA and AJA, a world's first for post production.

Editorial, Feature
Kylee Peña
Broadcasting
Project DAVID Creates Digital Content Preservation Tech

Project DAVID Creates Digital Content Preservation Tech

While there is a strong movement to preserve and restore old film, preservation of audiovisual digital content has been less of a priority. Project DAVID seeks to change that, by developing technology for prevention of damage, preservation and restoration for the vast libraries of digital content.

Editorial, Feature
Ryan Salazar
Broadcasting
NAB Show Conferences: Can We Talk?

NAB Show Conferences: Can We Talk?

NAB Show is billed as "the largest and most relevant educational program for media and entertainment professionals anywhere." With so much information, you'll want to do your research and find what's the best conference for you. In this condensed guide, Ryan Salazar names a few of the conferences he finds the most "interesting and intriguing" and provides links for your further perusal.

Editorial, Feature, People / Interview
Ryan Salazar
Broadcasting
NOT Your Father's AM

NOT Your Father's AM

At this time, over 2,200 radio stations are broadcasting a hybrid HD Radio signal, with more on the way, and with NAB approaching, Ryan Salazar is considering the future and applications of digital radio. Radio is going to become sexy again, and this is NOT your father's AM broadcast...

Editorial, Feature
Ryan Salazar
Broadcasting
NAB Labs Futures Park: FM & Smartphones - A Good Idea?

NAB Labs Futures Park: FM & Smartphones - A Good Idea?

NAB Labs Futures Park is a showplace designed to display the newest of advanced projects, demonstrations of high-tech media developments in progress, prototypes, and products not yet available for sale. One of the most exciting features in NAB Labs Futures Park will be the exploration of FM radio signals delivered via smartphones. Read up on what's new and be prepared to visit NAB with questions ready.

Editorial, Feature
Ryan Salazar
Broadcasting
Inter BEE keynote: Japan Will Broadcast 8K by 2020 Olympics

Inter BEE keynote: Japan Will Broadcast 8K by 2020 Olympics

It's not news that Japan has announced its intent to broadcast 8K by the 2020 Tokyo Olympics. In today's production environment, the advent of 4K is still news and 8K seems like a futuristic pipe dream. But when Creative COW Contributing Editor Debra Kaufman was in Japan in November, she learned just how serious Japan is in its intent to broadcast 8K -- and to do so sooner than originally announced.

Editorial, Feature, People / Interview
Debra Kaufman
Broadcasting
Will 2014 Be the Year of 4K?

Will 2014 Be the Year of 4K?

UltraHD, or 4K, has been making an appearance at trade shows for the last couple of years. At this year's IBC in Amsterdam, the demonstrations and products pushed forward the idea that 4K is a possibility for TV distribution and general production and post. Creative COW takes a look at the offerings and the opportunities.

Editorial, Feature, People / Interview
Debra Kaufman
MORE
© 2020 CreativeCOW.net All Rights Reserved
[TOP]