Media 100's 'Lossless' Compression: What Does It Mean?
COW Library : Media 100 : Ron Lindeboom : Media 100's 'Lossless' Compression: What Does It Mean?
A Lot of Confusion: Why?
There was some confusion on the Net and on the floor at NAB following Media 100's announcement that they would soon begin support of a new "lossless" compression system. Media 100's latest did not offer an "uncompressed" solution, rather it offered a lossless compression technology -- and that's where the confusion and some of the backlash came from.
Many users complained about the news and some decried it as "Marlboro marketing voodoo" that was meant to cover up for the engineers at Media 100 who weren't able to pull off uncompressed. But I think that this is a short-sighted view and misses most of the reason why things were done as they were at NAB 2001. To understand why I say that, I have to lay some groundwork before jumping ahead into dissecting this new technology.
As many reading this know, I have been both a vociferous Media 100 cheerleader and a well-thought Media 100 naysayer in the past. Because of this, I have my own thoughts on the matter and I've learned this much for sure: Regardless of any public or private pressure, Media 100 charts its own course -- like it or not. You can argue with the goal all you want but there is definitely someone steering the car here.
I have sometimes quipped that Media 100 is really "Johnco" and anyone who doesn't realize that this company reflects John Molinari's vision is missing a great deal that lies at the heart of Media 100. As many here on the Cow leadership team can attest to from discussions we've had over the years, I consider John one of the true visionaries in this market. Simply put: He's led the trends more times than he's followed them and as a creative visionary, he's passionate about his vision and wants things done that facilitate this view. He's intense, focused and has a "gut" for where things are going and he plays his hunches well. And contrary to what some believe, he's got a clear vision of where he's headed and I am not one who'd like to bet against him in a chess game. (And no, they don't advertise here and I don't own one of his systems anymore, so please spare me the diatribes about myopia and blind loyalty or whoredom.)
In keeping with his vision, John has always been very vocal about his view of the market. No, I do not mean vocal as in being a man of a lot of words who drops his every thought with each new breeze -- no, this is a man who says what he thinks, with words spread out over time. But you have to dig to find them and then connect the dots for yourself. If you do, then you begin to see a pattern that is at the heart of the way things are done in Marlboro Country.
John Molinari (right) is someone who has definite opinions about this industry and the position that his company plays in it. My own opinion of him forged from years of following his comments both public and private, is that he doesn't see Media 100 as many of his customers would like him to see it. He's never hidden this fact and at NAB two or three years ago, he made an announcement that (this is a loose paraphrase as I don't have his words handy) his target wasn't broadcast editors working on Avids. He was after everyone who used Photoshop and a computer and was on the Net. He was less interested in the upper 15% of the market than he was in the far broader market that reached into nearly every home and school on most all of the world's continents. When I heard this announcement, it surprised me and I thought that he was missing a vital segment that he'd need to win his campaign for total world domination. But once again, he proved me and most all of the self-appointed industry watchdogs wrong -- something that he's done more than a few times over the years.
John Molinari's vision, at least in my interpretation of it, is built around "everyman." It's the point he made long ago in Media 100's "Blood Secrets" that ran in an old issue of Videography -- that John and Videography graciously gave us his permission to reprint. In "Blood Secrets," John laid out a vision that democratised video and brought true broadcast quality video into the hands of many instead of being in the control of the few. It was why I became a vociferous fan and stayed that way for years. (It helped that I had come out of the aerospace industry back in 1993 and knew his father's company, Data Translation and how far ahead they were of everyone else when it came to imaging technology. I used to joke that "DT was the company that taught the CIA's satellites what your pores look like from space." It didn't take a genius to see what John would later unleash and back then he made me look psychic as he hit nearly every forecast I had made right on the money.)
Today, I do not believe that John's vision has changed much really. I still think that that is his goal and while he has to please stockholders and an increasingly more expectant base of users, his original vision is pretty much still intact. His destination is to put Media 100's legendary image quality (which once stood alone, though today it faces serious competition from every side and is even exceeded by some) into the hands of Joe Everyman. And in that is the key to understanding why Media 100's announcement was what it was.
The fact is -- and again this is a "like it or not" kinda thing -- lossless is in keeping with the original focus of John's vision. Uncompressed is great for studios with huge budget outlays for massive drive enclosures and whose work for film houses, etc., necessitates the need for uncompressed. But while Media 100 has always touted their long-vaunted picture quality, their goals of democratizing video have never been quite lost even in the corporate world of Wall Street. In my opinion, John's still the man of "Blood Secrets" and his focus is still as a New England Outsider who is working to bring storytelling tools to those outside of Hollywood and New York City. His Tewksbury cousins made be welcome in Hollywood circles and have the Oscars to prove it but John is instead wanting to video-tape, edit and broadcast 98.6% of the world's backyard barbeques. Like this or not, you only have to go back to NAB a few years back and examine his words to see that this is true. Still, with Media 100i, he broght to the market a system which met many more needs of broadcast users -- this in part, due to the ability to script the system for automating repetitive tasks, etc.
But with the latest news out of Marlboro, Molinari & Company have given Joe Everyman a system which includes the advantages of uncompressed -- i.e., the "lossless" image -- without paying the harddrive premium which comes with uncompressed. Even in the new world where drives sell for 20% of what they did just a few years ago, uncompressed is still a luxury that is in keeping with the 15% at the top of the market. While this segment is an ancillary part of Media 100's business, many users have long suspected that it is not the focus of the Media 100 Vision. Me, I think that it's a telling thing that Media 100 bought Digital Origin, Wired and Terran a few years back. They bought mass-market competitors, not their cocktail party upper market competitors. And this has angered many who do not seem to relate to what drives the Marlboro Man and his company of heretics and renegades.
As Philip Hodgetts (right, creator of the Media 100 Companion) points out in some of his own remarks at Creative Cow and elsewhere: "Once again, in terms of the codec there is no difference between uncompressed and lossless. Stuffit and other compression systems have long proved that. When I send an 8mb TIFF image to my printer in Australia for a brochure and I compress it to just over 2.5mbs, when it's uncompressed I expect the full 8mbs -- not the 2.5 it was compressed to for the sake of transferring it across the Net. Lossless is just analog to digital conversion with the redundancy taken out. Look at the Stuffit-type engine -- what comes out is what goes in. That is lossless compression and that is what Media 100 uses."
But Philip has also pointed out another interesting fact: the ability using the Media 100 scheme to seamlessly integrate legacy data from past "lossy" Media 100 projects into the same timeline with the new "lossless" footage. That's a big plus to many users. Another key facet is that this also allows access to legacy renders.
Once again, Our Affable Aussie, cuts through the crap to the heart of the matter...
Bigger Doesn't Always Mean Better:
Borrowing from one of Philip's points that he expressed elsewhere, I'd like to ask: What is going to look better, an image compressed with a lossless algorithm or an image that is uncompressed? Right. They are both the same.
Back in 1994, I remember that there was always an argument somewhere online -- usually between people on Digvid-L, Vidpro-L, the WWUG and the Media 100 and the Avid lists -- arguing about file size and picture. Back then the Media 100 algorithm was unrivaled for picture quality and yet it made smaller files. At the time, Radius, for example, made far bigger file sizes and yet Media 100 had the better image at the time. Size isn't always the answer.
As I pointed out in one of my own recent posts: "Media 100's picture was far greater back then and even at 60 and 80kb (the limit back then) people were getting projects snuck onto the air. I remember more than a few Media 100 Nubus systems that were quietly churning out broadcast material and most of the producers wouldn't let you know they were using them for fear they'd shoot the goose that was laying the golden eggs.
Point Number One: Even with much more compressed files, the Media 100 looked far, far better than files which were much less compressed. It was the quality of the compression algorithm that made the difference. It still does.
Point Number Two: It is quite possible to write algorithms that can track all of an image's data and yet compress the data significantly. When these images uncompress and the image contains 100% of the picture information, what do you call this? Right. Lossless compression."
As Philip and other users have pointed out, Stuffit and other compression tools have long proven that you can compress a file and not lose data. Look at it this way: When was the last time that you compressed a data file and opened it only to find that in order to get it compressed, Stuffit threw out the first 5 pages and the last chapter of your report??? It just doesn't happen that way. You compress it and you get your report. All the letters and the formating are there.
A great example of how it can work is thinking about a "talking head" shot against a bluescreen. Yes, you can bring it in totally uncompressed and have a huge file where all frames are exactly the same size no matter the frame composition. Or, you can use a "lossless" algorithm that is "smart" enough to see that there is a lot of blue that never changes -- so track that as a single value that let's say for example sake is 100,000 pixels. The algorithm would basically say that all the pixels between 1 and 100,000 are all blue. So that could be compressed as a mathmatic formula that tracks all non-moving blue to one value instead of tracking all pixels individually.
Then only the "moving" blue pixels would have to be "watched" as individual pixels -- you know the ones which are adjacent to the talking head and so their values are fluctuating as they blend with the hair and skin tones of the talking head as it moves through the field of blue pixels.
Using this formula, you can do a lot of compressing and still be "lossless" while still giving yourself the luxury of not having to bury yourself in drive space. (Theoretically, though often depending on the subject matter, you will have to bury yourself as if the subject matter includes a lot of movement -- say like shooting a lot of water rippling with sunlight sparkles on it -- you will be eating full frame a lot of the time.)
Is this the perfect solution for everyone? I doubt it. There are those who for multiple reasons, need the "marketing edge" that comes with the words "Uncompressed Images." But in the world of nonlinear editing, don't kid yourself: A lossless image and an uncompressed image are virtually identical -- sometimes absolutely identical.
But as to the cost of the upgrade to get there, that's your call and that's up to you to decide...
-- Ron Lindeboom