Using Apple FCPX for A DocumenTree
COW Library : Apple Final Cut Pro X : Michael Angelo : Using Apple FCPX for A DocumenTree
Editors note: We first came across Michael's story in Creative COW's Apple FCPX Techniques forum, in a spirited thread on the challenges of offline-online documentary workflows with FCPX. We asked him to expand on some of the technical aspects of that conversation, as well as to tell us more about the mysterious Treeman himself. Some of those details will have to wait for the finished film, but this is a remarkable tale for both sides of your brain.
After 25 years of working on other people's documentaries, commercials, feature films, I decided it was high time to start making my own original content, things that have special meaning for me.
I decided to make and gift short documentary films to local artists in Venice Beach that inspired me. Treeman is my second subject and it didn't take long to realize that A DocumenTree should be a feature film.
When I first started following the Treeman around I quickly noticed he is on a mission to not make money but rather inspire positive social change. He takes about 2 hours to get into Treeman, will often spend 10 hours spontaneously exploring a city on his big shoes (stilts), rarely asks for photos tips and will often come back with only $7 in his pocket.
So if he's not out to make money then why do it? What is is he getting out of it? This begs to question what do we receive when we give selflessly to others? A DocumenTree is an exploration of human nature and kindness told through the story of Treeman.
In keeping with the motif of gifting, the film is being made with 1000% volunteers, industry professionals, college film students, anyone we encounter that appreciates the story, theme, filmmaking and the high visual art aspect of the film.
We've been in production for a total of "Tree" years and are now in editorial. We are looking to staff up with more volunteers (they receive a ton of experience and training in all aspects of filmmaking) and are currently applying for fiscal sponsorship through the International Documentary Association (IDA) here in LA in order to qualify for more grants and enjoy other benefits in the program.
I make my living working mostly as an Autodesk Flame Artist, so building a feature pipeline and implementing FCPX with only volunteers has been a wild but rewarding ride.
I am no stranger to learning new edit systems as I started editing on 3/4 tape in college, cutting on Avid in the early 90s, then Light Works, then Sony 9100 online editing, then Autodesk Smoke, then legacy Final Cut Pro and finally focusing back on finishing in Autodesk Flame, offlining in Avid and legacy FCP as needed.
FCPX FOR A DOCUMENTREE INTRODUCTION
I chose FCPX despite the difficult transition editorially from Avid and FCP 7, because I see keywording and hash tagging as the future of media management. Given the amount of footage in our film it just made sense to me: over 10TB footage, with mixed source DSLR, RED camera, GoPro, archival sources, public domain, etc.
We sometimes had 5 sources recording as we were run-and-gun chasing a 10ft tree through the subways of New York city, and Treeman is also contributing years of his own footage. Over 1000 hours in all! I saw keywording as the only way to organize this mountain of media and the metadata heavy workflow made sense moving forward into the future.
Other factors that influenced the choice were the low price of $299 for 3 seats, the 64bit architecture, background rendering and the fact that support for legacy FCP was going away.
Now years later there are many things that I enjoy about FCPX, the use of roles, the speed, the plug ins and the constant product development. XMLs are working better now, RED work flow is solid, software improvements are enjoyed and it feels like Apple is paying more attention to the professional community now.
My favorite feature right now is the timeline index. We have a technique where, when logging, we strategically place markers on a word that is essential in a sentence to the ideas contained in that sound bite, a moment that is most likely to be included in the edit.
When editing I always have the timeline index up with just markers so I can see the whole edit timeline as a series of marker notes. I get a feel for the flow and order of the edits by simply reading thru the timeline index, no need to play down.
The multicam workflow is also pretty solid and since much of the film is shot with multiple sources simultaneously, we build multicams, log the multicam clips, and do much of the editing on the fly switching camera using hotkeys.
We are really trying to use the software as intended and build up the meta data up front before diving head first into editing. Little things like adding the clip angle to the metadata upon import really speeds things up when making multicams.
On the front end, building metadata slows down the process, but once you start editing, finding shots is unbelievably fast. As you can imagine smart collections are key for us when your digging through well over 1000 hours of footage, we live by them!
iMac (27-inch, Mid 2011)
3.4 GHz Intel Core i7
32 GB 1333 MHz DDR3
AMD Radeon HD 6970M 2048 MB
Pro Tools 11
Mbox Pro 3rd Generation
Autodesk Smoke for Mac 2016
"...and about a million 8TB Thunderbolt G-RAIDs"
DOCUMENTREE MEDIA MANAGEMENT
Given the longer timelines involved with cutting a feature we have chosen to transcode most of our footage to optimized and use the 3 pool method of media management per this thread, keeping each type of media on it's own drive for maximum flexibility. Most of the time we only use the Proxy drive for editing and only turn on the optimized or original media drives as needed:
Our RED footage plays natively and performs well without optimizing which is great, so we leave that as is on separate original media drives. We have mirrored back ups of these drives at a separate location in case of fire etc.
FCPX does a very good job of transcoding/optimizing media to ProRes but we had a problem of FCPX crashing while transcoding – plus it doesn't do true background processing, so it can take forever.
When we started the project 3 years ago we ran many tests with our DSLR 5D footage plus GoPro and found the only application where optimized transcodes looked as good as FCPX was 5DtoRGB. Everything else was of lower quality or had phasing issues. (We tested Magic Bullet Grinder, Quicktime Pro 7, Cineform, Apple Compressor just to name a few.) I'm sure much has changed since then, but 5DtoRGB did the trick and has decent batch features:
If you choose to transcode optimized media to their own drives per 3 pool method, make sure to uncheck "copy files to Final Cut Events folder" upon import. Otherwise you will create duplicates. Any "non-video" files like audio and pictures we chose "copy files to Final Cut Events folder," otherwise the audio and pictures wouldn't live with our proxy clips (see PROXY MEDIA below) and this can bite you further down the line if you want to take a proxy-only version of the show on the road.
Two of Treeman's many celebrity encounters, Spike Lee and Joseph Gordon-Levitt.
Once our original footage (less RED) was transcoded to ProRes, we import the ProRes, RED, audio files and pictures into FCPX and let FCPX create proxy media and move the necessary files to a separate Proxy/Library Files drive. We tell FCPX to store all this media, its cache files and backups into separate folders on the same drive with the library files but not inside the library file itself.
This "external media method" setting can be made in "File/Library Properties" and keeps the library files quite small in size. This creates maximum flexibility with regard to collaboration if you have duplicate systems with the same directory structure.
By having the library and proxies on the same drive we can easily move our offline proxy edit between multiple computers by simply mounting the drive to another system.
It is tricky keeping all your media locations straight when you start using FCPX, especially with the 3 pool method, so you do need to pay close attention to where you are pointing and copying media to. This article is a good read on the topic:
The most important thing to understand is that FCPX thinks differently. Ot works with "managed media" i.e. there is a lot of sensitive metadata that tells FCPX where the media is stored, and FCPX gets very unhappy when you simply copy files using the finder.
Coming from a traditional NLE background this can be pretty frustrating but the "managed media" aspect is what gives the application its greatest power. FCPX uses keywords to categorize shots rather than placing them into a single folder, very much like "tagging" media on the internet.
This is the future of media management in my opinion whether it's the internet, computer directories or in our case NLEs. It is especially useful for documentaries with tons and tons of footage to sort thru and organize. Because FCPX used "managed media", once your clips are imported, you should always copy libraries, projects (edits) and events from within the application when collaborating to truly make FCPX happy.
Interns training on FCPX
FCPX COLLABORATION – WHAT WORKS FOR US
Collaboration is a big topic for us as we have many interns logging footage remotely, editing meltdowns and we take the whole movie edit and source files with us when traveling on shoots. FCPX's "managed media" works more like a database so it's very particular about relinking media on other drives compared to a traditional NLE.
The safest way we have found to collaborate is to have a master edit system with the 3 pool drive system. Then from within the application (very important), create libraries for your collaborators onto an external drive.
Ideally these collaborator libraries should have the same directory structure as the master system (see PROXY MEDIA "external media method" above). After copying libraries, projects (edits) and events to the collaborator's drive, make sure to click "Consolidate" under "File/Library Properties" otherwise FCPX only moves pointer files and won't actually copy the proxy media.
The collaborating editor can then take these drives, do their work, return and mount the drive to share libraries, projects (edits) and events from within the application. You can share work without duplicating media by simply choosing not to transfer proxy or optimized media when prompted.
We've found it is best to have the optimized media drive connected to the master system when importing media as sometimes FCPX get confused about the relinks. Conversely we've had to turn the optimized media drive off when exporting libraries and events to external drives at proxy resolution as FCPX gets confused and will export optimized media as well.
FCPX COLLABORATION – WHAT SHOULD WORK
Ideally one should be able to simply post a "pointer file" on the web (no media), have someone pull it down, then have access to your work. If the remote collaborators have duplicate optimized media with the same directory structure ("external media method") then library files can be exchanged and with a simple relink work can be shared, but we've had challenges with these re-links.
Even so, who wants to travel with over 10TB of optimized media? Having to hook up your remote collaboration drive directly to your master system is a big pain and very impractical when collaborating over a great distance. Proxy collaboration drives are key if you want to collaborate remotely and nothing under the current version of FCPX is dependable.
Treeman with the Tommy Posse
We have had much success collaborating remotely with proxy-only drives in the past by "tricking" the software, so it can be done. The problem is that as new FCPX versions evolve these back doors have been shut and this kind of functionality is not being built into the application. I've had direct correspondence with the one of the FCPX product managers at Apple, we've tried many things but have had inconsistent results.
Below are some of these methods that we have used to work remotely with proxy only drives sharing only pointer files in case they become relevant again. When they worked it was fantastic, currently all of these methods are flawed.
Filmmaker Michael Angelo on location with Treeman in Las Vegas and the forests of Patchogue, NY
FCPX COLLABORATION – WHAT USED TO WORK – EXCHANGE FILES WITHIN PACKAGE CONTENTS
Early on, we were able to hack FCPX's lack of remote workflow by right clicking the Library file, going inside the package contents, grabbing the "CurrentVersion" event file, then posting it for an editor who could then replace that file inside their library package contents. Their library could even be created on a remote drive by copying in the finder and avoiding the "managed media" sensitivity of FCPX.
This method worked as long as we had duplicate media. There was no relinking of files necessary and occasionally you would have to re-render some clips.
The beauty of this method is the "CurrentVersion" event file is tiny, fits in an email. I really wish Apple could build this kind of functionality into the product natively. BTW this is a big no-no, messing around inside package contents, I don't recommend it despite our success.
We learned the trick here at Creative COW, though. https://forums.creativecow.net/thread/344/11304#11309
FCPX COLLABORATION – WHAT USED TO WORK – EXCHANGE LIBRARY FILES VIA FINDER
If your proxy media is stored externally (see PROXY MEDIA "external media method" above) and you have a mirror drive with the same directory structure and same drive name, you were able to simply exchange the library file and see each other's work remotely. This would work even if you had a proxy only drive and no re-linking was necessary.
Unfortunately, with more recent version of FCPX the "managed media" can tell that it's not the same drive despite having the same name. The proxies won't link. There is no way to relink proxies-only in FCPX, so if you are working remotely on a proxy only drive you are out of luck.
Here's what happens if you link to the proxies: all hell breaks loose:
FCPX COLLABORATION – WHAT KINDA WORKS NOW – XMLs
XML is the most successful method that we have found to share work remotely without having to hook up the remote drives and by only sending small pointer files. It seems to work much more like a traditional NLE with paths stemming from the library file not the drive so it is able to relink to mirrored directory.
The rub is that it makes mistakes like not transferring all your edits, losing any multicam sync changes, axis change errors and more. It is however reliable enough to get an idea of the collaborators edit decisions, or logging and keywording that is done. It's usually best to pull the XML from the collaborators library or event as pulling directly from the clips in latest versions of FCPX can change the case of keywords.
OFFLINE EDITING TOOLS
FCPX will make you pull your hair out over the traditional tools that are missing, but jump for joy at the incredible metadata and keyword workflow features. Learn how to properly name source material, set roles, keyword on import via your external directory structure before you start using FCPX. It's lightning fast compared to FCP 2009, with no rendering needed to see most things given the 64-bit architecture. It's really coming along considering that Apple is building this from the ground up.
There are some crazy things, like no way to undo a mark (insane), and if you get lots of keywords (like you will with a long form footage heavy show), you can't expand the keyword window to see them. You're stuck scrolling in a tiny window, but life goes on.
The magnetic timeline is really gonna take you a minute to get used to, best to not fight it and see where it takes you. Here's a good article: https://library.creativecow.net/austin_charlie/FCPX-Timeline/1
ONLINE EDITING TOOLS
People are constantly making new tools for the software and Apple is charging features pretty hard. The color corrector is the weak link so I'm guessing most people finish color outside of the app. You can do a lot inside the app, previewing is a snap but the tools are a bit too automatic for the high end finisher, really depends on how much control you desire and what kind of things you need to do. We will likely use a mix of FCPX, After Effects, Resolve and Flame.
ESSENTIAL FUNCTIONALITY MISSING
Undo mark does not exist. Getting the perfect mark can be a zen thing and if you accidentally lose it you can't get it back
Unable to edit keyword range numerically after range keyworded. The ability to edit these numeric values is so basic, save the time of deleting a keyword range and re-doing
Unable to mark on the fly on secondary storylines. Option-Bracket trims on the fly, but what if I want to mark a range inside the clip, delete it as a short pullup?
Remote collaboration workflow. Working remotely should only be a matter of exchanging a data pointer file via email or upload to the web, I cover this in detail already in my original post
Filter window needs to expand horizontally. Apple did finally make filter window expand vertically but it's still cutting off keywords horizontally and doesn't expand on x axis
Event viewer text search should isolate single lines of marker text. It works this way in timeline index, so painful for longer logged clips
Need event viewer column for unlinked or clips with errors. So painful to find the one unlinked clip in an event with 3000 clips
Ability to only view markers or keywords in event viewer – just like in timeline index turn off on individually via legend
Ability to customize color coding for keywords and marker icons like in legacy FCP. (Blue markers next to blue keywords very hard to read.)
Favorites should keep be added timeline index
File/Import should have the option for Column view.
The ability to only view markers, keywords, favorites, tasks etc., in both timeline index and event viewer, in any combination -- not just "all" or individually
STRANGE BEHAVIOR & BUGS
XML does reflect changes in multicam track edits nor compound clips embedded edits properly
Modifying a marker not using shift-M causes event viewer to scroll the top of the marker list. (So painful to marker a 1 hour multicam!)
Batch rename to original name does not restore to original file name on hard drive
Unable to mark using hotkeys when playing down full screen on a laptop
USING FCPX FOR A DOCUMENTREE
Since we just finished production this year, currently we are mostly focused on building up metadata, logging, editing string outs, still finding our story really. I'm not sure if I am completely ready to chime in on editing using the magnetic timeline.
With that said, the magnetic timeline is a whole new way of thinking, definitely finding a mix of plus and minuses. The magnetic timeline does have big advantages in swapping the order of shots quickly and the trackless system can help keep things in sync when moving many layers of footage given the attachment of secondary storylines.
However, trackless editing can get a bit chaotic and difficult to keep things straight as secondary storylines pop up and down vertically, versus the organization of a track based system where X footage lives on track Y. Audio editing is particularly cumbersome and unruly with the trackless system. I feel that we lose lots of time here.
Nonetheless we are still learning, building our workflow and excited to get deeper into the edit. I'm sure we will find workarounds and become more proficient once we have more time under our belt.
In the end, edit programs are ever evolving tools and the editing itself is really in the mind of the editor. I have a unique perspective sitting in both the offline and online chair for so long and my goal is to be proactive, positive and do what I can to support the tools and other users.
I humbly offer you and the group my experiences in the case that they are useful, noting I have much to learn and greatly appreciate any ideas folks have to address some of the challenges I've shared above. We are trying to push the software to the limit and are currently looking for interns and volunteers to join our team in completing A DocumenTree.