LIBRARY: Tutorials Reviews Interviews Editorials Features Business Authors RSS Feed

Renaissance Masters Go 3D with Nuke

CreativeCOW presents Renaissance Masters Go 3D with Nuke -- TV & Movie Appreciation Editorial


Steve Wright Digital FX
Los Angeles CA USA
CreativeCOW.net. All rights reserved.


In 2015 Sky 3D, an Italian all-3D television station, embarked on an epic project to present the city of Florence and the masterpieces of Renaissance art housed in the Uffizi Gallery in a spectacular stereoscopic 3D movie shown in 60 countries around the world. This would give movie-goers a chance to see some of the greatest works of art ever produced by man (and one woman ) with a never before seen intimacy that placed the viewer at eye level with the statue of David and face to face with the mighty works of Michelangelo and Leonardo da Vinci by using the latest Hollywood blockbuster techniques of 3D movie making.

Titled "Florence and the Uffizi Gallery 3D," much of the movie would be shot with state-of-the-art stereo camera rigs. But 27 works of art would be converted to stereo 3D, or "dimensionalized," from high resolution photographs using the same Hollywood tech used to convert "Clash of the Titans" to 3D. In this article we will see the range of techniques used for the stereo conversion and the unique problems associated with applying it to fine art.


17th century female painter Artemisia Gentilieschi painted Judith Slaying Holofernes
Madonna of the Harpies in 3D. 17th century female painter Artemisia Gentilieschi painted Judith Slaying Holofernes.


Some of the images herein are 3D so you will want to fetch your red and blue anaglyph 3D glasses to view them. The masterpiece Madonna of the Harpies by Andrea del Sarto shown here (time to don your glasses) is a classic example of one if the 27 works of art converted to stereo 3D. We can only view a small version here, but imagine the impact of seeing this in 3D on the big screen.


red and blue 3D anaglyph glasses
Red and blue anaglyph 3D glasses.


I first became aware of this project when I was contacted by a former student, Daniele Pugni, who was now the head of Sky Production & Creative Hub, the motion graphics department of Sky 3D. He had taken one of my courses on compositing feature film visual effects using Nuke, the dominant visual effects software in the entertainment industry. But now he needed to use Nuke on a whole new level – to dimensionalize classic works of art to convert them to 3D – so we began a series of personal training sessions so I could bring him up to speed on the arcane art of stereo conversion. The training was conducted remotely using desktop sharing to his office in Milan, Italy.


How 3D Movies Are Made
Before we begin I must confess that the subject of 3D movie-making has an unfortunate terminology conflict. On the movie-making (production) side it is referred to as "stereo". We have stereo camera rigs, stereographers, and stereo visual effects. On the display side (TV, movie theatre, photographs) it suddenly becomes 3D. We have 3D movies, 3D TV's, and 3D glasses. That is why in this article you will see the term "stereo" when we are discussing the making of the pictures but "3D" when we talk about viewing them.

Let us begin with a brief review of how stereo 3D movies are made. Our vision is stereoscopic (3D) because we get two slightly different views of a scene, one from each eye. To make a 3D movie we also need two points of view. The technique for shooting a stereo 3D movie is illustrated here. Starting on the left, we have the stereo camera rig with two cameras photographing an object from slightly different angles (the angles are exaggerated for illustration purposes). In the middle are the two views of the object seen at slightly different angles. These two views are then combined to form the 3D image on the right (3D glasses on).


Stereo camera rig (left). (Right) Two views, resulting 3D image.


For the observer to see the image in 3D the display system (photograph, TV, movie screen) must somehow show a different view to each eye. In the movie theatre big digital projectors project the two images with polarized light that the audience's 3D glasses can separate into the two views. For photographs like in this article we have to fall back on the ancient red and blue (red and cyan actually) anaglyph system which uses colored filters to separate the two views.


How Stereo Conversion Is Done
The movie industry's recent transition to digital production has lead to a boom in 3D movies because digital technology makes 3D much more practical and aesthetically much better than the old 35mm film technologies. But digital technology has also made a whole new idea possible, namely stereo conversion – converting a movie originally shot in 2D into a 3D movie. This is the technology that made the 3D versions of the Renaissance masterpieces. But how do you start with one view of a painting and turn it into the two slightly different views that are required to make the 3D version? An understanding of this process will reveal the conundrum presented to the digital artists on this particular project.


(Left) 2D shot converted to 3D. (Middle) Original 2D shot. (Right) Second view.


Let's look (with your 3D glasses) at how to make the converted 3D shot shown in the left panel of the illustration above. The middle panel shows a closeup of the original 2D shot of the boy. The right panel shows how the boy needs to be shifted to the right to create the second view for stereo conversion. However, when he is shifted right it reveals previously covered regions of the background which are marked in red.

If we were converting a feature film this would not be an issue. In visual effects we have a variety of methods for re-creating the missing regions of the background. However, if we are trying to dimensionalize a masterpiece of art, who are we to make up the missing pieces of a masterpiece? And that is the conundrum of the digital artists trying to dimensionalize these works of art. The artists on this job were forbidden to simply "invent" the missing background regions for historical works of art created by Renaissance masters.

There was a two-fold solution to this problem. The first was to keep the dimensionalization and 3D camera moves to a minimum so only small regions of the background are affected. The more the picture is "extruded" in depth and the greater the 3D camera moves the more background that must be restored. The second was to ever so gently stretch the pixels in the background to cover the affected regions. Between these two strategies a balance was struck that would not offend either the Renaissance masters or the viewing audience.


Leonardo da Vinci Self Portrait
There are a range of techniques used to dimensionalize pictures and which one to use is based on the content and complexity of the scene. On the "simpler" end of the stereo conversion spectrum is what is often referred to as the "rubber sheet" method which was used for the Leonardo da Vinci Self Portrait. It is as though the image to be converted is on a sheet of rubber that is vacuum formed over a mold. The mold provides the depth dimensionality for the stereo cameras while the image provides the picture content. Photographed by the stereo cameras from the two slightly different angles provides the two views used to create the stereo 3D effect.


(Left) Original photograph. (Middle) Hand painted depth map. (Right) Displaced 3D mesh.


How it is done is shown in the three illustrations above. On the left is the original artwork photographed at very high resolution. From this photograph an artist uses Photoshop to paint a "depth map" (middle grayscale image). The depth map represents the distance from the camera to the painting with brighter pixels being closer to the camera. This depth map is then used to displace the vertices of a 3D mesh shifting the bright pixels closer to the camera and darker pixels further from the camera as seen in the image on the right. This is the "mold" mentioned above upon which the original image will be "vacuum formed" to dimensionalize it when re-photographed by the stereo cameras.


Camera and 3D geometry setup in Nuke


The illustration above shows the 3D arrangement required to render each view. The rear-most layer is the original painting placed as the background. The da Vinci character is isolated using Photoshop then projected onto the displaced 3D mesh in front of the painting. This way the isolated and dimensionalized character is rendered separately so its edges can be artfully blended with the original painting background. In front of all that is the 3D camera used to render this view. A parallel setup is used for the other view but with the da Vinci element offset a bit to introduce the parallax shift required to create the stereo effect.

Here is the resulting stereo 3D image produced from the two views rendered in Nuke.



Converted da Vinci 3D image.


The Head of Medusa by Caravaggio
The head of Medusa by Michelangelo Caravaggio (not "the" Michelangelo) presented a truly unusual stereography problem. As you can see from the production photo here that it was painted on a convex shield (curved outward). However, Caravaggio, an avid bar-room brawler and miscreant, was also a master of realism and lighting. He actually painted it to make the convex shield appear concave (curved inward) with the Medusa head suspended within the curvature of the shield. This posed a vexing problem for the stereographers.


Head of Medusa painted on a shield.
Head of Medusa on shield.


Physically, the Medusa was painted on the surface of a shield that was curved outward but artistically it was intended to look like it was suspended within a shield that was curved inward. Daniel Pugni chose to go with the artistic intent rather than physical reality, so the stereo version would be made to look like the shield was concave and the Medusa head was suspended within it. But how might this be done?


3D model of the Head of Medusa painting.


To pull off this visual inversion required extraordinary skill and technique because the stereo cameras simply cannot be fooled. It was necessary to actually build a 3D model of what the Medusa and shield would look like if it were suspended within a concave shield, project the original Head of Medusa photograph onto the 3D geometry, then re-photograph it with a pair of stereo cameras. As you can see from the picture above the 3D model was very complex and much had to be inferred about the depth of each element by carefully interpreting the lighting painted into the original masterpiece.



Converted stereo 3D photo of Head of Medusa on shield.


The deliciously horrific 3D results can be seen in the resulting 3D image above. The shield appears concave and Medusa's head is floating within it just like Michelangelo Caravaggio envisioned it, but was never actually able to see it himself. A dramatic camera move was added to further emphasize the 3-dimensional horror.

One other unique issue in a project like this was that the stereo artists were not permitted to fix any of the defects in the art. These sculptures and paintings are hundreds of years old and have accumulated scratches and mars that are now considered part of the art. From a feature film standpoint, our natural instincts are to fix every blemish in every frame for a perfect viewing experience. But not this time. The Sky 3D stereographers had to stay their hand.

There is no feature film stereo conversion industry in Italy so this project represented a major step forward both technically and artistically. Sky 3D's team has successfully made the transition from a motion graphics department to a full-up visual effects department and have already embarked on their next exciting adventure for Sky 3D – another 3D spectacular movie "St. Peter's and the Papal Basilicas of Rome 3D" which will give the world a 3D tour of four magnificent cathedrals in Rome. With such spectacular subject matter we are expecting more great things from the stereo artists of Sky 3D where classic art meets Hollywood tech.






Steve Wright
Steve Wright is a 20 year visual effects veteran having worked on over 70 feature films. Since 2005 he has been a master trainer for Nuke visual effects compositing and has trained the staffs of major studios such as Disney Feature Animation, Prime Focus, Reliance MediaWorks, Pixar Animation, and many more. He is also the author of two popular books on digital compositing available on Amazon.com.


Links:

Courses on Lynda.com - http://www.lynda.com/Steve-Wright/127-1.html

Training Website - http://www.vfxio.com/

MDb filmography - http://www.imdb.com/name/nm0942824/

Facebook - https://www.facebook.com/nukeguru

LinkedIn - https://www.linkedin.com/in/stevewrightdigitalfx

Compositing Visual Effects – introductory book http://www.amazon.com/exec/obidos/ASIN/0240817818/wwwswdfxcom-20

Digital Compositing for Film and Video – advanced book http://www.amazon.com/exec/obidos/ASIN/024081309X/wwwswdfxcom-20




Related Articles / Tutorials:
TV & Movie Appreciation
Favreau, Technicolor & MPC Make The Jungle Book Come Alive

Favreau, Technicolor & MPC Make The Jungle Book Come Alive

Todd McCarthy, veteran film critic and historian, in his review of director Jon Favreau's new, stunning adaptation of Rudyard Kipling's The Jungle Book declared, "...the visual effects team led by Robert Legato and (MPC's) Adam Valdez has both created sumptuous settings that look as lifelike as any CGI ever presented in a studio feature and integrated both humans and animal characters in them in seamless ways."

Editorial, Feature
Jon Favreau
TV & Movie Appreciation
VFX Legion | Hardcore Henry breakdown reel

VFX Legion | Hardcore Henry breakdown reel

Remote post-production and visual effects studio VFX Legion has released its breakdown reel for the incendiary Hardcore Henry. The reel reveals the work that went into the first-person perspective action film, from augmenting violence to stitching shots together into one continuous sequence.

Editorial, Feature
COW News
TV & Movie Appreciation
The Sisterhood of the X-Files Fandom

The Sisterhood of the X-Files Fandom

As the first show to create a rabid, real-time internet fandom, devotion to "The X-Files" has been growing in intensity with each year since the original series finale, with a fanbase that is clever, thoughtful, and largely female. Not that there's any shortage of male X-Philes, but there's a generation of women who was inspired to technical careers by the Gillian Anderson's Dana Scully. Kylee Peña is among them, and additionally very specifically inspired by the production values of "The X-Files" to build a career in the technology of TV storytelling in particular. Here's Kylee's look at what it has meant to be a female fan of the art, technology, and empowerment of "The X-Files" in the 21st century.

Editorial, Feature
Kylee Peña
TV & Movie Appreciation
Peter Doyle: Supervising Visual Colourist at Technicolor

Peter Doyle: Supervising Visual Colourist at Technicolor

Peter Doyle, Supervising Visual Colourist at Technicolor, shares details of his upward spiraling career. His deep technical knowledge allows for a perfect blend of creativity and productivity in equal measure. Here he talks about his career, his aspirations, and his involvement in productions right from the outset.

Feature, People / Interview
FilmLight
TV & Movie Appreciation
Introducing Ultron: Trixter Builds The Avengers' Biggest Bad

Introducing Ultron: Trixter Builds The Avengers' Biggest Bad

At the heart of Marvel's biggest Avengers movie yet lies their greatest threat yet: Ultron, a self-constructing robot intelligence bent on destroying all of humanity. Munich's boutique-scale Trixter Film was given the critical task of introducing this epic-scale character, which they undertook from concept art through design, mocap, animation, compositing, and output.

Editorial, Feature, People / Interview
Tim Wilson
TV & Movie Appreciation
The Walk: Largest Use of Cloud Computing in Film History

The Walk: Largest Use of Cloud Computing in Film History

Award-winning VFX Company Joins UPP and Rodeo FX to Recapture the Legendary Walk Between the World Trade Center Towers. Lead VFX vendor Atomic Fiction needed a more efficient way to do the compute-intensive, and traditionally very expensive, processes of rendering. The company used their cloud-based software Conductor, which allows artists to offload the processing from their own computers and send it to the cloud. By the end of the project, Atomic Fiction had completed 9.1 million hours of processing in the cloud, which equates to over a millennium of processing time!

Editorial, Feature
Shaina Ostroff
TV & Movie Appreciation
Milk's Epic VFX For Jonathan Strange & Mr Norrell

Milk's Epic VFX For Jonathan Strange & Mr Norrell

Jonathan Strange & Mr Norrell, the acclaimed bestselling epic novel of magical realism set 200 years ago, was adapted by the BBC to a 7-episode miniseries that was itself quite epic, with over 1000 effects shots handled by London's Milk VFX. Creative COW's Tim Wilson spoke with Milk CEO Will Cohen about his team's work on the series, starting with his own enthusiasm as a fan of the novel. To use that word once more, it's an epic conversation about adapting novels, carefully managing budgeted creativity, and collaboration. Books, televised cinematic storytelling, VFX, good conversation, and magic: if any of those is your cup of tea, you won't want to miss this.

Editorial, Feature, People / Interview
Tim Wilson
TV & Movie Appreciation
FuseFX & Emmy-Nominated Work for American Horror Story

FuseFX & Emmy-Nominated Work for American Horror Story

Burbank-based effects house’s freaky work for hit FX series continues to draw accolades.

Editorial, Feature, People / Interview
FuseFX
TV & Movie Appreciation
Pixels: Going From 8-bits to Epic is No Game

Pixels: Going From 8-bits to Epic is No Game

Classic video game characters like Pac-Man and Donkey Kong sent by aliens to destroy Earth? No worries! Digital Domain and Sony Pictures Imageworks are on the case. It turns out that integrating 8-bit characters into a world recognizable as our own is a lot harder than it looks. It was also a lot of fun for everyone involved, and hearing about it from the two VFX supervisors will be a lot of fun for you too.

Feature, People / Interview
Tim Wilson
TV & Movie Appreciation
VFX from a tropical paradise

VFX from a tropical paradise

In today’s landscape, post-production companies are no longer restricted by geography. Husband and wife VFX team Lindsay and Jamie Hallett have based their business in Maui, while working on some of filmdom's biggest franchises and features: Avengers, Iron Man, X-Men, Ant-Man, American Sniper, and the latest installment of the Divergent seris, Insurgent. Here are some of their insights on remote collaboration. VERY remote.

Editorial, Feature, People / Interview
cineSync VFX
MORE
© 2016 CreativeCOW.net All Rights Reserved
[TOP]