Generative AI Is “Not To Replace The Human Aspect of Getting a Performance,” Asserts Digital Domain CTO

10.16.23Hanno Basse stated that a "human audience wants to connect with human beings," during a keynote Monday at the Society of Motion Picture and Television Engineers' Media Technology Summit.

Digital Domain CTO Hanno Basse sees a lot of potential for the use of generative AI and machine learning in visual effects, but “not to replace the human aspect of getting a performance” with actors.

“The human audience want to connect with human beings,” he asserted in a keynote Monday at the Society of Motion Picture and Television Engineers’ Media Technology Summit, during which he reviewed VFX studio Digital Domain’s recent work in digital humans, such as creating 43 minutes — several hundred shots — of actress Tatiana Maslany’s performance as the title character in Marvel’s She Hulk. But while this and other examples involved machine learning in its techniques, he emphasized, “every example has the performance of a real actor underlying it. That’s how you get the real emotion and performance. I don’t see that changing.”

The use of generative AI is a thorny topic in SAG-AFTRA’s current strike and contract negotiations, as it was for the WGA, which recently ended its strike and ratified a new contract with AI protections.

During his keynote at the Ray Dolby Ballroom in Hollywood, Basse provided an overview of tools such as DD’s proprietary Charlatan face swapping tech, which involves machine learning and was used on the Amazon series Citadel to create actor-driven performances on characters with the appearance of JFK, FDR and Winston Churchill. “We can turn anybody into anybody … and you will not be able to see the difference,” Basse said, noting the work isn’t easy and people won’t be able to do this at home “very soon, but I think it’s a matter of time.”

DD’s latest proprietary tools include Charlatan Geo, which can be used to enable markerless motion capture. “It’s very accurate and very fast,” Basse said, noting that this allows actors to work without a helmet camera. “It’s easier for the actors … and more cost effective to capture stuff that way.”

Basse believes generative AI and machine learning can assist with other types of VFX work, particularly for ideation and communication, as well as removing “grunt” work. “Often even very creative people don’t know what they want,” he said, noting that a 3D team might work for weeks or months on prototypes that might not be used in a finished project. “[Now] we can do this in 2D with generative AI and rapidly create 50-100 different [sample] versions. … The communication becomes so much simpler.”

DD also has developed a tool that determines how cloth (clothing) will move on a human body in a given pose. Basse reported that DC’s Blue Beetle was DD’s first major application of this technology, used in more than 300 shots in the movie. Without the machine learning tool, he added, “It would have been too expensive.”

Looking ahead, during the keynote Basse also reported on a developing generative AI-driven tool being designed by DD to convert “text to animation.” He admits there’s “a lot of work” still to do in this area. “We believe with these types of technology we can rapidly accelerate our animation work and take out the grunt work.”

Prototype work shown during the keynote included a developing autonomous virtual human, which was a digital robot (still in the uncanny valley) driven by DD’s Charlatan engine. Said Basse, “She can get snippy and upset if you push her too hard.”

The annual SMPTE Media Technology Summit runs through Thursday.