At the Games Developers Conference in San Francisco last week, video game publisher and developer Epic Games showed off something incredibly cool. The studio wants to build digital humans that are indistinguishable from real ones. And they’re very, very close to succeeding.
The company hosted a State of Unreal presentation to demonstrate the new capabilities of its graphics platform Unreal Engine. It had a few demos to show off, each amazing in their own way.
First off was a showcase of Epic Games’ ray-tracing technology. It features an elevator scene from the Star Wars Universe, showing off various types of real-time lighting and reflection effects, using Nvidia’s RTX technology. It’s been “a dream of the graphics and visualization industry for years,” Nvidia senior VP Tony Tamasi said, but it’s only now that we have graphics chips powerful enough to accomplish it
Next up is something called the ‘Osiris monologue’, which shows how facial animations can be mapped from a human actor onto a digital character. Working with 3Lateral’s ‘Meta Human Framework’ technology, they captured veteran actor Andy Serkis’ performance and digitally recreated it. Instead of modelling animations from individual frames captured of actors, game developers (and even movie CGI artists) can now map directly from a face. “Instead of shooting individual shapes, we are now shooting a performance,” a representative from Project Spotlight, the initiative working on this, said in a video.
It might already be hard for you to swallow how realistic Andy Serkis looks up there. In that case, you’re going to flip for this. The video below is the exact same performance. Except, instead of Serkis’ mug, Project Spotlight mapped his facial expressions and animations to 3Lateral’s fictional lizard alien character named Osiris Black. As you can tell, the detail of animation and realism is just astonishing. The best part? Both this video and the one above were rendered in real-time, no post-production editing whatsoever.
That in itself is a pretty amazing development for the entertainment industry. We now have at our fingertips graphics technology we’ve only ever dreamed of. So it’s saying something that the last entry on this list is somehow even better.
This last video is the result of a partnership between Epic Games, Tencent, Vicon, Cubic Motion, and 3Lateral. It’s basically full-body motion capture and facial mapping of actor Alexa Lee in real-time. Except, that’s not her face in the video, but instead that of actor Bingie Jiang. ‘Siren’ also part of Project Spotlight, is a look to what the future of interactive entertainment could be like. While you may be familiar with motion capture technology thanks to Hollywood, the rig on her head is what’s capturing the expressions of her face, to be rendered by the digital woman.
Imagine a future where VR games are now the standard, where sci-fi novels have told you that your digital avatar will look like whatever you want but will behave exactly as you do. That’s also the premise for ‘Ready Player One’, which was recently also released as a movie. And while we’re a long way from this technology releasing, a few improvements and miniaturisation of the hardware are all that’s needed. Fingers crossed.