Nvidia has created a realistic 3D model using AI in real time. Earlier this year, the graphics giant announced the great plans it has for artificial intelligence, and now its groundbreaking research has yielded some impressive results.
Neural networks are receiving more and more attention, resulting in more applications for real-time computing. The team taught their AI to render 3D models in real time, which includes rendering each object individually in the virtual world.
Since the 3D modeling and rendering process is nowhere near affordable and takes longer than one would expect, Nvidia used AI models which learned from video rendering and managed to render objects like buildings, trees and vehicles. This could help the company create 3D worlds for gaming, automotive architecture, robotics and virtual reality much faster than it would normally take. The network can mimic real-world locations in a 3D world.
“NVIDIA has been inventing new ways to generate interactive graphics for 25 years, and this is the first time we can do so with a neural network,” said Bryan Catanzaro, vice president of Applied Deep Learning Research at Nvidia, in a news release. “Neural networks — specifically generative models — will change how graphics are created. This will enable developers to create new scenes at a fraction of the traditional cost.”
In this case, Nvidia created a 3D model using AI which mimics a driving game, allowing players to navigate a busy urban scene. The neural network rendered the game to be interactive by using 3D sketches produced using gaming engines currently in use. Their work will be shown publicly at the NeurIPS 2018 conference, which will take place in Montreal.
The researchers taught the neural network to imitate and model the appearance of the real world, including shadows, lighting, materials and dynamics in certain circumstances.
“The capability to model and recreate the dynamics of our visual world is essential to building intelligent agents,” the researchers wrote in their paper. “Apart from purely scientific interests, learning to synthesize continuous visual experiences has a wide range of applications in computer vision, robotics, and computer graphics.”
The Verge notes that this research does raise some moral and ethical questions when it comes to using AI in modeling. AI can be abused to spread false information, which means this method of creating a realistic 3D model using AI could be used to spread propaganda. For example, AI could be used to create videos in which celebrities or politicians are caught saying or doing things they didn’t. However, the company told The Verge that it’s unlikely this will happen.
“Can [this technology] be used for creating content that’s misleading?” The Verge asked.
“Yes. Any technology for rendering can be used to do that,” Catanzaro said.
However, he added that the company is collaborating with various partners who use different methods to detect fake AI work, stressing that this is a “trust issue.”