NVIDIA’s first fully AI-generated Virtual World

Thanks to so-called GAN networks, Artificial Intelligence has made tremendous progress in image generation in recent months. Examples of visually stunning GAN results abound, including, for example, deceptively real burger photos by Deepmind or the first AI-generated portrait sold for $432K at Christie’s in New York on October 25th. The well-known Deepfakes AI for video manipulation and its various modifications use the GAN principle.

The most recent example comes from Nvidia, who unveiled the new AI system, which can automatically build 3D gaming environments modeled on videos and photos taken in real life. A team led by Bryan Catanzaro, Vice President of Applied Deep Learning at NVIDIA, used a conditional generative neural network as a starting point. From that data, they trained a neural network to render new 3D environments, after it was already trained on existing videos.

Nvidia’s AI also has a lot in common with a current feature of games design known as procedural generation, a popular function for games with large worlds, in which an engine is taught particular rules and an algorithm extrapolates from these rules to automatically create a game world. According to NVIDIA, this “AI breakthrough” will enable developers and artists to create new interactive 3D virtual worlds quicker and cheaper for automotive, gaming, and virtual reality, by training models on videos from the real world.

This new method is in the early stages of development. However, NVIDIA believes this technology will ultimately make it much cheaper and easier to create virtual environments once it’s fleshed out.