AI & Art: Creating Photorealistic Landscapes from Scribbles with NVIDIA GauGAN2

Artificial Intelligence Art refers to when "works of art" are created with the aid of artificial intelligence. To this end, many of these works are generated by GANs (Generative Adversarial Network), a type of machine learning model proposed by Ian Goodfellow and collaborators.

GANs are used to generate synthetic data. To do this, two models compete with each other (i.e., the generator and the discriminator). The generator model seeks to create data from the latent space of a learned distribution, and the discriminator model seeks to infer whether the generated data is artificial or natural. The goal of the generator is to fool the discriminator, which indirectly trains the generator. With training, both models become more proficient at their respective tasks (i.e., forging data and discerning synthetic data from natural data). After training, we have in our hands a generator model capable of creating extremely realistic data (e.g., images).

Want to try it for yourself?

NVIDIA's AI research team recently released a demo of their new model, GauGAN2, which is basically the state-of-the-art in GANs for image generation.

The model allows you to turn "doodles" (the kind you might have done in Windows Paint) into extremely realistic images of landscapes. This allows you, in a few minutes, to generate images of landscapes that until then lived only in your imagination. Let's look at some examples:

Input (Doodle):

Output (Image):

Input (Doodle):

Output (Image):

GauGAN2 is a multimodal model, which also allows images to be created from text commands.

Input ("A forest with snow falling on the trees"):

Interested? Go to this link and have fun!

In this link, you can see a tutorial on how to use the demo (don't be intimidated, it is extremely simple).