What is a GAN?

How a weird idea became the foundation of cutting-edge AI

T ake any course on machine learning and you’ll invariably encounter Generative Adversarial Networks, or GANs. Understanding them means mastering the surprising power of playing a computer out against itself.

It’s around five o’clock and you’ve just finished your homework.

‘I’m done!’

‘Great! Would you like to play a game?’

‘Yeah! Today we’re playing Monopoly.’

‘Ugh, Monopoly! To be honest, I hate it when one of the players gets all the properties and it becomes impossible for the other one to win.’

‘Come on! Please!’

‘All right then.’

Half an hour later, your and my properties are still quite evenly distributed.

‘See? Monopoly isn’t that bad.’

‘You’re right. I said earlier that because I’m having a problem with an algorithm that I’m building. Somehow one player is notoriously better than the other. I’m having a hard time to fix that.’

‘Huh? How can there be a player in an algorithm?’

‘When you’re dealing with GANs, you have players.’

‘What? What are GANs?’

‘Let me explain. Do you know what a neural network is?’

‘Ermm, no.’

‘A neural network is basically an algorithm that was made to recognize patterns. Facial recognition, for example, relies heavily on neural networks.

‘You give it a heap of unordered data — let’s say pictures of faces — and with time it learns what a human face looks like. This we call training.

‘Once we’ve trained the neural network, we can show it a random image, and it will decide whether it’s a human face or not — with impressive accuracy.’

‘Okay, so a neural network can tell you whether a face is a face. Now what’s a GAN?’

‘A GAN is when we pit two neural networks against one another. Like in a game.’

‘You let the computer play against itself? How stupid is that?’

‘It’s not stupid. It’s genius.’

‘You’ll have to explain that to me…’

‘The first neural network is called the Generator. It generates fake data points and passes them to its opponent. That’s the second network, the Discriminator. Its job is to tell which data point is real and which is fake.

‘First, we train the Discriminator on a set of real and fake data points. The discriminator guesses what is real and what is fake. After that, we unveil the real solution to it. Based on the feedback, the discriminator learns what is fake and what is not.’

‘Kind of like in Police Academy when the students have to tell apart real from fake cash?’

‘Exactly. But now the Generator comes into play. He’s the money forger in our game. We give him real and fake data points to train with. Based on these points, its job is to make new data points which fool the Discriminator.’

‘But wouldn’t it be much better to give the Generator only real data points? Because if you train it on fakes, then it will repeat the mistakes from those. If you let me forge a 300-dollar-bill, the police will catch it rather quickly. ’

‘You’re right! But we don’t want the Generator to get too good too fast. Otherwise the Discriminator can’t keep up.’

‘Oh, that’s the Monopoly problem? Where one player gets too much property in Monopoly? And then it gets boring because the other player can’t win anyway?’

‘Precisely. We want to mislead the Generator from time to time. So that the Discriminator has a chance to keep up.’

‘Okay, so let’s say the Generator has made fake data. Now it will fool the Discriminator, right?’

‘Well, it’s gonna try. The Discriminator will hopefully do a reasonable job at differentiating the real data points from the fake ones. It then informs the Generator about its decisions.’

‘Like when the police will tell a criminal whether he’s convicted or not?’

‘Yep. And based on this decision, the Generator will set to work again. It’ll make new fakes, but based on the feedback it will try to make them better so that they fool the Discriminator.

‘Then it’s the Discriminator’s turn again, and so on and so forth. And at the end of the day, we’ve used a GAN to make a heap of new data points!’

‘Okay cool. But why is that useful?’

‘You can use it for all kinds of things. For example, you can make beautiful new pictures, based on others.

‘You’ve probably heard of deep-fakes. These are pictures that are made with GANs. And have you heard about the apps with which you can age people’s faces?’

‘Pah. Gimmicks!’

‘Oh, they might be useful! For example to catch criminals many years later.

‘GANs are also used to make good descriptions of pictures. This is extremely useful for search engines, but also for visually impaired people. You’re basically teaching a computer how to see, and how to explain what it’s seeing!

‘And we haven’t reached the end. There are so many applications that are yet to be explored.’

‘If you say so… But you said earlier that you’re having a problem with your GAN.’

‘True. It’s the Monopoly problem — even though I give my Generator evough fakes to train on, it gets too good too fast. So the discriminator keeps giving up…’

‘Oh man!’

‘It’s quite a common issue with GANs. Research is still ongoing on how to balance the system out better.’

‘Okay, then I’ll let you get on with your problem. I’ve won this round of Monopoly anyway.’

‘Wait, what? I must have gotten rather distracted by thinking about my GANs… Damn, this round goes to you!’

‘Yippie! I’ll let you work on your GAN Monopoly now.’

‘Thank you. But wait for it — tomorrow’s my round of revenge!’