Learning from Simulation and Games

Sometimes images of objects-of-interest are not available. For instance, rare objects such as a tank. Can we train an object detection model on simulated images? Yes, we can! A model trained on the game GTA-5 detected these objects in YouTube footage.

The game GTA-5 was used to create synthetic images. The images look very photo-realistic. We changed viewpoints, viewing distances, and various weathers - to learn a model that’s robust to changing conditions.

Research was funded by the V1508 program and NATO RTG-144.