A dive into the different ways that Artificial Intelligence (AI) can and is being used in the VFX industry.
The community has long enjoyed the benefits of synthesizing training data with graphics, but the domain gap between real and synthetic data has remained a problem, especially for human faces. Researchers have tried to bridge this gap with data mixing, domain adaptation, and domain-adversarial training, but we show that it is possible to synthesize data with minimal domain gap, so that models trained on synthetic data generalize to real in-the-wild datasets.
In this presentation, the team describes how to combine a procedurally-generated parametric 3D face model with a comprehensive library of hand-crafted assets to render training images with unprecedented realism and diversity. We train machine learning systems for face-related tasks such as landmark localization and face parsing, showing that synthetic data can both match real data inaccuracy as well as open up new approaches where manual labeling would be impossible.
Image Credit: © Microsoft. All Rights Reserved.
Project Contributors: Erroll Wood, Tadas Baltrusaitis, Charlie Hewitt, Sebastian Dziadzio, Matthew Johnson, Virginia Estellers, Thomas J. Cashman, Jamie Shotton.
SPEAKER
Matthew Johnson
Principal Scientist // Microsoft
Matthew Johnson is a principal scientist working on computer vision and augmented reality at Microsoft's research lab in Cambridge, UK, where he has worked since 2014. Prior to working at Microsoft he was a research software engineer at Nokia on the Point and Find team, where he was a contributor to the HERE City Lens product. In 2011 he was hired by Unicorn Media as Vice President of Engineering, a position he held until it was acquired by Brightcove in 2014, where he was a Distinguished Engineer.
Matthew holds a Ph.D. in Computer Vision and Machine Learning from the University of Cambridge.
Website // Twitter // LinkedIn
MODERATOR
Virginia Estellers
Scientist // Microsoft Mixed Reality Lab
Researcher in the domain of machine learning and computer vision. In particular, she has experience in the following topics: sparsity and integration of prior knowledge in signal reconstruction, geometric models in image processing and development of fast and efficient minimization algorithms for imaging.
A dive into the different ways that Artificial Intelligence (AI) can and is being used in the VFX industry.
The community has long enjoyed the benefits of synthesizing training data with graphics, but the domain gap between real and synthetic data has remained a problem, especially for human faces. Researchers have tried to bridge this gap with data mixing, domain adaptation, and domain-adversarial training, but we show that it is possible to synthesize data with minimal domain gap, so that models trained on synthetic data generalize to real in-the-wild datasets.
In this presentation, the team describes how to combine a procedurally-generated parametric 3D face model with a comprehensive library of hand-crafted assets to render training images with unprecedented realism and diversity. We train machine learning systems for face-related tasks such as landmark localization and face parsing, showing that synthetic data can both match real data inaccuracy as well as open up new approaches where manual labeling would be impossible.
Image Credit: © Microsoft. All Rights Reserved.
Project Contributors: Erroll Wood, Tadas Baltrusaitis, Charlie Hewitt, Sebastian Dziadzio, Matthew Johnson, Virginia Estellers, Thomas J. Cashman, Jamie Shotton.
SPEAKER
Matthew Johnson
Principal Scientist // Microsoft
Matthew Johnson is a principal scientist working on computer vision and augmented reality at Microsoft's research lab in Cambridge, UK, where he has worked since 2014. Prior to working at Microsoft he was a research software engineer at Nokia on the Point and Find team, where he was a contributor to the HERE City Lens product. In 2011 he was hired by Unicorn Media as Vice President of Engineering, a position he held until it was acquired by Brightcove in 2014, where he was a Distinguished Engineer.
Matthew holds a Ph.D. in Computer Vision and Machine Learning from the University of Cambridge.
Website // Twitter // LinkedIn
MODERATOR
Virginia Estellers
Scientist // Microsoft Mixed Reality Lab
Researcher in the domain of machine learning and computer vision. In particular, she has experience in the following topics: sparsity and integration of prior knowledge in signal reconstruction, geometric models in image processing and development of fast and efficient minimization algorithms for imaging.
