Компьютерное зрение и распознавание образов
Computer Vision and Pattern Recognition (CVPR), IEEE, June 2016
Информация о проекте
Научные интституты: - University of Erlangen-Nuremberg, Max Planck Institute for Informatics, Stanford University
Исследователи: - Justus Thies, Michael Zollhцfer, Marc Stamminger, Christian Matthias NieЯnerTheobalt
О проекте:
Мы представляем новый подход к реконструкции лицевой активности (эмоции, выражение лица - движение лицевых мускулов и реалистичная передача теней, освещения) актера перенесенной поверх лица цели в потоковой передачи изображения (к примеру ролики с youtube или прямой трансляции) в режиме реального времени.
Более подробные детали по проекту на английском языке, а также на сайте одного из участников проекта.
We present a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam. Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion. To this end, we first address the under-constrained problem of facial identity recovery from monocular video by non-rigid model-based bundling. At run time, we track facial expressions of both source and target video using a dense photometric consistency measure. Reenactment is then achieved by fast and efficient deformation transfer between source and target. The mouth interior that best matches the re-targeted expression is retrieved from the target sequence and warped to produce an accurate fit. Finally, we convincingly re-render the synthesized target face on top of the corresponding video stream such that it seamlessly blends with the real-world illumination. We demonstrate our method in a live setup, where Youtube videos are reenacted in real time.
Более подробные детали по проекту на английском языке, а также на сайте одного из участников проекта.
We present a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam. Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion. To this end, we first address the under-constrained problem of facial identity recovery from monocular video by non-rigid model-based bundling. At run time, we track facial expressions of both source and target video using a dense photometric consistency measure. Reenactment is then achieved by fast and efficient deformation transfer between source and target. The mouth interior that best matches the re-targeted expression is retrieved from the target sequence and warped to produce an accurate fit. Finally, we convincingly re-render the synthesized target face on top of the corresponding video stream such that it seamlessly blends with the real-world illumination. We demonstrate our method in a live setup, where Youtube videos are reenacted in real time.
Личное мнение:
Это не первый подобный проект в этом направлении, но на сегодняшний день данный проект самый впечатляющий. С такими возможностями можно много достичь не только в манипулировании видео материалом, но и в системах управления, автоматизации.
Официальный сайт проекта одного из исследователей Christian Matthias - http://www.graphics.stanford.edu/~niessner...es2016face.html
Дополнительно, если кому интересно, о Техническом зрении в системах управления - http://www.iki.rssi.ru/books/2012tz.pdf