Makeup Lamps
Disney Research’s lab in Zurich has developed the first system for live dynamic augmentation of human faces. Using projector-based illumination, they are able to alter the appearance of human performers.
According to the research paper, the key challenge of live augmentation is latency — an image is generated according to a specific pose, but is displayed on a different facial configuration by the time it is projected. Therefore, their system aims at reducing latency during every step of the process, from capture, through processing, to projection. Using infrared illumination, an optically and computationally aligned high-speed camera detects facial orientation as well as expression. The estimated expression blendshapes are mapped onto a lower dimensional space, and the facial motion and non-rigid deformation are estimated, smoothed and predicted. Finally, the desired appearance is generated using factors such as time, global position, and expression.
The resulting effect is impressive and you can imagine how neat it will be to maybe see it used at a Disney theme park in the future.
Check out the video they released showcasing their technique.