From 2D to 3D real-time expression transfer for facial animation


Ekmen B., Ekenel H. K.

MULTIMEDIA TOOLS AND APPLICATIONS, cilt.78, ss.12519-12535, 2019 (SCI İndekslerine Giren Dergi) identifier identifier

  • Cilt numarası: 78 Konu: 9
  • Basım Tarihi: 2019
  • Doi Numarası: 10.1007/s11042-018-6785-8
  • Dergi Adı: MULTIMEDIA TOOLS AND APPLICATIONS
  • Sayfa Sayıları: ss.12519-12535

Özet

In this paper, we present a three-stage approach, which creates realistic facial animations by tracking expressions of a human face in 2D and transferring them to a human-like 3D model in real-time. Our calibration-free method, which is based on an average human face, does not require training. The tracking is performed using a single camera to enable several practical applications, for example, using tablets and mobile devices, and the expressions are transferred with a joint-based system to improve the quality and persuasiveness of animations. In the first step of the method, a joint-based facial rig providing mobility to pseudo-muscles is attached to the 3D model. The second stage covers the tracking of 2D positions of the facial landmarks from a single camera view and transfer of 3D relative movement data to move the respective joints on the model. The last step includes the recording of animation using a partially automated key-framing technique. Experiments on the extended Cohn-Kanade dataset using peak frames in frontal-view videos have shown that the presented method produces visually satisfying facial animations.