3D Face Animation Generation from Audio Using Convolutional Networks Evrişimsel Aǧlar ile Sesten 3B Yüz Animasyonu Üretilmesi

Ünlü T., İnceoğlu A., Yilmaz E. O., Sariel S.

30th Signal Processing and Communications Applications Conference, SIU 2022, Safranbolu, Türkiye, 15 - 18 Mayıs 2022 identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.1109/siu55565.2022.9864734
  • Basıldığı Şehir: Safranbolu
  • Basıldığı Ülke: Türkiye
  • Anahtar Kelimeler: 3D Face Animation from Audio, Deep Learning, Speech Processing
  • İstanbul Teknik Üniversitesi Adresli: Evet

Özet

© 2022 IEEE.3D facial animation generation from audio problem is drawing attention as it is demanded for generating artificial characters in games and movies. In the literature, several studies address this problem. However, the generated facial animations are far away from being realistic. In this work, we represent faces with Facial Action Coding System (FACS) and collect a 37-minute-long dataset. We develop convolutional and transformer based models. It is observed that the trained model is able to generate animations that can be used in video games and virtual reality applications, even with novel speaker audio data of speakers it has never seen in the training data.