Assessing Facial Expressions in Virtual Reality Environments

Catarina Runa Miranda, Verónica Costa Orvalho

2016

Abstract

Humans rely on facial expressions to transmit information, like mood and intentions, usually not provided by the verbal communication channels. The recent advances in Virtual Reality (VR) at consumer-level (Oculus VR 2014) created a shift in the way we interact with each other and digital media. Today, we can enter a virtual environment and communicate through a 3D character. Hence, to the reproduction of the users’ facial expressions in VR scenarios, we need the on-the-fly animation of the embodied 3D characters. However, current facial animation approaches with Motion Capture (MoCap) are disabled due to persistent partial occlusions produced by the VR headsets. The unique solution available for this occlusion problem is not suitable for consumer-level applications, depending on complex hardware and calibrations. In this work, we propose consumer-level methods for facial MoCap under VR environments. We start by deploying an occlusions-support method for generic facial MoCap systems. Then, we extract facial features to create Random Forests algorithms that accurately estimate emotions and movements in occluded facial regions. Through our novel methods, MoCap approaches are able to track non-occluded facial movements and estimate movements in occluded regions, without additional hardware or tedious calibrations. We deliver and validate solutions to facilitate face-to-face communication through facial expressions in VR environments.

Paper Citation

in Harvard Style

Runa Miranda C. and Costa Orvalho V. (2016). Assessing Facial Expressions in Virtual Reality Environments . In Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2016) ISBN 978-989-758-175-5, pages 486-497. DOI: 10.5220/0005716604860497

in Bibtex Style

@conference{visapp16,
author={Catarina Runa Miranda and Verónica Costa Orvalho},
title={Assessing Facial Expressions in Virtual Reality Environments},
booktitle={Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2016)},
year={2016},
pages={486-497},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005716604860497},
isbn={978-989-758-175-5},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP, (VISIGRAPP 2016)
TI - Assessing Facial Expressions in Virtual Reality Environments
SN - 978-989-758-175-5
AU - Runa Miranda C.
AU - Costa Orvalho V.
PY - 2016
SP - 486
EP - 497
DO - 10.5220/0005716604860497