REAL-TIME LIPTRACKING FOR SYNTHETIC FACE ANIMATION WITH FEEDBACK LOOP

Franck Luthon, Brice Beaumesnil

2006

Abstract

This article deals with facial segmentation and liptracking with feedback control for real-time animation of a synthetic 3D face model. Straightforward approaches consist in two successive steps: video analysis then synthesis. Our approach departs from the previous ones in that we build a global analysis/synthesis processing loop, where the image analysis needs the 3D synthesis and conversely. A first facial segmentation is computed according to which the 3D face model is positionned. Then the feedback loop, implemented from the 3D animated model back to the input pixel segmentation algorithm, helps to correct some (few) control points that were badly tracked, which are detected by measuring the vertical distance between lip contour and corresponding 3D face model. When this distance is too big, we re-enter into the image segmentation process and zoom-in inside a few regions of interest (ROI) where the algorithm is run again, with a new set of tuning parameters better suited to the pixel neighborhood context. In that way, the face segmentation is refined in order to extract more precise parameters. This approach is inspired from control theory with closed-loop systems. The contribution of the paper is to use simple image processing techniques, but to improve segmentation through the feedback loop. Results show that real-time and robust performances are achievable under real-world conditions, which are two key issues for face and lip tracking applications.

Paper Citation

in Harvard Style

Luthon F. and Beaumesnil B. (2006). REAL-TIME LIPTRACKING FOR SYNTHETIC FACE ANIMATION WITH FEEDBACK LOOP . In Proceedings of the First International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, ISBN 972-8865-40-6, pages 402-407. DOI: 10.5220/0001367604020407

in Bibtex Style

@conference{visapp06,
author={Franck Luthon and Brice Beaumesnil},
title={REAL-TIME LIPTRACKING FOR SYNTHETIC FACE ANIMATION WITH FEEDBACK LOOP},
booktitle={Proceedings of the First International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP,},
year={2006},
pages={402-407},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001367604020407},
isbn={972-8865-40-6},
}

in EndNote Style

TY - CONF
JO - Proceedings of the First International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP,
TI - REAL-TIME LIPTRACKING FOR SYNTHETIC FACE ANIMATION WITH FEEDBACK LOOP
SN - 972-8865-40-6
AU - Luthon F.
AU - Beaumesnil B.
PY - 2006
SP - 402
EP - 407
DO - 10.5220/0001367604020407