Precise Estimation of Reading Activities with Face Image and Read Aloud Voice

Kyota Aoki, Shuichi Tashiro, Shu Aoki

2017

Abstract

In Japanese public primary schools, every pupil may use an ICT device individually and simultaneously. In normal primary school, a few teachers must teach all pupils in a class. It is difficult to help all pupils to use an ICT device. For using ICT devices individually in a normal class, the ICT device help its’ user by itself. To help the user, the ICT device must understand the state of the user. To help a teacher, it must precisely understand the users' reading activities. Reading ability is the base of all subjects. It is important that pupils acqure reading ability. This paper proposes a method to recognize the precise reading activity of a user with read aloud voices and facial images, shows its implementation, and experimental results. With the cooperative analysis of a read aloud voice from a microphone and a movement of a mouth from a camera, our implementation enables to estimate the action of read aloud much more precisely. The timing of a read aloud action is estimated in phrase by phrase manner. In-vitro experiments confirm the performance of our implementation.

Paper Citation

in Harvard Style

Aoki K., Tashiro S. and Aoki S. (2017). Precise Estimation of Reading Activities with Face Image and Read Aloud Voice . In Proceedings of the 9th International Conference on Computer Supported Education - Volume 1: CSEDU, ISBN 978-989-758-239-4, pages 315-322. DOI: 10.5220/0006315603150322

in Bibtex Style

@conference{csedu17,
author={Kyota Aoki and Shuichi Tashiro and Shu Aoki},
title={Precise Estimation of Reading Activities with Face Image and Read Aloud Voice},
booktitle={Proceedings of the 9th International Conference on Computer Supported Education - Volume 1: CSEDU,},
year={2017},
pages={315-322},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006315603150322},
isbn={978-989-758-239-4},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Computer Supported Education - Volume 1: CSEDU,
TI - Precise Estimation of Reading Activities with Face Image and Read Aloud Voice
SN - 978-989-758-239-4
AU - Aoki K.
AU - Tashiro S.
AU - Aoki S.
PY - 2017
SP - 315
EP - 322
DO - 10.5220/0006315603150322