Digital Assisted Communication

Paula Escudeiro, Nuno Escudeiro, Marcelo Norberto, Jorge Lopes, Fernando Soares

2017

Abstract

The communication with the deaf community can prove to be very challenging without the use of sign language. There is a considerable difference between sign and written language as they differ in both syntax and semantics. The work described in this paper addresses the development of a bidirectional translator between several sign languages and their respective text, as well as the evaluation methods and results of those tools. A multiplayer game is using the translator is also described on this paper. The translator from sign language to text employs two devices, namely the Microsoft Kinect and 5DT Sensor Gloves in order to gather data about the motion and shape of the hands. This translator is being adapted to allow the communication with the blind as well. The Quantitative Evaluation Framework (QEF) and the ten-fold cross-validation methods were used to evaluate the project and show promising results. Also, the product goes through a validation process by sign language experts and deaf users who provide their feedback answering a questionnaire. The translator exhibits a precision higher than 90% and the projects overall quality rates are close to 90% based on the QEF.

Paper Citation

in Harvard Style

Escudeiro P., Escudeiro N., Norberto M., Lopes J. and Soares F. (2017). Digital Assisted Communication . In Proceedings of the 13th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST, ISBN 978-989-758-246-2, pages 395-402. DOI: 10.5220/0006377903950402

in Bibtex Style

@conference{webist17,
author={Paula Escudeiro and Nuno Escudeiro and Marcelo Norberto and Jorge Lopes and Fernando Soares},
title={Digital Assisted Communication},
booktitle={Proceedings of the 13th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,},
year={2017},
pages={395-402},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006377903950402},
isbn={978-989-758-246-2},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 13th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,
TI - Digital Assisted Communication
SN - 978-989-758-246-2
AU - Escudeiro P.
AU - Escudeiro N.
AU - Norberto M.
AU - Lopes J.
AU - Soares F.
PY - 2017
SP - 395
EP - 402
DO - 10.5220/0006377903950402