VISUAL SCENE AUGMENTATION FOR ENHANCED HUMAN PERCEPTION

Daniel Hahn, Frederik Beutler, Uwe D. Hanebeck

2005

Abstract

In this paper we present an assistive system for hearing-impaired people that consists of a wearable microphone array and an Augmented Reality (AR) system. This system helps the user in communication situations, where many speakers or sources of background noise are present. In order to restore the “cocktail party” effect multiple microphones are used to estimate the position of individual sound sources. In order to allow the user to interact in complex situations with many speakers, an algorithm for estimating the user’s attention is developed. This algorithm determines the sound sources, which are in the user’s focus of attention. It allows the system to discard irrelevant information and enables the user to focus on certain aspects of the surroundings. Based on the user’s hearing impairment, the perception of the speaker in the focus of attention can be enhanced, e.g. by amplification or using a speech-to-text conversion. A prototype has been built for evaluating this approach. Currently the prototype is able to locate sound beacons in three-dimensional space, to perform a simple focus estimation, and to present floating captions in the Augmented Reality. The prototype uses an intentionally simple user interface, in order to minimize distractions.

Paper Citation

in Harvard Style

Hahn D., Beutler F. and D. Hanebeck U. (2005). VISUAL SCENE AUGMENTATION FOR ENHANCED HUMAN PERCEPTION . In Proceedings of the Second International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO, ISBN 972-8865-30-9, pages 146-153. DOI: 10.5220/0001173701460153

in Bibtex Style

@conference{icinco05,
author={Daniel Hahn and Frederik Beutler and Uwe D. Hanebeck},
title={VISUAL SCENE AUGMENTATION FOR ENHANCED HUMAN PERCEPTION},
booktitle={Proceedings of the Second International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO,},
year={2005},
pages={146-153},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001173701460153},
isbn={972-8865-30-9},
}

in EndNote Style

TY - CONF
JO - Proceedings of the Second International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO,
TI - VISUAL SCENE AUGMENTATION FOR ENHANCED HUMAN PERCEPTION
SN - 972-8865-30-9
AU - Hahn D.
AU - Beutler F.
AU - D. Hanebeck U.
PY - 2005
SP - 146
EP - 153
DO - 10.5220/0001173701460153