Where is that Button Again?! – Towards a Universal GUI Search Engine

Sven Hertling, Markus Schröder, Christian Jilek, Andreas Dengel

2017

Abstract

In feature-rich software a wide range of functionality is spread across various menus, dialog windows, toolbars etc. Remembering where to find each feature is usually very hard, especially if it is not regularly used. We therefore provide a GUI search engine which is universally applicable to a large number of applications. Besides giving an overview of related approaches, we describe three major problems we had to solve, which are analyzing the GUI, understanding the users’ query and executing a suitable solution to find a desired UI element. Based on a user study we evaluated our approach and showed that it is particularly useful if a not regularly used feature is searched for. We already identified much potential for further applications based on our approach.

Paper Citation

in Harvard Style

Hertling S., Schröder M., Jilek C. and Dengel A. (2017). Where is that Button Again?! – Towards a Universal GUI Search Engine . In Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-220-2, pages 217-227. DOI: 10.5220/0006201402170227

in Bibtex Style

@conference{icaart17,
author={Sven Hertling and Markus Schröder and Christian Jilek and Andreas Dengel},
title={Where is that Button Again?! – Towards a Universal GUI Search Engine},
booktitle={Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2017},
pages={217-227},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006201402170227},
isbn={978-989-758-220-2},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Where is that Button Again?! – Towards a Universal GUI Search Engine
SN - 978-989-758-220-2
AU - Hertling S.
AU - Schröder M.
AU - Jilek C.
AU - Dengel A.
PY - 2017
SP - 217
EP - 227
DO - 10.5220/0006201402170227