Why you should Empirically Evaluate your AI Tool - From SPOSH to yaPOSH

Jakub Gemrot, Martin Černý, Cyril Brom

2014

Abstract

The autonomous agents community has been developing specific agent-oriented programming languages for more than two decades. Some of the languages have been considered by academia as possible tools for developing artificial intelligence (AI) for non-player characters in computer games. However, as most of the research related to the development of new AI languages within the agent community does not reach production quality, they are seldom adopted by the games industry. As our experience has shown, it is not only the actual language that matters. The toolchain supporting the language and its integration (or lack thereof) with a development environment can make or break the success of the language in practical applications. In this paper, we describe our methodology for evaluating AI languages and associated tools in practice based on controlled experiments with programmers and/or game designers. The methodology is demonstrated on our development and evaluation of SPOSH and yaPOSH high level agent behavior languages. We show that incomplete development support may prevent the tool from giving any benefit to developers at all. We also present our experience from transferring knowledge gained during yaPOSH development to actual AI design for an upcoming AAA game.

Paper Citation

in Harvard Style

Gemrot J., Černý M. and Brom C. (2014). Why you should Empirically Evaluate your AI Tool - From SPOSH to yaPOSH . In Proceedings of the 6th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-758-015-4, pages 461-468. DOI: 10.5220/0004818604610468

in Bibtex Style

@conference{icaart14,
author={Jakub Gemrot and Martin Černý and Cyril Brom},
title={Why you should Empirically Evaluate your AI Tool - From SPOSH to yaPOSH},
booktitle={Proceedings of the 6th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2014},
pages={461-468},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004818604610468},
isbn={978-989-758-015-4},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - Why you should Empirically Evaluate your AI Tool - From SPOSH to yaPOSH
SN - 978-989-758-015-4
AU - Gemrot J.
AU - Černý M.
AU - Brom C.
PY - 2014
SP - 461
EP - 468
DO - 10.5220/0004818604610468