TWO METHODS FOR EVALUATING DYNAMIC ONTOLOGIES

Jaimie Murdock, Cameron Buckner, Colin Allen

2010

Abstract

Ontology evaluation poses a number of difficult challenges requiring different evaluation methodologies, particularly for a “dynamic ontology” representing a complex set of concepts and generated by a combination of automatic and semi-automatic methods. We review evaluation methods that focus solely on syntactic (formal) correctness, on the preservation of semantic structure, or on pragmatic utility. We propose two novel methods for dynamic ontology evaluation and describe the use of these methods for evaluating the different taxonomic representations that are generated at different times or with different amounts of expert feedback. The proposed “volatility” and “violation” scores represent an attempt to merge syntactic and semantic considerations. Volatility calculates the stability of the methods for ontology generation and extension. Violation measures the degree of “ontological fit” to a text corpus representative of the domain. Combined, they support estimation of convergence towards a stable representation of the domain. No method of evaluation can avoid making substantive normative assumptions about what constitutes “correct” representation, but rendering those assumptions explicit can help with the decision about which methods are appropriate for selecting amongst a set of available ontologies or for tuning the design of methods used to generate a hierarchically organized representation of a domain.

Paper Citation

in Harvard Style

Murdock J., Buckner C. and Allen C. (2010). TWO METHODS FOR EVALUATING DYNAMIC ONTOLOGIES . In Proceedings of the International Conference on Knowledge Engineering and Ontology Development - Volume 1: KEOD, (IC3K 2010) ISBN 978-989-8425-29-4, pages 110-122. DOI: 10.5220/0003101601100122

in Bibtex Style

@conference{keod10,
author={Jaimie Murdock and Cameron Buckner and Colin Allen},
title={TWO METHODS FOR EVALUATING DYNAMIC ONTOLOGIES},
booktitle={Proceedings of the International Conference on Knowledge Engineering and Ontology Development - Volume 1: KEOD, (IC3K 2010)},
year={2010},
pages={110-122},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003101601100122},
isbn={978-989-8425-29-4},
}

in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Knowledge Engineering and Ontology Development - Volume 1: KEOD, (IC3K 2010)
TI - TWO METHODS FOR EVALUATING DYNAMIC ONTOLOGIES
SN - 978-989-8425-29-4
AU - Murdock J.
AU - Buckner C.
AU - Allen C.
PY - 2010
SP - 110
EP - 122
DO - 10.5220/0003101601100122