An Analysis of Factors Affecting Automatic Assessment based on Teacher-mediated Peer Evaluation - The Case of OpenAnswer

Maria De Marsico, Andrea Sterbini, Marco Temperini

2016

Abstract

In this paper we experimentally investigate the influence of several factors on the final performance of an automatic grade prediction system based on teacher-mediated peer assessment. Experiments are carried out by OpenAnswer, a system designed for peer assessment of open-ended questions. It exploits a Bayesian Network to model the students’ learning state and the propagation of information injected in the system by peer grades and by a (partial) grading from the teacher. The relevant variables are characterized by a probability distribution (PD) of their discrete values. We aim at analysing the influence of the initial set up of the PD of these variables on the ability of the system to predict a reliable grade for answers not yet graded by the teacher. We investigate here the influence of the initial choice of the PD for the student’s knowledge (K), especially when we have no information on the class proficiency on the examined skills, and of the PD of the correctness of student’s answers, conditioned by her knowledge, P(C|K). The latter is expressed through different Conditional Probability Tables (CPTs), in turn, to identify the one allowing to achieve the best final results. Moreover we test different strategies to map the final PD for the correctness (C) of an answer, namely the grade that will be returned to the student, onto a single discrete value.

Paper Citation

in Harvard Style

De Marsico M., Sterbini A. and Temperini M. (2016). An Analysis of Factors Affecting Automatic Assessment based on Teacher-mediated Peer Evaluation - The Case of OpenAnswer . In Proceedings of the 8th International Conference on Computer Supported Education - Volume 2: CSEDU, ISBN 978-989-758-179-3, pages 49-61. DOI: 10.5220/0005863200490061

in Bibtex Style

@conference{csedu16,
author={Maria De Marsico and Andrea Sterbini and Marco Temperini},
title={An Analysis of Factors Affecting Automatic Assessment based on Teacher-mediated Peer Evaluation - The Case of OpenAnswer},
booktitle={Proceedings of the 8th International Conference on Computer Supported Education - Volume 2: CSEDU,},
year={2016},
pages={49-61},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005863200490061},
isbn={978-989-758-179-3},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 8th International Conference on Computer Supported Education - Volume 2: CSEDU,
TI - An Analysis of Factors Affecting Automatic Assessment based on Teacher-mediated Peer Evaluation - The Case of OpenAnswer
SN - 978-989-758-179-3
AU - De Marsico M.
AU - Sterbini A.
AU - Temperini M.
PY - 2016
SP - 49
EP - 61
DO - 10.5220/0005863200490061