NEW ELECTRONIC MULTI-MEDIA ASSESSMENT SYSTEM

Simon Chester, Giles Tewkesbury, David Sanders

Systems Engineering Research Group, University of Portsmouth, Anglesea Road, Portsmouth, United Kingdom

Peter Salter

Counterpoint MTC Ltd., Worthing, United Kingdom

Keywords: e-Learning, Assessment.

Abstract: Assessment for learning is a new approach to assessment. Assessment for learning is the process of seeking

and interpreting evidence for use by learners and their teachers to raise pupils’ achievement. The key

principle of Assessment for Learning is that pupils will improve most if they understand the aim of their

learning, where they are in relation to this aim and how they can achieve the aim (or close the gap in their

knowledge). Studies have shown that student achievement has been raised significantly where this method

of assessment has been implemented. This paper examines existing electronic assessment systems and then

describes the creation of a new multi-user system that assists teachers in the implementation of assessment

for learning in the classroom.

1 INTRODUCTION

In order to develop a new system for electronic

assessment, an analysis of existing systems and

modes of assessment was first researched. A

literature search was conducted and five main

methods of assessment were reviewed. Assessment

for learning was identified as an assessment method

that required new tools to aid teachers to implement

this method in the classroom. A search for existing

assessment systems was also conducted and three

systems were identified. None of the systems

reviewed allowed teachers within the school to share

common information. Therefore, teachers’ time was

spent performing administrative tasks for their

assessment systems rather than teaching.

The new system presented here aims to overcome

the problem of entering and maintaining school-

wide information by introducing software tools to

share relevant information between teachers. The

new system also aims to provide teachers and

students with the functionality needed to implement

assessment for learning. This new system was

created by using a new centralised database system

and a central administration web application. The

results of this work are presented.

1.1 Modes of Assessment

There are five main types of assessment

(Assessment Reform Group, 2002):

• Formative assessment

• Summative assessment

• Ipsative assessment

• Self-assessment

• Assessment for learning

Formative assessment is that which takes place

on a regular basis involving both teacher and student

in discussion about the students learning and work

(Andrews, Jane as part of a review of: Torrance, H;

Pryor, J, 2004). Feedback that is specific and

diagnostic is given to individual students and the

teacher will be concerned with moving students on

as learners. This monitoring of students’ progress

provides teachers with an on-going record of their

students’ progress and also provides feedback on

their own effectiveness in planning and teaching.

Whilst formative assessment encourages teachers to

listen to their students and identify the next steps

required, it will not be effective unless the teacher is

clear about what it is that they want to find out and

what they are looking for (MacKrill, D, 2004).

320

Chester S., Tewkesbury G., Sanders D. and Salter P. (2006).

NEW ELECTRONIC MULTI-MEDIA ASSESSMENT SYSTEM.

In Proceedings of WEBIST 2006 - Second International Conference on Web Information Systems and Technologies - Society, e-Business and

e-Government / e-Learning, pages 320-324

DOI: 10.5220/0001258303200324

 SciTePress

Summative assessment is used to provide a

summary of a student’s progress at a given point in

time measured against criteria, such as National

Curriculum Levels (Wininger, Steven R, 2005). It

will include records of the formative assessments

made over time. A common form of summative

assessment is, for example, an end of year report,

and in music, an instrumental grade exam.

Summative assessment is important but as Harlen et

al commented “its prime purpose is not so much to

influence teaching”. MacKrill (2004) suggests that

Teachers have been reluctant to adopt summative

assessment because it has been closely associated

with unpopular Government driven data collection

initiatives.

Ipsative assessment informs students how they

have performed and progressed, compared with their

own previous performance or efforts (Harlen et al

1994). It is most effective when students themselves

are involved in the process and promotes

independent learning. However, it is often more

successful when students and teachers have access to

previous work. In subjects that are only timetabled

once a week, like music and drama for instance, this

is even more important but is difficult to implement

because of the type of work being assessed, which is

invariably practical in nature (MacKrill, D, 2004).

Self-assessment forms part of the process of

formative assessment and can improve student

motivation whilst providing the teacher with

important information regarding a student’s

understanding and perception (Daniel R, 2001).

Pratt & Stephens give self-assessment a priority in

their publication “supporting the National

Curriculum for music” (Pratt, G & Stephens, J.,

1995).

In the late 1990’s, a new approach to assessment

began to be considered by Black and William (1998)

who showed that where formative assessment was

used to promote learning, student achievement was

increased significantly. However, they noted that

this would require changes in classroom practice.

There is a clear difference between assessment of

learning and assessment for learning. Assessment of

learning frequently includes both formative and

summative assessment and involves assessing what

students have learnt, often including marks or grades

from tests and assignments (DfES, 2004).

Assessment for

learning has a different focus and is

defined as “the process of seeking and interpreting

evidence for use by learners and their teachers to

decide where the learners are in their learning, where

they need to go and how best to get there”

(Assessment Reform Group, 2002).

Improving learning through assessment depends

on five key factors:

• provision of effective feedback to students;

• active involvement of students in their own

learning;

• adjusting teaching to take account of the

results of assessment;

• recognition of the profound influence

assessment has on the motivation and self-

esteem of students, both of which are

crucial influences on learning;

• need for students to be able to assess

themselves and understand how to improve.

(Assessment Reform Group, 1999)

Assessment for Learning is an area for

development in schools that will require changes in

teacher practice (MacKrill, D., 2004).

1.2 Existing Electronic Assessment

Systems

The KAAN Keyboard And Audio Network System

(KAAN) has been produced by the collaborating

company. This system was created to assist teaching

of music in the classroom. From this, another

system was then developed to accommodate the

assessment of students’ work in a wider variety of

subjects. The new system was marketed by the

collaborating company under the brand name

‘eSAAMS’ (Electronic Student Assessment And

data Management System). The creation of these

systems is described in Lassauniere (2003).

Student Information Management System

(SIMS) is used by some schools in the UK. This

product, produced by Capita Educations Services

(Capita Education Services: SIMS, n.d.), provides

functionality to store a wide range of administrative

information (such as: students, classes, formal

assessments, staff, etc). Typically, this software is

primarily designed for school administrators.

Although it may be used by Teachers, typically it is

not used in the classroom.

AssessIT is an interactive software system,

produced by Pearson Phoenix (n.d.), which tracks

individual pupil attainment and group achievements.

NEW ELECTRONIC MULTI-MEDIA ASSESSMENT SYSTEM

321

It supports teachers when monitoring and measuring

progress and alerts them when pupils fall below

targets. The software places emphasis on

implementing initiatives and promoting individual

pupil attainment and group achievements.

1.3 Requirements of the New System

An electronic assessment system was made available

by the collaborating company at the start of this

work. However, it was designed for use by one

teacher at a time on a single PC. Although it was

possible for the software to be installed on many

computers, each instance of the software required its

own separate database. This caused problems

because these separate databases contained some

information that was common to the whole school

(not just a particular teacher) and there was no

mechanism for synchronising changes to this

information across the disparate databases. The

research described aimed to create a new system for

sharing information between teachers.

2 DESCRIPTION OF THE NEW

SYSTEM

The sharing of information involved setting-up a

new central database. Figure 1 shows the central

database on a server connected to a school’s

computer network. The database server was used as

a single location for storing school-wide

information. A number of database systems were

evaluated including ‘off-the-shelf’ products and

some open-source systems. Microsoft SQL Server

2000 was identified as the most suitable system

because it provided close integration with the

software language used. Initial systems analysis was

conducted and key concepts were identified. These

concepts were then used to create a new database

structure which was implemented on the central

database server.

Figure 1: Components of the new assessment syste

WEBIST 2006 - E-LEARNING

322

In the new system, the eSAAMS software

regularly performed a backup routine which made a

copy of the database to a backup storage location on

the server. This operation was performed by each

instance of the assessment software to the same

location on the network. The Multi-user Data

Collection system was contained on the server. This

new software component collected information from

these distributed database backups across teachers

PCs’ and synchronised it with data in the new

central database.

Finally, a new web-based application was created

to allow students to view a portfolio of their work

from any computer within a school’s computer

network. The web-application was located on the

server. Students interacted with the web-application

by using a web browser, as shown in Figure 1.

Much of the information in the central database

was to be sourced from the distributed teacher

databases. However, some information could not be

gathered in this manner since there was no way of

determining which distributed database was the

authoritative source. Therefore, a new central

administration system to allow teachers and school

administrators to create and manage school-wide

information centrally was created. This web-

application was integrated with the Intranet

ePortfolio system described previously in order to

create a single user interface for administering

Student information. These systems are shown in

Figure 1 as the ‘Intranet ePortfolio & Central

Administration Web Application’.

The eSAAMS electronic assessment software

allowed Teacher’s to work in an ‘offline’ mode

when disconnected from a network. It was therefore

necessary to create a software application for

copying centrally administered information from the

central database system to the database for each

electronic assessment system. This ‘eSAAMS

Importer’ system is shown in Figure 1.

In order for the software applications to be able

to communicate with the central database system, it

was necessary to create software components that

could be used to store and retrieve data from the

database. Different models for data-access were

evaluated and the application of an n-tier model was

investigated using Microsoft Visual Studio 2003 and

Microsoft Visual Studio 2005 (Microsoft Visual

Studio, n.d.).

In particular, methods for automatically

generating data-access code and database stored

procedures were investigated. In particular, one

code-generation tool was identified called ‘My

Generation’ (My Generation Software, n.d.). This

tool was designed to create templates for generating

database stored procedures and code to access a

database from visual studio.

3 RESULTS

The electronic assessment system made available by

the collaborating company at the start of this work

has been installed and tested in over 100 schools.

The software, known as eSAAMS (eSAAMS

Software, n.d.), allowed Teachers to capture and

record evidence of Student’s work. This information

was stored in a database on a Teacher’s computer.

Code generation software was used to create

templates that were then used to generate stored

procedures and re-usable software components to

access the central database. These components

would normally have to be created manually. The

use of code generation reduced the time needed to

create the data access software used by the new

software applications described here.

A new Multi-User Data Collection System was

created that collected information from many single-

user databases (one database per Teacher) and

compiled this into the central database (school-wide

information). This system was tested using a test to

fail method and approved by the collaborating

company.

An Intranet ePortfolio system for showing

student report information was then produced. The

system provided students with the ability to view a

portfolio of their work. This allowed Students to

view feedback from assessments of their work

without increasing the workload for teachers. The

provision of effective feedback to students is one of

the key factors in implementing Assessment for

Learning (Assessment Reform Group, 1999). This

software helped Teachers to implement Assessment

for Learning in the classroom. The information

displayed to Students was sourced from the central

database system. This system was tested and

approved by Teachers in 22 schools.

A new central administration system to allow

teachers and school administrators to create and

NEW ELECTRONIC MULTI-MEDIA ASSESSMENT SYSTEM

323

manage school-wide information centrally was

created. The system will be tested by the

collaborating company using a ‘test to fail’ method.

Also, a software application for copying centrally

administered information from the central database

system to the database for each electronic

assessment system has been created. The system will

be tested by the collaborating company using a ‘test

to fail’ method.

The completed system will be beta-tested in a

two schools. At each beta testing site, the system

will be used by at least two teachers and one school

administrator. User interface experience information

and usability information will be obtained through

questionnaires issued to users and by direct

observation. Also, information will be retrieved

from error log files on the server and teachers’ PC’s.

4 DISCUSSION AND

CONCULSIONS

A new electronic assessment system was created and

a number of its components were tested by the

collaborating company and classroom teachers. This

was the first intelligent multi-media assessment

system to assist teachers implement assessment for

learning practices in the classroom. Other systems,

described previously, have tended to be

administrative and designed for use by a school

administrator.

Information in the central database was sourced

from the backup databases by the new multi-user

data collection software. The information in these

databases was not synchronised so, inevitably,

information could be duplicated. Since each teacher

had equal administrative rights for the information

contained within their database, the multi-user data

collection software was unable to determine which

database had the correct version of each entity.

Therefore, information was processed on a ‘first-in

wins’ basis. This was undesirable because there was

no reliable way to determine which client database

would be used to source school-wide information. In

the future, new electronic assessment software to be

used by teachers in the classroom will be created.

Unlike the existing software, the information

captured will be committed directly to the central

database therefore avoiding unnecessary duplication

of data described above.

REFERENCES

Andrews, J. (Review of: Torrance, H; Pryor, J) (2004).

Investigating Formative Assessment: Teaching,

Learning and Assessment in the Classroom, Language

Testing, 21(3), 432-435.

Assessment Reform Group. (1999). Assessment for

Learning: Beyond the black box. Cambridge:

University of Cambridge.

Assessment Reform Group. (2002). Assessment for

Learning: 10 Principles. Cambridge: University of

Cambridge

Black, P. & William, D. (1998). Inside the black box:

Raising standards through classroom assessment.

Retrieved December 1, 2005, from King's College

London School of Education website:

http://www.kcl.ac.uk/depsta/education/publications/bl

ackbox.html.

Capita Education Services: SIMS. (n.d.). Retrieved

December 1, 2005, from Capita Education Services

website: http://home.capitaes.co.uk/sims/index.asp.

Daniel, R. (2001). Self-assessment in performance. British

Journal of Music Education, 18(3), 215-226.

DfES. (2004). Assessment for Learning: Guidance for

Senior leaders. London: DfES.

eSAAMS Software. (n.d.). Retrieved September 27, 2005,

from the eSAAMS website: http://www.esaams.co.uk.

Harlen, W., Gipps, C., Broadfoot, P. & Nuttall, D. (1994).

Assessment and the improvement of education. In

Moon, B. & Mayes, A.S. Teaching and Learning in

the Secondary School. London: Routledge / The Open

University.

Lassauniere, A. (2003). New Music Technology Systems

and Methods to Assist Teachers. Portsmouth:

University of Portsmouth.

MacKrill, D. (2004). MA Dissertation. University of

Sussex.

Microsoft Visual Studio. (n.d.). Retrieved September 27,

2005, from the Microsoft Developer Network website:

http://msdn.microsoft.com/vstudio.

MyGeneration Software. (n.d.). Retrieved September 27,

2005, from the MyGeneration software website:

http://www.mygenerationsoftware.com.

Pearson Phoenix. (n.d.). Retrieved 29

September 2005

from Pearson Phoenix web site:

http://www.pearsonphoenix.com.

Pratt, G & Stephens, J. (1995). Teaching Music in the

National Curriculum. Oxford: Heinemann.

Wininger, S. (2005). Using Your Tests to Teach:

Formative Summative Assessment. Teaching of

Psychology, 32(3), 164-166.

WEBIST 2006 - E-LEARNING

324