Annotating Arguments in a Parliamentary Corpus: An Experience

Mare Koit

Institute of Computer Science, University of Tartu, Narva Mnt 18, Tartu, Estonia

Keywords: Parliamentary Corpus, Argument, Annotation, Knowledge Representation.

Abstract: Estonian parliamentary corpus includes verbatim records of sessions held in the Parliament of Estonia

(Riigikogu) in 1995-2001. An important task of the Riigikogu is the passing of acts and resolutions. A bill

initiated in the Riigikogu will pass three readings, during which it is refined and amended. Negotiation is an

important part of parliamentary discussions. Arguments for and against of the bill and its amendments are

presented by the members of the Parliament in negotiation. In the paper, arguments used in negotiation are

considered. Every argument consists of one or more premises, and a claim (or conclusion). The arguments

and the relations between them (rebuttal, attack, and support) are determined with the aim to create a corpus

where arguments are annotated. Some problems are discussed in relation with annotation. Our further aim is

the automatic recognition of arguments and inter-argument relations in Estonian political texts.

1 INTRODUCTION

Parliament data has always been in the center of the

humanitarian and societal interest with its influential

language and content for the social and political

environment. There are many ongoing initiatives for

compiling digital collections of parliamentary

resources. An overview of the existing resources is

given in CLARIN-PLUS survey on parliament data

(Survey, 2020).

As a rule, parliamentary discussions include

numerous arguments. Analyzing argumentation from

a computational linguistics point of view has recently

led to a new ﬁeld called argumentation mining. The

review of Atkinson et al. (2017) considers the

development of artiﬁcial tools that capture the human

ability to argue. Such systems, being able

automatically extract arguments and relations

between them, can be used when modelling political

argumentation.

Stab and Gurevych (2014) present an approach to

model arguments, their components and relations in

persuasive essays in English. The annotation scheme

includes the annotation of claims and premises as

well as support and attack relations for capturing the

structure of argumentative discourse. The authors

conduct a manual annotation study with three

annotators on 90 persuasive essays. The corpus

updated in (Stab and Gurevych, 2017), consists of

402 argument-annotated persuasive essays.

Amgoud et al. (2015) propose a language for

representing arguments that captures the various

forms of arguments encountered in natural language,

and demonstrate that it is possible to represent attack

and support relations between arguments as formulas

of the same language.

Haddadan et al. (2018) introduce the annotation

guidelines deﬁned for annotating arguments in

political debates. Their annotation scheme includes

the annotation of claims and premises as the

components of arguments.

Menini et al. (2018) have created a corpus of

political speeches where argument pairs are annotated

with the support and attack relations.

Lawrence and Reed (2019) provide a review of

recent advances in argument mining techniques.

In the current paper, we examine negotiations in

the Estonian Parliament (Riigikogu) based on

verbatim records of the sittings. In the records,

repetitions and disﬂuencies are omitted, while

supplementary information such as speaker names are

added. The records (in Estonian) are accessible on the

Web. A corpus is formed that includes a part of the

records from 1995 to 2001 (in total, 13 million

tokens), both for download and on-line searching. We

are looking for arguments presented by the members

of the Parliament (MPs) in negotiations when

proceeding a bill. Our further aim is to create a corpus

where arguments and inter-argument relations are

annotated.

Koit, M.

Annotating Arguments in a Parliamentary Corpus: An Experience.

DOI: 10.5220/0010135102130218

In Proceedings of the 12th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2020) - Volume 2: KEOD, pages 213-218

ISBN: 978-989-758-474-9

213

Figure 1: General structure of proceedings on a bill (source: https://www.riigikogu.ee/en/introduction-and-history/riigikogu-

tasks-organisation-work/what-does-riigikogu/legislative-work/).

The corpus with annotated arguments will be used

to prepare the automatic annotation. We are looking

for formal indicators in Estonian political texts that

can be used to annotate arguments automatically.

The first attempt to analyze and model the formal

structure and relations of arguments in Estonian

political discourse was made in (Koit, 2020). The

current paper continues the analysis and brings out

some problems of annotating arguments.

The remainder of the paper is structured as

follows. In Section 2, we examine randomly selected

discussions in the Riigikogu by using verbatim

records of the sittings. We consider the arguments

presented by the MPs, annotate the premises and

claims of the arguments and determine the inter-

argument relations (attack, rebuttal, and support). We

also consider some problems of annotating the

arguments and relations. Section 3 discusses creating

a corpus where arguments are annotated. Further aim

of creating the corpus is to prepare the automatic

recognition of arguments (argument mining). Section

4 draws conclusions and figures out future work.

2 ARGUMENTS IN POLITICAL

NEGOTIATIONS

In this section, we examine discussions in the

Riigikogu based on verbatim records of the sittings.

We consider the arguments presented by the members

of the parliament in negotiations and determine the

inter-argument relations.

2.1 Empirical Material

Our empirical material is formed by the records of the

Parliament of Estonia – Riigikogu. The records are

accessible on the Web (cf. Riigikogu, 2020). An

important task of the Riigikogu is the passing of acts

and resolutions. Acts are the result of work in

multiple stages. The first stage of legislation involves

the drafting of a bill. During the second stage, the bill

is initiated in the Riigikogu. The bill will then pass

three readings, during which it is refined and

amended. The proceeding of a bill is managed by the

relevant leading committee. After having been passed

by the Riigikogu, the act is sent to the President of the

Republic for proclamation, and is then published in

KEOD 2020 - 12th International Conference on Knowledge Engineering and Ontology Development

214

State Gazette (Riigi Teataja). The general structure on

proceedings of a bill is shown in Figure 1.

The readings have a predetermined structure (cf.

Koit et al., 2019). Every reading includes negotiation.

Figure 2 shows the structure of the 1

reading. The

and the 3

readings also include voting on

amendments and final voting, respectively.

- - 1

reading

- - initiator – Government

Presenter (Minister): Report

{

MP: Question

Presenter: Giving information

}

Co-presenter (a member of leading committee): Report

{

MP: Question

Co-presenter: Giving information

}

- - negotiation

{

MP: argument

}

Figure 2: The structure of readings in the Riigikogu. The

winding brackets ‘{‘ and ‘}’ connect a part that can be

repeated; ‘- -’ starts a comment; MP – any member of

Riigikogu.

For this paper, arguments and inter-argument

relations were annotated in a part of the corpus

(27,768

tokens in total). The following examples are

taken from proceedings on the bill of alcohol. It

should be mentioned that we currently limit us with

annotating the arguments in the negotiation part of the

proceedings and do not consider the arguments that

have been presented in the reports. The examples in

Subsections 2.2 and 2.3 are taken from different

readings.

2.2 Annotating Arguments

An argument is a series of statements in a natural

language, called the premises, intended to determine

the degree of truth of another statement – the claim

(or the conclusion). These parts can be presented in

one or more sentences. There are three types of

relations between the arguments: attack, support, and

rebuttal (Amgoud et al., 2015).

When analyzing persuading essays, Stab and

Gurevych (2014) distinguish the major claim of the

essay and the claim of an (arbitrary) argument. In

parliamentary discussions, we can make similar

distinctions. The major claim, together with its

premises, is given in the beginning of the 1

reading,

in the report of Minister and it is always ‘to approve

the bill’. As a rule, the claim of a supporting argument

presented in following negotiation, coincides with the

major claim. The claim of a rebutting argument is

opposite: ‘do not approve the bill’. The claim of an

attacking argument depends on a previous argument

that is under attack.

In the analyzed negotiations, premises and claims

of arguments and inter-argument relations were

manually annotated by the author of the paper,

following (Stab and Gurevich, 2013), (Amgoud et al.,

2015), and (Haddadan et al., 2018). Therefore, the

current annotation is rather subjective. Still, as a

further work, we plan to involve more annotators as

well as to calculate the inter-annotator agreement.

Following (Amgoud et al, 2015), we use an

annotation scheme where begins and ends of the

components of arguments are labelled. The next

examples illustrate annotations of arguments and

relations. The first two arguments are against and for

the major claim, respectively. The major claim is ‘to

approve the bill on alcohol’.

Example 1: Argument rebutting the major claim.

- - rebutting

Täielikult puudub seaduseelnõus sotsiaalne

dimensioon. […]

The social dimension is fully missing in the bill.

[…]

</premise>

<claim>Seaduse vastuvõtmisega sel kujul

näitame oma rahulolevat suhtumist sellesse, et meil

alkoholi palju tarbitakse, ja sellesse, et meile on

pigem tähtsam saada miljard krooni riigikassasse kui

arvestada seda, mida alkohol teeb rahva tervisele,

perekondadele, kuidas ta soodustab kuritegevust.

If we approve the bill in the existing form then we

express our satisfaction with the high consumption of

alcohol and demonstrate that it is more important for

us to get a billion Kroon for the budget than taking

into account how the alcohol influences the health of

people and families and how it contributes to

delinquency.

</claim>

</argument>

Example 2: Argument supporting the major claim.

- - supporting

[…] joomarlus on Eesti rahvuslik õnnetus. […]

Excessive drinking is a national disaster in

Estonia. […]

</premise>

<claim>

Meil kui rahvaesindajatel on vaja näidata eelkõige

oma suhtumist ja arusaamist selle probleemi

olulisusest Eesti jaoks ja ka tahet seda probleemi

lahendada.

Annotating Arguments in a Parliamentary Corpus: An Experience

215

We as public delegates must express our

understanding of the importance of the problem for

Estonia as well as our intention to solve this problem.

</claim>

</argument>

The next three arguments support or attack some

presented amendments, respectively. It should be

mentioned that the amendments’ motions are not

accessible on the Web therefore it is unclear how they

have been formulated.

Example 3: Argument supporting (some)

amendments.

- - supporting

Poliitiku […] vastutus ja kohustus seisneb selles,

et tagada riigis sellised seadused, mis toimiksid ja

garanteeriksid elujõu kõigile, kes meie riigis elavad,

ja lõppkokkuvõttes turvalisuse kogu rahvale. […]

[…] Responsibility and obligation of a politician

is to guarantee such laws that ensure normal life for

all inhabitants of the country and the safety for all

people. […]

</premise>

<claim>

Ma kutsun täna kõiki hääletama selliste

ettepanekute poolt (ja neid on palju), mis tagavad

Eestis turvalisuse, mis tagavad selle, et meil joodaks

vähem, et alkohol ei oleks nii kättesaadav, et

sotsiaalselt ebaküps noor ei võiks seda iga kell igalt

poolt hankida.

I invite you all to vote for these (numerous)

amendments that will provide the safety in Estonia, in

order to decrease drinking, to decrease the

availability of alcohol and ensure that socially

immature young people can’t get it every time

everywhere.

</claim>

</argument>

Example 4: Argument attacking (some) amendments.

- - attacking

Me näeme, et Eestis on alkoholi liigtarbimine

probleem, kuid probleeme ei ole võimalik lahendada

keelamisega. […]

We see that drinking is a problem in Estonia but

the problems can’t be solved by prohibition. […]

</premise>

<claim>

Seetõttu ei toeta me ettepanekuid, mis

vähendavad alkoholimüügi võimalusi ja samal ajal

võiksid kaasa tuua salaalkoholi leviku kasvu.

Therefore, we do not support the amendments for

reducing the sale of alcohol that can bring along the

growth of spreading the illegal alcohol.

</claim>

</argument>

Example 5: Argument attacking an amendment. The

premise comes after the claim.

- - attacking

<claim>

[…] ma ei ole nõus sõnastusega, et alkohol on

toidugrupp või kuulub toidugruppi.

[…] I don’t accept the definition of alcohol as a

food group.

</claim>

On selge, et siin on tegemist alkoholiäriga,

tarbimisele ja tootmisele paremate võimaluste

loomisega, siin ei ole arvestatud inimeste tervisega.

It is clear that there is the alcohol business that

creates better opportunities for consumption and

production, which don’t account the health of people.

</premise>

</argument>

2.3 Problems of Annotation

Following (Stab and Gurevych, 2013) and (Haddadan

et al., 2018) we started with annotation of claims of

arguments and determined whether the argument is

supporting, attacking or rebutting another argument

or one of its components. Then we determined

premises and linked them with claims. Haddadan et

al. (2018) have suggested some indicator phrases

which were commonly used while making claims or

premises. Stab and Gurevych (2017) also give certain

linguistic patterns for recognizing the components of

arguments.

In the negotiations on the bill of alcohol, 14

arguments (out of 28) have exact indicators to

recognize premises and/or claims, e.g. kui … siis (if

… then), sest, sellepärast et, seetõttu (because, in

that). Nevertheless, the argumentation in the political

debates is not always nicely signalled linguistically,

or even and intuitively clear (cf. Visser et al., 2018).

MPs in the Riigikogu often use figurative language

(Example 6).

Example 6. An emotional claim.

Aga kas selle eest peaks siis risti lööma ainult

putkad ja bensiinijaamad, kas nemad on siis Jeesus

Kristuse rollis, kes kogu alkoholi õuduse ja patu

peavad kinni maksma?

But should we only crucify booths and service

stations, do they have the role of Saviour who buys

out the horror and enormity of alcohol?

KEOD 2020 - 12th International Conference on Knowledge Engineering and Ontology Development

216

Some talks are ironic or sarcastic (Example 7).

Example 7. An ironic claim.

See seadus on väga hea alkoholiäri seadus.

This law is very good just for alcohol business.

Some MPs in the Riigikogu implement special

strategies to present their arguments (cf. Abbott et al.,

2016), including rhetorical questions or conditional

(Example 8).

Example 8. Rhetorical question and conditional in

argument.

Samuti, miks ei võiks riik oma alkoholi- ja

alkoholimonopoli poodidega minna Eesti

suurematesse keskustesse, et pakkuda turul

konkurentsi alkoholile, mille kasumimarginaali on

eraettevõtjad küllaltki kõrgeks ajanud? Why not

would the state open alcohol shops in bigger centres,

in order to initiate competition with private

entrepreneurs?

</premise>

Siis Then

<claim>

oleks võimalik saada kätte suhteliselt odavamat

alkoholi, kvaliteetset alkoholi, mitte salaviina. it

would be possible to receive relatively cheaper and

better alcohol, instead of illegal vodka.

</claim>

</argument>

The components of the arguments have the order

either ‘premise(s)-claim’ (Examples 1-4, 8) or ‘claim-

premise(s)’ (Example 5). Likewise, there are some

nested arguments where one argument is a premise of

another, e.g. Example 9 (cf. Amgoud et al., 2015).

Example 9. Nested arguments.

Siin öeldi, et kui It was said that if

me teatud reegleid karmistame, we introduce

some sanctions

</premise>

siis then

<claim>

kohe läheb taksoviinamajandus lahti, nagu oli

Gorbatšovi ajal. alcohol business from taxi-cabs will

start like it was in Gorbachov’s time.

</claim>

</argument1>

<claim>

Kuid siis olid teised ajad ja teised suhtumised. But

then the times and the attitudes were different.

</claim>

</argument0>

A problem is how to recognize the inter-argument

relations. We have determined the attack, rebuttal and

support relations between an argument and the claim

of another argument (incl. the major claim), e.g.

Examples 1 to 5, like in (Stab and Gurevych, 2017).

However, it has been difficult to determine another

argument related to the argument under consideration

like in (Amgoud et al., 2015). To do this, all

arguments (incl. the ones presented in the reports and

discussions preceding the negotiation) have to be

annotated. However, we are here only looking for the

arguments presented in negotiations.

3 DISCUSSION

The paper describes an experience of annotating

arguments in Estonian parliamentary discourse. The

empirical material is formed by the corpus that

consists of verbatim records of sittings held in the

Riigikogu. The components of arguments (premises

and claims) and inter-argument relations (attack,

support, and rebuttal) are annotated in a part of the

corpus.

As an example, proceedings on the bill of alcohol

are considered. The discussions in the Riigikogu have

been intensive, the total number of questions asked

after reports presented by the representatives of the

government and the leading committee is 81. The

number of presented amendments is 97 (still, only 39

of them were approved by MPs after voting). In total,

28 arguments have been presented by 12 MPs in

negotiations. The arguments supporting the bill are

prevailing over the counterarguments (by their

weightiness, not the number) and the act is approved

in the Parliament.

Every argument consists of two parts – one or

more premises and a claim. In our parliamentary

discussions, the presented arguments typically

include more than one premise in many sentences.

That is different as compared with persuasive essays

where premise(s) and a claim are often located in the

same sentence (Stab and Gurevych, 2014, 2017). In

the negotiations on the bill of alcohol, 21 arguments

(out of 28) consist of more than one sentence.

Microstructures of arguments (basic, convergent,

serial, divergent, and linked) proposed in (Stab and

Gurevych, 2017) have not been considered in our

parliamentary corpus, it needs an additional study.

Our current work on annotation of arguments is only

the first step towards creating an argument corpus.

More annotators must be involved in order to achieve

reliable results. Some tools can be used for (manual)

annotation of the arguments and for visualising the

Annotating Arguments in a Parliamentary Corpus: An Experience

217

inter-argument relations, e.g. OVA+ (Janier et al.,

2014).

The deﬁnition of suitable NLP methods for the

automatic identiﬁcation of the argument components

and the relations between them also needs an

additional study. A challenging further research

question is a comparative study of political

argumentation in Estonian parliament and in other

parliaments as well as in different political cultures

and different languages.

4 CONCLUSIONS

Verbatim records of sittings of the Parliament of

Estonia can be accessed online. Readings in the

Riigikogu have a predetermined structure, including

negotiations as one part. In this paper, discussions on

the bill of alcohol are considered in order to illustrate

arguments and their relations. The arguments used in

the process of adopting the act and inter-argument

relations are annotated. The structure of arguments

and the relations are analyzed. Some problems of

annotation are considered.

This study is a step towards automatic analysis of

political arguments in Estonian parliamentary

discussions. The current task has been the

development of the annotation scheme and creation

of a corpus with annotated arguments and inter-

argument relations. Future work includes the

ﬁnalization of the annotation process of a dataset of

political debates, and the deﬁnition of suitable NLP

methods for the automatic identiﬁcation of these

argument components and the inter-argument

relations.

ACKNOWLEDGEMENTS

This work was supported by the European Union

through the European Regional Development Fund

(Centre of Excellence in Estonian Studies).

REFERENCES

Abbott, R., Ecker, B., Anand, P., and Walker. M.A., 2016.

Internet Argument Corpus 2.0: An SQL schema for

Dialogic Social Media and the Corpora to go with it. In

Proceedings of LREC, 4445-4452.

Amgoud, L., Besnard, P., and Hunter, A., 2015. Logical

Representation and Analysis for RC-Arguments. In

IEEE 27th International Conference on Tools with

Artificial Intelligence (ICTAI), 104–110.

https://doi.org/10.1109/ICTAI.2015.28

Atkinson, K., Baroni, P., Giacomin, M., Hunter, A.,

Prakken, H., Reed, C., Simari, G., Thimm, M., Villata,

S. 2017. Towards Artiﬁcial Argumentation. In AI

Magazine, 38(3). https://doi.org/10.1609/aimag.v38i3.-

2704

Haddadan, S., Cabrio, E., and Villata, S., 2018. Annotation

of Argument Components in Political Debates Data. In

S. Kübler & H. Zinsmeister (Eds.), Workshop on

Annotation in Digital Humanities, 12–16.

Janier, M., Lawrence, J., and Reed, C., 2014. OVA+: An

Argument Analysis Interface. In Frontiers in Artificial

Intelligence and Applications, vol, 266, 463–464.

https://doi.org/10.3233/978-1-61499-436-7-463

Koit, M. Arguments in Parliamentary Negotiation: a Study

of Verbatim Records. In Proceedings of the 12th

International Conference on Agents and Artificial

Intelligence ICAART-2020, vol. 2, 822−828.

Koit, M., Õim, H., Roosmaa, T., 2019. How Do the

Members of a Parliament Negotiate? Analysing

Verbatim Records. In 11th International Joint

Conference on Knowledge Discovery, Knowledge

Engineering and Knowledge Management (IC3K 2019),

vol. 2, 329−335.

Lawrence, J., Reed, C., 2019. Argument Mining: A Survey.

In Computational Linguistics, vol. 45 (4), 765–818.

https://doi.org/10.1162/COLI.a.00364

Menini, S., Cabrio, E., Tonelli, S., and Villata, S., 2018.

Never retreat, never retract: Argumentation analysis for

political speeches. In The Thirty-Second AAAI

Conference on Artificial Intelligence (AAAI-18), 4889–

4896.

Riigikogu, 2020. https://www.riigikogu.-ee/en/

Stab, C. and Gurevych, I., 2017. Parsing Argumentation

Structures in Persuasive Essays. In Computational

Linguistics, vol. 43 (3), 619–659.

https://doi.org/10.1162/COLI.a.00295

Stab, C. and Gurevych, I. (2014). Annotating Argument

Components and Relations in Persuasive Essays. In J.

Tsujii & J. Hajic (Eds.), COLING 2014, the 25th

International Conference on Computational

Linguistics: Technical Papers, 1501–1510, Dublin,

Ireland.

Stab, C. and Gurevych, I., 2013. Guidelines for Annotating

Argument Components and Relations in Persuasive

Essays. Technische Universität Darmstadt, Germany.

Survey, 2020. https://www.clarin.eu/resource-

families/parliamentary-corpora

Visser, J., Lawrence, J., Wagemans, J., and Reed, C., 2018.

An annotated corpus of argument schemes in US

election debates. In 9th Conference of the International

Society for the Study of Argumentation (ISSA), 1101–

1111.

KEOD 2020 - 12th International Conference on Knowledge Engineering and Ontology Development

218