Informativeness-based Keyword Extraction from Short Documents

Mika Timonen, Timo Toivanen, Yue Teng, Chao Chen, Liang He

2012

Abstract

With the rise of user created content on the Internet, the focus of text mining has shifted. Twitter messages and product descriptions are examples of new corpora available for text mining. Keyword extraction, user modeling and text categorization are all areas that are focusing on utilizing this new data. However, as the documents within these corpora are considerably shorter than in the traditional cases, such as news articles, there are also new challenges. In this paper, we focus on keyword extraction from documents such as event and product descriptions, and movie plot lines that often hold 30 to 60 words. We propose a novel unsupervised keyword extraction approach called Informativeness-based Keyword Extraction (IKE) that uses clustering and three levels of word evaluation to address the challenges of short documents. We evaluate the performance of our approach by using manually tagged test sets and compare the results against other keyword extraction methods, such as CollabRank, KeyGraph, Chi-squared, and TF-IDF. We also evaluate the precision and effectiveness of the extracted keywords for user modeling and recommendation and report the results of all approaches. In all of the experiments IKE out-performs the competition.

Paper Citation

in Harvard Style

Timonen M., Toivanen T., Teng Y., Chen C. and He L. (2012). Informativeness-based Keyword Extraction from Short Documents . In Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: SSTM, (IC3K 2012) ISBN 978-989-8565-29-7, pages 411-421. DOI: 10.5220/0004130704110421

in Bibtex Style

@conference{sstm12,
author={Mika Timonen and Timo Toivanen and Yue Teng and Chao Chen and Liang He},
title={Informativeness-based Keyword Extraction from Short Documents},
booktitle={Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: SSTM, (IC3K 2012)},
year={2012},
pages={411-421},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004130704110421},
isbn={978-989-8565-29-7},
}

in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Knowledge Discovery and Information Retrieval - Volume 1: SSTM, (IC3K 2012)
TI - Informativeness-based Keyword Extraction from Short Documents
SN - 978-989-8565-29-7
AU - Timonen M.
AU - Toivanen T.
AU - Teng Y.
AU - Chen C.
AU - He L.
PY - 2012
SP - 411
EP - 421
DO - 10.5220/0004130704110421