DISTL: Distributed In-Memory Spatio-Temporal Event-based Storyline Categorization Platform in Social Media

Manu Shukla, Ray Dos Santos, Andrew Fong, Chang-Tien Lu

2016

Abstract

Event analysis in social media is challenging due to endless amount of information generated daily. While current research has put a strong focus on detecting events, there is no clear guidance on how those storylines should be processed such that they would make sense to a human analyst. In this paper, we present DISTL, an event processing platform which takes as input a set of storylines (a sequence of entities and their relationships) and processes them as follows: (1) uses different algorithms (LDA, SVM, information gain, rule sets) to identify events with different themes and allocates storylines to them; and (2) combines the events with location and time to narrow down to the ones that are meaningful in a specific scenario. The output comprises sets of events in different categories. DISTL uses in-memory distributed processing that scales to high data volumes and categorizes generated storylines in near real-time. It uses Big Data tools, such as Hadoop and Spark, which have shown to be highly efficient in handling millions of tweets concurrently.

Paper Citation

in Harvard Style

Shukla M., Dos Santos R., Fong A. and Lu C. (2016). DISTL: Distributed In-Memory Spatio-Temporal Event-based Storyline Categorization Platform in Social Media . In Proceedings of the 2nd International Conference on Geographical Information Systems Theory, Applications and Management - Volume 1: GISTAM, ISBN 978-989-758-188-5, pages 39-50. DOI: 10.5220/0005831200390050

in Bibtex Style

@conference{gistam16,
author={Manu Shukla and Ray Dos Santos and Andrew Fong and Chang-Tien Lu},
title={DISTL: Distributed In-Memory Spatio-Temporal Event-based Storyline Categorization Platform in Social Media},
booktitle={Proceedings of the 2nd International Conference on Geographical Information Systems Theory, Applications and Management - Volume 1: GISTAM,},
year={2016},
pages={39-50},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005831200390050},
isbn={978-989-758-188-5},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 2nd International Conference on Geographical Information Systems Theory, Applications and Management - Volume 1: GISTAM,
TI - DISTL: Distributed In-Memory Spatio-Temporal Event-based Storyline Categorization Platform in Social Media
SN - 978-989-758-188-5
AU - Shukla M.
AU - Dos Santos R.
AU - Fong A.
AU - Lu C.
PY - 2016
SP - 39
EP - 50
DO - 10.5220/0005831200390050