Crowdsourcing Reliable Ratings for Underexposed Items

Beatrice Valeri, Shady Elbassuoni, Sihem Amer-Yahia

2016

Abstract

We address the problem of acquiring reliable ratings of items such as restaurants or movies from the crowd. A reliable rating is a truthful rating from a worker that is knowledgeable enough about the item she is rating. We propose a crowdsourcing platform that considers workers’ expertise with respect to the items being rated and assigns workers the best items to rate. In addition, our platform focuses on acquiring ratings for items that only have a few ratings. Traditional crowdsourcing platforms are not suitable for such a task for two reasons. First, ratings are subjective and there is no single correct rating for an item which makes most existing work on predicting the expertise of crowdsourcing workers inapplicable. Second, in traditional crowdsourcing platforms there is no control over task assignment by the requester. In our case, we are interested in providing workers with the best items to rate based on their estimated expertise for the items and the number of ratings the items have. We evaluate the effectiveness of our system using both synthetic and real-world data about restaurants.

Paper Citation

in Harvard Style

Valeri B., Elbassuoni S. and Amer-Yahia S. (2016). Crowdsourcing Reliable Ratings for Underexposed Items . In Proceedings of the 12th International Conference on Web Information Systems and Technologies - Volume 2: WEBIST, ISBN 978-989-758-186-1, pages 75-86. DOI: 10.5220/0005770700750086

in Bibtex Style

@conference{webist16,
author={Beatrice Valeri and Shady Elbassuoni and Sihem Amer-Yahia},
title={Crowdsourcing Reliable Ratings for Underexposed Items},
booktitle={Proceedings of the 12th International Conference on Web Information Systems and Technologies - Volume 2: WEBIST,},
year={2016},
pages={75-86},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005770700750086},
isbn={978-989-758-186-1},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Conference on Web Information Systems and Technologies - Volume 2: WEBIST,
TI - Crowdsourcing Reliable Ratings for Underexposed Items
SN - 978-989-758-186-1
AU - Valeri B.
AU - Elbassuoni S.
AU - Amer-Yahia S.
PY - 2016
SP - 75
EP - 86
DO - 10.5220/0005770700750086