Dynamic estimation of worker reliability in crowdsourcing for regression tasks: Making it work
详细信息 查看全文
- 作者：Alexey Tarasov ; tarasovs.aleksejs@gmail.com" class="auth_mail ; Sarah Jane Delany ; Brian Mac Namee
- 关键词：Supervised machine learning ; Crowdsourcing ; Worker reliability ; Multi-armed bandits ; Emotion recognition
- 刊名：Expert Systems with Applications
- 出版年：15 October 2014
- 年：2014
- 卷：41
- 期：14
- 页码：6190-6210
- 全文大小：1542 K
文摘
- •
We propose DER3, a novel approach to estimate worker reliability in crowdsourcing dynamically.
- •
Constant rater availability is not required.
- •
A large number of tasks with correct answers known in advance is not required.
- •
DER3 is specifically suited for regression tasks, but can also be used in different settings.
- •
DER3 leads to the results of high accuracy and low costs in different application domains.