We propose DER3, a novel approach to estimate worker reliability in crowdsourcing dynamically.
Constant rater availability is not required.
A large number of tasks with correct answers known in advance is not required.
DER3 is specifically suited for regression tasks, but can also be used in different settings.
DER3 leads to the results of high accuracy and low costs in different application domains.