Find truth in the hands of the few: acquiring specific knowledge with crowdsourcing |
| |
Authors: | Tao HAN Hailong SUN Yangqiu SONG Yili FANG Xudong LIU |
| |
Affiliation: | 1. SKLSDE Lab, School of Computer Science and Engineering, Beihang University, Beijing 100191, China2. Beijing Advanced Innovation Center for Big Data and Brain Computing, Beihang University, Beijing 100191, China3. Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Clearwater Bay, Hong Kong 999077, China4. School of Computer and Information Engineering, Zhejiang Gongshang University, Hangzhou 310018, China |
| |
Abstract: | Crowdsourcing has been a helpful mechanism to leverage human intelligence to acquire useful knowledge.However, when we aggregate the crowd knowledge based on the currently developed voting algorithms, it often results in common knowledge that may not be expected. In this paper, we consider the problem of collecting specific knowledge via crowdsourcing. With the help of using external knowledge base such as WordNet, we incorporate the semantic relations between the alternative answers into a probabilisticmodel to determine which answer is more specific. We formulate the probabilistic model considering both worker’s ability and task’s difficulty from the basic assumption, and solve it by the expectation-maximization (EM) algorithm. To increase algorithm compatibility, we also refine our method into semi-supervised one. Experimental results show that our approach is robust with hyper-parameters and achieves better improvement thanmajority voting and other algorithms when more specific answers are expected, especially for sparse data. |
| |
Keywords: | crowdsourcing knowledge acquisition EM algorithm label aggregation |
|
| 点击此处可从《Frontiers of Computer Science》浏览原始摘要信息 |
|
点击此处可从《Frontiers of Computer Science》下载全文 |
|