基于Web的大规模语料库构建方法

低成本、短周期构建大规模语料库是目前研究工作的难点之一。该文提出一种建设大规模语料库的新方法，主要解决如何基于Web构建大规模的语料库及对语料库纠错，从而提高其质量。该方法利用网格技术的大规模计算能力与Wiki的开放编辑环境去收集和处理语料，根据可信度模型挑选出不可信的语料并由人工进行校对，计算校对后结果的可信度，选择出最可信的结果作为正确语料存储到语料库中。

Construction Approach of Large-scale Corpus Based on Web

Nowadays, it’s hard to build a large-scale corpus with low cost and short period. A new approach is provided to build that on Web. It mainly focuses on how to build a large-scale corpus on Web and then how to correct the mistakes in the corpus. The language materials are collected and processed based on grid and Wiki. The untrustworthy language materials in the corpus are picked out to be checked manually on Wiki according to their trustworthiness. After the check finishes, the approach calculates the trustworthiness of each checked result and selects ones with highest trustworthiness as the correct result.