| 基于余弦距离的局部敏感哈希的KNN算法在中文文本上的快速分类 |
| |
| 引用本文： | 戴上平,冯鹏,刘盛英杰,舒红.基于余弦距离的局部敏感哈希的KNN算法在中文文本上的快速分类[J].计算机工程与科学,2015,37(10):1971-1976. |
| |
| 作者姓名： | 戴上平 冯鹏 刘盛英杰 舒红 |
| |
| 作者单位： | ;1.华中师范大学计算机学院;2.测绘遥感信息工程国家重点实验室 |
| |
| 基金项目： | 武汉市政府资助项目（基于网格的社区宜居环境分析研究） |
| |
| 摘 要： | 文本分类是文本挖掘中最重要的研究内容之一。为了克服目前以距离衡量的近似分类算法在海量数据下耗费大量时间的缺陷,提出了结合基于余弦距离的局部敏感哈希的方式将KNN算法在TF-IDF下对中文文本进行快速分类。同时结合文本数据的特性给出了不同的哈希函数级联方式分别进行实验。在实验过程采用了布尔向量的方式规避重复访问,使分类的结果在可以允许的范围内,分类速度比原始KNN提高了许多。
|
| 关 键 词： | 文本分类 局部敏感哈希 TF-IDF KNN 布尔向量 |
| 收稿时间： | 2015-07-25 |
| 修稿时间： | 2015-10-25 |
A fast KNN algorithm for Chinese text classification based on the LSH of cosine distance |
| |
| DAI Shang ping,FENG Peng,LIU SHEN Ying jie,SHU Hong.A fast KNN algorithm for Chinese text classification based on the LSH of cosine distance[J].Computer Engineering & Science,2015,37(10):1971-1976. |
| |
| Authors: | DAI Shang ping FENG Peng LIU SHEN Ying jie SHU Hong |
| |
| Affiliation: | (1.School of Computer Science,Central China Normal University,Wuhan 430079;2.National Key Laboratory of Surveying and Remote Sensing I nformation Engineering,Wuhan 430079,China) |
| |
| Abstract: | Text classification is one of the most important study spots in text mining.In order to overcome the drawback that the classification algorithm based on distance brings high query time cost,a K Nearest Neighbors (KNN) algorithm based on the Locality Sensitive Hashing (LSH) of cosine distance under TF IDF is proposed,which can classify Chinese text quickly.Besides,by combing the properties of the text data,experiments with different parameters are carried out.In the experiments,boolean vectors are used to avoid the duplication calculation.Compared with the original KNN,our algorithm can increase the speed of classification in ensuring the accuracy. |
| |
| Keywords: | text classification Locality Sensitive Hashing (LSH) TF-IDF KNN boolean vector |
| 本文献已被 万方数据 等数据库收录！ |
| 点击此处可从《计算机工程与科学》浏览原始摘要信息 |
|
点击此处可从《计算机工程与科学》下载全文 |
