一种自动分类的网页搜索排序算法

针对传统网页排序算法Okapi BM25通常会出现网页与查询关键词领域无关的领域漂移现象，以及改进算法需要人工建立领域向量的问题，提出了一种基于BM25和Softmax回归分类模型的网页搜索排序算法。该方法首先对网页文本进行数据预处理并利用词袋模型进行网页文本的向量表示，之后通过少量的网页数据来训练Softmax回归分类模型，来预测测试网页数据的类别分数，并与BM25信息检索的分数结合在一起，得到最终的网页排序结果。实验结果显示该检索算法无须人工建立领域向量，即可达到很好的网页排序结果。

Web page search ranking algorithm using automatic classification

In the traditional Web page ranking algorithm Okapi BM25, there exists a problem that the retrieval results are independent to the domain keywords, and the improved algorithm needs to build the domain vector manually. To address this issue, we propose a web page ranking algorithm based on BM25 and softmax regression classification model. In this method, we first encode the web page text with the bag-of-words model, and then train the softmax regression classification model by a small amount of web data to predict the category scores of the test web data. Finally we combine the category scores and the BM25 information retrieval scores to get the final ranking of web page results. Experiment results show that our method can meet the user''s information need better without even manually creating the domain vector.