面向Web电子产品信息分布式检索系统的设计与实现

为了从这些海量信息中获取“有用的、满足用户需求的信息”,提出一个基于Hadoop和Lucene技术的分布式检索系统架构处理Web电子产品信息检索。利用Hadoop的Map和Reduce实现分布式索引文件的存储,通过Lucene检索技术实现索引文件的访问,从而提高信息检索的效率。并且针对Lucene_Hadoop架构存在粗粒度检索问题,提出了一种细粒度检索方法,减少了系统建立索引的时间。实验表明基于Hadoop和Lucene的分布式检索系统在Web电子产品信息中具有较高的检索性能。     

全文检索系统中动态索引技术的研究

全文检索是一种非常有效的信息检索技术,本文通过分析全文检索系统中静态索引技术的优缺点,以及影响动态性能的因素,提出一种基于互关联后继树模型的动态索引技术,该技术在不影响查询效率等性能的情况下,很好地解决了索引的更新问题,提高了索引的动态性能。     

全文检索系统中动态索引技术的研究

全文检索是一种非常有效的信息检索技术,本文通过分析全文检索系统中静态索引技术的优缺点,以及影响动态性能的因素,提出一种基于互关联后继树模型的动态索引技术,该技术在不影响查询效率等性能的情况下,很好地解决了索引的更新问题,提高了索引的动态性能。     

一种基于主成分的多表图像哈希检索方法

大数据时代的到来,快速而准确的索引算法对信息检索至关重要。针对基于随机投影构成的单表哈希检索方法导致搜索性能低的问题,提出一种基于主成分的多表图像哈希检索方法。为了得到高效的哈希编码保证不同语义样本特征的区分性,首先通过主元分析方法保留训练集具有区分性图像特征,此外利用特征聚类作为学习哈希投影的指引构建多个索引表;其次采用正交旋转矩阵对哈希投影进行优化,保证了相同语义的样本具有相似的哈希码。最后分别在CIFAR-10和Caltech-256数据集上与相关方法进行比较,实验结果表明提出的方法提高了检索性能。     

基于基频的朝鲜语方言辨识方法的研究

该文提出了一种基于基音频率特征的中国朝鲜族语言、韩国朝鲜语和朝鲜朝鲜语方言的自动辨识方法。首先,选择具有良好区分度的基频移位差分系数作为三个方言的特征参数;其次,设计和采用了分层支持向量机分类器,并进一步引入投票法确定最佳的分类结果。实验结果表明该文提取的特征参数具有良好的区分性和较强的稳定性,该文提出的方言辨识方法比传统的移位差分倒谱系数特征方法识别率高,可以有效解决朝鲜朝鲜语、韩国朝鲜语和中国朝鲜族语言的方言辨识问题。     

基于关键词设置的P2P信息检索

在分析以往结构化P2P基于单关键词分割的信息检索基础上,结合利用用户频繁出现的请求关键词的特点,提出结构化P2P中的一个基于请求关键词设置的索引框架KS-PIR。采用通过关键词联合的新特征项设置来扩充索引特征,并删节与特征项相关的索引记录表至恒定长度的方法,在检索质量提高的同时,又保证了可接受的带宽消耗,系统具有很好的扩展性。     

基于Spark框架的图书馆文献信息检索方法

由于传统信息检索方法检索时间长，检索误差率高，研究基于Spark框架的图书馆文献信息检索方法。首先，预处理文献信息数据，结合信息相似度，提取特征数据集合中的敏感数据获得属性特征。其次，基于Spark框架批处理数据，训练待测样本进行目标跟踪，寻找最优解集。对信息的特征进行查询，提取其中的关键词并在已建立的文本索引结构上进行检索和融合，运用K-Means聚类方法对融合结果进行重新排序。采用布尔查询机制构造文献检索模型，将不同信息数据多重倒排文献进行索引从而完成信息检索。实验结果表明，运用该方法检索时间最短，误差率最低，完成图书馆文献信息检索方法的优化。     

词性标注对信息检索系统性能的影响

在信息检索中引入NLP 技术是信息检索发展的主要趋势,本文将NLP 中较为成熟的词性标注技术加入信息检索,采用大规模TREC 数据集,试图发现词性标注对检索系统性能的影响。笔者在SMART 检索系统上使用不同标注集、不同索引项权重进行了检索实验。实验表明,在信息检索中加入词性标注信息可能会对某些特定Topic 和Document 的检索效果有所改进,但词性标注的影响能力弱于索引项权重选择的影响能力。词性标注对检索性能的影响涉及到Topic 和Document 中的具体用词,普遍规律有待进一步研究。     

基于复合结构的高效索引在线更新策略

倒排索引结构已被广泛地应用在信息检索系统中，倒排索引离线的生成和更新方法已不适合在线更新。文中研究了在线索引更新方法，分析了合并更新、插入更新、复合更新等方法，提出一种结合“插入更新”和“合并更新”优点，并采用多级结构的改进复合更新策略。使用磁盘操作复杂度来衡量更新策略的性能，对几种常用的更新策略和复合更新策略在大量记录下的性能进行理论和实验分析。结果显示，改进复合更新策略具有较好的效率。     

潜在语义索引中特征优化技术的研究

潜在语义索引被广泛应用于信息检索、文本分类、自动问答等领域中。潜在语义索引是一种降维方法,它把共现特征映射到同一维空间上,而非共现特征映射到不同的空间上。在潜在语义索引的语义空间中,共现特征通过文档内部以及文档之间的特征传递关系获得。该文认为这种特征传递关系会引入一些不存在的共现特征,从而降低潜在语义索引的性能,应该对这种特征传递关系进行一些选择,削除不存在的共现特征信息。该文采用文档频率对文档集合进行特征选择,用Complete-Link聚类算法在两个公开语料上进行三个实验,实验结果显示,保留文档频度的10%～15%时,其F1值分别提高了6.577 0%,1.992 8%和3.361 4%。     

基于N-GRAM的朝鲜文索引方法与性能评价

在为检索信息而索引朝鲜文时,通常的做法是对语句和词素进行分析后把名词抽取成索引词.但由于分析的模糊性,若在对词素进行分析时出现参考的词典上不存在的未登录词的情况,就很难抽取正确的索引词.N-gram不需要对词进行语言的分析,因而索引速度快,而且对词素分析词典里不存在的未登录词的分析有利,所以对分析复合名词有很好的效果.但与别的分析方法相比,使用N-gram时会出现因索引词抽取得太多而导致存储空间使用率低和索引效率的下降.为了克服N-gram的缺点,本文提出了一种新的朝鲜语自动索引方法.该方法首先把体词与谓词抽取成索引词后,再利用语句类型规则对词素分析中失败的语句进行助词分离,最后在未登录词处理中使用N-gram的索引方法.对比分析和性能评价表明,所提出的方法是有效的.     

Resolving Ambiguous Segmentation of Korean Compound Nouns Using Statistics and Rules

Korean compound nouns may be written as a sequence of characters without blanks between unit nouns. For Korean processing systems, Korean compound nouns have to be first segmented into a sequence of unit nouns. However, the segmentation task is difficult because a sequence of characters may be ambiguously segmented to several sequences of appropriate unit nouns. Moreover, this task is not trivial because Korean compound nouns may include many unknown unit nouns.
This paper proposes a new method for KCNS (Korean Compound Noun Segmentation) and reports on the appliccation of such a segmentationtechnique to enhance the performance of an information retrieval system. According to our method, compound nouns are first segmented by using a dictionary and structure patterns. If they are ambiguously segmented, we resolve the ambiguities by using statistical information and a preference rule. Moreover, we employ three kinds of heuristics in order to segment compound nouns with unknown unit nouns.
To evaluate KCNS, we use three kinds of data from various domains. Experimental results show that the precision of KCNS's output is approximately 96% on average, regardless of domains. The effectiveness of using the segmented unit nouns provided by KCNS for indexing is proved by improving retrieval performance of our information retrieval system.     

ALSBIR: A local-structure-based image retrieval

The general-purpose shape retrieval problem is a challenging task. Particularly, an ideal technique, which can work in clustered environment, meet the requirements of perceptual similarity measure on partial query and overcoming dimensionality curse and adverse environment, is in demand. This paper reports our study on one local structural approach that addresses these issues. Shape representation and indexing are two key points in shape retrieval. The proposed approach combines a novel local-structure-based shape representation and a new histogram indexing structure. The former makes possible partial shape matching of objects without the requirement of segmentation (separation) of objects from complex background, while the latter has an advantage on indexing performance. The search time is linearly proportional to the input complexity. In addition, the method is relatively robust under adverse environments. It is able to infer retrieval results from incomplete information of an input by first extracting consistent and structurally unique local neighborhood information from inputs or models, and then voting on the optimal matches. Thousands of images have been used to test the proposed concepts on sensitivity analysis, similarity-based retrieval, partial query and mixed object query. Very encouraging experimental results with respect to efficiency and effectiveness have been obtained.     

一种新颖的对比子图索引算法

针对当前图索引算法存在的问题,提出一种基于对比子图索引框架,开发冗余感知机制,选择一个小型的具有明显区分力的索引特征集,改善索引性能。实验结果表明,该算法对不同的包容搜索载荷能达到近优化的修剪力,与传统图搜索方法相比,具有明显的索引性能优势。     

Exploiting concept clusters for content-based information retrieval

Current approaches to index weighting for information retrieval from texts are based on statistical analysis of the texts' contents. A key shortcoming of these indexing schemes, which consider only the terms in a document, is that they cannot extract semantically exact indexes that represent the semantic content of a document. To address this issue, we proposed a new indexing formalism that considers not only the terms in a document, but also the concepts. In the proposed method, concepts are extracted by exploiting clusters of terms that are semantically related, referred to as concept clusters. Through experiments on the TREC-2 collection of Wall Street Journal documents, we show that the proposed method outperforms an indexing method based on term frequency (TF), especially in regard to the highest-ranked documents. Moreover, the index term dimension was 53.3% lower for the proposed method than for the TF-based method, which is expected to significantly reduce the document search time in a real environment.     

一种新的监督潜在语义模型

Sprinkling方法是一种集成了训练样本类别信息的监督潜在语义模型。但是该方法特征权重采用词频,降低了文本分类效果,同时该模型并没有考虑不同样本对分类的贡献能力,而是认为样本对分类的贡献相同,另外,该模型采用多个特征映射一个类别来加强类别知识对分类的贡献。为此,文章在Sprinkling方法的基础上提出了一种新的监督潜在语义模型。实验结果表明,该文方法的总体性能优于原始的Sprinkling方法,在特征数为1 100时,获得了最高分类精度,提高幅度达到1.71%。     

基于时态编码和线序划分的时态XML 索引

研究了一种基于时态编码和线序划分的时态XML索引机制.首先,提出一种基于扩展先序编码的时态编码方案,通过该编码可确定结点间的结构关系;其次,在深入分析时间区间关系的基础上引入线序划分的概念,并讨论了获取线序划分的算法;然后,建立了整合路径结构信息和时态约束信息的时态结构摘要,并在此基础上建立了时态XML索引结构—TempSumIndex,同时研究了基于TempSumlndex的时态XML查询和增量式更新算法;最后,对TempSumlndex和现有时态XML索引技术的基本性能进行了详细的实验评估.实验结果表明,TempSumlndex具有更为优越的性能.     

A New Korean Corpus-Based Text-to-Speech System

This paper describes a new Korean Text-to-Speech (TTS) system based on a large speech corpus. Conventional concatenative TTS systems still produce machine-like synthetic speech. The poor naturalness is caused by excessive prosodic modification using a small speech database. To cope with this problem, we utilized a dynamic unit selection method based on a large speech database without prosodic modification. The proposed TTS system adopts triphones as synthesis units. We designed a new sentence set maximizing phonetic or prosodic coverage of Korean triphones. All the utterances were segmented automatically into phonemes using a speech recognizer. With the segmented phonemes, we achieved a synthesis unit cost of zero if two synthesis units were placed consecutively in an utterance. This reduces the number of concatenating points that may occur due to concatenating mismatches. In this paper, we present data concerning the realization of major prosodic variations through a consideration of prosodic phrase break strength. The phrase break was divided into four kinds of strength based on pause length. Using phrase break strength, triphones were further classified to reflect major prosodic variations. To predict phrase break strength on texts, we adopted an HMM-like Part-of-Speech (POS) sequence model. The performance of the model showed 73.5% accuracy for 4-level break strength prediction. For unit selection, a Viterbi beam search was performed to find the most appropriate triphone sequence, which has the minimum continuation cost of prosody and spectrum at concatenating boundaries. From the informal listening test, we found that the proposed Korean corpus-based TTS system showed better naturalness than the conventional demisyllable-based one.     

基于互信息置信度的网格连续汉语语音检索*

针对目前生活中涌现的海量语音数据,人们对语音检索技术准确度的要求越来越高。主要研究了汉语连续语音检索任务中,基于转换音节网格的研究方法。针对语音检索系统中置信度计算的问题,提出了一种基于音节间互信息的置信度计算方法,并将其用于网格结构的语音检索系统中。该方法能够有效地利用上下文之间的互信息量,从而更准确、合理地描述汉语语言模型。实验结果表明,用提出的方法建立转换音节网格来进行语音检索,其检出率（FOM）比后验概率法和N-best法有较大幅度的提高。得到的汉语语音检索系统其FOM最高可以达到83.7%。     

一种面向图包容搜索的图索引模型

由于图模型的广泛采用,图数据的快速包容搜索在许多不同领域广泛应用。给定一个模型图集D和一个查询图集q,传统的图搜索旨在检索所有包含q的图(qg),与此不同,包容搜索有其自身的索引特性,针对这些特性进行系统地研究,并提出一种基于对比子图的索引模型（csgIndex）：使用一个冗余感知特征选择过程,csgIndex能挑选出一个鲜明的具有区分力的对比子图集,并最大化其索引能力。对实时测试数据的实验结果显示,csgIndex对不同的包容搜索荷载能达到近优化修剪力,相较于传统图搜索方法表现出明显的索引性能优势。