共查询到20条相似文献,搜索用时 69 毫秒
1.
解决多音字字-音转换的一种统计学习方法 总被引:4,自引:1,他引:4
字-音转换是语音合成系统中的一个重要模块,其中多音词和以单字词形式存在的多音字读音的确定一直是个没有很好解决的问题。本文通过对大量标注有正确拼音的语料的统计,指出着重解决41个重点多音字和22个重点多音词就可基本解决字-音转换的问题。本文采用基于扩展的随机复杂度的随机决策列表方法自动提取多音字(词)的读音规则,将字-音转换的错误率由8.8‰降低到4.4‰。规则的训练和测试的材料的标注是一个耗费人力和时间的工作,而训练材料的数量和质量又直接影响最终的结果。本文提出一种半自动的语料标注流程,可以节省将近一半的人工和时间。 相似文献
2.
基于PAT TREE统计语言模型与关键词自动提取 总被引:8,自引:0,他引:8
未登录关键词的识别是中文信息处理中的一个关键问题。文章利用PAT TREE实现了一种可变长统计语言模型,由于不存在n元统计语言模型的截断效应,从而对待提取的关键词的长度没有限制。在该模型的基础上,通过相关性检测,从540M汉语语料中自动提取出了12万个关键词候选字串。最后,经过分析和筛选,候选字串的准确度由82.3%上升到96.1%。实验表明,基于PAT TREE的统计语言模型是实现未登录词提取的有力工具。 相似文献
3.
We present a unified probabilistic framework for statistical language modeling which can simultaneously incorporate various aspects of natural language, such as local word interaction, syntactic structure and semantic document information. Our approach is based on a recent statistical inference principle we have proposed—the latent maximum entropy principle—which allows relationships over hidden features to be effectively captured in a unified model. Our work extends previous research on maximum entropy methods for language modeling, which only allow observed features to be modeled. The ability to conveniently incorporate hidden variables allows us to extend the expressiveness of language models while alleviating the necessity of pre-processing the data to obtain explicitly observed features. We describe efficient algorithms for marginalization, inference and normalization in our extended models. We then use these techniques to combine two standard forms of language models: local lexical models (Markov N-gram models) and global document-level semantic models (probabilistic latent semantic analysis). Our experimental results on the Wall Street Journal corpus show that we obtain a 18.5% reduction in perplexity compared to the baseline tri-gram model with Good-Turing smoothing.Editors: Dan Roth and Pascale Fung 相似文献
4.
5.
6.
从自然语言描述文本中提取网络攻击知识存在语义鸿沟,导致TTPs威胁情报自动化利用低。为提高威胁情报自动分析效率,设计并实现了基于ATT&CK的APT攻击语义规则。首先,构建带标签的有向图语义规则模型,对自然语言文本描述的攻击技术进行知识化描述;其次,定义语义规则,阐释网络实体属性及其逻辑运算关系的形式化描述方法;最后,利用关键词组识别、知识抽取等自然语言处理技术,从攻击技术文本中抽取形成123个APT攻击语义规则,涵盖ATT&CK的115项技术和12种战术。利用模拟场景采集的APT攻击日志数据,对语义规则进行验证,实验结果表明,语义规则检出率达到93.1%,并具备一定的攻击上下文信息还原能力,可有效支撑威胁检测分析。 相似文献
7.
一个基于互信息的规则量化方法 总被引:4,自引:0,他引:4
规则量化问题是基于规则与统计相结合的计算语言模型的关键性问题,首先描述了一个基于规则与统计相结合的计算语言模型,从而引出了规则结合度的概念,并提出一种基于互信息的规则量化方法,该方法既给出了一条规则的实用性的定量评价,又为基于规则与统计相结合的计算语言模型提供了一个重要的规则参量,通过实验指出该方法对系统规则集的筛选与计算语言模型中规则与统计结合机制的优化的重要指导意义。 相似文献
8.
针对基于层次短语翻译模型的统计机器翻译使用上下文信息有限,时态翻译质量不高的问题,提出一种融合时态特征的日英统计机器翻译方法。该方法通过引入翻译规则的时态分类约束信息,解码器可以根据每条规则的潜在时态分类,为相应时态的句子匹配到最合适的规则进行翻译。首先从双语训练语料中抽取时态特征构建最大熵分类模型,然后再抽取包含各类时态信息的层次短语规则的时态特征,最后将规则的时态分类结果作为一类新特征,融入基于层次短语的翻译系统中。实验结果表明,与基线系统相比,该方法在多个测试集上提高了翻译质量,在一定程度上解决了日英层次短语模型的时态翻译问题。 相似文献
9.
一种多知识源汉语语言模型的研究与实现 总被引:7,自引:0,他引:7
针对汉语语言模型中知识获取不足的问题,提出了一种统计与多种形式规则信息结合的机制,将规则的表示量化,提出语法语义规则的概念,通过扩充词网络,对其于最大可能性的n元概率值合理调整,将短语构成规则,二元语法语义规则,最少分词原则等融入统计模型框架,构成多知识源语言模型,模型应用于智能拼音汉字转换系统,明显提高了音字转换正确率,并适于处理长距离和递归语言现象。 相似文献
10.
11.
12.
中文信息检索系统中的查询语句包含中文字、拼音、英文等多种形式,而有些查询语句过长,不利于纠错处理。现有的查询纠错方法不能很好的解决中文检索系统中的混合语言与中文长查询的问题。为了解决上述两个问题,该文提出了一种支持混合语言的并行纠错方法。该方法通过对混合语言统一编码,建立统一编码语言模型和异构字符词典树,并根据语言特点制定相应的编辑规则对查询词语进行统一处理,其中,针对中文长查询,提出双向并行的纠错模型。为了并行处理查询语句,我们在字符词典树和语言模型的基础上提出了逆向字符词典树和逆向语言模型的概念。模型中使用的训练语料库是从用户查询日志、网页点击日志、网页链接信息等文件中提取的高质量文本。实验表明,与单向查询纠错相比,支持混合语言的并行纠错方法在准确率上提升了9%,召回率降低了3%,在速度上提升了40%左右。 相似文献
13.
高阶N-gram语言模型在OCR后处理方面有着广泛的应用,但也面临着因模型复杂度大导致的数据稀疏,以及耗费较多的时空资源等问题。该文针对印刷体汉字识别的后处理,提出了一种基于字节的语言模型的后处理算法。通过采用字节作为语言模型的基本表示单位,模型的复杂度大大降低,从而数据稀疏问题得到很大程度上缓解。实验证明,采用基于字节的语言模型的后处理系统能够以极少的时空开销获取很好的识别性能。在有部分分割错误的测试集上,正确率从88.67%提高到了98.32%,错误率下降了85.18%,运行速度较基于字以及基于词的系统有了大幅的提升,提高了后处理系统的综合性能;与目前常用的基于词的语言模型后处理系统相比,新系统能够节省95%的运行时间和98%的内存资源,但系统识别率仅降低了1.11%。 相似文献
14.
Guang-Hong Wang Ping Jiang Zu-Ren Feng 《国际自动化与计算杂志》2006,3(3):282-290
A support vector rule based method is investigated for the construction of motion controllers via natural language training. It is a two-phase process including motion control information collection from natural language instructions, and motion information condensation with the aid of support vector machine (SVM) theory. Self-organizing fuzzy neural networks are utilized for the collection of control rules, from which support vector rules are extracted to form a final controller to achieve any given control accuracy. In this way, the number of control rules is reduced, and the structure of the controller tidied, making a controller constructed using natural language training more appropriate in practice, and providing a fundamental rule base for high-level robot behavior control. Simulations and experiments on a wheeled robot are carried out to illustrate the effectiveness of the method. 相似文献
15.
准确地建立待解决问题的可拓模型是可拓策略生成的关键步骤。目前的可拓策略生成系统在建立可拓模型时因自然语言理解的困难,未能充分理解用户需求,所以较难自动建立问题的可拓模型。提出了解析用户自然语言需求语句、并自动建立可拓模型的方法。该方法的核心包括4步:1)对用户需求语句进行组块分析得到短语序列;2)对短语序列进行分类;3)使用匹配规则抽取分类后的短语,得到便于计算机处理的需求信息;4)结合数据库技术进行可拓模型的建立。以租房问题为案例,实现了该方法。实验结果表明,该方法能较好地理解用户需求信息并成功建立租房问题可拓模型。 相似文献
16.
中文人称名词短语单复数自动识别 总被引:2,自引:1,他引:1
名词短语的单复数信息在共指消解中是必不可少的特征. 与英语不同, 中文属于汉藏语系, 名词本身不能明显体现单复数信息, 需要借助其所在的名词短语来进行体现. 本文在自动内容抽取(Automatic content extraction, ACE)语料上抽取得到人称名词短语的单复数信息, 分别采用了基于规则和机器学习的方法来进行人称名词短语的单复数自动识别. 基于规则的方法, 在一些知识资源的基础上定义了规则模板库, 每条规则采用槽和槽值的方法来进行体现; 机器学习方法采用最大熵模型组合考察了词形、词性、词义、数量关系等特征. 两种方法分别达到了48.24\%和87.48\%的正确率. 实验结果显示, 基于规则的方法能够保证精确率而不能保证召回率, 机器学习的方法可以更好地完成单复数信息的识别任务. 相似文献
17.
Yuqing Gao Bowen Zhou Zijian Diao Jeffrey Sorensen Michael Picheny 《Machine Translation》2002,17(3):185-212
We present MARS (Multilingual Automatic tRanslation System), a research prototype speech-to-speech translation system. MARS is aimed at two-way conversational spoken language translation between English and Mandarin Chinese for limited domains, such as air travel reservations. In MARS, machine translation is embedded within a complex speech processing task, and the translation performance is highly effected by the performance of other components, such as the recognizer and semantic parser, etc. All components in the proposed system are statistically trained using an appropriate training corpus. The speech signal is first recognized by an automatic speech recognizer (ASR). Next, the ASR-transcribed text is analyzed by a semantic parser, which uses a statistical decision-tree model that does not require hand-crafted grammars or rules. Furthermore, the parser provides semantic information that helps further re-scoring of the speech recognition hypotheses. The semantic content extracted by the parser is formatted into a language-independent tree structure, which is used for an interlingua based translation. A Maximum Entropy based sentence-level natural language generation (NLG) approach is used to generate sentences in the target language from the semantic tree representations. Finally, the generated target sentence is synthesized into speech by a speech synthesizer.Many new features and innovations have been incorporated into MARS: the translation is based on understanding the meaning of the sentence; the semantic parser uses a statistical model and is trained from a semantically annotated corpus; the output of the semantic parser is used to select a more specific language model to refine the speech recognition performance; the NLG component uses a statistical model and is also trained from the same annotated corpus. These features give MARS the advantages of robustness to speech disfluencies and recognition errors, tighter integration of semantic information into speech recognition, and portability to new languages and domains. These advantages are verified by our experimental results. 相似文献
18.
复句是汉语语法的重要实体单位,关系词的自动识别是复句标识的基础,对复句的标识以及篇章的研究有重要意义。在对汉语复句语料库进行广泛分析的基础上,从复句关系词所在的环境和关系词的组合搭配方面进行特征的提取,对提取的特征进行形式化描述。采用互信息和信息增益相结合的方式进行特征选择以及冗余特征的消除;使用贝叶斯模型对特征集合进行训练和测试;将基于统计过程的结果转化为规则,形成规则库,并根据规则进行关系词自动识别。实验结果显示,本方法获得了较高的识别正确率,具有可行性和有效性。 相似文献
19.
Rainfall prediction model using soft computing technique 总被引:6,自引:0,他引:6
K. W. Wong P. M. Wong T. D. Gedeon C. C. Fung 《Soft Computing - A Fusion of Foundations, Methodologies and Applications》2003,7(6):434-438
Rainfall prediction in this paper is a spatial interpolation problem that makes use of the daily rainfall information to
predict volume of rainfall at unknown locations within area covered by existing observations. This paper proposed the use
of self-organising map (SOM), backpropagation neural networks (BPNN) and fuzzy rule systems to perform rainfall spatial interpolation
based on local method. The SOM is first used to separate the whole data space into some local surface automatically without
any knowledge from the analyst. In each sub-surface, the complexity of the whole data space is reduced to something more homogeneous.
After classification, BPNNs are then use to learn the generalization characteristics from the data within each cluster. Fuzzy
rules for each cluster are then extracted. The fuzzy rule base is then used for rainfall prediction. This method is used to
compare with an established method, which uses radial basis function networks and orographic effect. Results show that this
method could provide similar results from the established method. However, this method has the advantage of allowing analyst
to understand and interact with the model using fuzzy rules. 相似文献