分布式系统中的故障处理

对一种实时分布式系统中的故障处理机制进行了研究,详细描述了该系统中故障管理的概念,故障定义、故障分类与级别,故障检测、故障滤波、故障确认、故障相关分析、故障定位、故障隔离,健康监控、故障管理的功能与实现。任何系统可按其需求选择其中介绍的一种或多种技术组合。     

二次信息过滤中的分类算法研究与应用

二次信息过滤是根据搜索引擎返回的网页,对过滤结果进行理解、分析并进一步过滤,以使结果优化。二次过滤中的样本分类处理中,通过对基于Bayesian算法的网页识别技术的研究,探讨了基于改进的Bayesian算法的网页识别技术,用于对二次过滤中的整篇文档进行进一步的分析以提高过滤效果。     

协同过滤算法在推荐系统中的应用

介绍了协同过滤算法,并对算法进行了改进,解决了用户稀疏的情况下传统算法的不足,同时通过引入评分阈值,显著提高了个性化协同过滤算法的推荐精度。     

邮件过滤算法的比较

探讨了邮件过滤器的体系,介绍了6种常用的过滤算法.在4个公用的数据集上,利用3个评价指标对这6种算法进行了全面的比较.实验的主要结论有:集成过滤器的方法有利于提高过滤效果;与其它两个数据集相比,PU1和Ling-spam数据集用于评价过滤器时会得出相对乐观的结果;NB和k-NN的表现不稳定,对数据集的敏感程度较高.对于邮件过滤研究者了解、改进已有算法,提出新的算法有一定的参考价值.     

支持向量机在反垃圾邮件过滤中的应用

垃圾邮件对计算机系统的安全和人们的生活造成了严重的威胁,反垃圾邮件问题已经成为的具有重要现实意义的研究课题.针对垃圾邮件过滤本质是分类问题,提出了一种基于服务器前端的反垃圾邮件过滤方法,它采用了改进的v支持向量机算法对邮件内容进行分类,过滤垃圾邮件.研究结果表明该方法与直接的支持向量机增量算法相比,提高了过滤的准确率,具有一定的应用价值.     

协同过滤算法中的相似性度量方法研究

衡量用户的相似性是协同过滤算法的核心内容,用户间相似性的准确率对个性化推荐的结果会有显著影响。通过对用户-项目评分记录的分析,在比较pearson和jaccard相似性的基础上对相似性度量方法进行改进,并将该改进方法应用于MovieLens站点提供的数据集进行实证分析。实证研究表明,改进后的算法可以提高个性化推荐的准确性,并在一定程度上克服数据稀疏性对推荐质量的影响。     

基于增量协同过滤和潜在语义分析的混合推荐算法

为了解决传统新闻推荐系统定期更新推荐算法不能根据用户喜好的变化进而动态地调整推荐列表的问题,提出了一种混合推荐算法(IULSACF)。该算法包含了2个关键部分:基于项目的增量更新协同过滤算法和基于关键词频率的潜在语义分析算法。该混合推荐算法在基于项目的增量更新协同过滤模块中,通过对项目相似度列表增量更新来动态地调整推荐列表,并结合潜在语义分析算法来确保所推荐文章的相关性。实验结果表明,所提出的IULSACF算法在各项评价指标上均优于传统的推荐方法。     

基于Spark的混合协同过滤算法改进与实现

针对传统协同过滤在推荐过程中存在的稀疏性、扩展性以及个性化问题,通过引入算法集成的思想,旨在优化和改进一种新型的基于Spark平台下的混合协同过滤。借鉴了Stacking集成学习思想,将多个弱推荐器线性加权组合,形成综合性强的推荐器。首先,算法基于近邻协同过滤,结合分类、流行度、好评度等对近邻相似度计算策略进行优化,旨在改善相似度的合理性以及相似度计算的复杂度,在一定程度上改善了评分稀疏性的问题;同时,该算法结合Spark分布式计算平台,充分借鉴分布式平台的优点,利用其流式处理以及分布式存储结构等特性,设计并实现一种推荐算法的增量迭型,解决协同过滤算法扩展性和实时性问题。实验数据采用UCI公用数据集MovieLens和NetFlix电影评分数据,实验结果表明,改进算法在推荐个性化、准确率以及扩展性上都有不错的表现,较以前同类型算法均有不同程度的提高,为推荐系统的应用提供一种可行的算法集成方案。     

一种基于GridGIS的增量式协同过滤算法

空间数据的广泛应用需要高效的推荐系统来管理,以增加空间数据的可用性。用户协同过滤(Collaborative Filtering)是推荐系统中发展最为迅速的方法之一,也是在电子商务领域应用最广泛的方法。在研究传统协同过滤算法的基础上提出了一种减轻数据稀疏性对推荐效果产生的负面影响的方法。提出了一种基于项目相似度的数据填充方法,其目的在于当原始数据集比较稀疏时为算法提供足够的数据支持。经实验证明,改进算法在空间数据集上比传统方法有更好的预测性能和运行效率。     

中文垃圾邮件过滤系统中的特征提取算法

针对垃圾邮件过滤,首先对获取的垃圾邮件及合法邮件进行分词,预处理,构建文本矢量,然后用四种常用的特征词提取方法进行矢量降维,再在此基础上,给出了一种综合性的特征词提取算法,即按照各个评估函数的排序结果,取它们交集的前n个特征词作为候选词进行分类测试,仿真比较了各个算法中n对分类结果的影响,从而验证了该算法的有效性。     

Robust multiple‐fault detection filter

A new robust multiple‐fault detection and identification algorithm is determined. Different from other algorithms which explicitly force the geometric structure by using eigenstructure assignment or geometric theory, this algorithm is derived from solving an optimization problem. The output error is divided into several subspaces. For each subspace, the transmission from one fault, denoted the associated target fault, is maximized while the transmission from other faults, denoted the associated nuisance fault, is minimized. Therefore, each projected residual of the robust multiple‐fault detection filter is affected primarily by one fault and minimally by other faults. The transmission from process and sensor noises is also minimized so that the filter is robust with respect to these disturbances. It is shown that, in the limit where the weighting on each associated nuisance fault transmission goes to infinity, the filter recovers the geometric structure of the restricted diagonal detection filter of which the Beard–Jones detection filter and unknown input observer are special cases. Filter designs can be obtained for both time‐invariant and time‐varying systems. Copyright © 2002 John Wiley & Sons, Ltd.     

Optimal stochastic fault detection filter

A fault detection and identification algorithm, called optimal stochastic fault detection filter, is determined. The objective of the filter is to detect a single fault, called the target fault, and block other faults, called the nuisance faults, in the presence of the process and sensor noises. The filter is derived by maximizing the transmission from the target fault to the projected output error while minimizing the transmission from the nuisance faults. Therefore, the residual is affected primarily by the target fault and minimally by the nuisance faults. The transmission from the process and sensor noises is also minimized so that the filter is robust with respect to these disturbances. It is shown that the filter recovers the geometric structure of the unknown input observer in the limit where the weighting on the nuisance fault transmission goes to infinity. Further, the asymptotic behavior of the filter near the limit is determined by using a perturbation method. Filter designs can be obtained for both time-invariant and time-varying systems.     

EMD滤波在煤矿电动机故障诊断中的应用

针对煤矿电动机信号故障特征难以提取的问题,提出将EMD滤波方法应用于煤矿电动机的振动信号分析,通过分析电动机故障特征频率的分布特征,采取不同的EMD滤波方法来去除振动信号中的噪声。现场应用结果表明,该方法可以有效降低噪声,提高信号的信噪比。     

多传感器系统中基于扩展卡尔曼滤波器的在线故障检测

随着多传感器系统的广泛应用,在线故障对于系统性能影响严重,如何使得多传感器系统具有自主故障检测与诊断能力成为首要问题。根据非线性多传感器系统的输入信号、输出信号和故障阵列,建立一种具有多输入多输出处理和自调节加强功能的扩展卡尔曼滤波器（ EKF）的故障分析模型,在此基础上,提出了一种适用于多传感器系统的在线故障检测算法及其在传感器节点上的实施架构。实验结果表明：所提算法在低并发故障和高并发故障环境下均具有高准确度故障报告能力。此外,在温度传感器上实施所提算法,温度监测值的对比结果验证了所提算法比传统算法具有更好的系统性能保证能力。     

双线性检测滤波器及其故障可检测条件

研究了双线性检测滤波器的故障可检测条件. 设计了一种检测滤波器, 并得到了其故障完全可检测的严格条件. 结果表明要使检测滤波器故障完全可检测, 系统独立的传感器和独立的状态数至少和系统故障种类相同. 这个结果符合线性空间的性质. 最后利用所构造的滤波器分析了算例.     

An adaptive particle filter for soft fault compensation of mobile robots

Soft fault compensation plays an important role in mobile robot locating, mapping, and navigating. It is difficult to achieve fast and accurate compensation for mobile robots because they are usually highly non-linear, non-Gaussian systems with limited computation and memory resources. An adaptive particle filter is presented to compensate two kinds of soft faults for mobile robots, i.e., noise or factor faults of dead reckoning sensors and slippage of wheels. Firstly, the kinematics models and the fault models are discussed, and five kinds of residual features are extracted to detect soft faults. Secondly, an adaptive particle filter is designed for fault compensation, and two kinds of adaptive scheme are discussed： 1） the noise variances of linear speed and yaw rate are adjusted according to residual features; 2） the particle number is adapted according to Kullback-Leibler divergence （KLD） of two approximate distribution denoted with two particle sets with different particles, i.e., increasing particle number if the KLD is large and decreasing particle number if the KLD is small. The theoretic proof is given and experimental results show the efficiency and accuracy of the presented approach.     

BPSO优化算法在含DG配电网中的故障诊断研究

针对含分布式电源(distributed generation,DG)的配电网故障检测,传统的故障检测方法容易出现检测精度低、适用性差、容错力低等问题;为此,提出了一种改进的二进制粒子群算法(binary particle swarm optimization,BPSO)进行故障检测,该算法是在BPSO算法的基础上,重新对自适应度值进行定义确保种群寻找到最为准确的最优解,解决BPSO算法容易陷入"早熟"的情况;通过IEEE33节点进行仿真分析,实验结果表明:采用改进的BPSO算法可以有效对故障区段进行定位,验证了改进的BPSO算法的有效性和正确性;同时,当存在信息畸变时,改进的BPSO算法比一般算法具有更强的容错能力。     

Discrete time-varying fault detection filter for non-uniformly sampled-data systems

This paper investigates the fault detection problem for non-uniformly sampled-data systems. No periodic assumption is made for the sampling instants. In contrast to most currently available resuks that are limited to strictly proper systems, measurement noises are considered. With the operators introduced to capture the inter-sampling behaviors of disturbances and faults, an omine fault detection algorithm is first derived to optimize the ratio-type design objective. It is then equivalently transformed into a recursive algorithm consisting of a discrete time-varying fault detection filter and the corresponding residual evaluation function. As repeated computation of the parity vectors is avoided, the proposed fault detection filter can help reduce the online computational burden with comparison to the existing parity relation based fault detection method.     

Cluster-based distributed dynamic cuckoo filter system for Redis

ABSTRACT

With the exponential growth of network data storage scale, the issue of uniform distribution and efficient retrieval of data in the distributed storage systems such as the Redis cluster has received increasing attention in recent years. In view of the existing problems in scalability, usability and other aspects of the solution in current researches, we propose the distributed dynamic cuckoo filter system based on Redis cluster. On one hand, we introduce an efficient hash indexing structure–dynamic cuckoo filter (DCF), which only stores the fingerprint information of data, and has the automatically scalable capacity to meet the demand of data storage on a dynamic scale. On the other hand, we use an improved consistent hashing algorithm to construct Redis cluster and use the thorough communication mechanism of Redis cluster to achieve the data sharing and efficient utilisation of multi-machine filters. The scheme proposed in this paper can take the time and space efficiency into account, greatly promote the retrieval performance of massive data, and improve the reliability and availability of Redis cluster.     

基于多目标约束的鲁棒故障检测滤波器设计

针对包含未知输入的线性时不变系统,研究了其鲁棒故障检测滤波器设计问题。由于故障信号往往分布在有限频域段内,设计的目标包括使特定有限频域段上的复合性能指标最小化以及满足区域极点配置的要求。一个基于LMI的方法被提出用于解决该设计问题。该方法的优点在于求解过程中可以获取给定有限频域段上的频域指标的真实值,并可求得满足目标的最优解。因此,设计的故障检测滤波器可以获得良好的故障检测性能。一个基于某国产歼击机的设计实例验证了该方法的有效性。