Mining non-redundant diverse patterns: an information theoretic perspective

The discovery of diversity patterns from binary data is an important data mining task. In this paper, we propose the problem of mining highly diverse patterns called non-redundant diversity patterns (NDPs). In this framework, entropy is adopted to measure the diversity of itemsets. In addition, an algorithm called NDP miner is proposed to exploit both monotone properties of entropy diversity measure and pruning power for the efficient discovery of non-redundant diversity patterns. Finally, our experimental results are given to show that the NDP miner can efficiently identify non-redundant diversity patterns.     

Mining non-redundant diverse patterns: an information theoretic perspective

The discovery of diversity patterns from binary data is an important data mining task. In this paper, we propose the problem of mining highly diverse patterns called non-redundant diversity patterns (NDPs). In this framework, entropy is adopted to measure the diversity of itemsets. In addition, an algorithm called NDP miner is proposed to exploit both monotone properties of entropy diversity measure and pruning power for the efficient discovery of non-redundant diversity patterns. Finally, our experimental results are given to show that the NDP miner can efficiently identify non-redundant diversity patterns.     

Hierarchical fuzzy rule based systems using an information theoretic approach

This paper proposes a novel anytime algorithm for the construction of a Hierarchical Fuzzy Rule Based System using an information theoretic approach to specialise rules that do not effectively model the decision space. The amount of uncertainty tolerated within the decision provides a single tuneable parameter to control the trade off between accuracy and interpretability. The algorithm is empirically compared with existing methods of function approximation and is demonstrated on a mobile robot application in simulation.     

Distributed multimedia information systems: an end-to-end perspective

Emerging Web-based applications require distributed multimedia information system (DMIS) infrastructures. Examples of such applications abound in the domains of medicine, entertainment, manufacturing, e-commerce, as well as military and critical national infrastructures. Development of DMIS for such applications need a broad range of technological solutions for organizing, storing, and delivering multimedia information in an integrated, secure and timely manner with guaranteed end-to-end (E2E) quality of presentation (QoP). DMIS are viewed as catalysts for new research in many areas, ranging from basic research to applied technology. This view is a result of the fact that no single monolithic end-to-end architecture for DMIS can meet the wide spectrum of characteristics and requirements of various Web-based multimedia applications. One size does not fit all in this medium of communication. Management of integrated end-to-end QoP and ensuring information security in DMIS, when viewed in conjunction with real world constraints and system-wide performance requirements, present formidable research and implementation challenges. These challenges encompass all the sub-system components of a DMIS. The ultimate objective of achieving a comprehensive end-to-end QoP management relies on the performance and allocation of resources of each of the DMIS sub-system components including networks, databases, and end-systems. In this paper, we elaborate on these challenges and present a high level distributed architecture aimed at providing the critical functionality for a DMIS.

Internet-based information systems use in organizations: an information studies perspective

Abstract. This case study investigates various ways in which different internet-based information systems (IS) are used by organizational participants. Borrowing theoretical insights on information behaviour accumulated over 50 years of information studies research, a conceptual framework is presented to help understand and assess the social and organizational impacts of internet-based IS. The framework describes the use of internet-based IS as a dynamic cycle of information needs–seeking–use activity situated in the context of a firm's information environment. Research questions pertain to the process of how individuals in organizations seek and use information from internet-based IS to satisfy information needs. In terms of information needs, this involves understanding the problem situations that lead participants to use internet-based IS, as well as the characteristics of those problems beyond subject matter. With respect to information seeking, this involves analysing how information from internet-based systems is displayed and formatted to signal their potential usefulness. In terms of information use, this involves how information obtained from internet-based systems is used in practice to resolve or redefine problems. Both quantitative and qualitative research methods are used. Data collection involves web tracking to identify significant episodes of internet-based IS activity, as well as one-on-one interviews to explore the context behind these episodes. Results suggest that it is possible and valuable to identify scenarios of internet-based IS use dominant in an organizational work setting. Doing so can help to identify ways to improve the situated use of internet-based IS that ameliorate the information needs–seeking–use cycle in firms.     

Quadratically constrained information theoretic analysis

Recently Zhang and Brockett extended the framework of‘minimum discrimination information’ (MDI) estimation techniques to include quadratic constraints. They claimed their approach was quite different from the usual Lagrange duality theory. We show that the dual problem obtained by Zhang and Brockett is actually a geometric dual. Hence the quadratically constrained MDI estimation can be enriched by the theory of generalized geometric programming.     

Extreme entropy machines: robust information theoretic classification

Most existing classification methods are aimed at minimization of empirical risk (through some simple point-based error measured with loss function) with added regularization. We propose to approach the classification problem by applying entropy measures as a model objective function. We focus on quadratic Renyi’s entropy and connected Cauchy–Schwarz Divergence which leads to the construction of extreme entropy machines (EEM). The main contribution of this paper is proposing a model based on the information theoretic concepts which on the one hand shows new, entropic perspective on known linear classifiers and on the other leads to a construction of very robust method competitive with the state of the art non-information theoretic ones (including Support Vector Machines and Extreme Learning Machines). Evaluation on numerous problems spanning from small, simple ones from UCI repository to the large (hundreds of thousands of samples) extremely unbalanced (up to 100:1 classes’ ratios) datasets shows wide applicability of the EEM in real-life problems. Furthermore, it scales better than all considered competitive methods.     

自适应的非支配排序遗传算法

针对带精英策略的非支配排序遗传算法不能根据环境变化自适应地动态调整运行参数,难以实现对解空间的高效搜索,提出一种自适应的非支配排序遗传算法.所提出算法根据运行阶段、运行代数和当前临时种群非支配个体数动态调整进化个体的运行参数,通过提高进化算子的自适应能力使算法具有自适应性.经实验对比,所提出算法在收敛性、多样性两方面确有提升,可以有效提高原算法的搜索能力.     

Vector quantization using information theoretic concepts

The process of representing a large data set with a smaller number of vectors in the best possible way, also known as vector quantization, has been intensively studied in the recent years. Very efficient algorithms like the Kohonen self-organizing map (SOM) and the Linde Buzo Gray (LBG) algorithm have been devised. In this paper a physical approach to the problem is taken, and it is shown that by considering the processing elements as points moving in a potential field an algorithm equally efficient as the before mentioned can be derived. Unlike SOM and LBG this algorithm has a clear physical interpretation and relies on minimization of a well defined cost function. It is also shown how the potential field approach can be linked to information theory by use of the Parzen density estimator. In the light of information theory it becomes clear that minimizing the free energy of the system is in fact equivalent to minimizing a divergence measure between the distribution of the data and the distribution of the processing elements, hence, the algorithm can be seen as a density matching method.     

Achievable sensitivity bounds for MIMO control systems via an information theoretic approach

This paper deals with an MIMO feedback control system that has two channels with additive noises and studies the effects of the noises on the input and output signals of the plant. We derive achievable bounds of integral type for sensitivity-like properties of the system based on an information theoretic approach. These bounds are generalizations of Bode’s integral formula for the case that the feedback system includes nonlinear elements.     

Understanding information interactions in diffusion: an evolutionary game-theoretic perspective

Social networks are fundamental mediums for diffusion of information and contagions appear at some node of the network and get propagated over the edges. Prior researches mainly focus on each contagion spreading independently, regardless of multiple contagions’ interactions as they propagate at the same time. In the real world, simultaneous news and events usually have to compete for user’s attention to get propagated. In some other cases, they can cooperate with each other and achieve more influences.In this paper, an evolutionary game theoretic framework is proposed to model the interactions among multiple contagions. The basic idea is that different contagions in social networks are similar to the multiple organisms in a population, and the diffusion process is as organisms interact and then evolve from one state to another. This framework statistically learns the payoffs as contagions interacting with each other and builds the payoff matrix. Since learning payoffs for all pairs of contagions IS almost impossible (quadratic in the number of contagions), a contagion clustering method is proposed in order to decrease the number of parameters to fit, which makes our approach efficient and scalable. To verify the proposed framework, we conduct experiments by using real-world information spreading dataset of Digg. Experimental results show that the proposed game theoretic framework helps to comprehend the information diffusion process better and can predict users’ forwarding behaviors with more accuracy than the previous studies. The analyses of evolution dynamics of contagions and evolutionarily stable strategy reveal whether a contagion can be promoted or suppressed by others in the diffusion process.     

Squeeziness: An information theoretic measure for avoiding fault masking

Fault masking can reduce the effectiveness of a test suite. We propose an information theoretic measure, Squeeziness, as the theoretical basis for avoiding fault masking. We begin by explaining fault masking and the relationship between collisions and fault masking. We then define Squeeziness and demonstrate by experiment that there is a strong correlation between Squeeziness and the likelihood of collisions. We conclude with comments on how Squeeziness could be the foundation for generating test suites that minimise the likelihood of fault masking.     

Measuring observability by generalized information theoretic quantities

A normalized measure is established to provide the quantitative information about the degree of observability for the discrete-time, stochastically autonomous system. This measure is based on the generalized information theoretic quantities （generalized entropy, mutual information） of the system state and the observations, where the system state can be a discrete or a continuous random vector. Some important properties are presented. For the linear case, the explicit formula for the degree of observability is derived, and the equivalence between the proposed measure and the traditional rank condition is proved. The curves for the degree of observability are depicted in a simple example.     

Graded consequence: an institution theoretic study

We develop a general study of graded consequence (of many-valued logic) in an institution theoretic (in the sense of Goguen and Burstall) style. This means both syntax and semantics are considered fully abstract, as well as the satisfaction between them. Our approach contrasts to other approaches on many-valued logic in that it is a multi-signature one, in the spirit of institution theory. We consider graded consequence at three different conceptual levels: entailment, semantic, and closure operators, and explore several interpretations between them. We also study logical connectors and quantifiers both at the entailment and semantic level, compactness and soundness properties.     

IMSP: An information theoretic approach for multi-dimensional sequential pattern mining

Sequential pattern mining is an important data mining problem with broad applications. While the current methods are inducing sequential patterns within a single attribute, the proposed method is able to detect them among different attributes. By incorporating the additional attributes, the sequential patterns found are richer and more informative to the user. This paper proposes a new method for inducing multi-dimensional sequential patterns with the use of Hellinger entropy measure. A number of theorems are proposed to reduce the computational complexity of the sequential pattern systems. The proposed method is tested on some synthesized transaction databases. Dr. Chang-Hwan Lee is a full professor at the Department of Information and Communications at DongGuk University, Seoul, Korea since 1996. He has received his B.Sc. and M.Sc in Computer Science and Statistics from Seoul National University in 1982 and 1988, respectively. He received his Ph.D. in Computer Science and Engineering from University of Connecticut in 1994. Prior to joining DongGuk University in Korea, he had worked for AT&T Bell Laboratories, Middletown, USA. (1994-1995). He also had been a visiting professor at the University of Illinois at Urbana-Champaign (2000-2001). He is author or co-author of more than 50 refereed articles on topics such as machine learning, data mining, artificial intelligence, pattern recognition, and bioinformatics.