Toward value difference metric with attribute weighting

In distance metric learning, recent work has shown that value difference metric (VDM) with a strong attribute independence assumption outperforms other existing distance metrics. However, an open question is whether VDM with a less restrictive assumption can perform even better. Many approaches have been proposed to improve VDM by weakening the assumption. In this paper, we make a comprehensive survey on the existing improved approaches and then propose a new approach to improve VDM by attribute weighting. We name the proposed new distance function as attribute-weighted value difference metric (AWVDM). Moreover, we propose a modified attribute-weighted value difference metric (MAWVDM) by incorporating the learned attribute weights into the conditional probability estimates of AWVDM. AWVDM and MAWVDM significantly outperform VDM and inherit the computational simplicity of VDM simultaneously. Experimental results on a large number of UCI data sets validate the performance of AWVDM and MAWVDM.     

A Criterion for Extractability of Mutual Information for a Triple of Strings

We say that the mutual information of a triple of binary strings a, b, c can be extracted if there exists a string d such that a, b, and c are independent given d, and d is simple conditional to each of the strings a, b, and c. It is proved that the mutual information between a, b, and c can be extracted if and only if the values of the conditional mutual informations I(a : b|c), I(a : c|b), and I(b : c|a) are negligible. The proof employs a non-Shannon-type information inequality (a generalization of the recently discovered Zhang–Yeung inequality).     

One-Way Functions Using Algorithmic and Classical Information Theories

We prove several results relating injective one-way functions, time-bounded conditional Kolmogorov complexity, and time-bounded conditional entropy. First we establish a connection between injective, strong and weak one-way functions and the expected value of the polynomial time-bounded Kolmogorov complexity, denoted here by?E(K ^t (x|f(x))). These results are in both directions. More precisely, conditions on?E(K ^t (x|f(x))) that imply that?f is a weak one-way function, and properties of?E(K ^t (x|f(x))) that are implied by the fact that?f is a strong one-way function. In particular, we prove a separation result: based on the concept of time-bounded Kolmogorov complexity, we find an interval in which every function?f is a necessarily weak but not a strong one-way function. Then we propose an individual approach to injective one-way functions based on Kolmogorov complexity, defining Kolmogorov one-way functions and prove some relationships between the new proposal and the classical definition of one-way functions, showing that a Kolmogorov one-way function is also a deterministic one-way function. A relationship between Kolmogorov one-way functions and the conjecture of polynomial time symmetry of information is also proved. Finally, we relate?E(K ^t (x|f(x))) and two forms of time-bounded entropy, the unpredictable entropy?H ^unp, in which ??one-wayness?? of a function can be easily expressed, and the Yao⁺ entropy, a measure based on compression/decompression schema in which only the decompressor is restricted to be time-bounded.     

Predictive complexity and information

The notions of predictive complexity and of corresponding amount of information are considered. Predictive complexity is a generalization of Kolmogorov complexity which bounds the ability of any algorithm to predict elements of a sequence of outcomes. We consider predictive complexity for a wide class of bounded loss functions which are generalizations of square-loss function. Relations between unconditional KG(x) and conditional KG(x|y) predictive complexities are studied. We define an algorithm which has some “expanding property”. It transforms with positive probability sequences of given predictive complexity into sequences of essentially bigger predictive complexity. A concept of amount of predictive information IG(y:x) is studied. We show that this information is noncommutative in a very strong sense and present asymptotic relations between values IG(y:x), IG(x:y), KG(x) and KG(y).     

Information-theoretic characterizations of recursive infinite strings

Loveland and Meyer have studied necessary and sufficient conditions for an infinite binary string x to be recursive in terms of the program-size complexity relative to n of its n-bit prefixes x_n. Meyer has shown that x is recursive iff ?c, ?n, K(x_n?n) ? c, and Loveland has shown that this is false if one merely stipulates that K(x_n?n) ? c for infinitely many n. We strengthen Meyer's theorem. From the fact that there are few minimal-size programs for calculating n given result, we obtain a necessary and sufficient condition for x to be recursive in terms of the absolute program-size complexity of its prefixes: x is recursive iff ?c, ?n, K(x_n) ? K(n) + c. Again Loveland's method shows that this is no longer a sufficient condition for x to be recursive if one merely stipulates that K(x_n) ? K(n) + c for infinitely many n.     

An improved approximation algorithm for the metric maximum clustering problem with given cluster sizes

The input to the metric maximum clustering problem with given cluster sizes consists of a complete graph G=(V,E) with edge weights satisfying the triangle inequality, and integers c₁,…,cp. The goal is to find a partition of V into disjoint clusters of sizes c₁,…,cp, maximizing the sum of weights of edges whose two ends belong to the same cluster. We describe an approximation algorithms for this problem with performance guarantee that approaches 0.5 when the cluster sizes are large.     

A Faster Algorithm for Computing the Principal Sequence of Partitions of a Graph

We consider the following problem: given an undirected weighted graph G=(V,E,c) with nonnegative weights, minimize function c(δ(Π))−λ|Π| for all values of parameter λ. Here Π is a partition of the set of nodes, the first term is the cost of edges whose endpoints belong to different components of the partition, and |Π| is the number of components. The current best known algorithm for this problem has complexity O(|V|²) maximum flow computations. We improve it to |V| parametric maximum flow computations. We observe that the complexity can be improved further for families of graphs which admit a good separator, e.g. for planar graphs.     

An attribute value frequency-based instance weighting filter for naive Bayes

Due to its simplicity, efficiency and efficacy, naive Bayes (NB) continues to be one of the top 10 data mining algorithms. A mass of improved approaches to NB have been proposed to weaken its conditional independence assumption. However, there has been little work, up to the present, on instance weighting filter approaches to NB. In this paper, we propose a simple, efficient, and effective instance weighting filter approach to NB. We call it attribute (feature) value frequency-based instance weighting and denote the resulting improved model as attribute value frequency weighted naive Bayes (AVFWNB). In AVFWNB, the weight of each training instance is defined as the inner product of its attribute value frequency vector and the attribute value number vector. The experimental results on 36 widely used classification problems show that AVFWNB significantly outperforms NB, yet at the same time maintains the computational simplicity that characterizes NB.     

Online Metric Tracking and Smoothing

We consider the online smoothing problem, in which a tracker is required to maintain distance no more than Δ≥0 from a time-varying signal f while minimizing its own movement. The problem is determined by a metric space (X,d) with an associated cost function c:?→?. Given a signal f ₁,f ₂,…∈X the tracker is responsible for producing a sequence a ₁,a ₂,… of elements of X that meet the proximity constraint: d(f _i ,a _i )≤Δ. To complicate matters, the tracker is on-line—the value a _i may only depend on f ₁,…,f _i —and wishes to minimize the cost of his travels, ∑c(d(a _i ,a _i+1)). We evaluate such tracking algorithms competitively, comparing this with the cost achieved by an optimal adversary apprised of the entire signal in advance. The problem was originally proposed by Yi and Zhang (In: Proceedings of the 20th annual ACM-SIAM symposium on discrete algorithms (SODA), pp. 1098–1107. ACM Press, New York, 2009), who considered the natural circumstance where the metric spaces are taken to be ? ^k with the ? ₂ metric and the cost function is equal to 1 unless the distance is zero (thus the tracker pays a fixed cost for any nonzero motion).

We begin by studying arbitrary metric spaces with the “pay if you move” metric of Yi and Zhang (In: Proceedings of the 20th annual ACM-SIAM symposium on discrete algorithms (SODA), pp. 1098–1107. ACM Press, New York, [2009]) described above and describe a natural randomized algorithm that achieves a O(logb _Δ)-competitive ratio, where b _Δ=max _x∈X |B _Δ(x)| is the maximum number of points appearing in any ball of radius Δ. We show that this bound is tight.

We then focus on the metric space ? with natural families of monotone cost functions c(x)=x ^p for some p≥0. We consider both the expansive case (p≥1) and the contractive case (p<1), and show that the natural lazy algorithm performs well in the expansive case. In the contractive case, we introduce and analyze a novel deterministic algorithm that achieves a constant competitive ratio depending only on p. Finally, we observe that by slightly relaxing the guarantee provided by the tracker, one can obtain natural analogues of these algorithms that work in continuous metric spaces.

     

Approximate Solutions of Polynomial Equations

In this paper, we introduce “approximate solutions" to solve the following problem: given a polynomial F(x, y) over Q, where x represents an n -tuple of variables, can we find all the polynomials G(x) such that F(x, G(x)) is identically equal to a constant c in Q ? We have the following: let F(x, y) be a polynomial over Q and the degree of y in F(x, y) be n. Either there is a unique polynomial g(x)  ∈ Q [ x ], with its constant term equal to 0, such that F(x, y)  = ∑_j = 0ⁿc_j(y − g(x))^jfor some rational numbers c_j, hence, F(x, g(x)  + a)  ∈ Q for all a ∈ Q, or there are at most t distinct polynomials g₁(x),⋯ , g_t(x), t ≤ n, such that F(x, g_i(x))  ∈ Q for 1  ≤ i ≤ t. Suppose that F(x, y) is a polynomial of two variables. The polynomial g(x) for the first case, or g₁(x),⋯ , g_t(x) for the second case, are approximate solutions of F(x, y), respectively. There is also a polynomial time algorithm to find all of these approximate solutions. We then use Kronecker’s substitution to solve the case of F(x, y).     

Substitutions into propositional tautologies

We prove that there is a polynomial time substitution (y₁,…,yn):=g(x₁,…,xk) with k?n such that whenever the substitution instance A(g(x₁,…,xk)) of a 3DNF formula A(y₁,…,yn) has a short resolution proof it follows that A(y₁,…,yn) is a tautology. The qualification “short” depends on the parameters k and n.     

Conditional probability and fuzzy information

The main subject of this paper is the embedding of fuzzy set theory—and related concepts—in a coherent conditional probability scenario. This allows to deal with perception-based information—in the sense of Zadeh—and with a rigorous treatment of the concept of likelihood, dealing also with its role in statistical inference. A coherent conditional probability is looked on as a general non-additive “uncertainty” measure m(·)=P(E|·) of the conditioning events. This gives rise to a clear, precise and rigorous mathematical frame, which allows to define fuzzy subsets and to introduce in a very natural way the counterparts of the basic continuous T-norms and the corresponding dual T-conorms, bound to the former by coherence. Also the ensuing connections of this approach to possibility theory and to information measures are recalled.     

Extracting Kolmogorov complexity with applications to dimension zero-one laws

We apply results on extracting randomness from independent sources to “extract” Kolmogorov complexity. For any α,?>0, given a string x with K(x)>α|x|, we show how to use a constant number of advice bits to efficiently compute another string y, |y|=Ω(|x|), with K(y)>(1-?)|y|. This result holds for both unbounded and space-bounded Kolmogorov complexity.We use the extraction procedure for space-bounded complexity to establish zero-one laws for the strong dimensions of complexity classes within ESPACE. The unbounded extraction procedure yields a zero-one law for the constructive strong dimensions of Turing degrees.     

Information storage and retrieval — mathematical foundations II (combinatorial problems)

The following combinatorial problem is fundamental for the design of file organizations minimizing simultaneously the storage space and the access time: For a given family $\text{M}$ of subsets of a finite set X, find a partial function S:X→X (if there is one), such that every M?$\text{M}$ can be written as M=lcubx,S(x),?,S^|M|-1 (x)rcub for a suitable x?X. We study the class of families admitting such a function S, as well as some of its subclasses defined by imposing certain restrictions on S (these restrictions reflect the structure of the storage medium to be used for storing a file).     

Parabolic equations with double variable nonlinearities

The paper is devoted to the study of the homogeneous Dirichlet problem for the doubly nonlinear parabolic equation with nonstandard growth conditions:

ut=div(a(x,t,u)|u^|α(x,t)|∇u^|p(x,t)−2∇u)+f(x,t)     

Geometric pattern matching for point sets in the plane under similarity transformations

We consider the following geometric pattern matching problem: Given two sets of points in the plane, P and Q, and some (arbitrary) δ>0, find a similarity transformation T (translation, rotation and scale) such that h(T(P),Q)<δ, where h(⋅,⋅) is the directional Hausdorff distance with L_∞ as the underlying metric; or report that none exists. We are only interested in the decision problem, not in minimizing the Hausdorff distance, since in the real world, where our applications come from, δ is determined by the practical uncertainty in the position of the points (pixels). Similarity transformations have not been dealt with in the context of the Hausdorff distance and we fill the gap here. We present efficient algorithms for this problem imposing a reasonable separation restriction on the points in the set Q. If the L_∞ distance between every pair of points in Q is at least 8δ, then the problem can be solved in O(mn²logn) time, where m and n are the numbers of points in P and Q respectively. If the L_∞ distance between every pair of points in Q is at least cδ, for some c, 0<c<1, we present a randomized approximate solution with expected runtime O(n²c−4ε−8log⁴mn), where ε>0 controls the approximation. Our approximation is on the size of the subset, B⊆P, such that h(T(B),Q)<δ and |B|>(1−ε)|P| with high probability.     

On the oriented chromatic number of Halin graphs

An oriented k-coloring of an oriented graph G is a mapping such that (i) if xy∈E(G) then c(x)≠c(y) and (ii) if xy,zt∈E(G) then c(x)=c(t)⇒c(y)≠c(z). The oriented chromatic number of an oriented graph G is defined as the smallest k such that G admits an oriented k-coloring. We prove in this paper that every Halin graph has oriented chromatic number at most 9, improving a previous bound proposed by Vignal.     

On the random generation and counting of weak order extensions of a poset with given class cardinalities

In previous work, we have proposed a simple algorithm to generate random linear extensions of a partially ordered set (poset). A closely related problem is the random generation of so-called weak order extensions of a poset. Such an extension can be informally characterized as a linear order on the equivalence classes of a partition of the poset, not contradicting the underlying poset order. The generation of linear extensions can then be seen as a special case of the generation of weak order extensions where each equivalence class degenerates into a singleton. If no a priori knowledge about the underlying partition is available, time complexity increases tremendously. In first instance, we therefore restrict to the generation of weak order extensions with given class cardinalities, a problem encountered in the context of ranking algorithms. It will be shown that a first random weak order extension can be generated in O(w²(P)·|I(P)|) time, while every subsequent extension with the same class cardinalities can be obtained in O(w(P)·|P|) time, where |I(P)| denotes the number of ideals of the poset P, and w(P) the width of the poset P. Additionally, the number of weak order extensions obeying the specified class cardinalities can also be obtained in the stated O(w²(P)·|I(P)|) time.     

Weighted aggregation operators based on minimization

Minimization based aggregation operators A_g,D are discussed. Special attention is paid to weighting function g based cases related to some fixed dissimilarity function D. When D₂(x,y)=(x-y)², we recognize mixture operators and we recall some sufficient conditions for g ensuring the monotonicity of A_g,D. For D₁(x,y)=|x-y| and non-decreasing (non-increasing) g, A_g,D is shown to be the upper (lower) median whenever A_g,D is an aggregation operator.