Multi-task clustering via domain adaptation

Clustering is a fundamental topic in pattern recognition and machine learning research. Traditional clustering methods deal with a single clustering task on a single data set. However, in many real applications, multiple similar clustering tasks are involved simultaneously, e.g., clustering clients of different shopping websites, in which data of different subjects are collected for each task. These tasks are cross-domains but closely related. It is proved that we can improve the individual performance of each clustering task by appropriately utilizing the underling relation. In this paper, we will propose a new approach, which performs multiple related clustering tasks simultaneously through domain adaptation. A shared subspace will be learned through domain adaptation, where the gap of distributions among tasks is reduced, and the shared knowledge will be transferred through all tasks by exploiting the strengthened relation in the learned subspace. Then the object is set as the best clustering in both the original and learned spaces. An alternating optimization method is introduced and its convergence is theoretically guaranteed. Experiments on both synthetic and real data sets demonstrate the effectiveness of the proposed approach.     

Process optimization via constraints adaptation

In the framework of real-time optimization, measurement-based schemes have been developed to deal with plant-model mismatch and process variations. These schemes differ in how the feedback information from the plant is used to adapt the inputs. A recent idea therein is to use the feedback information to adapt the constraints of the optimization problem instead of updating the model parameters. These methods are based on the observation that, for many problems, most of the optimization potential arises from activating the correct set of constraints. In this paper, we provide a theoretical justification of these methods based on a variational analysis. Then, various aspects of the constraint-adaptation algorithm are discussed, including the detection of active constraints and convergence issues. Finally, the applicability and suitability of the constraint-adaptation algorithm is demonstrated with the case study of an isothermal stirred-tank reactor.     

基于迁移子空间的半监督领域自适应方法

传统子空间学习方法在对齐领域总体分布时往往忽略样本类别信息,若原始样本判别力不足,将难以保证投影后子空间中样本的判别性.针对该问题,提出迁移子空间的半监督领域自适应方法.通过充分利用样本类别标签先验信息,在得到具有判别性子空间的同时充分挖掘重构矩阵中蕴含的鉴别信息,增强子空间跨领域特征表达的鉴别力和鲁棒性,提高模型的分...     

On the lower bound for a quadratic problem on the Stiefel manifold

We study the problem of finding the global minimum of a homogeneous quadratic function of special kind over the Stiefel manifold. For two variants of this problem, a low bound is proposed that is the dual Lagrange bound in the quadratic statement obtained using a family of redundant restrictions. The dual bound is proved to be exact in the case where the problem is considered over Boolean variables. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 5, pp. 95–103, September–October 2008.     

Visual domain adaptation via transfer feature learning

One of the serious challenges in computer vision and image classification is learning an accurate classifier for a new unlabeled image dataset, considering that there is no available labeled training data. Transfer learning and domain adaptation are two outstanding solutions that tackle this challenge by employing available datasets, even with significant difference in distribution and properties, and transfer the knowledge from a related domain to the target domain. The main difference between these two solutions is their primary assumption about change in marginal and conditional distributions where transfer learning emphasizes on problems with same marginal distribution and different conditional distribution, and domain adaptation deals with opposite conditions. Most prior works have exploited these two learning strategies separately for domain shift problem where training and test sets are drawn from different distributions. In this paper, we exploit joint transfer learning and domain adaptation to cope with domain shift problem in which the distribution difference is significantly large, particularly vision datasets. We therefore put forward a novel transfer learning and domain adaptation approach, referred to as visual domain adaptation (VDA). Specifically, VDA reduces the joint marginal and conditional distributions across domains in an unsupervised manner where no label is available in test set. Moreover, VDA constructs condensed domain invariant clusters in the embedding representation to separate various classes alongside the domain transfer. In this work, we employ pseudo target labels refinement to iteratively converge to final solution. Employing an iterative procedure along with a novel optimization problem creates a robust and effective representation for adaptation across domains. Extensive experiments on 16 real vision datasets with different difficulties verify that VDA can significantly outperform state-of-the-art methods in image classification problem.     

基于施蒂费尔流形和线性优化重采样的粒子滤波器

为了解决粒子退化问题,提出一种基于施蒂费尔流形和线性优化重采样的粒子滤波算法.将系统模型置于施蒂费尔流形之上,采用朗之万分布描述过程转移概率分布,用矩阵正态分布表示似然函数分布,从而得到一种较为通用的重要性概率密度函数选择方法;同时,将重采样中抛弃的粒子与复制的粒子按照一定的线性组合方式产生新粒子.仿真结果表明.该算法具有较高的滤波效率、滤波精度和较强的滤波鲁棒性.     

Plantwide optimization via real-time optimization with persistent parameter adaptation

Determining the optimization scope is a major issue whenever implementing Real-time Optimization (RTO). Ideally, the optimization problem should encompass the whole plant and not a single unit, which represents only a local subset of the problem. However, if the standard RTO method, the two-step approach (TS), is applied to the entire plant, the whole system needs to be at steady-state (SS) in order to initiate the optimization cycle. This condition is rarely found in practice. One alternative is to apply Real-time Optimization with Persistent Parameter Adaptation (ROPA). ROPA is an RTO variant that integrates online estimators to the standard TS framework and avoids the need of waiting for steady-state to trigger the optimization cycle. However, the problem shifts to obtaining a dynamic model of the entire plant, which can be challenging and time consuming. This paper proposes a variant of ROPA, named asynchronous ROPA (asROPA), where the plant-wide model is partitioned into submodels and, depending on their characteristics, their parameters are updated using either online or steady-state estimators. Consequently, it is not necessary to obtain a dynamic model for the whole process. This asynchronous updating strategy allows the plant-wide model to be up-to-date to the process and the plant-wide optimization can be scheduled at any arbitrary time. The new strategy is applied to a case study consisting of a system whose model can be partitioned into a separation and a reaction submodel. The plant-wide results indicate that asROPA reacts much faster to the disturbances in comparison to the TS approach, improving the overall economic performance and is able to drive the system to the plant-wide optimum. Additionally, a strategy for partitioning the process and choosing the estimation strategy for each partition is proposed.     

Interpretable clustering: an optimization approach

Machine Learning - State-of-the-art clustering algorithms provide little insight into the rationale for cluster membership, limiting their interpretability. In complex real-world applications, the...     

A non-monotone linear search algorithm with mixed direction on Stiefel manifold

In this paper, we propose a non-monotone line search method for solving optimization problems on Stiefel manifold. The main novelty of our approach is that our method uses a search direction based on a linear combination of descent directions and a Barzilai–Borwein line search. The feasibility is guaranteed by projecting each iterate on the Stiefel manifold through SVD (singular value decomposition) factorizations. Some theoretical results for analysing the algorithm are presented. Finally, we provide numerical experiments for comparing our algorithm with other state-of-the-art procedures. The code is available online. The experimental results show that the proposed algorithm is competitive with other approaches and for particular problems, the computational performance is better than the state-of-the-art algorithms.     

Unsupervised visual domain adaptation via discriminative dictionary evolution

Pattern Analysis and Applications - This work focuses on unsupervised visual domain adaptation which is still challenging in visual recognition. Most of the attention has been dedicated to seeking...     

基于跨域流形正则化特征域适应的浮选工况识别

针对浮选过程的故障工况信息不足难以建立准确识别模型,导致调整浮选生产工况不及时,从而无法正常稳定运行的问题,提出一种基于跨域流形正则化特征域适应方法.该方法将已有相似完备浮选过程积累的丰富工况信息作为源域迁移至未建模的不完备浮选过程的目标域中,首先,通过最大域内类密度和局部流形正则化约束分别保留原始判别信息和维持域内邻域结构信息不变,从而提取完备工况与不完备工况域间的特征并投影至公共子空间;然后,由最大均值差异缩小源域与目标域间分布差异,建立分类识别模型,再结合D-S证据理论,融合浮选过程泡沫的静态特征与动态特征信息,提高对不完备浮选过程工况识别的泛化能力,保证得到较好的识别分类效果;最后,通过仿真实验验证所提出方法的有效性.     

渐进式分离的开放集模糊域自适应算法

域自适应的目的是利用有标记（源）域中的信息来提高未标记（目标）域模型的分类性能,且这种方法已经取得了不错的成果。然而在具有开放性的现实场景下,目标域通常包含源域中未观察到的未知类样本,这被称为开放集域自适应问题。传统的域自适应算法对这样具有挑战性的场景设定无能为力,因此提出了渐进式分离的开放集模糊域自适应算法。首先,基于引进隶属度的开放集模糊域自适应算法,探索了逐步分离目标域中已知类和未知类样本的方法;然后,仅将从目标域中分离出的已知类与源域对齐,从而减小两个域之间的分布差异,进行模糊域自适应。所提算法很好地解决了由于未知类和已知类之间的不匹配而导致的负迁移所带来的影响。在Office数据集上的6组域自适应转化实验结果表明,与传统的域自适应算法比较,所提算法在图像分类中的精度有显著的提升,验证了该算法可以逐步增强域自适应分类模型的准确性和鲁棒性。     

Robust domain adaptation

We derive a generalization bound for domain adaptation by using the properties of robust algorithms. Our new bound depends on λ-shift, a measure of prior knowledge regarding the similarity of source and target domain distributions. Based on the generalization bound, we design SVM variants for binary classification and regression domain adaptation algorithms.     

An unconstrained H 2 model order reduction optimisation algorithm based on the Stiefel manifold for bilinear systems

In this paper, the optimal H ₂ model order reduction (MOR) problem for bilinear systems is explored. The orthogonality constraint of the cost function generated by the H ₂ MOR error makes it is posed not on the Euclidean space, but can be discussed on the Stiefel manifold. Then, the H ₂ optimal MOR problem of bilinear systems is turned into the unconstrained optimisation on the Stiefel manifold. The explicit expression of the gradient for the cost function on this manifold is derived. Full use of the geometry properties of this Stiefiel manifold, we propose a feasible and effective iterative algorithm to solve the unconstrained H ₂ minimisation problem. Moreover, the convergence of our algorithm is rigorously proved. Finally, two practical examples related to bilinear systems demonstrate the effectiveness of our algorithm.     

Towards Interpretable Defense Against Adversarial Attacks via Causal Inference

Machine Intelligence Research - Deep learning-based models are vulnerable to adversarial attacks. Defense against adversarial attacks is essential for sensitive and safety-critical scenarios....     

Geometry-aware domain decomposition for T-spline-based manifold modeling

This paper presents a new and effective method to construct manifold T-splines of complicated topology/geometry. The fundamental idea of our novel approach is the geometry-aware object segmentation, by which an arbitrarily complicated surface model can be decomposed into a group of disjoint components that comprise branches, handles, and base patches. Such a domain decomposition simplifies objects of arbitrary topological type into a family of genus-zero/one open surfaces, each of which can be conformally parameterized into a set of rectangles. In contrast to the conventional decomposition approaches, our method can guarantee that the cutting locus are consistent on the parametric domain. As a result, the resultant T-splines of decomposed components are automatically glued and have high-order continuity everywhere except at the extraordinary points. We show that the number of extraordinary points of the domain manifold is bounded by the number of segmented components. Furthermore, the entire mesh-to-spline data conversion pipeline can be implemented with full automation, and thus, has potential in shape modeling and reverse engineering applications of complicated real-world objects.     

Self-corrected unsupervised domain adaptation

Unsupervised domain adaptation (UDA), which aims to use knowledge from a label-rich source domain to help learn unlabeled target domain, has recently attracted much attention. UDA methods mainly concentrate on source classification and distribution alignment between domains to expect the correct target prediction. While in this paper, we attempt to learn the target prediction end to end directly, and develop a Self-corrected unsupervised domain adaptation (SCUDA) method with probabilistic label correction. SCUDA adopts a probabilistic label corrector to learn and correct the target labels directly. Specifically, besides model parameters, those target pseudo-labels are also updated in learning and corrected by the anchor-variable, which preserves the class candidates for samples. Experiments on real datasets show the competitiveness of SCUDA.     

Layer-wise domain correction for unsupervised domain adaptation

Deep neural networks have been successfully applied to numerous machine learning tasks because of their impressive feature abstraction capabilities. However, conventional deep networks assume that the training and test data are sampled from the same distribution, and this assumption is often violated in real-world scenarios. To address the domain shift or data bias problems, we introduce layer-wise domain correction (LDC), a new unsupervised domain adaptation algorithm which adapts an existing deep network through additive correction layers spaced throughout the network. Through the additive layers, the representations of source and target domains can be perfectly aligned. The corrections that are trained via maximum mean discrepancy, adapt to the target domain while increasing the representational capacity of the network. LDC requires no target labels, achieves state-of-the-art performance across several adaptation benchmarks, and requires significantly less training time than existing adaptation methods.     

A manifold imbedding algorithm for optimization problems

A manifold imbedding algorithm is described for the solution of two-point boundary value problems arising in optimization problems.