Adversarial attacks on text classification models using layer-wise relevance propagation

Due to the nested nonlinear structure inside neural networks, most existing deep learning models are treated as black boxes, and they are highly vulnerable to adversarial attacks. On the one hand, adversarial examples shed light on the decision-making process of these opaque models to interrogate the interpretability. On the other hand, interpretability can be used as a powerful tool to assist in the generation of adversarial examples by affording transparency on the relative contribution of each input feature to the final prediction. Recently, a post-hoc explanatory method, layer-wise relevance propagation (LRP), shows significant value in instance-wise explanations. In this paper, we attempt to optimize the recently proposed explanation-based attack algorithms (EAAs) on text classification models with LRP. We empirically show that LRP provides good explanations and benefits existing EAAs notably. Apart from that, we propose a LRP-based simple but effective EAA, LRPTricker. LRPTricker uses LRP to identify important words and subsequently performs typo-based perturbations on these words to generate the adversarial texts. The extensive experiments show that LRPTricker is able to reduce the performance of text classification models significantly with infinitesimal perturbations as well as lead to high scalability.     

Optimal feature selection for support vector machines

Selecting relevant features for support vector machine (SVM) classifiers is important for a variety of reasons such as generalization performance, computational efficiency, and feature interpretability. Traditional SVM approaches to feature selection typically extract features and learn SVM parameters independently. Independently performing these two steps might result in a loss of information related to the classification process. This paper proposes a convex energy-based framework to jointly perform feature selection and SVM parameter learning for linear and non-linear kernels. Experiments on various databases show significant reduction of features used while maintaining classification performance.     

Random classification noise defeats all convex potential boosters

A broad class of boosting algorithms can be interpreted as performing coordinate-wise gradient descent to minimize some potential function of the margins of a data set. This class includes AdaBoost, LogitBoost, and other widely used and well-studied boosters. In this paper we show that for a broad class of convex potential functions, any such boosting algorithm is highly susceptible to random classification noise. We do this by showing that for any such booster and any nonzero random classification noise rate η, there is a simple data set of examples which is efficiently learnable by such a booster if there is no noise, but which cannot be learned to accuracy better than 1/2 if there is random classification noise at rate η. This holds even if the booster regularizes using early stopping or a bound on the L ₁ norm of the voting weights. This negative result is in contrast with known branching program based boosters which do not fall into the convex potential function framework and which can provably learn to high accuracy in the presence of random classification noise.     

并行集成具有高可解释的TSK模糊分类器

针对分层Takagi-Sugeno-Kang (TSK)模糊分类器可解释性差,以及当增加或删除一个TSK模糊子分类器时Boosting模糊分类器需要重新训练所有TSK模糊子分类器等问题,提出一种并行集成具有高可解释的TSK模糊分类器EP-Q-TSK.该集成模糊分类器每个TSK模糊子分类器可以使用最小学习机(LLM)被并行地快速构建.作为一种新的集成学习方式,该分类器利用每个TSK模糊子分类器的增量输出来扩展原始验证数据空间,然后采用经典的模糊聚类算法FCM获取一系列代表性中心点,最后利用KNN对测试数据进行分类.在标准UCI数据集上,分别从分类性能和可解释性两方面验证了EP-Q-TSK的有效性.     

The collocating local volatility framework – a fresh look at efficient pricing with smile

ABSTRACT

It is a market practice to price exotic derivatives, like callable basket options, with the local volatility model [B. Dupire, Pricing with a smile, Risk 7 (1994), pp. 18–20; E. Derman and I. Kani, Stochastic implied trees: Arbitrage pricing with stochastic term and strike structure of volatility, Int. J. Theor. Appl. Finance 1 (1998), pp. 61–110.] which can, contrary to stochastic volatility frameworks, handle multi-dimensionality easily. On the other hand, a well-known limitation of the nonparametric local volatility model is the necessity of a short-stepping simulation, which, in high dimensions, is computationally expensive. In this article, we propose a new local volatility framework called the collocating local volatility (CLV) model which allows for large Monte Carlo steps and therefore it is computationally efficient. The CLV model is by its construction guaranteed to be almost perfectly calibrated to implied volatility smiles/skews at a given set of expiries. Additionally, the framework allows to control forward volatilities without affecting the fit to plain vanillas. The model requires only a fraction of a second for complete calibration to simple vanilla products.     

结合域适应学习的糖尿病视网膜病变分级诊断

目的 传统的糖尿病视网膜病变(糖网)(diabetic retinopathy, DR)依赖于早期病理特征的精确检测,但由于数据集缺乏病灶标记区域导致无法有效地建立监督性分类模型,引入其他辅助数据集又会出现跨域数据异质性问题;另外,现有的糖网诊断方法大多无法直观地从语义上解释医学模型预测的结果。基于此,本文提出一种端到端式结合域适应学习的糖网自动多分类方法,该方法协同注意力机制和弱监督学习加强优化。方法 首先,利用已标记病灶区域的辅助数据训练病灶检测模型,再将目标域数据集的糖网诊断转化为弱监督学习问题,依靠多分类预测结果指导深度跨域生成对抗网络模型,提升跨域的样本图像质量,用于微调病灶检测模型,进而过滤目标域中一些无关的病灶样本,提升多分类分级诊断性能。最后,在整体模型中融合注意力机制,从医学病理诊断角度提供可解释性支持其分类决策。结果 在公开数据集Messidor上进行糖网多分类评估实验,本文方法获得了71.2%的平均准确率和80.8%的AUC(area under curve)值,相比于其他多种方法具有很大优势,可以辅助医生进行临床眼底筛查。结论 结合域适应学习的糖网分类方法在没有...     

A hierarchical approach to multi-class fuzzy classifiers

In this paper we present a hierarchical approach for generating fuzzy rules directly from data in a simple and effective way. The fuzzy classifier results from the union of fuzzy systems, employing the Wang and Mendel algorithm, built on input regions increasingly smaller, according to a multi-level grid-like partition. Key parameters of the proposed method are optimized by means of a genetic algorithm. Only the necessary partitions are built, in order to guarantee high interpretability and to avoid the explosion of the number of rules as the hierarchical level increases. We apply our method to real-world data collected from a photovoltaic (PV) installation so as to linguistically describe how the temperature of the PV panel and the irradiation relate to the class (low, medium, high) of the energy produced by the panel. The obtained mean and maximum classification percentages on 30 repetitions of the experiment are 97.38% and 97.91%, respectively. We also apply our method to the classification of some well-known benchmark datasets and show how the achieved results compare favourably with those obtained by other authors using different techniques.     

Interpretable fuzzy partitioning of classified data with variable granularity

Fuzzy rule-based systems are effective tools for acquiring knowledge from data and represent it in a linguistically interpretable form. To achieve interpretability, input features are granulated in fuzzy partitions. A critical design decision is the selection of the granularity level for each input feature. This paper presents an approach, called DC^* (Double Clustering with A^*), for automatically designing interpretable fuzzy partitions with optimal granularity. DC^* is specific for classification problems and is mainly based on a two-stage process: the first stage identifies clusters of multidimensional samples in order to derive class-labeled prototypes; in the second stage the one-dimensional projections of such prototypes are further clustered along each dimension simultaneously, thus minimizing the number of clusters for each feature. Moreover, the resulting one-dimensional clusters provide information to define fuzzy partitions that satisfy a number of interpretability constraints and exhibit variable granularity levels. The fuzzy sets in each partition can be labeled by meaningful linguistic terms and used to represent knowledge in a natural language form. Experimental results on both synthetic and real data show that the derived fuzzy partitions can be exploited to define very compact fuzzy rule-based systems that exhibit high linguistic interpretability and good classification accuracy.     

Randomly coloring simple hypergraphs

We study the problem of constructing a (near) uniform random proper q-coloring of a simple k-uniform hypergraph with n vertices and maximum degree Δ. (Proper in that no edge is mono-colored and simple in that two edges have maximum intersection of size one.) We show that if for some α<1 we have Δ?nα and q?Δ(1+α)/kα then Glauber dynamics will become close to uniform in time from a random (improper) start. Note that for k>1+α−1 we can take q=o(Δ).     

Interpretability of first-order linear temporal logics in fork algebras

In this paper we prove theorems on the interpretability of the first-order temporal logics LTL and TL into Fork Algebras. This result is part of a research project on the interpretability of logics in Fork Algebras, and has important applications towards the relational specification of properties of systems within the Ar_gentum tool.     

The iterated defect correction methods for singular two-point boundary value problems

In this paper, the iterated defect correction (IDeC) techniques based on the centered Euler method for the equivalent first order system of the singular two-point boundary value problem in linear case (x ^α y′(x))′ = f(x), y(0) = a,y(1) = b, where 0 < α < 1 are considered. By using the asymptotic expansion of the global error, it is analyzed that the IDeC methods improved the approximate results by means of IDeC steps and the degree of the interpolating polynomials used. Some numerical examples from the literature are given in illustration of this theory.     

A fully discrete local discontinuous Galerkin method for one-dimensional time-fractional Fisher's equation

In this paper, we consider the local discontinuous Galerkin (LDG) finite element method for one-dimensional time-fractional Fisher's equation, which is obtained from the standard one-dimensional Fisher's equation by replacing the first-order time derivative with a fractional derivative (of order α, with 0<α<1). The proposed LDG is based on the LDG finite element method for space and finite difference method for time. We prove that the method is stable, and the numerical solution converges to the exact one with order O(h^k+1+τ^2?α), where h, τ and k are the space step size, time step size, polynomial degree, respectively. The numerical experiments reveal that the LDG is very effective.     

Medical diagnosis of cardiovascular diseases using an interval-valued fuzzy rule-based classification system

ObjectiveTo develop a classifier that tackles the problem of determining the risk of a patient of suffering from a cardiovascular disease within the next 10 years. The system has to provide both a diagnosis and an interpretable model explaining the decision. In this way, doctors are able to analyse the usefulness of the information given by the system.MethodsLinguistic fuzzy rule-based classification systems are used, since they provide a good classification rate and a highly interpretable model. More specifically, a new methodology to combine fuzzy rule-based classification systems with interval-valued fuzzy sets is proposed, which is composed of three steps: (1) the modelling of the linguistic labels of the classifier using interval-valued fuzzy sets; (2) the use of the K_α operator in the inference process and (3) the application of a genetic tuning to find the best ignorance degree that each interval-valued fuzzy set represents as well as the best value for the parameter α of the K_α operator in each rule.ResultsThe suitability of the new proposal to deal with this medical diagnosis classification problem is shown by comparing its performance with respect to the one provided by two classical fuzzy classifiers and a previous interval-valued fuzzy rule-based classification system. The performance of the new method is statistically better than the ones obtained with the methods considered in the comparison. The new proposal enhances both the total number of correctly diagnosed patients, around 3% with respect the classical fuzzy classifiers and around 1% vs. the previous interval-valued fuzzy classifier, and the classifier ability to correctly differentiate patients of the different risk categories.ConclusionThe proposed methodology is a suitable tool to face the medical diagnosis of cardiovascular diseases, since it obtains a good classification rate and it also provides an interpretable model that can be easily understood by the doctors.     

计算机专业基于课程群的EIP-CDIO项目设计

CDIO的教育模式致力于构建学生的可扩展知识体系、个人能力、人际交往能力和系统竞争力,汕头大学工学院计算机系实施以设计为导向的CDIO三级项目设计体系。本文阐述此三级项目设计的基本概念和方法,结合具体教学实践,介绍在计算机科学与技术专业进行的基于课程群的二级项目设计理念和实施过程。此项教学改革在提高学生对课程体系的理解,项目构思、设计、实施和操作的能力,以及团队合作精神等方面取得显著效果。     

Large dicliques in a directed inhomogeneous random graph

Our main interest in this paper is the large dicliques in a directed inhomogeneous random graph model G(n,α, Φ) on n vertices, which has power-law out/in-degree distributions with scaling exponent α>0 and community structures involved in the homophyly matrix Φ. We show that there is a major difference in the size of the largest diclique ω ^d (G(n,α, Φ)) between the case α<2 and α>2 with an intermediate result for α=2. In addition, we show that a simple algorithm with high probability finds a large diclique of size ω ^d (G(n,α, Φ)) in a polynomial time. Our simulation results reveal that the connections between different subcommunities are essential for the formation of large clusters in the networks.     

Improved Functional Mappings via Product Preservation

In this paper, we consider the problem of information transfer across shapes and propose an extension to the widely used functional map representation. Our main observation is that in addition to the vector space structure of the functional spaces, which has been heavily exploited in the functional map framework, the functional algebra (i.e., the ability to take pointwise products of functions) can significantly extend the power of this framework. Equipped with this observation, we show how to improve one of the key applications of functional maps, namely transferring real‐valued functions without conversion to point‐to‐point correspondences. We demonstrate through extensive experiments that by decomposing a given function into a linear combination consisting not only of basis functions but also of their pointwise products, both the representation power and the quality of the function transfer can be improved significantly. Our modification, while computationally simple, allows us to achieve higher transfer accuracy while keeping the size of the basis and the functional map fixed. We also analyze the computational complexity of optimally representing functions through linear combinations of products in a given basis and prove NP‐completeness in some general cases. Finally, we argue that the use of function products can have a wide‐reaching effect in extending the power of functional maps in a variety of applications, in particular by enabling the transfer of high‐frequency functions without changing the representation size or complexity.     

A multi-objective genetic optimization for fast,fuzzy rule-based credit classification with balanced accuracy and interpretability

Credit classification is an important component of critical financial decision making tasks such as credit scoring and bankruptcy prediction. Credit classification methods are usually evaluated in terms of their accuracy, interpretability, and computational efficiency. In this paper, we propose an approach for automatic designing of fuzzy rule-based classifiers (FRBCs) from financial data using multi-objective evolutionary optimization algorithms (MOEOAs). Our method generates, in a single experiment, an optimized collection of solutions (financial FRBCs) characterized by various levels of accuracy-interpretability trade-off. In our approach we address the complexity- and semantics-related interpretability issues, we introduce original genetic operators for the classifier's rule base processing, and we implement our ideas in the context of Non-dominated Sorting Genetic Algorithm II (NSGA-II), i.e., one of the presently most advanced MOEOAs. A significant part of the paper is devoted to an extensive comparative analysis of our approach and 24 alternative methods applied to three standard financial benchmark data sets, i.e., Statlog (Australian Credit Approval), Statlog (German Credit Approval), and Credit Approval (also referred to as Japanese Credit) sets available from the UCI repository of machine learning databases (http://archive.ics.uci.edu/ml). Several performance measures including accuracy, sensitivity, specificity, and some number of interpretability measures are employed in order to evaluate the obtained systems. Our approach significantly outperforms the alternative methods in terms of the interpretability of the obtained financial data classifiers while remaining either competitive or superior in terms of their accuracy and the speed of decision making.     

Empirical taxonomy of services and service products in electronic commerce

In this study, we newly classify services and service products in EC into four categories: Mass Services, Interactive Services, Supporting Services, and Professional Services. Five selected variables reflecting the characteristics of the services, and two factorized dimensions, (1) proportion of on-line services, and (2) need of on-line interaction, are utilized for the classification. In addition, the relationships with customer purchase intentions in EC are investigated as a result of the classification. For this purpose, a customer survey was conducted on respondent groups who frequently purchase goods or services by EC and who also had advanced knowledge of the services and EC. Statistical methods of factor analysis, cluster analysis, and analysis of variance were utilized for analysis of the data.     

Unsupervised law article mining based on deep pre-trained language representation models with application to the Italian civil code

Modeling law search and retrieval as prediction problems has recently emerged as a predominant approach in law intelligence. Focusing on the law article retrieval task, we present a deep learning framework named LamBERTa, which is designed for civil-law codes, and specifically trained on the Italian civil code. To our knowledge, this is the first study proposing an advanced approach to law article prediction for the Italian legal system based on a BERT (Bidirectional Encoder Representations from Transformers) learning framework, which has recently attracted increased attention among deep learning approaches, showing outstanding effectiveness in several natural language processing and learning tasks. We define LamBERTa models by fine-tuning an Italian pre-trained BERT on the Italian civil code or its portions, for law article retrieval as a classification task. One key aspect of our LamBERTa framework is that we conceived it to address an extreme classification scenario, which is characterized by a high number of classes, the few-shot learning problem, and the lack of test query benchmarks for Italian legal prediction tasks. To solve such issues, we define different methods for the unsupervised labeling of the law articles, which can in principle be applied to any law article code system. We provide insights into the explainability and interpretability of our LamBERTa models, and we present an extensive experimental analysis over query sets of different type, for single-label as well as multi-label evaluation tasks. Empirical evidence has shown the effectiveness of LamBERTa, and also its superiority against widely used deep-learning text classifiers and a few-shot learner conceived for an attribute-aware prediction task.