机器学习中的隐私攻击与防御

大数据时代丰富的信息来源促进了机器学习技术的蓬勃发展,然而机器学习模型的训练集在数据采集、模型训练等各个环节中存在的隐私泄露风险,为人工智能环境下的数据管理提出了重大挑战.传统数据管理中的隐私保护方法无法满足机器学习中多个环节、多种场景下的隐私保护要求.分析并展望了机器学习技术中隐私攻击与防御的研究进展和趋势.首先介绍了机器学习中隐私泄露的场景和隐私攻击的敌手模型,并根据攻击者策略分类梳理了机器学习中隐私攻击的最新研究;介绍了当前机器学习隐私保护的主流基础技术,进一步分析了各技术在保护机器学习训练集隐私时面临的关键问题,重点分类总结了5种防御策略以及具体防御机制;最后展望了机器学习技术中隐私防御机制的未来方向和挑战.     

普适计算认证过程基于粗糙集的隐私保护策略

在普适计算环境中,用户要获得需要的服务,需要向对应的服务提供商提供一定的认证信息,而这些认证信息中往往包含有用户不希望泄漏的隐私信息。为了对这些隐私信息进行保护,本文提出了认证过程中基于粗糙集的隐私保护策略：用户将认证信息扩展成粗糙集提供给服务提供商;服务提供商根据策略从粗糙集中提取用户的真实认证信息对用户请求进行认证。该策略充分利用了粗糙集合的不确定性,能够有效地防止用户隐私泄漏。     

The <Emphasis Type="Italic">k</Emphasis>-anonymity and <Emphasis Type="Italic">l</Emphasis>-diversity approaches for privacy preservation in social networks against neighborhood attacks

Recently, more and more social network data have been published in one way or another. Preserving privacy in publishing social network data becomes an important concern. With some local knowledge about individuals in a social network, an adversary may attack the privacy of some victims easily. Unfortunately, most of the previous studies on privacy preservation data publishing can deal with relational data only, and cannot be applied to social network data. In this paper, we take an initiative toward preserving privacy in social network data. Specifically, we identify an essential type of privacy attacks: neighborhood attacks. If an adversary has some knowledge about the neighbors of a target victim and the relationship among the neighbors, the victim may be re-identified from a social network even if the victim’s identity is preserved using the conventional anonymization techniques. To protect privacy against neighborhood attacks, we extend the conventional k-anonymity and l-diversity models from relational data to social network data. We show that the problems of computing optimal k-anonymous and l-diverse social networks are NP-hard. We develop practical solutions to the problems. The empirical study indicates that the anonymized social network data by our methods can still be used to answer aggregate network queries with high accuracy.     

Philosophy, privacy, and pervasive computing

Philosophers and others concerned with the moral good of personal privacy most often see threats to privacy raised by the development of pervasive computing as primarily being threats to the loss of control over personal information. Two reasons in particular lend this approach plausibility. One reason is that the parallels between pervasive computing and ordinary networked computing, where everyday transactions over the Internet raise concerns about personal information privacy, appear stronger than their differences. Another reason is that the individual devices which can become linked in a pervasive computing environment: PDAs, GPS sensors, RFID chips/readers, publicly-located video surveillance cameras, Internet-enabled mobile phones, and the like, each raise threats to individual privacy. Without discounting the value of this approach, this paper aims to propose an alternative; and, as a result of recasting the threat to individual privacy from pervasive computing, to identify other, and deeper, moral goods that pervasive computing puts at risk that otherwise might remain concealed. In particular, I argue that pervasive computing threatens to compromise what I call existential autonomy: the right to decide for ourselves at least some of the existential conditions under which we form and develop our ways of life, including our relations to information technology. From this perspective, some moral goods at stake in protecting privacy in an environment of pervasive computing emerge that have less to do with furthering human well-being through the promotion of self-identity and subjectivity, than with stimulating curiosity, receptivity to difference, and, most broadly, openness to the world.     

SPEP： Security Protocol to Enhance Privacy in Pervasive Computing

Pervasive computing world implies privacy, particularly the bad publicity around invasions of privacy, could block the incredible potential of pervasive computing. A brief introduction of the understanding of privacy, and how they relate to,the world of pervasive computing are provided in this paper. With the pseudonyms and Bit Commitment techniques, a Security Protocol--SPEP is designed and built. SPEP provides privacy with confidentiality, two-party data authentication, and evidence of data freshness. A secure node-to-node key agreement protocol is implemented, which is practical in minimal hardware of pervasive computing environment and easily matches the data rate of the network.     

A privacy preserving technique for distance-based classification with worst case privacy guarantees

There has been relatively little work on privacy preserving techniques for distance based mining. The most widely used ones are additive perturbation methods and orthogonal transform based methods. These methods concentrate on privacy protection in the average case and provide no worst case privacy guarantee. However, the lack of privacy guarantee makes it difficult to use these techniques in practice, and causes possible privacy breach under certain attacking methods. This paper proposes a novel privacy protection method for distance based mining algorithms that gives worst case privacy guarantees and protects the data against correlation-based and transform-based attacks. This method has the following three novel aspects. First, this method uses a framework to provide theoretical bound of privacy breach in the worst case. This framework provides easy to check conditions that one can determine whether a method provides worst case guarantee. A quick examination shows that special types of noise such as Laplace noise provide worst case guarantee, while most existing methods such as adding normal or uniform noise, as well as random projection method do not provide worst case guarantee. Second, the proposed method combines the favorable features of additive perturbation and orthogonal transform methods. It uses principal component analysis to decorrelate the data and thus guards against attacks based on data correlations. It then adds Laplace noise to guard against attacks that can recover the PCA transform. Third, the proposed method improves accuracy of one of the popular distance-based classification algorithms: K-nearest neighbor classification, by taking into account the degree of distance distortion introduced by sanitization. Extensive experiments demonstrate the effectiveness of the proposed method.     

普适家庭医疗系统中的隐私保护模型

针对普适家庭医疗系统(UHMS)中用户的隐私保护问题,提出UHMS中一种基于改进的角色访问控制策略、P3P技术标准和第三方可信机构的认证隐私保护模型——uPPM。模型基本实体包括用户、隐私控制中心和家庭医疗中心,其实质是将信息隐藏技术、隐私偏好策略和代理技术有机结合以适用于普适计算的特性——上下文感知。基于BAN逻辑分析论证了该模型的正确性和安全性。     

本地化差分隐私研究综述

大数据时代信息技术不断发展,个人信息的隐私问题越来越受到关注,如何在数据发布和分析的同时保证其中的个人敏感信息不被泄露是当前面临的重大挑战.中心化差分隐私保护技术建立在可信第三方数据收集者的假设基础上,然而该假设在现实中不一定成立.基于此提出的本地化差分隐私作为一种新的隐私保护模型,具有强隐私保护性,不仅可以抵御具有任意背景知识的攻击者,而且能够防止来自不可信第三方的隐私攻击,对敏感信息提供了更全面的保护.介绍了本地化差分隐私的原理与特性,总结和归纳了该技术的当前研究工作,重点阐述了该技术的研究热点：本地化差分隐私下的频数统计、均值统计以及满足本地化差分隐私的扰动机制设计.在对已有技术深入对比分析的基础上,指出了本地化差分隐私保护技术的未来研究挑战.     

System insecurity in the Internet age

The general-purpose computing environment that characterizes the PC and Internet was not designed for privacy or integrity. Surveying a variety of Internet targets, the author discusses likely attackers and their techniques. He offers defense mechanisms for protecting system integrity and blocking such attacks     

普适计算的可信研究*

随着应用规模的不断扩展以及具有无处不在性和便捷性的普适计算模式的特点,对普适计算环境下的可信保障提出了更高的要求。普适计算的可信保证是无时无处不在而又不可见的计算方式得以实施的重要保障,是普适计算领域的研究热点。从访问控制、隐私保护和容错三个方面阐述了普适计算的可信技术,并指出了该领域的未来研究发展趋势。     

联邦学习安全防御与隐私保护技术研究

联邦学习(federated learning, FL)在多个参与方不直接进行数据传输的前提下共同完成模型训练,充分发挥各方数据价值;然而,由于联邦学习的固有缺陷以及存储和通信的安全问题,其在实际应用场景中仍面临多种安全与隐私威胁。首先阐述了FL面临的安全攻击和隐私攻击;然后针对这两类典型攻击分别总结了最新的安全防御机制和隐私保护手段,包括投毒攻击防御、后门攻击防御、搭便车攻击防御、女巫攻击防御以及基于安全计算与差分隐私的防御手段。通过对联邦学习的现有风险和相应防御手段的系统梳理,展望了联邦学习未来的研究挑战与发展方向。     

A reliable recommendation and privacy-preserving based cross-layer reputation mechanism for mobile cloud computing

Mobile cloud computing (MCC) is gaining popularity due to anywhere anytime data access. However, at the same time it also introduces the new privacy and security threats that have become an obstacle to the widespread use and popularity of MCC. In this paper, we propose a reliable recommendation and privacy preserving based cross-layer reputation mechanism (RP-CRM) to provide secure and privacy-aware communication process in wireless mesh networks (WMNs) based MCC (WM-MCC). RP-CRM integrates the cross-layer design with recommendation reputation reliability evaluation mechanism and the privacy preserving scheme to identify and manage the internal malicious nodes and protect the security and privacy against internal multi-layer attack, bad mouthing attack and information disclosure attack. Simulation results and performance analysis demonstrate that RP-CRM can provide rapid and accurate malicious node identification and management, and provide security and privacy protection against aforementioned attacks more effectively and efficiently.     

普适环境下基于生物加密的认证机制

普适计算的出现对网络通信中的安全和隐私提出了新的挑战,传统的认证技术已经不能满足普适环境的安全需求。提出了一种普适环境中用于完成服务使用者与提供者之间双向认证及密钥建立的机制。该机制高度融合了生物加密技术和Diffie-Hellman密钥交换技术,在不泄露用户隐私的情况完成双向认证。该机制提供了安全的建立密钥的算法,并且通过使用生物加密技术实现了访问控制策略的区别对待。经分析证明,该协议能很好地抵抗各种攻击,尤其是拒绝服务（DoS）攻击。     

抗电路板级物理攻击的操作系统防御技术研究

计算设备处理和存储日益增多的敏感信息,如口令和指纹信息等,对安全性提出更高要求.物理攻击技术的发展催生了一种通过攻击电路板级硬件组件来获取操作系统机密信息的攻击方法：电路板级物理攻击.该类攻击具有工具简单、成本低、易流程化等特点,极容易被攻击者利用形成黑色产业,是操作系统面临的新安全威胁和挑战.在处理器上扩展内存加密引擎可抵抗该类攻击,但是目前大部分计算设备并未配备该硬件安全机制.学术界和产业界提出软件方式抗电路板级物理攻击的操作系统防御技术,该类技术已成为近年来的研究热点.深入分析了该类技术的研究进展,总结其技术优势和不足,并探讨其发展趋势.首先,介绍了电路板级物理攻击的定义、威胁模型、现实攻击实例.之后,介绍软件方式抗电路板级物理攻击的操作系统防御技术所依赖的一些基础技术.然后,对该类防御技术的研究进展按照保护范围进行分类总结和归纳.最后,分析了该类防御技术的优势与不足,给出工程实现建议,并探讨该类防御技术未来的研究趋势.     

Guest Editors' Introduction: Security & Privacy

Security and privacy concerns touch on all aspects of pervasive computing, including hardware, operating systems, networks, databases, user interfaces, and applications. The seven articles selected for this special issue draw on ideas from many of these fields and provide a flavor of the kinds of security and privacy challenges and opportunities in pervasive computing. This article is part of a special issue on security and privacy.     

Privacy-Aware Autonomous Agents for Pervasive Healthcare

This article is part of a special issue on Intelligent Agents in Healthcare. Hospitals are natural candidates for pervasive computing technology, but they have significant privacy requirements. Autonomous agents can help developers design privacy-aware systems for pervasive healthcare environments. An extension of the SALSA agent framework incorporates customizable privacy mechanisms to adapt applications to a certain quality-of-privacy level, thereby harnessing the properties of autonomous agents. An implementation of a context-aware pervasive hospital application shows how this framework can help developers manage user privacy.     

A differential privacy framework for matrix factorization recommender systems

Recommender systems rely on personal information about user behavior for the recommendation generation purposes. Thus, they inherently have the potential to hamper user privacy and disclose sensitive information. Several works studied how neighborhood-based recommendation methods can incorporate user privacy protection. However, privacy preserving latent factor models, in particular, those represented by matrix factorization techniques, the state-of-the-art in recommender systems, have received little attention. In this paper, we address the problem of privacy preserving matrix factorization by utilizing differential privacy, a rigorous and provable approach to privacy in statistical databases. We propose a generic framework and evaluate several ways, in which differential privacy can be applied to matrix factorization. By doing so, we specifically address the privacy-accuracy trade-off offered by each of the algorithms. We show that, of all the algorithms considered, input perturbation results in the best recommendation accuracy, while guaranteeing a solid level of privacy protection against attacks that aim to gain knowledge about either specific user ratings or even the existence of these ratings. Our analysis additionally highlights the system aspects that should be addressed when applying differential privacy in practice, and when considering potential privacy preserving solutions.     

Embedded security in a pervasive world

Embedded systems have become an integral part of our everyday life. Devices like vehicles, household appliances, and cell phones are already equipped with embedded microcontrollers. The networking of the myriads of embedded devices gives rise to the brave new world of pervasive computing. Pervasive computing offers enormous advantages and opportunities for users and businesses through new applications, increased comfort, and cost reduction. One often overlooked aspect of pervasive computing, however, are new security threats.This article describes security issues in current and future pervasive security scenarios, ranging from privacy threats and unreliable products to loss of revenue. We also highlight the opportunities, such as new business models, which are enabled through strong embedded security solutions. Current research issues are also summarized. As case studies, we introduce security aspects in future automotive systems and in ad-hoc networks.     

Privacy protection in pervasive systems: State of the art and technical challenges

Pervasive and mobile computing applications are dramatically increasing the amount of personal data released to service providers as well as to third parties. Data includes geographical and indoor positions of individuals, their movement patterns as well as sensor-acquired data that may reveal individuals’ physical conditions, habits, and, in general, information that may lead to undesired consequences like unsolicited advertisement or more serious ones like discrimination and stalking.In this survey paper, at first we consider representative classes of pervasive applications, and identify the requirements they impose in terms of privacy and trade-off with service quality. Then, we review the most prominent privacy preservation approaches, we discuss and summarize them in terms of the requirements.Finally, we take a more holistic view of the privacy problem by discussing other aspects that turn out to be crucial for the widespread adoption of privacy enhancing technologies. We discuss technical challenges like the need for tools augmenting the awareness of individuals and to capture their privacy preferences, as well as legal and economic challenges. Indeed, on one side privacy solutions must comply to ethical and legal requirements, and not prevent profitable business models, while on the other side it is unlikely that privacy preserving solutions will become practical and effective without new regulations.     

云计算环境安全综述

伴随云计算技术的飞速发展,其所面临的安全问题日益凸显,在工业界和学术界引起了广泛的关注.传统的云基础架构中存在较高安全风险,攻击者对虚拟机的非法入侵破坏了云服务或资源的可用性,不可信的云存储环境增大了用户共享、检索私有数据的难度,各类外包计算和云应用需求带来了隐私泄露的风险.该文从云计算环境下安全与隐私保护技术的角度出发,通过介绍云虚拟化安全、云数据安全以及云应用安全的相关研究进展,分析并对比典型方案的特点、适用范围及其在安全防御和隐私保护方面的不同效用,讨论已有工作的局限性,进而指出未来发展趋势和后续研究方向.