Dynamic replica placement and selection strategies in data grids— A comprehensive survey

Data replication techniques are used in data grid to reduce makespan, storage consumption, access latency and network bandwidth. Data replication enhances data availability and thereby increases the system reliability. There are two steps involved in data replication, namely, replica placement and replica selection. Replica placement involves identifying the best possible node to duplicate data based on network latency and user request. Replica selection involves selecting the best replica location to access the data for job execution in the data grid. Various replica placement and selection algorithms are available in the literature. These algorithms measure and analyze different parameters such as bandwidth consumption, access cost, scalability, execution time, storage consumption and makespan. In this paper, various replica placement and selection strategies along with their merits and demerits are discussed. This paper also analyses the performance of various strategies with respect to the parameters mentioned above. In particular, this paper focuses on the dynamic replica placement and selection strategies in the data grid environment.     

云存储中动态副本放置机制研究

数据副本管理是云计算系统管理的重要组成部分,在云计算系统的海量数据处理过程中,针对目前已知的数据存放与资源调度算法存在考虑副本动态性和可靠性的不足,提出了一种动态的副本放置机制。该机制基于区域结构,考虑数据处理时其副本的数量和放置位置,以及副本的产生对于内存和带宽等系统资源的开销:首先根据云存储中的副本信息,对被访问频率高且访问平均响应时间长的数据信息进行复制,并给出副本数量的计算方法;考虑缩小副本分布的节点选择范围,提出动态的副本放置算法——DRA,将一定范围内的节点根据提出的域的划分,进行放置筛选,以存放数据副本。实验结果表明,提出的动态放置机制不仅减少了低访问率副本对系统存储空间的浪费;同时也减少了高访问率副本所需跨节点的传输延迟,有效提高了云存储系统中的数据文件的访问效率、负载的均衡水平,以及云存储系统的可靠性和可用性。     

云存储智能多数据副本放置机制

数据副本管理机制是云存储系统的重要组成部分。为了提高云存储系统的可伸缩性、可靠性,同时改善用户访问时间,通常采用多数据副本机制,并且需要解决数据副本放置问题。为此,提出了一种用于云存储系统的智能多数据副本放置机制。该机制基于p-中心模型,以最小化访问代价为优化目标,基于遗传算法（genetic algorithm,GA）确定优化的数据副本放置方案,基于生物地理学优化（biogeography-based optimization,BBO）算法确定用户访问请求对数据副本的优化分配。基于CloudSim进行了仿真实现和性能评价,结果表明,云存储智能多数据副本放置机制是可行和有效的。     

A method for the optimum selection of datacenters in geographically distributed clouds

The optimal selection of a datacenter is one of the most important challenges in the structure of a network for the wide distribution of resources in the environment of a geographically distributed cloud. This is due to the variety of datacenters with different quality-of-service (QoS) attributes. The user’s requests and the conditions of the service-level agreements (SLAs) should be considered in the selection of datacenters. In terms of the frequency of datacenters and the range of QoS attributes, the selection of the optimal datacenter is an NP-hard problem. A method is therefore required that can suggest the best datacenter, based on the user’s request and SLAs. Various attributes are considered in the SLA; in the current research, the focus is on the four important attributes of cost, response time, availability, and reliability. In a geo-distributed cloud environment, the nearest datacenter should be suggested after receiving the user’s request, and according to its conditions, SLA violations can be minimized. In the approach proposed here, datacenters are clustered according to these four important attributes, so that the user can access these quickly based on specific need. In addition, in this method, cost and response time are taken as negative criteria, while accessibility and reliability are taken as positive, and the multi-objective NSGA-II algorithm is used for the selection of the optimal datacenter according to these positive and negative attributes. In this paper, the proposed method, known as NSGAII_Cluster, is implemented with the Random, Greedy and MOPSO algorithms; the extent of SLA violation of each of the above-mentioned attributes are compared using four methods. The simulation results indicate that compared to the Random, Greedy and MOPSO methods, the proposed approach has fewer SLA violations in terms of the cost, response time, availability, and reliability of the selected datacenters.     

面向流媒体基于蚁群的副本选择算法

本文针对流媒体Cloud-P2P存储模式中的副本选择,提出一种基于蚁群算法的改进算法（replica selection algorithm in Cloud-P2P based on ant colony algorithm,C²P²RSA²）,建立副本选择度量标准（副本节点的网络带宽、网络延时等）与蚁群信息素的映射,定义了副本信息素概率,最后得到一组副本资源的最优解.实验表明,与PARSA算法（Pheromone-base Ant colony Replica adaptive Selection Algorithm in cloud storage）和最佳副本选择算法比较,在平均访问时间增加2%–5%的情况下,本文的算法对云副本节点的负载率减少15%–25%.     

Understanding and improvement of the selection of replica servers in key–value stores

In current large-scale distributed key–value stores, the tail latency of the hundreds of key–value access operations generated by an end-user request determines the response time of this request. Therefore, this tail latency has great impacts on the user experience and revenue. Replica selection algorithms, which select the best replica server for the service of each key–value access operation as much as possible, is the key to cut the tail latency of these key–value access operations. This paper summarizes current replica selection algorithms, including both the algorithms employed by current key–value stores and the classic algorithms of other similar systems. These algorithms are classified into three categories: information-agnostic, client-independence and feedback, according to their demanded information. As a step further, simulation-based performance analysis of these algorithms is conducted. The result brings us the insights that the response time (RPT) is useful to measure the service rate, but will lead to the herd behaviors. Moreover, the number of outstanding key–value access operations (OSKs) is helpful to both the selection of the fastest replica server and the avoidance of herd behaviors. Based on these insights, we design the L2 algorithm by assembling the basic ideas of the Least OSK algorithm and the Least RPT algorithm. The L2 algorithm is much simpler than the recently proposed C3 algorithm and has a similar best performance with C3 as confirmed by the simulation and experimental results.     

Resource pricing and offloading decisions in mobile edge computing based on the Stackelberg game

We propose a new approach for the organic integration of edge cloud offloading decision and Stackelberg game pricing to address the problem that the current Stackelberg games all allocate edge cloud computing resources equally and ignore the difference of different users’ demand for computing resources. Firstly, the Stackelberg game theory is used to establish a model of the optimal amount of data to be offloaded by users and the optimal number of computing resource blocks to be purchased, which converts the multivariate offloading decision problem of users into a univariate optimization problem, simplifies the offloading decision problem of users, and proves the existence of Nash equilibrium. Secondly, the KKT condition is applied to realize the offloading decision of users to purchase the optimal computing resource blocks. The upper and lower bounds of edge cloud pricing are established. Finally, a dynamic programming-based offloading (DPPO) algorithm for edge cloud pricing is proposed to achieve the optimal pricing of edge cloud utility and maximize each user’s own utility. The simulation results show that the proposed method not only achieves the equilibrium of edge cloud utility and user utility, but also has good convergence and scalability. The DPPO algorithm yields better results than with different pricing and offloading strategies.

     

基于上下文和角色的云计算访问控制模型

云计算环境的开放性和动态性容易引发安全问题,数据资源的安全和用户的隐私保护面临严峻考验。针对云计算中用户和数据资源动态变化的特性,提出了一种基于上下文和角色的访问控制模型。该模型综合考虑云计算环境中的上下文信息和上下文约束,将用户的访问请求和服务器中的授权策略集进行评估验证,能够动态地授予用户权限。给出用户访问资源的具体实现过程,经分析比较,进一步阐明该模型在访问控制方面具有较为突出的优点。该方案不仅能够降低管理的复杂性,而且能限制云服务提供商的特权,从而有效地保证云资源的安全。     

Novel dynamic load balancing algorithm for cloud-based big data analytics

Big data analytics in cloud environments introduces challenges such as real-time load balancing besides security, privacy, and energy efficiency. This paper proposes a novel load balancing algorithm in cloud environments that performs resource allocation and task scheduling efficiently. The proposed load balancer reduces the execution response time in big data applications performed on clouds. Scheduling, in general, is an NP-hard problem. Our proposed algorithm provides solutions to reduce the search area that leads to reduced complexity of the load balancing. We recommend two mathematical optimization models to perform dynamic resource allocation to virtual machines and task scheduling. The provided solution is based on the hill-climbing algorithm to minimize response time. We evaluate the performance of proposed algorithms in terms of response time, turnaround time, throughput metrics, and request distribution with some of the existing algorithms that show significant improvements.

     

基于属性加密的雾协同云数据共享方案

为解决现有的属性加密数据共享方案粗粒度和开销大等问题,提出一种能保证数据隐私且访问控制灵活的雾协同云数据共享方案(FAC-ABE)。设计属性加密机制,将数据的访问控制策略分为个性化和专业化两种。通过个性化的访问策略,根据用户的经验和偏好,将数据共享给相应的云端。利用雾节点对数据分类,将共享的数据分流,保障数据共享给专业的云服务器。安全分析结果表明,该方案能保障数据机密性,实现更细粒度的访问控制。实验结果表明,用户能将加密开销转移到雾节点上,降低了云端用户开销。     

云计算下基于动态用户信任度的属性访问控制

为便于对云中资源的管理,云计算环境通常会被划分成逻辑上相互独立的安全管理域,但资源一旦失去了物理边界的保护会存在安全隐患。访问控制是解决这种安全问题的关键技术之一。针对云计算环境多域的特点,提出了一种基于动态用户信任度的访问控制模型(CT-ABAC),以减少安全域的恶意推荐的影响并降低恶意用户访问的数量。在CT-ABAC模型中,访问请求由主体属性、客体属性、权限属性、环境属性和用户信任度属性组成,模型采用动态细粒度授权机制,根据用户的访问请求属性集合来拒绝或允许本次访问。同时,该模型扩展了用户信任度属性,并考虑时间、安全域间评价相似度、惩罚机制对该属性的影响。仿真实验结果表明,CT-ABAC模型能够有效地降低用户的恶意访问,提高可信用户的成功访问率。     

G-TOPSIS: a cloud service selection framework using Gaussian TOPSIS for rank reversal problem

The dynamic resource requirement of applications has forced a large number of business organizations to join the cloud market and provide cloud services. It has posed a challenge for cloud users to select the best service providers and to minimize losses occurring due to its improper selection. This paper aims to propose a robust rank reversal technique for order of preference by similarity to ideal solution (TOPSIS) method based on Gaussian distribution and used to develop a cloud service selection framework. The proposed framework ranks cloud services based on the quality of services provided by cloud service providers and cloud user’s priority. A case study is performed on a real dataset obtained from CloudHarmony to show the effectiveness and correctness of the proposed framework. The results obtained demonstrate that the proposed framework ranks cloud services similar to TOPSIS-based frameworks. A sensitivity analysis has also been performed to check its robustness in six different cases causing rank reversal and found that the proposed framework is robust to handle rank reversal phenomenon in all the scenarios in comparison with other studies available in the literature.

     

蚊子产卵交配和模拟退火的混合优化数据副本放置机制

为了提高云存储系统的可扩展性、可靠性,同时改善用户访问能力,通常为其配备多数据副本机制,则不仅需要为数据副本选择合适的存储场地,而且需要实现用户访问请求对数据副本的优化分配。为此,提出了一种基于蚊子产卵交配和模拟退火混合优化数据副本放置机制。该机制以最小化总代价为优化目标,基于蚊子产卵交配思想确定数据副本候选放置方案,基于模拟退火进一步求精得到最优解。基于CloudSim,对该机制进行了仿真实现和性能评价,并且与现有的机制进行了对比分析,结果表明,该机制具有更好的性能,是可行和有效的。     

基于微服务架构FPGA云平台的并发请求调度机制

基于微服务架构的现场可编程门阵列（FPGA）云平台在被大规模推广后积累了许多用户。针对FPGA云平台存在大量用户并发请求的问题,建立一种基于优先级调度的自定义参数响应指数计算模型。将5个请求关键影响因素作为自定义参数,采用层次分析法确定各参数权重,根据响应指数函数计算各请求的响应指数。在该模型的基础上,设计一种高并发请求调度（HCRS）算法,通过响应指数阈值对请求类别进行划分,使得高优先级请求优先得到处理,次优先级请求加入先进先出队列等待,低优先级请求暂时挂起,从而缩短请求响应时间以及请求响应延时,缓解由高并发请求带来的硬件节点资源分配压力。在真实运营的FPGA云平台中实现该算法并在实际环境中进行测试,结果表明,在并发请求量相同时,与先来先服务调度算法相比,HCRS算法的平均响应延时降低29 074 ms,平均请求响应时间缩短12 605 ms,其在提升系统吞吐量与并发度的同时可以有效优化硬件节点资源利用率。     

云存储副本优化选择策略

为了提高云计算环境中系统的整体数据调度效率,对云存储系统中的副本选择问题进行研究,提出一种基于蚁群觅食原理的云存储副本优化选择策略。该策略利用蚁群算法在解决优化问题上的优势,将自然环境中蚁群的觅食过程与云存储中的副本选择过程相结合;再充分应用信息素的动态变化规律以及高斯概率分布特性优化副本的选择方式,得出一组副本资源的最优解,从而为数据请求响应合适的副本。在OptorSim仿真平台上对该算法进行实现,实验结果表明该算法具有不错的表现,如在平均作业用时这一性能指标上相比原始蚁群算法提升了18.7%,从而在一定程度上减少了副本选择过程的时间消耗,降低了网络负载。     

视频点播中基于用户行为的自适应资源搜索机制*

根据用户访问行为特征,提出一种新的自适应数据搜索结构U-VoD,该机制根据用户的自主访问行为自适应地组成行为相似团体,不需要历史访问记录就能实时地反映用户的访问模式与数据块的流行度。同时,基于搜索结构所学习的信息,设计一种有效的数据流行度感知的预存策略。大量的实验结果表明,U-VoD在频繁的用户交互情景下,极大地降低了用户VCR操作引起的跳转时延,并减轻了对服务器的访问压力。     

A dynamic replica management strategy in data grid

Data Grid provides scalable infrastructure for storage resource and data files management, which supports several large scale applications. Due to limitation of available resources in grid, efficient use of the grid resources becomes an important challenge. Replication is a technique used in data grid to improve fault tolerance and to reduce the bandwidth consumption. This paper proposes a Dynamic Hierarchical Replication (DHR) algorithm that places replicas in appropriate sites i.e. best site that has the highest number of access for that particular replica. It also minimizes access latency by selecting the best replica when various sites hold replicas. The proposed replica selection strategy selects the best replica location for the users' running jobs by considering the replica requests that waiting in the storage and data transfer time. The simulated results with OptorSim, i.e. European Data Grid simulator show that DHR strategy gives better performance compared to the other algorithms and prevents unnecessary creation of replica which leads to efficient storage usage.     

Online optimization for scheduling preemptable tasks on IaaS cloud systems

In Infrastructure-as-a-Service (IaaS) cloud computing, computational resources are provided to remote users in the form of leases. For a cloud user, he/she can request multiple cloud services simultaneously. In this case, parallel processing in the cloud system can improve the performance. When applying parallel processing in cloud computing, it is necessary to implement a mechanism to allocate resource and schedule the execution order of tasks. Furthermore, a resource optimization mechanism with preemptable task execution can increase the utilization of clouds. In this paper, we propose two online dynamic resource allocation algorithms for the IaaS cloud system with preemptable tasks. Our algorithms adjust the resource allocation dynamically based on the updated information of the actual task executions. And the experimental results show that our algorithms can significantly improve the performance in the situation where resource contention is fierce.     

Adaptive popularity-driven replica placement in hierarchical data grids

Data grids support access to widely distributed storage for large numbers of users accessing potentially many large files. Efficient access is hindered by the high latency of the Internet. To improve access time, replication at nearby sites may be used. Replication also provides high availability, decreased bandwidth use, enhanced fault tolerance, and improved scalability. Resource availability, network latency, and user requests in a grid environment may vary with time. Any replica placement strategy must be able to adapt to such dynamic behavior. In this paper, we describe a new dynamic replica placement algorithm, Popularity Based Replica Placement (PBRP), for hierarchical data grids which is guided by file “popularity”. Our goal is to place replicas close to clients to reduce data access time while still using network and storage resources efficiently. The effectiveness of PBRP depends on the selection of a threshold value related to file popularity. We also present Adaptive-PBRP (APBRP) that determines this threshold dynamically based on data request arrival rates. We evaluate both algorithms using simulation. Results for a range of data access patterns show that our algorithms can shorten job execution time significantly and reduce bandwidth consumption compared to other dynamic replication methods.     

Caching-based task scheduling for edge computing in intelligent manufacturing

Tasks have high requirements for response delay and security in intelligent manufacturing. Industrial data have the characteristics of high privacy. However, cloud services are difficult to implement for low latency-sensitive applications and privacy data tasks. Therefore, the offloading technology in edge computing can offload the computing tasks of terminal devices to the edge of the network, which can effectively reduce the delay and match the needs of intelligent manufacturing. Unreasonable task scheduling cannot meet the needs of real-time scheduling between edge servers and cloud servers. In this paper, we establish a joint low-delay optimization model of task scheduling and dynamic replacement-release caching (DRRC) mechanism, which couples a privacy selection strategy for tasks to protect privacy. Tasks are scheduled to different location by the privacy of sensitive data, which can improve the security of data and meet the calculation request of different tasks. DRRC mechanism caches tasks according to the size of the task and replaces it with the weight of the task data, and adds automatic release mechanism. To solve the task scheduling strategy, we design the improved genetic-differential evolution algorithm. Extensive simulations reveal that the proposed algorithm has a better performance in minimizing latency compared with other scheduling algorithms. At the same time, the caching mechanism has a better hit rate.