首页 | 本学科首页   官方微博 | 高级检索  
相似文献
 共查询到20条相似文献,搜索用时 62 毫秒
1.
异构环境下云计算数据副本动态管理研究   总被引:4,自引:0,他引:4  
针对现有HDFS(Hadoop Distributed File System)数据副本数目固定,副本放置时未考虑节点异构性问题,本文提出一种异构环境下云计算数据副本动态管理模型DMDR.在副本创建时,DMDR根据节点性能选择最优副本放置节点;在系统运行过程中,基于灰色预测技术,DMDR根据数据的最近访问特征预测数据访问热度,并动态调整副本数目.实验表明,在异构环境下,较之现有HDFS副本管理机制,DMDR可有效减少节点之间数据传输,提高负载均衡,降低作业执行时间.  相似文献   

2.
副本管理策略对于分布式存储系统的可用性、可靠性和系统整体性能有至关重要的作用。本文针对基于文件的动态副本调整策略的不足,提出了一种基于热点数据块的动态副本调整策略。根据时间局部性原理和数据访问规律,通过对历史访问周期和当前周期赋予不同的权重,数据块下一周期的预测进行访问频率计算,接下来基于计算出的预测访问频率对数据块进行热点判定。结合HDFS中数据访问规律近似二八定律的特点和热点数据块的判定结果,来确定数据块的调整阈值。最后,分3个步骤对基于热点数据块的动态副本调整策略进行性详细设计。实验结果表明,本文提出的基于热点数据块的动态副本调整策略在数据访问效率和集群存储资源利用率两方面有了明显提升。  相似文献   

3.
在Hadoop分布式文件系统(HDFS)云存储环境下,网络带宽和节点性能有限且动态变化,现有的副本选择策略无法根据环境的变化选择最合适副本.针对这一问题,提出一种综合考虑了网络带宽、节点I/O性能以及节点存储空间等因素,基于灰色马尔可夫链预测模型的副本选择策略,以此在系统可用性和负载均衡性之间寻求一个平衡.最后通过仿真实验,验证了该策略的可行性与有效性.  相似文献   

4.
《计算机工程》2017,(8):82-89
针对Hadoop分布式文件系统(HDFS)的写性能在执行效率上的不足,提出一种在节点性能异构环境中对HDFS数据块进行并发传输的调度算法。该算法实时监控HDFS集群中每个节点的资源状态和内存缓存队列,动态地将接收节点与转发节点进行配对传输,使全系统节点的网卡和磁盘并发工作,缩短了所有副本写入分布式文件系统的时间。将数据写入磁盘后请求下一个数据块,保证数据安全性,同时也使得各个节点获得与自身性能相匹配的副本数,使性能异构的系统能达到较高的写入速度。性能测试结果表明,使用该算法的分布式文件系统的写入性能较原始的HDFS提高了1倍。  相似文献   

5.
李晓恺  代翔  李文杰  崔喆 《计算机应用》2012,32(8):2150-2158
为了让Hadoop分布式文件系统(HDFS)达到更高的存储效率以及更加优化的负载均衡能力,针对HDFS的多副本存储技术提出了改进方案——Noah。Noah引入了编码和译码模块,对HDFS中的block进行编码分解,生成更多数量的数据分片(section),并随机地分散保存到集群当中,替代原有系统的多副本容灾策略;在集群出现节点失效的情况下,通过收集与失效block相关的任意70%左右的section进行原始数据的恢复;同时根据分布式集群运行情况以及对副本数目需求的不同采用动态副本策略。通过相关的集群实验,表明Noah在容灾效率、负载均衡、存储成本以及安全性上对HDFS作了相应的优化。  相似文献   

6.
针对教育网格中数据资源共享的这个关键问题,主要在数据复制技术方面作了研究.在讨论了一种合理的副本目录管理模型后,提出了一种动态的副本创建策略,该策略提高了副本放置的效率,并能够适应用户请求的动态变化.  相似文献   

7.
随着大数据时代的到来,分布式存储技术应运而生。目前主流大数据技术Hadoop的HDFS分布式存储系统的元数据存储架构上一直存在可扩展性差和写延迟高等问题,其在官方2.0版本中针对可扩展性的解决方案(Fe-deration)仍不完美,仅解决了原有HDFS扩展性的问题,在元数据分配的问题上没有考虑NameNode的异构性能差异,也未解决NameNode集群动态负载均衡的问题。针对该情况,提出了一种动态负载均衡的分布NameNode算法,通过元数据多副本异构节点的动态适应性备份,使元数据在考虑节点性能及负载的情况下实现了动态分布,保证了元数据服务器集群的性能;同时结合缓存策略及自动恢复机制,提高了元数据的读写性及可用性。该算法在试验验证中达到了较为理想的效果。  相似文献   

8.
《微型机与应用》2015,(16):21-23
Hadoop分布式文件系统默认采用三副本策略实现较为简单,未对数据节点负载进行充分考虑。为了改善HDFS中集群负载的均衡性,提高数据节点的资源利用率,提出一种优化的副本放置策略。该策略综合考虑数据节点的实时负载信息和工作进程数,选择负载最小的节点存放数据。实验结果表明,与默认策略相比,优化的Hadoop副本放置策略能使副本分布更加合理,集群的均衡性更加良好,并能减少数据上传响应时间。  相似文献   

9.
针对传统远程文件备份系统,备份数据存储在单节点服务器存在的存储空间受限、多用户情况下的读写性能以及备份数据单副本问题,提出了一种基于HDFS的远程文件备份系统的设计方案。将用户的备份数据分布式存储于多台不同的数据存储服务器,元数据存储在单独的控制服务器。该存储架构可以有效解决单存储服务器存储空间受限的问题,改善面对多用户并发读性能问题,提供了文件多副本存储策略,并且该系统增强了备份文件存储的安全性。  相似文献   

10.
袁丽娜 《计算机科学》2017,44(Z11):397-399, 431
HDFS默认的数据副本放置策略仅仅只根据磁盘空间使用单个指标进行负载衡量,无法实现各节点真正的负载均衡。提出了一种基于性能的副本负载均衡放置改进策略,从磁盘空间负载能力、CPU处理能力、内存处理能力、磁盘读写处理能力、带宽等5个方面考究节点实际工作负载,并定义了一个负载能力模型。实验结果表明,该改进策略比默认策略能更好地实现副本的均衡放置。  相似文献   

11.
Abstract This paper describes an approach to the design of interactive multimedia materials being developed in a European Community project. The developmental process is seen as a dialogue between technologists and teachers. This dialogue is often problematic because of the differences in training, experience and culture between them. Conditions needed for fruitful dialogue are described and the generic model for learning design used in the project is explained.  相似文献   

12.
European Community policy and the market   总被引:1,自引:0,他引:1  
Abstract This paper starts with some reflections on the policy considerations and priorities which are shaping European Commission (EC) research programmes. Then it attempts to position the current projects which seek to capitalise on information and communications technologies for learning in relation to these priorities and the apparent realities of the marketplace. It concludes that while there are grounds to be optimistic about the contribution EC programmes can make to the efficiency and standard of education and training, they are still too technology driven.  相似文献   

13.
融合集成方法已经广泛应用在模式识别领域,然而一些基分类器实时性能稳定性较差,导致多分类器融合性能差,针对上述问题本文提出了一种新的基于多分类器的子融合集成分类器系统。该方法考虑在度量层融合层次之上通过对各类基多分类器进行动态选择,票数最多的类别作为融合系统中对特征向量识别的类别,构成一种新的自适应子融合集成分类器方法。实验表明,该方法比传统的分类器以及分类融合方法识别准确率明显更高,具有更好的鲁棒性。  相似文献   

14.
Development of software intensive systems (systems) in practice involves a series of self-contained phases for the lifecycle of a system. Semantic and temporal gaps, which occur among phases and among developer disciplines within and across phases, hinder the ongoing development of a system because of the interdependencies among phases and among disciplines. Such gaps are magnified among systems that are developed at different times by different development teams, which may limit reuse of artifacts of systems development and interoperability among the systems. This article discusses such gaps and a systems development process for avoiding them.  相似文献   

15.
This paper presents control charts models and the necessary simulation software for the location of economic values of the control parameters. The simulation program is written in FORTRAN, requires only 10K of main storage, and can run on most mini and micro computers. Two models are presented - one describes the process when it is operating at full capacity and the other when the process is operating under capacity. The models allow the product quality to deteriorate to a further level before an existing out-of-control state is detected, and they can also be used in situations where no prior knowledge exists of the out-of-control causes and the resulting proportion defectives.  相似文献   

16.
Going through a few examples of robot artists who are recognized worldwide, we try to analyze the deepest meaning of what is called “robot art” and the related art field definition. We also try to highlight its well-marked borders, such as kinetic sculptures, kinetic art, cyber art, and cyberpunk. A brief excursion into the importance of the context, the message, and its semiotics is also provided, case by case, together with a few hints on the history of this discipline in the light of an artistic perspective. Therefore, the aim of this article is to try to summarize the main characteristics that might classify robot art as a unique and innovative discipline, and to track down some of the principles by which a robotic artifact can or cannot be considered an art piece in terms of social, cultural, and strictly artistic interest. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008  相似文献   

17.
《计算机科学》2007,34(4):148-148
Recent years have seen rapid advances in various grid-related technologies, middleware, and applications. The GCC conference has become one of the largest scientific events worldwide in grid and cooperative computing. The 6th international conference on grid and cooperative computing (GCC2007) Sponsored by China Computer Federation (CCF),Institute of Computing Technology, Chinese Academy of Sciences (ICT) and Xinjiang University ,and in Cooperation with IEEE Computer Soceity ,is to be held from August 16 to 18, 2007 in Urumchi, Xinjiang, China.  相似文献   

18.
Although there are many arguments that logic is an appropriate tool for artificial intelligence, there has been a perceived problem with the monotonicity of classical logic. This paper elaborates on the idea that reasoning should be viewed as theory formation where logic tells us the consequences of our assumptions. The two activities of predicting what is expected to be true and explaining observations are considered in a simple theory formation framework. Properties of each activity are discussed, along with a number of proposals as to what should be predicted or accepted as reasonable explanations. An architecture is proposed to combine explanation and prediction into one coherent framework. Algorithms used to implement the system as well as examples from a running implementation are given.  相似文献   

19.
为了设计一种具有低成本、低功耗、易操作、功能强且可靠性高的煤矿井下安全分站,针对煤矿安全生产实际,文章提出了采用MCS-51系列单片机为核心、具有CAN总线通信接口的煤矿井下安全监控分站的设计方案;首先给出煤矿井下安全监控分站的整体构架设计,然后着重阐述模拟量输入信号处理系统的设计过程,最后说明单片机最小系统及其键盘、显示、报警、通信等各个组成部分的设计;为验证设计方案的可行性与有效性,使用Proteus软件对设计内容进行仿真验证,设计的煤矿井下安全监控分站具有瓦斯、温度等模拟量参数超标报警功能和电机开停、风门开闭等开关量指示功能;仿真结果表明:设计的煤矿井下安全监控分站具有一定的实际应用价值.  相似文献   

20.
This paper provides the author's personal views and perspectives on software process improvement. Starting with his first work on technology assessment in IBM over 20 years ago, Watts Humphrey describes the process improvement work he has been directly involved in. This includes the development of the early process assessment methods, the original design of the CMM, and the introduction of the Personal Software Process (PSP)SM and Team Software Process (TSP){SM}. In addition to describing the original motivation for this work, the author also reviews many of the problems he and his associates encountered and why they solved them the way they did. He also comments on the outstanding issues and likely directions for future work. Finally, this work has built on the experiences and contributions of many people. Mr. Humphrey only describes work that he was personally involved in and he names many of the key contributors. However, so many people have been involved in this work that a full list of the important participants would be impractical.  相似文献   

设为首页 | 免责声明 | 关于勤云 | 加入收藏

Copyright©北京勤云科技发展有限公司  京ICP备09084417号