|
|||||
|
|
| 基于Spark的并行信任进化算法 | |
| 引用本文： | 黄冬平,周夏冰,刘冠峰. 基于Spark的并行信任进化算法[J]. 计算机应用与软件, 2021, 38(3): 243-248. DOI: 10.3969/j.issn.1000-386x.2021.03.037 |
| 作者姓名： | 黄冬平 周夏冰 刘冠峰 |
| 作者单位： | 苏州大学计算机科学与技术学院 江苏 苏州 215006;上交所技术有限责任公司 上海 200120;苏州大学计算机科学与技术学院 江苏 苏州 215006;苏州大学计算机科学与技术学院 江苏 苏州 215006 |
| 基金项目： | 国家自然科学基金青年基金项目;江苏高校优势学科建设工程项目 |
| 摘 要： | 为了提高信任进化(MET)算法在大数据环境下的时效性,提出一种基于Spark的并行MET算法——SparkMET.采用主从式架构,由主节点进行变异、交叉、选择过程,从节点计算适应度值.针对数据倾斜问题,提出一种新的数据分区策略——LBP算法,在SparkMET计算适应度值之前,根据合适的分区标签将数据重分区.在Epin... |
| 关 键 词： | MET Spark 并行计算 数据倾斜 |
PARALLEL TRUST EVOLUTION ALGORITHM BASED ON SPARK |
|
| Huang Dongping,Zhou Xiabing,Liu Guanfeng. PARALLEL TRUST EVOLUTION ALGORITHM BASED ON SPARK[J]. Computer Applications and Software, 2021, 38(3): 243-248. DOI: 10.3969/j.issn.1000-386x.2021.03.037 | |
| Authors: | Huang Dongping Zhou Xiabing Liu Guanfeng |
| Affiliation: | (School of Computer Science and Technology,Soochow University,Suzhou 215006,Jiangsu,China;Shanghai Stock Exchange Technology Limited Liability Company,Shanghai 200120,China) |
| Abstract: | In order to improve the timeliness of trust evolution(MET)algorithm in big data environment,a parallel MET algorithm based on Spark-SparkMET is proposed.It adopted a master-slave architecture,in which the master node performed mutation,intersection,and selection processes,and the slave nodes calculated fitness values.Aiming at the data skew problem,a new data partitioning strategy-LBP algorithm was proposed.It repartitioned the data according to the appropriate partition label before SparkMET calculates the fitness value.Epinions dataset was used to conduct experiments.The experimental results show that the SparkMET algorithm can effectively improve the computational efficiency of the MET algorithm. |
| Keywords: | MET Spark Parallel computing Data skew |
| 本文献已被 维普 万方数据 等数据库收录！ | |