Robust evaluation method of communication network based on the combination of complex network and big data

Neural Computing and Applications - The essence of big data may be the science of complex network, which should be one of the basic theories of big data, and the direct object of mobile...     

面向制造大数据的安全存储交换技术

为解决制造大数据的安全存储和交换问题,提出一种面向制造大数据的存储交换模式,重点分析了制造大数据在存储和交换时的安全风险,提出一种面向制造大数据的安全存储交换模式。使用可搜索加密技术、基于属性的访问控制技术和进程度量方法等关键技术解决数据存储和交换时的泄露风险,并总结分析了这些关键技术的现状和不足,展望未来发展。     

Design for a manufacturing data base for management use

The current state of the art in the field of data base management systems (DBMS) ranges from simple microprocessor-based information lists to complex international concerns costing millions of dollars. This paper describes an example of the use of this technology to create a manufacturing data base that contains information on eight plants located throughout the United States which manufacture plastic molded products. This work was performed on an IBM 3033 mainframe using a commercial data base software package. A presentation of the system design is given in the form of flowcharts and examples of system output reports are shown. Applications which are discussed are bill of materials processing, standards maintenance, performance measurement and pricing and job costing applications. Consideration is given to the various costs involved in the creation of the data base as compared to the benefits received by management. Finally, areas for planned expansion and improvement are given, including a proposed interface with an existing mini-computer based plant information system.     

Correction to: Quality 4.0: a review of big data challenges in manufacturing

Journal of Intelligent Manufacturing - A correction to this paper has been published: https://doi.org/10.1007/s10845-021-01786-z     

面向大数据任务的调度方法

针对在大数据的处理过程中,对大数据任务的划分和资源分配缺乏合理性的问题,提出一种面向大数据任务的调度方法。该方法首先引入了调度理论用于处理大数据任务,帮助建立合理的大数据任务管理体系并规范大数据任务处理流程;然后,基于大数据任务的本质对数据集进行分析处理,引入决策表进行属性约简,以减小大数据分析任务的数据量和提高大数据分析效率;最后,采用模糊综合评价方法,将模糊综合评价的结果作为对任务调度的依据,以提高任务资源分配合理性。在UCI（University of California Irvine）数据集上进行测试,实验结果表明,该调度算法在平均预测准确度上比朴素贝叶斯（NB）算法高7.42个百分点,比误差反向传播（BP）算法高5.16个百分点,比均方根传递（RMSProp）算法高3.74个百分点。而对于特征数较多的数据集,所提算法在预测精度上较其他算法有显著提高。所提算法在平均调度长度比（SLR）上较HCPFS（Heterogeneous Critcal Path First Synthesis）算法和HIPLTS（Heterogeneous Improved Priority List for Task Scheduling）算法分别下降了12.14%和4.56%,在平均加速比上分别提升了7.14%和42.56%,表明该算法能有效提高大数据系统中任务调度的效率。综合比较分析,所提方法具有较高的预测精度,且高效可靠。     

面向大数据任务的调度方法

针对在大数据的处理过程中，对大数据任务的划分和资源分配缺乏合理性的问题，提出一种面向大数据任务的调度方法。该方法首先引入了调度理论用于处理大数据任务，帮助建立合理的大数据任务管理体系并规范大数据任务处理流程；然后，基于大数据任务的本质对数据集进行分析处理，引入决策表进行属性约简，以减小大数据分析任务的数据量和提高大数据分析效率；最后，采用模糊综合评价方法，将模糊综合评价的结果作为对任务调度的依据，以提高任务资源分配合理性。在UCI（University of California Irvine）数据集上进行测试，实验结果表明，该调度算法在平均预测准确度上比朴素贝叶斯（NB）算法高7.42个百分点，比误差反向传播（BP）算法高5.16个百分点，比均方根传递（RMSProp）算法高3.74个百分点。而对于特征数较多的数据集，所提算法在预测精度上较其他算法有显著提高。所提算法在平均调度长度比（SLR）上较HCPFS（Heterogeneous Critcal Path First Synthesis）算法和HIPLTS（Heterogeneous Improved Priority List for Task Scheduling）算法分别下降了12.14%和4.56%，在平均加速比上分别提升了7.14%和42.56%，表明该算法能有效提高大数据系统中任务调度的效率。综合比较分析，所提方法具有较高的预测精度，且高效可靠。     

制造大数据相关技术架构分析

针对大数据技术给制造业带来的机遇和挑战,通过分析制造大数据的研究现状和产生,给出了制造大数据的定义。依据制造大数据的处理流程构建其技术架构,并介绍了相应的关键技术。最后列举了几种典型的应用场景,指出了制造大数据面临的挑战并展望下一步发展方向。     

大数据融合模型的智能化网络安全检测方法

针对传统智能化网络安全检测平台处理数据效率低、误差大等问题,文章提出一种新型的解决方案;该方案基于大数据融合模型构建新型的智能化网络安全检测平台,采用卡尔曼滤波算法、采用数据融合分类算法和模糊推理算法3种方法结合构建出数据融合模型来对网络安全检测数据进行运算与处理;其中,采用卡尔曼滤波算法进行改进,对原始网络安全检测数据进行滤波降低噪声干扰,提高数据的精准度;通过SAE稀疏自动编码器自主提取网络安全检测数据的特征信息,之后K-means聚类算法对SAE稀疏自动编码器输出的数据进行处理,通过模糊推理算法调整权值;试验表明,文章所提方案克服了现有技术存在的不足,显著提高了处理数据效率和精准度,在数据量为2 TB的环境下,本研究方法的误差低至6.9％.     

Reliability evaluation for a manufacturing network with multiple production lines

This paper focuses on performance evaluation of a manufacturing system with multiple production lines based on the network-analysis perspective. Due to failure, partial failure, or maintenance, the capacity of each machine is stochastic (i.e., multi-state). Hence, the manufacturing system can be constructed as a stochastic-flow network, named manufacturing network herein. This paper intends to measure the probability that the manufacturing network can satisfy customers’ orders. Such a probability is referred to as the system reliability. A graphical representation is first proposed to transform a manufacturing system into a manufacturing network. Thereafter, we decompose the manufacturing network into general processing paths and reworking paths. Three algorithms are subsequently developed for different scenarios and multiple production lines to generate the minimal capacity vectors that machines should provide to satisfy demand. The system reliability can be derived in terms of such capacity vectors afterwards.     

Reliability evaluation for a manufacturing network with multiple production lines

This paper focuses on performance evaluation of a manufacturing system with multiple production lines based on the network-analysis perspective. Due to failure, partial failure, or maintenance, the capacity of each machine is stochastic (i.e., multi-state). Hence, the manufacturing system can be constructed as a stochastic-flow network, named manufacturing network herein. This paper intends to measure the probability that the manufacturing network can satisfy customers’ orders. Such a probability is referred to as the system reliability. A graphical representation is first proposed to transform a manufacturing system into a manufacturing network. Thereafter, we decompose the manufacturing network into general processing paths and reworking paths. Three algorithms are subsequently developed for different scenarios and multiple production lines to generate the minimal capacity vectors that machines should provide to satisfy demand. The system reliability can be derived in terms of such capacity vectors afterwards.     

Atrak: a MapReduce-based data warehouse for big data

As warehouse data volumes expand, single-node solutions can no longer analyze the immense volume of data. Therefore, it is necessary to use shared nothing architectures such as MapReduce. Inter-node data segmentation in MapReduce creates node connectivity issues, network congestion, improper use of node memory capacity and inefficient processing power. In addition, it is not possible to change dimensions and measures without changing previously stored data and big dimension management. In this paper, a method called Atrak is proposed, which uses a unified data format to make Mapper nodes independent to solve the data management problem mentioned earlier. The proposed method can be applied to star schema data warehouse models with distributive measures. Atrak increases query execution speed by employing node independence and the proper use of MapReduce. The proposed method was compared to established methods such as Hive, Spark-SQL, HadoopDB and Flink. Simulation results confirm improved query execution speed of the proposed method. Using data unification in MapReduce can be used in other fields, such as data mining and graph processing.     

E-government multimedia big data platform for disaster management

Multimedia Tools and Applications - By efficiently managing and utilizing the city’s multimedia big data, 3D analysis and visualization of the city’s multimedia information can be...     

Framework for implementing big data analytics in Indian manufacturing: ISM-MICMAC and Fuzzy-AHP approach

Information Technology and Management - Manufacturing firms generate a massive amount of data points because of higher than ever connected devices and sensor technology adoption. These data points...     

支持增量式更新的大数据特征学习模型

大数据具有高速变化特性,其内容与分布特征均处于动态变化之中,目前的前馈神经网络模型是一种静态学习模型,不支持增量式更新,难以实时学习动态变化的大数据特征。针对这个问题,提出一种支持增量式更新的大数据特征学习模型。通过设计一个优化目标函数对参数进行快速增量式更新,为了在更新过程中保持网络的原始知识,最小化平方误差函数。对于特征变化频繁的数据,通过增加隐藏层神经元数目网络对结构进行更新,使得更新后的网络能够实时学习动态变化大数据的特征。在对网络参数与结构更新之后,通过权重矩阵SVD分解对更新后的网络结构进行优化,删除冗余的网络连接,增强网络模型的泛化能力。实验结果表明提出的模型能够在尽可能保持网络模型原始知识的基础上,通过不断更新神经网络的参数与结构实时学习动态大数据的特征。     

Secure sharing of big digital twin data for smart manufacturing based on blockchain

With the rapid development of digital twin technology, a large amount of digital twin data named as big digital twin data (BDTD), is generated in the lifecycle of equipment, which is supposed to be used in digital twin enabled applications. However, in the implementation of these applications, data sharing problem which is caused by the lack of data security as well as trust among stakeholders of equipment, limits data using value. It is a novel way to introduce blockchain technology into digital twin to solve the problem. However, current methods cannot fulfill the requirements of exponential growth and timely sharing of BDTD. Therefore, a blockchain-based framework for secure sharing of BDTD is proposed to solve the problems. Cloud storage is integrated into the framework, with which, BDTD is encrypted and stored in Cloud, while the hash of BDTD and transaction records are stored in blockchain. Some rules of generating new block are designed to improve the processing speed of blockchain. An algorithm for optimal sampling rate selection is presented to maximize total social benefits of the participants of BDTD sharing. Simulation results show that the algorithm is better than traditional method for maximizing the total social benefits. Furthermore, a protype system is developed and evaluated based on Fabric test network. Evaluation results show that BDTD can be shared securely multiple times per second through the framework, which demonstrates the feasibility of the framework in supporting timely sharing of BDTD.     

基于客观权重确定的数据中心网络性能评估方法

针对大规模数据中心网络如何有效监控网络、发现网络性能瓶颈和潜在故障点,为网络性能优化提供支持成为新的研究课题.然而影响网络性能的因素众多,性能因素的影响程度存在差异,如何给出一个准确的性能评估一直是比较困难的问题.针对上述问题,提出了网络性能评估指标体系,在此基础上进一步提出了一种基于客观权重确定的数据中心网络性能评估方法(PE-OWD).该方法通过采用基于客观权值确定方法动态计算性能权值,利用基于历史参数分布的数据归一化方法,建立了完善的网络性能健康度评估模型.针对天河2真实的网络环境,对网络设备的性能指标进行评估,验证了网络性能评估方法的有效性.     

P-DOT: a model of computation for big data

In response to the high demand of big data analytics, several programming models on large and distributed cluster systems have been proposed and implemented, such as MapReduce, Dryad and Pregel. However, compared with high performance computing areas, the basis and principles of computation and communication behaviour of big data analytics is not well studied. In this paper, we review the current big data computational model DOT and DOT Advanced, and propose a more general and practical model p-DOT (p-phases DOT). p-DOT is not a simple extension, but with profound significance: for general aspects, any big data analytics job execution expressed in DOT model or bulk synchronous parallel model can be represented by it; for practical aspects, it considers I/O behaviour to evaluate performance overhead. Moreover, we provide a cost function of p-DOT implying that the optimal number of machines is near-linear to the square root of input size for a fixed algorithm and workload, and certify that the processing paradigm of p-DOT is scalable and fault-tolerant. Finally, we demonstrate the effectiveness of the model through several experiments.     

零售商品关联大数据稀疏网络的快速聚类算法

关联规则方法被广泛应用于分析零售企业交易数据,以此指导品类管理、门店布局陈列和商品促销等运营决策,但面对电子商务网站非常巨大的数据量,仍存在效率低下的问题.对此,提出商品关联大数据稀疏网络快速聚类算法.首先,利用单步链表结构存储零售商品的共同购买关系矩阵;其次,对商品关联大数据稀疏网络的低度商品节点进行剪枝,降低搜索空间;再次,利用模糊k均值聚类对商品关联大数据稀疏网络进行快速聚类,并利用高连接度值商品节点被低连接度值商品节点分割的思想对剩余节点聚类;最后,将所提算法应用到亚马逊网站商品交易数据分析中,取得了良好的效果.