一种海量数据生成方法

目前还没有得到广泛认可的DBMS数据生成框架.本文发展已有的数据生成方法,建立一种海量数据生成框架.提供了自定义的函数和表达式,在数据序列的基础上进行迭代操作,并在数据序列迭代和RDBMS间建立联系,加入数据非一致性程度控制机制,分析了多个迭代节点简单引用和复杂引用的情况,建立起迭代模型,给出了多个迭代节点有多个引用的解决方法,尽管有一个附加的负载,但可以避免缓冲.提出把迭代可转换为SQL的数据生成语言,可灵活生成不同的数据模式以及多粒度非一致性数据.实验参照测试基准数据模式,结果表明方法是有效的.

Method for Generating Massive Data

At present,there is not exist a flexible data generation framework which is generally accepted.This paper extended existing data generation methods to create a framework for generating massive data.The work presented user-defined functions and expressions to execute iteration operations based on data sequences,established connection between data sequences and RDBMS.A mechanism was added to control the inconsistency degree of data.The work analysed the cases that several iteration nodes had simple references and complex references for establishing an iteration mode,which resolved the problem that several iteration nodes had multi-references.The mode produced an additional overload,but it avoided buffering.A data generation language was presented to transfer iterations to SQL for generating different data schemas and multiple-grain inconsistent data.The experiments refer to data schemas of benchmarks,the results show that the approach is efficient.