基于ANN的改进Spark系统在空管大数据处理中的应用

针对Spark系统参数量巨大且手动调整参数具有耗时、效率低下等问题，提出一种基于人工神经网络（Artificial Neural Network, ANN）的方法来对Spark系统的配置参数进行自动调整，保障在处理空管大数据时的速度和性能。使用Dell PowerEdge T430服务器测试了空管大数据中5种常用的不同大小的数据集，以验证该方法。研究表明，与默认参数配置相比，该方法可将Spark系统的性能平均提高约35％。随着数据集大小的增加，性能呈现进一步提高的趋势。该方法可以有效地保障Spark系统的参数调整效率，达到高效处理空管大数据的目的。

Application of Improved Spark System Based on ANN in Big Data Processing ofAir Traffic Management#br#

A new method based on artificial neural network (ANN) was proposed to automatically adjust the configuration parameters of the Spark system to improve the performance of the Spark system in processing ATM big data and solving time-consuming and inefficient issues. The Dell PowerEdge T430 server was used to test 5 commonly used datasets of different sizes in air traffic control big data to verify the method. Research shows that compared with the default parameter configuration, this method can improve the performance of the Spark system by about 35% on average. As the size of the dataset increases, the performance shows a trend of further improvement. This method can effectively guarantee the parameter adjustment efficiency of the Spark system and achieve the purpose of efficiently processing the ATM big data.