基于Hadoop与RabbitMQ的人脸识别算法测试平台的设计与实现

针对人脸识别算法研究过程中测试效率低下的问题，基于分布式技术，设计并实现了通用的分布式大数据测试平台。为了提高人脸识别算法的大数据测试的执行效率，提高测试结果统计计算的执行效率，基于RabbitMQ设计分布式并行执行架构，利用Hadoop集群的MapReduce框架进行分布式并行计算。利用Java语言的Spring框架开发测试平台，将测试代码与测试图片托管于Hadoop集群的HDFS文件系统，实现了测试业务与测试平台的分离，提高了平台的通用性。该测试平台不仅实现了单个测试任务的分布式执行而且满足多个测试任务同时执行，可对测试任务以及测试相关的代码与数据进行有效的管理。与传统测试方法相比，该平台测试效率提高10余倍，测试图片的数量越大测试效率提升越明显。该测试平台具有业务通用性、容量可扩展性，对于其他人工智能算法的大量数据测试具有借鉴意义与参考价值。

Design and Implementation of Test Platform for Face Recognition Algorithm Based on Hadoop and RabbitMQ

Aiming at the problem of low test efficiency in the research of face recognition algorithm,a general distributed big data test platform is designed and implemented based on distributed technology.In order to improve the execution efficiency of big data test of face recognition algorithm,and improve the execution efficiency of statistical calculation of test results,a distributed parallel execution architecture is designed based on RabbitMQ,and the MapReduce framework of Hadoop cluster is used.The test platform is developed by using the spring framework of Java language.On this test platform,the test codes and test pictures are hosted in the HDFS file system of Hadoop cluster,which realizes the separation of test business and test platform,also improves the universality of the platform.The test platform not only realizes the distributed execution of a single test task,but also satisfies the simultaneous execution of multiple test tasks,and effectively manages the test tasks and related test codes and test data.Compared with traditional test methods,the test efficiency is improved by more than 10 times.The larger the number of test pictures,the more obvious the improvement of test efficiency.The test platform has business universality and capacity scalability.It has reference significance and reference value for big data test of other artificial intelligence algorithms.