首页 | 本学科首页   官方微博 | 高级检索  
     

基于Spark的实时视频分析系统
引用本文:郑健,冯瑞.基于Spark的实时视频分析系统[J].计算机系统应用,2017,26(12):51-57.
作者姓名:郑健  冯瑞
作者单位:复旦大学 计算机科学技术学院, 上海 201203,复旦大学 上海市智能信息处理重点实验室, 上海 201203;复旦大学 上海视频技术与系统工程研究中心, 上海 201203
基金项目:国家科技支撑计划(2013BAH09F01);临港地区智能制造产业专项(ZN2016020103)
摘    要:视频监控技术在交通管理、公共安全、智慧城市等方面有着广泛的应用前景,且向着智能识别、实时处理、大数据分析的方向发展. 本文针对大规模实时视频监控提出了新的解决方案. 基于Spark streaming流式计算、分布式存储及OLAP框架,使多路视频处理在可扩展性、容错性及数据多维聚合分析上具有明显的优势. 系统根据视频处理算法划分为单机处理与分布式处理. 并将视频图像处理与数据分析耦合,利用Kafka消息队列与Spark streaming完成对多路视频输出数据的进一步操作. 结合分布式存储方案,并利用OLAP框架实现对海量数据实时多维聚合分析与高效实时查询.

关 键 词:Spark  视频分析  数据分析  实时计算
收稿时间:2017/3/20 0:00:00

Scalable Real-Time Video Analysis System Based on Spark
ZHENG Jian and FENG Rui.Scalable Real-Time Video Analysis System Based on Spark[J].Computer Systems& Applications,2017,26(12):51-57.
Authors:ZHENG Jian and FENG Rui
Affiliation:School of Computer Science, Fudan University, Shanghai 201203, China and Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai 201203, China;Shanghai Engineering Research Center for Video Technology and System, Fudan University, Shanghai 201203, China
Abstract:The video surveillance technology has a wide application prospect in traffic management, public safety, intelligent city, and is developing towards intelligent recognition, real-time processing, and large data analysis. In this paper, we propose a new system for large-scale real-time video surveillance. The system is based on Spark streaming, distributed storage and OLAP framework so that multi-channel video processing has obvious advantages in scalability, fault tolerance and data analysis of the multi-dimensional polymer. According to video processing algorithm, the processing module is divided into single machine processing and distributed processing. The video processing is separated from the data analysis, and the further operation of the multi-channel video output data is completed by using Kafka message queue and Spark streaming. Combining the distributed storage technology with OLAP framework, the system achieves real-time multi-dimensional data analysis and high-performance real-time query.
Keywords:Spark  video analysis  data analysis  realtime computation
点击此处可从《计算机系统应用》浏览原始摘要信息
点击此处可从《计算机系统应用》下载全文
设为首页 | 免责声明 | 关于勤云 | 加入收藏

Copyright©北京勤云科技发展有限公司  京ICP备09084417号