数字视频中字幕检测及提取的研究和实现

首先进行文字事件检测，然后进行边缘检测、阈值计算和边缘尺寸限制，最后依据文字像素密度范围进一步滤去非文字区域的视频字幕．提出的叠加水平和垂直方向边缘的方法，加强了检测到的文字的边缘；对边缘进行尺寸限制过滤掉了不符合文字尺寸的边缘；进一步，提出像素密度α的概念，并指出文字区域的像素密度α应在某一阈值范围之内(αmin≤α≤αmax)．通过像素密度α滤去了非文字区域，应用投影法最终确定视频字幕所在区域．以上方法的结合保证了提出的算法的正确率和鲁棒性．选用不同类型的视频素材对文中算法进行实验，并与其他方法进行比较，得出文中算法具有较高的正确率和较快的计算速度．     

视频中的字幕提取

提出一种综合运用文字边缘特征、颜色信息以及视频时空特性的字幕提取方法。通过边缘检测获取字幕位置进而得到文字颜色,采用全局混合高斯模型对颜色建模,建模完成后直接利用模型从视频文字变化帧中提取文字颜色层。在判断字幕是否变化时,提出了“与”掩码图的方法。实验结果表明,对于复杂背景下包含1~2种颜色字幕颜色的视频,该方法具有良好的提取效果。     

基于笔画相关加权的视频图像文字识别

为了提取影视视频图像中的字幕信息,提出一套鲁棒的方法：首先采用图像的边缘特征对字幕信息进行区域定位,并给出结合边缘信息的方法对图像文字进行二值化;其次,采用投影法和区域生成方法定位单个文字;最后,充分考虑到文字笔画的拓扑结构,进行相邻子网格笔画结构相关性的判定,并采用笔画模糊隶属度完成弹性网格特征的提取。该方法在复杂的背景图像中能够有效得到文字的二值图像,并保证了提取特征的稳定性、健壮性,对二值化后的影视字幕的识别率达到92.1%,实验结果表明了方法的有效性。     

一种有效的新闻视频主题字幕检测方法

提出一种新的有效的新闻视频主题字幕检测方法。利用像素的空间量化亮度值计算局部帧差变化,以便检测新闻视频字幕的出现帧和消失帧,并建立4条规则来进一步区分主题字幕和非主题字幕,然后对同一主题字幕所在帧利用多帧结合的方法生成两个候选的主题字幕文字检测帧,选择其中之一进行文字区域的定位和提取;最后用投影法分割字符,并对投影法产生的错误分割字符的常见情况,提出相应的解决方法。实验结果表明了该方法的有效性。     

基于多帧图像的视频文字跟踪和分割算法

视频中文字的提取是视频语义理解和检索的重要信息来源．针对视频中的静止文字时间和空间上的冗余特性,以文字区域的边缘位图为特征对检测结果作精化,并提出了基于二分搜索法的快速文字跟踪算法,实现了对文字对象快速有效的定位．在分割阶段,除了采用传统的灰度融合图像进行文字区域增强方法,还结合边缘位图对文字区域进行进一步的背景过滤．实验表明,文字的检测精度和分割质量都有很大提高．     

基于角点检测和自适应阈值的新闻字幕检测

目前用于提取新闻视频帧中字幕的方法准确率和检测速度普遍较低,尤其对于分辨率和对比度较小的标题文字,检测效果很差。针对上述问题,提出一种基于角点检测和自适应阈值的字幕检测方法。该方法利用角点检测确定标题帧中的文字区域并进行灰度变换,利用自适应阈值的方法对其进行二值化,得到OCR可识别的文字图片。实验表明,该方法可以快速有效地提取出分辨率和对比度较小的新闻视频标题字幕。     

基于颜色聚类和多帧融合的视频文字识别方法

提出一种基于颜色聚类和多帧融合的视频文字识别方法,首先,在视频文字检测模块,综合考虑了文字区域的两个显著特征:一致的颜色和密集的边缘,利用近邻传播聚类算法,根据图像中边缘颜色的复杂程度,自适应地把彩色边缘分解到若干边缘子图中去,使得在各个子图中检测文字区域更为准确.其次,在视频文字增强模块,基于文字笔画强度图过滤掉模糊的文字区域,并综合平均融合和最小值融合的优点,对在不同视频帧中检测到的、包含相同内容的文字区域进行融合,能够得到背景更为平滑、笔画更为清晰的文字区域图像.最后,在视频文字提取模块,通过自适应地选取具有较高文字对比度的颜色分量进行二值化,能够取得比现有方法更好的二值化结果;另一方面,基于图像中背景与文字的颜色差异,利用颜色聚类的方法去除噪声,能够有效地提高文字识别率.实验结果表明,该方法能够比现有方法取得更好的文字识别结果.     

基于支持向量机的垃圾邮件过滤方法

目前用于提取新闻视频帧中字幕的方法准确率和检测速度普遍较低,尤其对于分辨率和对比度较小的标题文字,检测效果很差.针对上述问题,提出一种基于角点检测和自适应阈值的字幕检测方法.该方法利用角点检测确定标题帧中的文字区域并进行灰度变换,利用自适应阈值的方法对其进行二值化,得到OCR可识别的文字图片.实验表明,该方法可以快速有效地提取出分辨率和对比度较小的新闻视频标题字幕.     

基于灰度差分和二维最大熵阈值的新闻字幕检测*

针对新闻视频帧中文本区域的定位提取问题,提出了一种有效的字幕定位提取方法。通过灰度差分和变异灰度直方图对新闻视频帧字幕区域定位,再经改进的二维最大熵阈值方法对分割出的文字区域进行二值化,得到可识别的文字图片。最后对文本定位和OCR识别情况进行了算法对比。实验表明：与传统的投影法和最大熵方法相比,该方法可有效地提高文本定位的查全率和OCR的识别率。     

两阶段的视频字幕检测和提取算法

视频字幕检测和提取是视频理解的关键技术之一。文中提出一种两阶段的字幕检测和提取算法,将字幕帧和字幕区域分开检测,从而提高检测效率和准确率。第一阶段进行字幕帧检测:首先,根据帧间差算法进行运动检测,对字幕进行初步判断,得到二值化图像序列;然后,根据普通字幕和滚动字幕的动态特征对该序列进行二次筛选,得到字幕帧。第二阶段对字幕帧进行字幕区域检测和提取:首先,利用Sobel边缘检测算法初检文字区域;然后,利用高度约束等剔除背景,并根据宽高比区分出纵向字幕和横向字幕,从而得到字幕帧中的所有字幕,即静止字幕、普通字幕、滚动字幕。该方法减少了需要检测的帧数,将字幕检测效率提高了约11%。实验对比结果证明, 相比单一使用帧间差和边缘检测的方法,该方法在F值上提升约9%。     

改进的视频帧中文本区域定位算法

针对视频帧图像背景复杂、字体大小变化较大等特点,提出一种改进的视频帧中文本区域定位算法。算法设计并实现一个交叉点检测算法,利用交叉点密度等边缘特征去除大部分非文字边缘以降低背景边缘对文本区域的影响,并对剩余边缘进行膨胀以形成候选文本区域,结合文本区域特征和支持向量机算法区分文本区域和非文本区域。实验表明,该算法可以提取视频帧中90%以上的文本区域,并且文本区域定位准确率达92.0%。     

基于空白条方向拟合的复杂文本图像倾斜检测

针对扫描背景不定且含有图表信息的复杂文本图像,提出了一种有效的倾斜检测方法。该方法首先通过对梯度图像的统计分析,自适应地选取到了包含文字的特征子区;在特征子区内,论文把文字行间的空白条带看作一条隐含的线,用优化理论计算出空白条带的倾斜角度,这也就是文本的倾斜角度。实验结果表明,该倾斜检测方法不受扫描背景、边界大小、文本布局及行间距等情况的限制,具有速度快、精度高、适应性强的特点。     

Text detection in chart images

Common OCR (Optical Character Recognition) systems fail to detect and recognize small text strings of few characters, in particular when a text line is not horizontal. Such text regions are typical for chart images. In this paper we present an algorithm that is able to detect small text regions regardless of string orientation and font size or style. We propose to use this algorithm as a preprocessing step for text recognition with a common OCR engine. According to our experimental results, one can get up to 20 times better text recognition rate, and 15 times higher text recognition precision when the proposed algorithm is used to detect text location, size and orientation, before using an OCR system. Experiments have been performed on a benchmark set of 1000 chart images created with the XML/SWF Chart tool, which contain about 14000 text regions in total.     

一种轻量化中文指路标志的文本识别算法

针对中文交通指路标志中多方向、多角度的文本提取与识别困难的问题,提出了一种融合了卷积神经网络与传统机器学习方法的轻量化中文交通指路标志文本提取与识别算法。首先,对YOLOv5l目标检测网络进行轻量改进,提出了YOLOv5t网络用以提取指路标志牌中的文本区域;然后,结合投影直方图法与多项式拟合法的M-split算法,对提取到的文本区域进行字符分割;最后,使用MobileNetV3轻量化网络对文本进行识别。提出的算法在自制数据集TS-Detect上进行近景文本识别,精度达到了901%,检测速度达到了40 fps,且权重文件大小仅有24.45 MB。实验结果表明,提出的算法具有轻量化、高精度的特性,能够完成复杂拍摄条件下的实时中文指路标志文本提取与识别任务。     

AUTOMATIC TEXT LOCATION IN IMAGES AND VIDEO FRAMES

Textual data is very important in a number of applications such as image database indexing and document understanding. The goal of automatic text location without character recognition capabilities is to extract image regions that contain only text. These regions can then be either fed to an optical character recognition module or highlighted for a user. Text location is a very difficult problem because the characters in text can vary in font, size, spacing, alignment, orientation, color and texture. Further, characters are often embedded in a complex background in the image. We propose a new text location algorithm that is suitable in a number of applications, including conversion of newspaper advertisements from paper documents to their electronic versions, World Wide Web search, color image indexing and video indexing. In many of these applications, it is not necessary to extract all the text, so we emphasize on extracting important text with large size and high contrast. Our algorithm is very fast and has been shown to be successful in extracting important text in a large number of test images.     

ImPrEd: An Improved Force‐Directed Algorithm that Prevents Nodes from Crossing Edges

PrEd [ [Ber00] ] is a force‐directed algorithm that improves the existing layout of a graph while preserving its edge crossing properties. The algorithm has a number of applications including: improving the layouts of planar graph drawing algorithms, interacting with a graph layout, and drawing Euler‐like diagrams. The algorithm ensures that nodes do not cross edges during its execution. However, PrEd can be computationally expensive and overly‐restrictive in terms of node movement. In this paper, we introduce ImPrEd: an improved version of PrEd that overcomes some of its limitations and widens its range of applicability. ImPrEd also adds features such as flexible or crossable edges, allowing for greater control over the output. Flexible edges, in particular, can improve the distribution of graph elements and the angular resolution of the input graph. They can also be used to generate Euler diagrams with smooth boundaries. As flexible edges increase data set size, we experience an execution/drawing quality trade off. However, when flexible edges are not used, ImPrEdproves to be consistently faster than PrEd.     

Toward a computer vision-based wayfinding aid for blind persons to access unfamiliar indoor environments

Independent travel is a well-known challenge for blind and visually impaired persons. In this paper, we propose a proof-of-concept computer vision-based wayfinding aid for blind people to independently access unfamiliar indoor environments. In order to find different rooms (e.g. an office, a laboratory, or a bathroom) and other building amenities (e.g. an exit or an elevator), we incorporate object detection with text recognition. First, we develop a robust and efficient algorithm to detect doors, elevators, and cabinets based on their general geometric shape, by combining edges and corners. The algorithm is general enough to handle large intra-class variations of objects with different appearances among different indoor environments, as well as small inter-class differences between different objects such as doors and door-like cabinets. Next, to distinguish intra-class objects (e.g. an office door from a bathroom door), we extract and recognize text information associated with the detected objects. For text recognition, we first extract text regions from signs with multiple colors and possibly complex backgrounds, and then apply character localization and topological analysis to filter out background interference. The extracted text is recognized using off-the-shelf optical character recognition software products. The object type, orientation, location, and text information are presented to the blind traveler as speech.     

Summarization of Visual Content in Instructional Videos

In instructional videos of chalk board presentations, the visual content refers to the text and figures written on the boards. Existing methods on video summarization are not effective for this video domain because they are mainly based on low-level image features such as color and edges. In this work, we present a novel approach to summarizing the visual content in instructional videos using middle-level features. We first develop a robust algorithm to extract content text and figures from instructional videos by statistical modelling and clustering. This algorithm addresses the image noise, nonuniformity of the board regions, camera movements, occlusions, and other challenges in the instructional videos that are recorded in real classrooms. Using the extracted text and figures as the middle level features, we retrieve a set of key frames that contain most of the visual content. We further reduce content redundancy and build a mosaicked summary image by matching extracted content based on K-th Hausdorff distance and connected component decomposition. Performance evaluation on four full-length instructional videos shows that our algorithm is highly effective in summarizing instructional video content.     

一种织物彩色图像的分割算法

提出了一种针对织物彩色图像的分割算法。该算法在颜色聚类的基础上,把每种颜色用 不同的序号来表示,生成一个与原图同大小的序号矩阵,然后根据矩阵中序号的分布进行初始分割并 产生种子区域,再把初始分割未能确定的像素指派到种子区域中,最后根据颜色信息合并过分割的区 域,并对区域边缘进行平滑,得到最终的分割结果。实验表明,该算法能较有效地实现各类彩色织物 图像的分割。     

基于系数关系和分形维数的隐藏算法

提出了一种基于系数关系和分形维数的信息隐藏算法，该算法将原始载体图像分割成固定大小的图像块，利用分形维数与图像之间的关系确定每个图像块的特点，结合人类视觉系统（HVS）的特性，选择代表边缘和纹理特征的图像块，对它们进行离散余弦变换，将信息嵌入到变换系数的关系中，隐藏的信息可以是字符序列，数字序列，文本文件，二值图像，灰度图像以及彩色图像等，实验结果表明该算法透明性好，保密性高，对几何攻击，滤波，噪声和压缩等处理具有一定的稳健性。