Analysis‐preserving video microscopy compression via correlation and mathematical morphology

The large amount video data produced by multi‐channel, high‐resolution microscopy system drives the need for a new high‐performance domain‐specific video compression technique. We describe a novel compression method for video microscopy data. The method is based on Pearson's correlation and mathematical morphology. The method makes use of the point‐spread function (PSF) in the microscopy video acquisition phase. We compare our method to other lossless compression methods and to lossy JPEG, JPEG2000, and H.264 compression for various kinds of video microscopy data including fluorescence video and brightfield video. We find that for certain data sets, the new method compresses much better than lossless compression with no impact on analysis results. It achieved a best compressed size of 0.77% of the original size, 25× smaller than the best lossless technique (which yields 20% for the same video). The compressed size scales with the video's scientific data content. Further testing showed that existing lossy algorithms greatly impacted data analysis at similar compression sizes. Microsc. Res. Tech. 78:1055–1061, 2015. © 2015 Wiley Periodicals, Inc.     

航空视频图像的实时压缩算法

提出一种航空视频图像快速编码方案,在工控计算机上实现两路PAL制式彩色视频的同时实时压缩存储。方案选择离散余弦变换（DCT）来对图像去相关性,在比较各种DCT方法后,选择AAN算法进行优化改进,并给出优化的依据。方案同时优化了可变长编码(VLC)中VLC表的检索方式。通过对视频压缩算法中运算量最大的这两个算法模块的优化改进,在P4 3.0G处理器上实现8ms内压缩一帧768x576彩色图像。算法在32:1的压缩比下,PSNR值为30.18,接近小波压缩算法在该压缩比下的PSNR值,同时保证压缩质量。实验结果表明编码方案在压缩速度、压缩率与压缩质量三者间取得平衡,能很好的满足航空视频图像实时压缩的应用。     

稳定运动物体视频的特征方法

将背景特征块配对和直方图聚类运动矢量滤波相结合,提出了一种稳定运动物体视频的方法。用得到的全局运动参量补偿摄像机的帧间运动,运用帧差法分割出前景块和背景块,把参考帧背景上的特征块与当前帧上的特征块配对估计下一轮的帧间运动。用一块对多块的匹配策略,把参考块和以参考块为中心的搜索窗内的当前帧上的最佳匹配块配对,建立稀疏运动矢量场。然后,运用直方图聚类方法滤除矢量场中未分离出的前景块矢量和误匹配矢量。最后,用多个包含运动物体的实际视频序列对提出方法进行测试,并与其它先进稳像算法和技术仿真比较。结果表明:提出方法的全局平均帧间变换保真度可达31.05dB,接近或优于上述先进算法和技术。提出的方法对包含运动物体的视频具有较强的鲁棒性,可以去除帧间高频抖动并有效改善视频质量。     

Image enhancement and post-processing for low resolution compressed video

This research paper recommends the point spread function(PSF)forecasting technique based on the projection onto convex set(POCS)and regularization to acquire low resolution images.As the environment for the production of user created contents(UCC)videos(one of the contents on the Internet)becomes widespread,resolution reduction and image distortion occurs,failing to satisfy users who desire high quality images.Accordingly,this research neutralizes the coding artifact through POCS and regularization processes by:1)factoring the local characteristics of the image when it comes to the noise that results during the discrete cosine transform(DCT)and quantization process;and 2)removing the blocking and ring phenomena which are problems with the existing video compression.Moreover,this research forecasts the point spread function to obtain low resolution images using the above-mentioned methods.Thus,a method is suggested for minimizing the errors found among the forecasting interpolation pixels.Low-resolution image quality obtained through the experiment demonstrates that significant enhancement was made on the visual level compared to the original image.     

结合人眼对比度敏感视觉特性的视频质量客观评价

结合人眼对亮度、色度、对比度以及运动目标的感知特性,提出了一种基于人眼对视频内容感知的视频质量客观评价方法。该方法将视频分为空域和时域信息分别描述,并利用人眼感知特性,从视频的亮度、色度、对比度以及目标运动4个方面提取特征,计算其强度。然后以人眼对比度敏感值作为强度的权重因子求和,构建人眼感知视频内容模型。最后,分别以此模型模拟人眼感知源视频和失真后的视频,计算每对应单元的所有像素之间和运动矢量之间的强度差;以强度差作为视频质量评价的分数,构建视频质量客观评价模型。采用LIVE数据库中的6个源视频和48个测试视频进行了质量评价实验,并与视频质量专家组(VQEG)推荐的5个较好的视频质量客观评价模型进行了对比分析。结果表明:提出模型的视频质量评价结果与主观评价结果之间的线性相关性系数达到0.8705,显示了较好的一致性,评价效果优于5个典型的模型。     

An overview of state‐of‐the‐art image restoration in electron microscopy

In Life Science research, electron microscopy (EM) is an essential tool for morphological analysis at the subcellular level as it allows for visualization at nanometer resolution. However, electron micrographs contain image degradations such as noise and blur caused by electromagnetic interference, electron counting errors, magnetic lens imperfections, electron diffraction, etc. These imperfections in raw image quality are inevitable and hamper subsequent image analysis and visualization. In an effort to mitigate these artefacts, many electron microscopy image restoration algorithms have been proposed in the last years. Most of these methods rely on generic assumptions on the image or degradations and are therefore outperformed by advanced methods that are based on more accurate models. Ideally, a method will accurately model the specific degradations that fit the physical acquisition settings. In this overview paper, we discuss different electron microscopy image degradation solutions and demonstrate that dedicated artefact regularisation results in higher quality restoration and is applicable through recently developed probabilistic methods.     

Automated and semi-automated cell tracking: addressing portability challenges

Cell tracking is a key task in the high-throughput quantitative study of important biological processes, such as immune system regulation and neurogenesis. Variability in cell density and dynamics in different videos, hampers portability of existing trackers across videos. We address these potability challenges in order to develop a portable cell tracking algorithm. Our algorithm can handle noise in cell segmentation as well as divisions and deaths of cells. We also propose a parameter-free variation of our tracker. In the tracker, we employ a novel method for recovering the distribution of cell displacements. Further, we present a mathematically justified procedure for determining the gating distance in relation to tracking performance. For the range of real videos tested, our tracker correctly recovers on average 96% of cell moves, and outperforms an advanced probabilistic tracker when the cell detection quality is high. The scalability of our tracker was tested on synthetic videos with up to 200 cells per frame. For more challenging tracking conditions, we propose a novel semi-automated framework that can increase the ratio of correctly recovered tracks by 12%, through selective manual inspection of only 10% of all frames in a video.     

基于相机溯源的潜在不良视频通话预警

目前主流的不良视频检测大多基于视频内容分析,属于计算密集型任务,不利于巨量视频通话流的实时检测。为此,本文试图从新的视角来阻断潜在不良视频的传播,即实时视频流相机溯源。和传统检测内容的方法不同,本文试图通过检测产生不良视频的相机源来阻断潜在不良视频的传播,即一旦发现某个正在视频通话的手机产生过不良视频,就发出安全警告进行阻断。该思路的基本假设是拍摄过不良视频的手机拥有者有更大概率利用同一部手机拍摄不良视频。该思路的核心问题是寻找一种实时、可靠的相机溯源方法。为此,本文主要聚焦以下三方面工作:建立了一个包含100部视频的数据库用于算法评估,数据库中的视频来自25部不同型号、不同品牌的相机,每一部视频注明了相机来源;建立了一种简单、有效的视频相机溯源机制,实现相机的在线实时溯源;提出了一种多相机指纹特征集成决策模型,实现可靠的相机溯源。实验结果显示,所提相机溯源机制能满足相机溯源的实时性要求,并且所提多相机指纹特征集成决策模型显著优于现有的单一模型,对于安卓手机,其视频相机溯源准确率达到98.161%,验证了该思路的可行性。     

基于循环一致性对抗网络的室内火焰图像场景迁移

基于深度学习的视频火灾探测模型的训练依赖于大量的正负样本数据,即火灾视频和带有干扰的场景视频。由于很多室内场合禁止点火,导致该场景下的火灾视频样本不足。本文基于生成对抗网络,将其他相似场景下录制的火焰迁移到指定场景,以此增广限制性场合下的火灾视频数据。文中提出将火焰内核预先植入场景使之具备完整的内容信息,再通过添加烟雾和地面反射等风格信息,完成场景与火焰的融合。该方法克服了现有多模态图像转换方法在图像转换过程中因丢失信息而造成的背景失真问题。同时为减少数据采集工作量,采用循环一致性生成对抗网络以解除训练图像必须严格匹配的限制。实验表明,与现有多模态图像转换相比,本文方法可以保证场景中火焰形态的多样性,迁移后的场景具有较高的视觉真实性,所得结果的FID与LPIPS值最小,分别为119.6和0.134 2。     

目标运动轨迹匹配式的红外-可见光视频自动配准

为实现精确的红外-可见光视频序列的自动配准,提出了一种新的基于目标轨迹线匹配的配准方法。首先,利用运动目标检测技术提取目标前景,并由基于相关滤波器(KCF)的多目标跟踪算法对每个前景顶点进行跟踪,进而获取每个目标的运动轨迹。此后,为每条轨迹线建立归一化运动方向描述子与归一化运动幅度描述子,通过时序分析、方向描述子匹配及幅值描述子匹配建立分步约束的匹配机制,完成轨迹线匹配工作。最后,采用迭代更新的方式获取最佳全局配准矩阵,实现对异源视频的配准。在LITIV数据库上的9组视频上进行测试验证,实验的结果表明:本文配准算法的重叠率误差一般小于0.2,接近或已超过手动的Ground-Truth矩阵。通过充分利用目标的运动信息,该算法实现了精确的红外-可见光图像序列配准。     

MRT letter: Visual attention driven framework for hysteroscopy video abstraction

Diagnostic hysteroscopy is a popular method for investigating the regions in the female reproductive system. The videos generated by hysteroscopy sessions of patients are recurrently archived in medical libraries. Gynecologists often need to browse these libraries in search of similar cases or for reviewing old videos of a patient. Diagnostic hysteroscopy videos contain a lot of information with abundant redundancy. Key frame extraction‐based video summarization can be used to reduce this huge amount of data. Moreover, key frames can be used for browsing and indexing of hysteroscopy videos. In this article, a domain specific visual attention driven framework for summarization of hysteroscopy videos is proposed. The visual attention model is materialized by computing saliency based on color, texture, and motion. The experimental results, in comparison with other techniques, demonstrate the efficacy of the proposed framework. Microsc. Res. Tech. 76:559–563, 2013. © 2013 Wiley Periodicals, Inc.     

A robust digital watermarking scheme for video copyright protection in the wavelet domain

This paper proposes a novel digital watermarking method for video based on a multi-resolution wavelet decomposition. The watermark used is a binary image. This watermark is embedded in the wavelet coefficients of the LH, HL and HH sub-bands of the second wavelet decomposition level by quantization. Every bit of the watermark is spread over a number of wavelet coefficients with the use of a key. The resulting watermarking scheme can be used for public watermarking applications, where the original image is not available for watermark extraction. We have tested the resilience of the watermarking algorithm against a series nine different attacks for different videos and improved the decoding BER by redundant embedding of the same watermark in different frames and by using an error correction code. Experimental results show that the embedded watermark is invisible and robust to attacks.     

Performance evaluation of image segmentation algorithms on microscopic image data

In our paper, we present a performance evaluation of image segmentation algorithms on microscopic image data. In spite of the existence of many algorithms for image data partitioning, there is no universal and ‘the best’ method yet. Moreover, images of microscopic samples can be of various character and quality which can negatively influence the performance of image segmentation algorithms. Thus, the issue of selecting suitable method for a given set of image data is of big interest. We carried out a large number of experiments with a variety of segmentation methods to evaluate the behaviour of individual approaches on the testing set of microscopic images (cross‐section images taken in three different modalities from the field of art restoration). The segmentation results were assessed by several indices used for measuring the output quality of image segmentation algorithms. In the end, the benefit of segmentation combination approach is studied and applicability of achieved results on another representatives of microscopic data category – biological samples – is shown.     

Confocal microscopic image sequence compression using vector quantization and three-dimensional pyramid

The three-dimensional (3-D) pyramid compressor project at the University of Glasgow has developed a compressor for images obtained from confocal laser scanning microscopy (CLSM) device. The proposed method using a combination of image pyramid coder and vector quantization techniques has good performance at compressing confocal volume image data. An experiment was conducted on several kinds of CLSM data using the presented compressor compared with other well-known volume data compressors, such as MPEG-1. The results showed that the 3-D pyramid compressor gave a higher subjective and objective image quality of reconstructed images at the same compression ratio, and presented more acceptable results when applying image processing filters on reconstructed images.     

Development of fan‐shaped tracker for single particle tracking

With the development of super‐resolution fluorescence microscopy, complex dynamic processes in living cells can be observed and recorded with unprecedented temporal and spatial resolution. Single particle tracking (SPT) is the most important step to explore the relationship between the spatio‐temporal dynamics of subcellular molecules and their functions. Although previous studies have developed SPT algorithms to quantitatively analyze particle dynamics in cell, traditional tracking methods have poor performance when dealing with intersecting trajectories. This can be attributed to two main reasons: (a) they do not have point compensation process for overlapping objects; (b) they use inefficient motion prediction models. In this paper, we present a novel fan‐shaped tracker (FsT) algorithm to reconstruct the trajectories of subcellular vesicles in living cells. We proposed a customized point compensation method for overlapping objects based on the fan‐shaped motion trend of the particles. Furthermore, we validated the performance of the FsT in both simulated time‐lapse movies with variable imaging quality and in real vesicle moving images. Meanwhile, we compared the performance of FsT with other five state‐of‐the‐art tracking algorithms by using commonly defined measures. The results showed that our FsT achieves better performance in high signal‐to‐noise ratio conditions and in tracking of overlapping objects. We anticipate that our FsT method will have vast applications in tracking of moving objects in cell.     

基于帧间颜色梯度的背景建模

在分析了交通场景视频特点的基础上,根据视频流中帧间像素颜色变化情况来确定背景像素值。提出以帧间像素梯度值作为背景像素的判断条件来建立背景模型的方法。在计算中将图像进行分块,通过求子区域颜色梯度变化值来确定其背景,然后根据各子区域的计算结果重建背景模型。实验结果表明:平均经过100帧图像初始化后,即可完成背景建模,初始化帧数比其它方法少,平均每帧图像的处理时间为15.566 ms。该方法背景建模的速度和初始化帧数都满足交通场景视频监控系统的要求。     

Compression of fluorescence microscopy images based on the signal-to-noise estimation

Modern microscopic techniques like high-content screening (HCS), high-throughput screening, 4D imaging, and multispectral imaging may involve collection of thousands of images per experiment. Efficient image-compression techniques are indispensable to manage these vast amounts of data. This goal is frequently achieved using lossy compression algorithms such as JPEG and JPEG2000. However, these algorithms are optimized to preserve visual quality but not necessarily the integrity of the scientific data, which are often analyzed in an automated manner. Here, we propose three observer-independent compression algorithms, designed to preserve information contained in the images. These algorithms were constructed using signal-to-noise ratio (SNR) computed from a single image as a quality measure to establish which image components may be discarded. The compression efficiency was measured as a function of image brightness and SNR. The alterations introduced by compression in biological images were estimated using brightness histograms (earth's mover distance (EMD) algorithm) and textures (Haralick parameters). Furthermore, a microscope test pattern was used to assess the effect of compression on the effective resolution of microscope images.     

基于视觉感知的率失真优化算法

率失真优化在视频编码中起着关键作用,其目的是在压缩效率和视频质量失真之间取得平衡。现有的率失真算法主要针对视频中时间和空间冗余的消除,未充分考虑视频内容的主观感知冗余。本文提出了一种基于视觉感知的率失真优化算法,通过数据驱动的JND预测模型推导拉格朗日乘数因子,并使用显著性模型优化拉格朗日乘子权重系数,最终融合应用于率失真优化,并采用SW-SSIM评估视频质量,实现视频编码的感知优化。实验结果表明,与AVS3标准率失真算法相比,本文所提算法平均节省12.15%的码率,SW-SSIM提高了0.004 3,有效降低了视频内容中的感知冗余,提高了视频感知质量和编码性能。     

多模态特征融合与多任务学习的特种视频分类

特种视频(本文特指暴力视频)的智能分类技术有助于实现网络信息内容安全的智能监控。针对现有特种视频多模态特征融合时未考虑语义一致性等问题,本文提出了一种基于音视频多模态特征融合与多任务学习的特种视频识别方法。首先,提取特种视频的表观信息和运动信息随时空变化的视觉语义特征及音频信息语义特征;然后,构建具有语义保持的共享特征子空间,以实现音视频多种模态特征的融合;最后,提出基于音视频特征的语义一致性度量和特种视频分类的多任务学习特种视频分类理论框架,设计了对应的损失函数,实现了端到端的特种视频智能识别。实验结果表明,本文提出的算法在Violent Flow和MediaEval VSD 2015两个数据集上平均精度分别为97.97%和39.76%,优于已有研究。结果证明了该算法的有效性,有助于提升特种视频监控的智能化水平。