基于全局运动仿射模型的视频目标移除

给出一种基于全局运动仿射模型的视频目标移除方法.用仿射模型描述非移除区域的全局运动,通过提取相邻帧的匹配对进行模型参数估计;对于每一帧中待修补的移除区域,根据仿射模型之间的递推关系和运动一致性准则,从其它帧中寻找该区域的信息并进行修补.实验表明,方法能够较好的对运动前景、台标等目标进行移除.     

H.266/VVC中一种改进的仿射运动估计算法

通用视频编码标准H.266/VVC通过引入多种新的编码技术,如仿射运动补偿预测、自适应运动矢量精度、多核变换等,以支持360°视频和HDR视频的编解码,从而为用户提供最优的视频质量,但是在H.266/VVC帧间预测过程中,仿射运动估计计算复杂度高导致编码时间显著增加。针对该问题,提出一种改进的仿射运动估计算法。通过对仿射高级矢量预测（AAMVP）候选列表的构建过程进行改进,并构建一种AAMVP候选列表候选项筛选准则,使得列表的候选项更接近编码块真实的运动矢量,从而缩短编码时间。同时对仿射运动估计中迭代搜索最优仿射运动矢量的迭代过程进行优化,以加快迭代搜索速度。实验结果表明,在低时延的编码器配置下,相比VVC原始算法,当BD-BR增加了0.023%时,该算法的总体编码时间平均缩短13%,在保证编码质量的前提下能够有效降低编码的计算复杂度。     

一种基于单环结构的扩展基本层FGS视频编码方法

可分级编码是解决Internet流视频应用中网络带宽不断波动的一种有效方法，所以MPEG-4标准中采用了FGS(fine granularity scalability)编码方法来获得精细颗粒可分级能力，但其代价是编码效率的下降。为解决此问题，现提出在增强层中采用运动补偿的MC加FGS(motion compensation加FGS)结构，用于去除FGS方案中增强层在时域上的冗余，以提高FGS方案编码效率的双环和单环两种方法。在比较了两种结构各自的优缺点后，选定了一种复杂度小、实现简单、效率高的单环结构，并提出了对单环结构的缺陷进行改善的方法。实验结果表明，该方法的编码性能优于MPEG-4 FGS方法。     

A flexibility coupled hypercube multiprocessor for high level vision

In general, message passing multiprocessors suffer from communication overhead between processors and shared memory multiprocessors suffer from memory contention. Also, in computer vision tasks, data I/O overhead limits performance. In particular, high level vision tasks, which are complex and require nondeterministic communication, are strongly affected by these disadvantages. This paper proposes a flexibly (tightly/loosely) coupled hypercube multiprocessor (FCHM) for high level vision to alleviate these problems. A variable address space memory scheme in which a set of adjacent memory modules can be merged into a shared memory module by a dynamically partitionable hypercube topology is proposed. The architecture is quantitatively analyzed using computational models and simulated on the Intel’s Personal SuperComputer (iPSC/I), a hypercube multiprocessor. A parallel algorithm for exhaustive search is simulated on FCHM using the iPSC/I showing significant performance improvements over that of the iPSC/I. This research was supported in part by IBM corporation.     

基于可分级视频编码的双向双维运动信息可分级*

为进一步提高基于一种运动信息可分级模型的可分级视频编码的编码效率,减小运动信息可分级的最低码率限制,对该运动可分级模型的二维多向性进行了具体研究与改进,更新了运动估计的流程,提出了多向上的两个运动可分级维度的运动分级等级一致性原则以及二维数据的渐进式存储存取结构,从而实现了二维多向上的运动可分级。实验测试结果证明该二维多向的运动信息模型优于不可分级的运动信息,能有效提高可分级视频编码系统的效率。     

基于CAC的数字视频安全编码研究

对于视频压缩领域,基于上下文的自适应编码(CAC)是一类新出现的高效熵编码方法。为了对数字视频进行实时安全编码,提出了一种基于CAC的数字视频安全编码方案,并首先建立了以CAC安全编码器为核心的安全编码链,然后给出了基于上下文的自适应二进制算术安全编码(CABASC)和基于上下文的自适应变长安全编码(CAVLSC)两种安全编码操作。实验结果表明,该方案具有较好的安全性、实时性和软件易实现性,可作为进一步研究CAC安全编码的基础。     

结合率失真优化的自适应全局运动估计方法

全局运动估计的关键在于全局运动模型的选择。结合率失真优化理论,提出一种自适应全局运动估计方法来达到编码优化的目的。该方法的主要思路是：对同一帧图像,分别使用平移运动模型、六参数运动模型和十二参数运动模型进行编码,用率失真优化算法计算3种运动模型下的拉格朗日代价函数值,拉格朗日代价函数值最小的运动模型被选为最佳的当前帧的运动模型。实验证明,该方法具有较好的鲁棒性,对不同分辨率的视频序列均有不同程度的编码增益。     

Multicoloring of grid-structured PDE solvers on shared-memorymultiprocessors

In order to execute a parallel PDE (partial differential equation) solver on a shared-memory multiprocessor, we have to avoid memory conflicts in accessing multidimensional data grids. A new multicoloring technique is proposed for speeding sparse matrix operations. The new technique enables parallel access of grid-structured data elements in the shared memory without causing conflicts. The coloring scheme is formulated as an algebraic mapping which can be easily implemented with low overhead on commercial multiprocessors. The proposed multicoloring scheme bas been tested on an Alliant FX/80 multiprocessor for solving 2D and 3D problems using the CGNR method. Compared to the results reported by Saad (1989) on an identical Alliant system, our results show a factor of 30 times higher performance in Mflops. Multicoloring transforms sparse matrices into ones with a diagonal diagonal block (DDB) structure, enabling parallel LU decomposition in solving PDE problems. The multicoloring technique can also be extended to solve other scientific problems characterized by sparse matrices     

基于多模态输入的对抗式视频生成方法

视频生成是计算机视觉和多媒体领域一个重要而又具有挑战性的任务.现有的基于对抗生成网络的视频生成方法通常缺乏一种有效可控的连贯视频生成方式.提出一种新的多模态条件式视频生成模型.该模型使用图片和文本作为输入,通过文本特征编码网络和运动特征解码网络得到视频的运动信息,并结合输入图片生成连贯的运动视频序列.此外,该方法通过对输入图片进行仿射变换来预测视频帧,使得生成模型更加可控、生成结果更加鲁棒.在SBMG(single-digit bouncing MNIST gifs),TBMG(two-digit bouncing MNIST gifs)和KTH(kungliga tekniska hgskolan human actions)数据集上的实验结果表明：相较于现有的视频生成方法,生成结果在目标清晰度和视频连贯性方面都具有更好的效果.另外定性评估和定量评估(SSIM(structural similarity index)与PSNR(peak signal to noise ratio)指标)表明提出的多模态视频帧生成网络在视频生成中起到了关键作用.     

Encoding of Affine Motion Vectors

An affine motion model provides better motion representation than a translational motion model. Therefore, it is a good candidate for advanced video compression algorithms, requiring higher compression efficiency than current algorithms. One disadvantage of the affine motion model is the increased number of motion vector parameters, therefore increased motion vector bit rate. We develop and analyze several simulation based approaches of entropy coding for orthonormalized affine motion vector (AMV) coefficients, by considering various context-types and coders. In our work we expand the traditional idea of a context type by introducing four new context types. We compare our method of contexts-type and coder selection with context quantization. The best of our contexts-type and coder solutions produces 4% to 15% average AMV bit-rate reductions over the original VLC approach. For more difficult content AMV bit rate reduction up to 26% is reported.     

面向视频会议的运动估计快速算法

针对H.264/AVC视频压缩算法中运动估计模块计算复杂度高的问题,提出了一种基于积分图像的运动估计快速算法.算法通过对单元块进行预计算求和,在块匹配运算时使用单元块级绝对差值.该算法能够在保证视频编码质量的前提下,大幅度提高运动估计模块的计算效率.算法适用于视频会议等要求视频实时压缩的应用.     

An optimized parallel order scheme of the deblocking filtering process for enhancing the performance of the HEVC standard using GPUs

In HEVC, deblocking filtering (DF) is responsible for about 20% of the time consumed to perform video compression. In a typical parallel DF scheme, a set of horizontal and vertical edges are processed using deblocking filters. In conventional parallel DF schemes, deblocking filters could be applied to the same edges more than once. Moreover, some edges are assigned to cores to be filtered even though those edges are not designated to be filtered. Accordingly, the used parallel hardware architecture requires more on-chip memory modules. Those challenges negatively affect HEVC performance resulting in an increase in computational complexity. In this paper, an optimized parallel DF scheme is proposed for HEVC using graphical processing units (GPUs). The proposed scheme outperforms competing ones in terms of reducing the decoding time of all frames of video sequences by average speed-up factors of 2.83 and 2.45 using the all-intra and low-delay video coding configuration modes, respectively. The proposal does not change the rate-distortion between the decoded video sequences and their original sequences.     

Efficient data parallel algorithms for multidimensional array operations based on the EKMR scheme for distributed memory multicomputers

Array operations are useful in a large number of important scientific codes, such as molecular dynamics, finite element methods, climate modeling, atmosphere and ocean sciences, etc. In our previous work, we have proposed a scheme of extended Karnaugh map representation (EKMR) for multidimensional array representation. We have shown that sequential multidimensional array operation algorithms based on the EKMR scheme have better performance than those based on the traditional matrix representation (TMR) scheme. Since parallel multidimensional array operations have been an extensively investigated problem, we present efficient data parallel algorithms for multidimensional array operations based on the EKMR scheme for distributed memory multicomputers. In a data parallel programming paradigm, in general, we distribute array elements to processors based on various distribution schemes, do local computation in each processor, and collect computation results from each processor. Based on the row, column, and 2D mesh distribution schemes, we design data parallel algorithms for matrix-matrix addition and matrix-matrix multiplication array operations in both TMR and EKMR schemes for multidimensional arrays. We also design data parallel algorithms for six Fortran 90 array intrinsic functions: All, Maxval, Merge, Pack, Sum, and Cshift. We compare the time of the data distribution, the local computation, and the result collection phases of these array operations based on the TMR and the EKMR schemes. The experimental results show that algorithms based on the EKMR scheme outperform those based on the TMR scheme for all test cases.     

A lossy 3D wavelet transform for high-quality compression of medical video

In this paper, we present a lossy compression scheme based on the application of the 3D fast wavelet transform to code medical video. This type of video has special features, such as its representation in gray scale, its very few interframe variations, and the quality requirements of the reconstructed images. These characteristics as well as the social impact of the desired applications demand a design and implementation of coding schemes especially oriented to exploit them. We analyze different parameters of the codification process, such as the utilization of different wavelets functions, the number of steps the wavelet function is applied to, the way the thresholds are chosen, and the selected methods in the quantization and entropy encoder. In order to enhance our original encoder, we propose several improvements in the entropy encoder: 3D-conscious run-length, hexadecimal coding and the application of arithmetic coding instead of Huffman. Our coder achieves a good trade-off between compression ratio and quality of the reconstructed video. We have also compared our scheme with MPEG-2 and EZW, obtaining better compression ratios up to 119% and 46%, respectively for the same PSNR.     

Motion-Based Selection of Relevant Video Segments for Video Summarization

We present a method for motion-based video segmentation and segment classification as a step towards video summarization. The sequential segmentation of the video is performed by detecting changes in the dominant image motion, assumed to be related to camera motion and represented by a 2D affine model. The detection is achieved by analysing the temporal variations of some coefficients of the 2D affine model (robustly) estimated. The obtained video segments supply reasonable temporal units to be further classified. For the second stage, we adopt a statistical representation of the residual motion content of the video scene, relying on the distribution of temporal co-occurrences of local motion-related measurements. Pre-identified classes of dynamic events are learned off-line from a training set of video samples of the genre of interest. Each video segment is then classified according to a Maximum Likelihood criterion. Finally, excerpts of the relevant classes can be selected for video summarization. Experiments regarding the two steps of the method are presented on different video genres leading to very encouraging results while only low-level motion information is considered.     

Fast Motion Estimation on Graphics Hardware for H.264 Video Encoding

The video coding standard H.264 supports video compression with a higher coding efficiency than previous standards. However, this comes at the expense of an increased encoding complexity, in particular for motion estimation which becomes a very time consuming task even for today's central processing units (CPU). On the other hand, modern graphics hardware includes a powerful graphics processing unit (GPU) whose computing power remains idle most of the time. In this paper, we present a GPU based approach to motion estimation for the purpose of H.264 video encoding. A small diamond search is adapted to the programming model of modern GPUs to exploit their available parallel computing power and memory bandwidth. Experimental results demonstrate a significant reduction of computation time and a competitive encoding quality compared to a CPU UMHexagonS implementation while enabling the CPU to process other encoding tasks in parallel.     

XOR Storage Schemes for Frequently Used Data Patterns

In parallel image processing and numerical analysis, various matrix manipulation operations are used intensively. In the past decades, many parallel storage schemes, called skewing schemes, have been proposed to provide simultaneous access to various data patterns (slices of a matrix). The existing storage schemes have the following limitations: (1) The address generation mechanism is dependent on the size of the matrix to be processed, thus the system hardware must be changed to efficiently process different sized matrices. (2) Many schemes have limitations on the machine size and image size (N × N), such as N must be an even power of 2. (3) As more and more frequently used data patterns have been recognized, most schemes can only provide parallel access to a limited range of data patterns. (4) With existing routing techniques, the data alignment (connecting each memory module to a proper processor) may require special hardware. This paper proposes several storage schemes (EE, MG, EE-MG, and EE*MG). They employ only exclusive-or operations for address generation which can be completed in constant time. The address generation mechanism is independent of the matrix size so that different sized matrices can be processed efficiently on a fixed-size machine. The system uses N memory modules where N is any (even or odd) power of 2. These schemes cover more data patterns than any other scheme yet proposed. Patterns of N elements that can be accessed in one memory cycle include diagonals, blocks with various shapes, points scattered over various blocks, and chessboards with various shapes. Data alignment requirements can be easily realized on a general-purpose interconnection network, such as a hypercube or MINs.     

Parallel algorithms for algebraic Riccati equations

The matrix sign function is the basis of a parallel algorithm for solving the generalized algebraic Riccati equation. Three forms of the algorithm were implemented and tested on a distributed memory hypercube multiprocessor. Performance results indicate that the method is an excellent means of solving large-scale problems on a parallel computer.     

一种H.264/AVC中的快速运动估计算法

在UMHexagonS算法的基础上,最新的H.264/AVC视频编码采用一种简单、有效的混合快速运动估计算法。该文介绍一种新的快速搜索算法,初步研究和分析其综合性能的优越性,并提出改进策略,如增加终止搜索的判定、进一步划分运动类型、采用自适应的方向性搜索模板。实验结果表明,改进的策略能减少“简化UMHexagonS算法”8%~30%的搜索时间,而码率和率失真性能的变化可以忽略不计,对一些特殊序列,率失真性能略优于“简化UMHexagonS算法”。     

A Motion Compensated Lifting Wavelet Codec for 3D Video Coding

A motion compensated lifting (MCLIFT) framework for the 3D wavelet video coding is proposed in this paper. By using bi-directional motion compensation in each lifting step of the temporal direction, the video frames are effectively de-correlated. With the proper entropy coding and bit-stream packaging schemes, the MCLIFT wavelet video coder is scalable at frame rate and quality level. Experimental results show that the MCLIFT video coder outperforms the 3D wavelet video coder without motion by an average of 0.9-1.3dB, and outperforms MPEG-4 coder by an average of 0.2-0.6dB.