Advanced free viewpoint video streaming techniques

Free-viewpoint video is a new type of interactive multimedia service allowing users to control their viewpoint and generate new views of a dynamic scene from any perspective. The uniquely generated and displayed views are composed from two or more high bitrate camera streams that must be delivered to the users depending on their continuously changing perspective. Due to significant network and computational resource requirements, we proposed scalable viewpoint generation and delivery schemes based on multicast forwarding and distributed approach. Our aim was to find the optimal deployment locations of the distributed viewpoint synthesis processes in the network topology by allowing network nodes to act as proxy servers with caching and viewpoint synthesis functionalities. Moreover, a predictive multicast group management scheme was introduced in order to provide all camera views that may be requested in the near future and prevent the viewpoint synthesizer algorithm from remaining without camera streams. The obtained results showed that even 42 % traffic decrease can be realized using distributed viewpoint synthesis and the probability of viewpoint synthesis starvation can be also significantly reduced in future free viewpoint video services.     

快速3维坐标变换的绘制算法

目的 基于深度图的绘制(DIBR)是一种新型的虚拟视点生成技术,在诸多方面得到了广泛的应用。然而,该技术还不能满足实时性的绘制需求。为了在保证绘制质量不下降的前提下,尽可能地提高绘制速度,提出了一种高效的3D-Warping(3维坐标变换)算法。方法 主要在以下3个方面进行了改进:1)引入了深度—视差映射表技术,避免了重复地进行视差求取操作。2)对深度平坦的像素块进行基于块的3D-Warping,减少了映射的次数。对深度非平坦像素块中的像素点采取传统的基于像素点的3D-Warping,保证了映射的准确性。3)针对两种不同的3D-Warping方式,分别提出了相应的插值算法。在水平方向上,改进的像素插值算法对紧邻插值和Splatting(散射)插值算法进行了折中,只在映射像素点与待插值像素点很近的情况下才进行紧邻插值,否则进行Splatting插值;在深度方向上,它对Z-Buffer(深度缓存)技术进行了改进,舍弃了与前景物体太远的映射像素点,而对其他映射像素点按深度值进行加权操作。结果 实验结果表明,与标准绘制方案的整像素精度相比,绘制时间平均节省了72.05%;与标准绘制方案的半像素精度相比,PSNR平均提高了0.355dB,SSIM平均提高了0.00115。结论 改进算法非常适用于水平设置相机系统的DIBR技术中的整像素精度绘制,对包含大量深度平坦区域的视频序列效果明显,不但能够提高绘制的速度,而且可以有效地改善绘制的客观质量。     

Automatic camera control in virtual environments augmented using multiple sparse videos

Automated virtual camera control has been widely used in animation and interactive virtual environments. We have developed a multiple sparse camera based free view video system prototype that allows users to control the position and orientation of a virtual camera, enabling the observation of a real scene in three dimensions (3D) from any desired viewpoint. Automatic camera control can be activated to follow selected objects by the user. Our method combines a simple geometric model of the scene composed of planes (virtual environment), augmented with visual information from the cameras and pre-computed tracking information of moving targets to generate novel perspective corrected 3D views of the virtual camera and moving objects. To achieve real-time rendering performance, view-dependent textured mapped billboards are used to render the moving objects at their correct locations and foreground masks are used to remove the moving objects from the projected video streams. The current prototype runs on a PC with a common graphics card and can generate virtual 2D views from three cameras of resolution 768×576 with several moving objects at about 11 fps.     

A novel terminal-controlled handover scheme in heterogeneous wireless networks

In the next generation heterogeneous wireless networks, a mobile terminal (MT) with a multi-interface may have network access from different service providers using various technologies. In spite of this heterogeneity, seamless intersystem mobility is a mandatory requirement. One of the major challenges for seamless mobility is the creation of a vertical handover scheme, which is for users that move between different types of networks. In this article a seamless mobility handover scheme is presented. The novelty of the proposed scheme is that handover is fully controlled by the terminal. To improve the performance of the proposed handover scheme, a network discovery algorithm with fuzzy logic and a handover decision algorithm using multi criteria decision making (MCDM) based on vague sets are derived, which are both user-centric. With these algorithms, the handover scheme is power-saving, cost-aware, and performance-aware. Simulation results show that the novel handover scheme can perform network discovery in time to reduce handover dropping rate and present unnecessary activation of its interface to save the battery power, while effectively choose the optimum network through making trade-off among the user preference, network condition, and system performance.     

IEEE 802.21-based seamless multicast streaming with dynamic playback control

With the explosion of video streaming available on the Internet, online multimedia applications become more popular in our days and the video quality of the Internet multimedia applications is directly affected by the network transmission state, which will be worse while switching the ongoing network connection from one wireless interface to another heterogeneous wireless technology, such as IEEE 802 families, UMTS (universal mobile telecommunications system) network and 3GPP LTE (3rd generation partnership project-long term evolution). In order to perform seamless handover between heterogeneous wireless networks, IEEE group proposed the “IEEE 802.21 standard” and defined a middleware function called “media independent handover function (MIHF)” to smooth the handover. In this paper, a dynamic playback control for multicasting streaming based on IEEE 802.21 is proposed to reduce the influence of handover between heterogeneous networks. In addition, we evaluate three different rate modes for seamless multicasting streams. The simulation results show that different playback frame rate modes, including the usual, incremental and linear modes, can achieve different video quality and can extend the playing time for handover video playback.     

Optimization for adaptive bandwidth reservation in wireless multimedia networks

In future wireless multimedia networks, user mobility management for seamless connection regarding realtime multimedia applications is one of the most important problems. In this paper we propose an opportunity-cost concept-based approach for adaptive bandwidth reservation with admission control for handover calls utilizing network traffic information. Excessive reservation guarantees low blocking probability of handover calls at the cost of high blocking probability of new calls. According to our survey, however, it may degrade bandwidth utilization while no prioritization for handover admissions degrades quality of service (QoS) for ongoing calls. We consider both QoS assurance and bandwidth utilization in order to optimize the amount of bandwidth to reserve for handover admissions. We believe that our scheme could be utilized as a guideline for cost-effective radio resource allocation in mobile multimedia networks.     

基于快速连续蚁群算法的QoS切换决策机制

随着人们对移动网络服务需求的日益增长,用户需要在复杂的异构网络中实现不同接入技术之间的无缝切换,因此选择总最佳连接并提供服务质量保证的切换决策成为下一代互联网研究的关键内容.为了更好地满足用户的需求,在综合考虑网络信息的不确定性、用户服务质量需求、用户偏好、用户愿意支付的费用以及网络供应商的收益等因素的前提下,设计了基于快速连续蚁群算法的切换决策机制,并引入协商博弈和Nash均衡思想来解决用户之间的串谋问题以保证切换决策的公平性.仿真结果表明,设计的切换决策机制在满足用户服务质量需求的同时,不仅使双方的效用最大,并且最契合用户的偏好,能够很好地解决下一代互联网中支持总最佳连接的切换决策问题.     

Optimizing throughput performance of FMIPv6 over legacy 802.11 networks using iterative scanning

Predictive handover protocols like FMIPv6 have been proposed to provide seamless handover to MNs with minimum handover latency and packet losses. An essential prerequisite of FMIPv6 protocol is the advance discovery of candidate access points while the MN is still connected to its Present Access Router, a process which is carried out by the underlying data link layer protocol. The FMIPv6 standard only specifies the seamless handover operation at the network layer and does not take into consideration the impact of the relevant mechanism, such as the advance access point discovery process, on the overall seamless operation in terms of packet losses. At the data-link layer, the MN may repeatedly preform channel scanning operations to discover the in-range candidate APs. During the scan process, a MN is unable to transmit or receive packets leading to service quality degradation.This paper will analyze the implication of the IEEE 802.11 scanning process, that is carried out in advance to the actual initiation of the FMIPv6 handover process as part of the candidate access points discovery process. The impact of such a scan process is assessed on the overall seamlessness of the FMIPv6 protocol and its impact on the user’s QoE. In this regard, an iterative channel scanning solution approach would be described, which has been proposed to offset the adverse affects of 802.11 channel scanning process on the overall seamlessness of the FMIPv6 protocol in a legacy 802.11 network, especially during the candidate AP discovery process. A detailed evaluation of the performance of our proposed scheme will be presented based on extensive simulation analysis.     

A flexible vehicle surround view camera system by central-around coordinate mapping model

The surround view camera system is an emerging driving assistant technology that can assist drivers in parking by providing top-down view of surrounding situations. Such a system usually consists of four wide-angle or fish-eye cameras that mounted around the vehicle, and a bird-eye view is synthesized from images of these cameras. Commonly there are two fundamental problems for the surround view synthesis, geometric alignment and image synthesis. Geometric alignment performs fish-eye calibration and computes the image perspective transformation between the bird-eye view and images from the surrounding cameras. Image synthesis technique dedicates to seamless stitch between adjacent views and color balancing. In this paper, we propose a flexible central-around coordinate mapping (CACM) model for vehicle surround view synthesis. The CACM model calculates perspective transformation between a top-view central camera coordinate and the around camera coordinates by a marker point based method. With the transformation matrices, we could generate the pixel point mapping relationship between the bird-eye view and images of the surrounding cameras. After geometric alignment, an image fusion method based on distance weighting is adopted for seamless stitch, and an effective overlapping region brightness optimization method is proposed for color balancing. Both the seamless stitch and color balancing can be easily operated by using two types of weight coefficient under the framework of the CACM model. Experimental results show that the proposed approaches could provide a high-performance surround view camera system.

     

Adaptive VoIP playout scheduling: assessing user satisfaction

Delay and packet loss dramatically affect the quality of voice-over-IP (VoIP) calls and depend on the playout buffer scheme implemented at the receiver. The choice of playout algorithm can't be based on statistical metrics without considering the perceived end-to-end conversational speech quality. The authors present a method for evaluating various playout algorithms that extends the E-model concept by estimating user satisfaction from time-varying transmission impairments. This article evaluates several playout algorithms and shows a correspondence between the authors' results and those obtained via statistical loss and delay metrics.     

5G异构网络中基于群组的切换认证方案

随着5G网络的发展,各类网络服务质量极大提升的同时网络环境也愈加复杂,从而带来了一系列安全挑战。切换认证可以解决用户在不同类型网络间的接入认证问题,但现存方案仍存在一些不足,还需要解决如全局切换认证、密钥协商、隐私保护、抵抗伪装攻击、抵抗中间人攻击、抵抗重放攻击以及群组用户切换效率等问题。针对这些问题,提出了一个5G异构网络中基于群组的切换认证方案。在所提出的方案中,注册域服务器在区块链上为每个用户存入一个通行证,任何实体都可以利用该通行证对用户进行认证,从而实现全局切换认证。对于群组用户,各用户分别设置可聚合的认证参数,验证者通过验证聚合签名实现对群组用户的批量验证。新方案不仅提升了群组用户切换时的效率,同时还满足上述安全性要求。基于形式化分析软件AVISPA的分析结果表明,所提出的方案是安全的。性能分析表明,所提出的方案执行批量验证时的效率比现存方案至少提升了89.8%。     

A curved ray camera for handling occlusions through continuous multiperspective visualization

Most images used in visualization are computed with the planar pinhole camera. This classic camera model has important advantages such as simplicity, which enables efficient software and hardware implementations, and similarity to the human eye, which yields images familiar to the user. However, the planar pinhole camera has only a single viewpoint, which limits images to parts of the scene to which there is direct line of sight. In this paper we introduce the curved ray camera to address the single viewpoint limitation. Rays are C1-continuous curves that bend to circumvent occluders. Our camera is designed to provide a fast 3-D point projection operation, which enables interactive visualization. The camera supports both 3-D surface and volume datasets. The camera is a powerful tool that enables seamless integration of multiple perspectives for overcoming occlusions in visualization while minimizing distortions.     

Free Appearance-Editing with Improved Poisson Image Cloning

In this paper, we present a new edit tool for the user to conveniently preserve or freely edit the object appearance during seamless image composition. We observe that though Poisson image editing is effective for seamless image composition. Its color bleeding (the color of the target image is propagated into the source image) is not always desired in applications, and it provides no way to allow the user to edit the appearance of the source image. To make it more flexible and practical, we introduce new energy terms to control the appearance change, and integrate them into the Poisson image editing framework. The new energy function could still be realized using efficient sparse linear solvers, and the user can interactively refine the constraints. With the new tool, the user can enjoy not only seamless image composition, but also the flexibility to preserve or manipulate the appearance of the source image at the same time. This provides more potential for creating new images. Experimental results demonstrate the effectiveness of our new edit tool, with similar time cost to the original Poisson image editing.     

Analytical study of intradomain handover in multiple-mobile-routers-based multihomed NEMO networks

The need to stay connected to the Internet “anytime” and “anywhere” is becoming more and more required in the recent years with the development of wireless networks. The NEMO basic support protocol was developed to grant the Internet connection for an entire mobile network, like vehicle or aircraft network. In NEMO a Mobile Router manages the mobility of the entire network on behalf of the other nodes in order to reduce the overhead of signaling messages. The critical part of mobility support is to reduce the handover latency, particularly when a Mobile Router is not reachable. This handover period degrades the performance of most delay sensitive applications. Referring to our proposition for NEMO handover improvement which is conceived for multiple-mobile-routers-based multihomed NEMO network, we find out a mathematical model to study the intradomain handover performance. We derive the mathematical model to analyze interruption interval (the time interval during which none of mobile routers can transport packets) during the intradomain handover. We study the impact of varying critical parameters, like vehicle speed, overlapping coverage distance, distance between mobile routers, and scanning frequency. The obtained results reveal that the seamless handover can be achieved if network configuration parameters are correctly chosen.     

基于移动用户位置预测的垂直切换算法

针对异构无线网络环境中现有垂直切换算法存在难以实现移动用户接入网络的无缝切换,无法向用户提供稳定通信服务的问题,提出一种基于移动用户位置预测的垂直切换算法。首先利用用户移动轨迹的相似性,使用用户轨迹数据离线训练LSTM模型以学习各用户普遍具有的移动规律;然后在线加载LSTM模型进行用户位置预测,从而使用模糊逻辑分析计算下一时刻用户对应候选网络的回报值,并选取回报值最高的网络进行切换。实验结果表明,该算法与已有垂直切换算法相比,在不同的用户移动速度和用户规模条件下的切换次数、切换失败次数和切换时延均有明显下降,内存消耗较低,能够实现无缝切换。     

An efficient playout smoothing mechanism for layered streaming in P2P networks

Layered video streaming in peer-to-peer (P2P) networks has drawn great interest, since it can not only accommodate large numbers of users, but also handle peer heterogeneity. However, there’s still a lack of comprehensive studies on chunk scheduling for the smooth playout of layered streams in P2P networks. In these situations, a playout smoothing mechanism can be used to ensure the uniform delivery of the layered stream. This can be achieved by reducing the quality changes that the stream undergoes when adapting to changing network conditions. This paper complements previous efforts in throughput maximization and delay minimization for P2P streaming by considering the consequences of playout smoothing on the scheduling mechanisms for stream layer acquisition. The two main problems to be considered when designing a playout smoothing mechanism for P2P streaming are the fluctuation in available bandwidth between peers and the unreliability of user-contributed resources—particularly peer churn. Since the consideration of these two factors in the selection and scheduling of stream layers is crucial to maintain smooth stream playout, the main objective of our smoothing mechanism becomes the determination of how many layers to request from which peers, and in which order. In this work, we propose a playout smoothing mechanism for layered P2P streaming. The proposed mechanism relies on a novel scheduling algorithm that enables each peer to select appropriate stream layers, along with appropriate peers to provide them. In addition to playout smoothing, the presented mechanism also makes efficient use of network resources and provides high system throughput. An evaluation of the performance of the mechanism demonstrates that the proposed mechanism provides a significant improvement in the received video quality in terms of lowering the number of layer changes and useless chunks while improving bandwidth utilization.     

下一代高速铁路异构网络切换安全认证

近年来,随着高速铁路无线通信技术的快速发展, GSM-R无线通信系统将逐步向LTE-R系统演进。在此演进过程中存在GSM-R和LTE-R长期共存的局面,如何实现高速铁路无线通信异构网络之间的快速切换和安全认证成为铁路无线通信研究的热点问题。针对高速铁路无线通信异构网络切换认证过程中,存在安全性低和认证开销高等问题,提出了一种适用于下一代高速铁路异构网络的轻量级切换安全认证方案。首先,采用哈希函数等操作生成切换请求Token和异构网络切换认证码PASS,实现了用户身份匿名性和可追溯性等安全要求,并且高速列车无需多次注册就可实现异构网络间的无缝切换。其次,设计了基于椭圆曲线密钥交换的轻量级切换算法,完成了高速列车与目标基站的相互认证和密钥协商,降低了计算开销和通信开销,实现了会话协商密钥的前后向安全性。最后,采用形式化方式BAN逻辑进行了安全性验证,并使用朔黄铁路LTE-R线路实测数据进一步对本文所提方案的有效性进行了验证,分析得出所提方案能够满足可追溯性、匿名性、抗伪装用户攻击、抗中间人攻击和抗重放攻击等安全特性。性能分析表明,本文方案在通信开销和计算开销方面较比较方法性能更优,能够满足...     

Automatic Registration of Multi‐Projector Domes Using a Single Uncalibrated Camera

In this paper we present a novel technique for easily calibrating multiple casually aligned projectors on spherical domes using a single uncalibrated camera. Using the prior knowledge of the display surface being a dome, we can estimate the camera intrinsic and extrinsic parameters and the projector to display surface correspondences automatically using a set of images. These images include the image of the dome itself and a projected pattern from each projector. Using these correspondences we can register images from the multiple projectors on the dome. Further, we can register displays which are not entirely visible in a single camera view using multiple pan and tilted views of an uncalibrated camera making our method suitable for displays of different size and resolution. We can register images from any arbitrary viewpoint making it appropriate for a single head‐tracked user in a 3D visualization system. Also, we can use several cartographic mapping techniques to register images in a manner that is appropriate for multi‐user visualization. Domes are known to produce a tremendous sense of immersion and presence in visualization systems. Yet, till date, there exists no easy way to register multiple projectors on a dome to create a high‐resolution realistic visualizations. To the best of our knowledge, this is the first method that can achieve accurate geometric registration of multiple projectors on a dome simply and automatically using a single uncalibrated camera.