Taking advantage of contextualized interactions while users watch TV

While watching TV, viewers use the remote control to turn the TV set on and off, change channel and volume, to adjust the image and audio settings, etc. Worldwide, research institutes collect information about audience measurement, which can also be used to provide personalization and recommendation services, among others. The interactive digital TV offers viewers the opportunity to interact with interactive applications associated with the broadcast program. Interactive TV infrastructure supports the capture of the user–TV interaction at fine-grained levels. In this paper we propose the capture of all the user interaction with a TV remote control—including short term and instant interactions: we argue that the corresponding captured information can be used to create content pervasively and automatically, and that this content can be used by a wide variety of services, such as audience measurement, personalization and recommendation services. The capture of fine grained data about instant and interval-based interactions also allows the underlying infrastructure to offer services at the same scale, such as annotation services and adaptative applications. We present the main modules of an infrastructure for TV-based services, along with a detailed example of a document used to record the user–remote control interaction. Our approach is evaluated by means of a proof-of-concept prototype which uses the Brazilian Digital TV System, the Ginga-NCL middleware.     

MPEG-2节目流分析及解复用器设计

在多媒体应用中,为满足不同的应用需求,常需要对若干路音视频基本流进行系统复用,而要完成复合系统流的解码并播放又需对其解复用,因而进行码流的分析十分必要。MPE6—2系统标准作为业界遵循的规范,有传输流和节目流两种编码形式,与固定长度包结构的传输流不同,节目流包长可变,通过对MPE6—2标准的研究并结合实际节目流的比特流结构分析,实现了节目流解复用器的软件编程,应用于系统的分析和测试中。     

Design and integration of the OpenCore-based mobile TV framework for DVB-H/T wireless network

In the array of mobile communication techniques, the application of a mobile phone combined with television is a new technique under development. As TV program is a real-time video/audio service, in comparison with either traditional video/audio file downloads or network video/audio streams, there are more technical difficulties to be overcome, in particular, how to satisfy the playback functions of TV programs in hand-held device. OpenCore is a multimedia framework, which has recently been widely applied in hand-held devices, but it does not offer functions of mobile TV. To solve this problem, this study incorporates the function of mobile TV into the OpenCore framework, in order to support both formats of TV signals, i.e. DVB-H and DVB-T. The incorporated function, DVB-H/T, has different characteristics, so that users can select TV signals according to their receiving environments and fulfill their needs in TV programs selection.     

Synchronization-Sensitive Frame Estimation: Video Quality Enhancement

Network transmission is liable to errors and data loss. In movie transmission, packets of video frames are subject to loss or even explicit elimination for many reasons including congestion handling and the achievement of higher compression. Not only does the loss of video frames cause significant reduction in video quality, but it could also cause a loss of synchronization between the audio and video streams. If not corrected, this cumulative loss can seriously degrade the motion picture's quality beyond viewers' tolerance. In this paper, we study and classify the effect of audio-video de-synchronization. Afterwards, we develop and examine the performance and appropriateness of the application of many client-based techniques in the estimation of lost frames using the existing received frames, without the need for retransmissions or error control information. The estimated frames are injected at their appropriate locations in the movie stream to restore the loss. The objective is to enhance video quality by finding a very close estimate to the original frames at a suitable computation cost, and to contribute to the restoration of synchronization within the tolerance level of viewers.     

Bus and Buffer Usage in In-Home Digital Networks: Applying the Dantzig–Wolfe Decomposition

In an in-home digital network several data streams (audio, video) may run simultaneously over a shared communication device, e.g. a bus. The burstiness of a data stream can be reduced by buffering data at the sending and receiving side, thereby allowing a lower bus share allocation for the stream. In this paper we present an algorithm that determines how much of the bus capacity and buffer space should be allocated to each stream, in order to have a feasible transmission schedule for each stream. Furthermore, the algorithm determines a transmission schedule for each stream, indicating how much data is transmitted over time. We model the problem as a linear program and apply a Dantzig–Wolfe decomposition such that the multiple-stream problem can be solved by repeatedly solving single-stream problems. For these single-stream problems we briefly describe efficient algorithms to solve them.     

Using high-level models for the creation of staged participatory multimedia events on TV

Broadcasted television shows are becoming more interactive. Some broadcast TV shows allow even home viewers without professional equipment to be part of them. In this paper we present an approach that takes this concept even further. In the proposed kind of participation television viewers will not only participate in the show through interaction or video streams, but also be able to create and host their own show. The core of the presented approach consists of the use of high-level models to describe the different aspects of the television show, and a generic runtime environment. This paper discusses this type of participation television, Staged Participatory Multimedia Events, and the supporting runtime environment in more detail. It also introduces the tool and the models that are used to support graphical creation of the structure and appearance of Staged Participatory Multimedia Events.     

An alternative paradigm for scalable on-demand applications:evaluating and deploying the Interactive Multimedia Jukebox

Straightforward, one-way delivery of audio/video through television sets has existed for many decades. In the 1980s, new services like pay-per-view and video-on-demand were touted as the “killer applications” for interactive TV. However, the hype quickly died away, leaving only hard technical problems and costly systems. As an alternative, we propose a new jukebox paradigm offering flexibility in how programs are requested and scheduled for playout. The jukebox scheduling paradigm offers flexibility ranging from complete viewer control (true video-on-demand), to complete service provider control (traditional broadcast TV). We first describe our proposed jukebox paradigm and relate it to other on-demand paradigms. We also describe several critical research issues, including the one-to-many delivery of content, program scheduling policies, server location, and the provision of advanced services like VCR-style interactivity and advanced reservations. In addition, we present our implementation of a jukebox-based service called the Interactive Multimedia Jukebox (IMJ). The IMJ provides scheduling via the World Wide Web (WWW) and content delivery via the Multicast Backbone (MBone). For the IMJ, we present usage statistics collected during the past couple of years. Furthermore, using this data and a simulation environment, we show that jukebox systems have the potential to scale to very large numbers of viewers     

Community Streaming With Interactive Visual Overlays: System and Optimization

Community streaming is an enhanced form of joint content viewing where a sense of community is reinforced by the addition of interactive visual overlays, controlled in real-time by viewers, on top of a shared video stream. As a concrete example, we describe a community video system called ECHO, where personalized avatars are overlaid on top of a real-time encoded video stream of an Internet game for multicast consumption. Recognizing that only the visual overlays are generated live, we propose schemes that encode and schedule the live and non-live portions of the overlaid video separately in order to exploit the difference in delay sensitivity of the two, leading to video streams that contain two sub-streams with different delay constraints. We show that, in the known channel case, a low complexity ldquoearliest deadline firstrdquo packet scheduling algorithm minimizes receiver buffer delay. We also analyze the case where multiple streams are multiplexed, which allows us to quantify the potential gains of allowing different delay constraints for different sub-streams. We show that a ldquowater fillingrdquo strategy maximizes the total number of streams that can be supported. Simulation results show that the bandwidth necessary to maintain low-latency for visual overlays is reduced by about 40% when our proposed sub-stream approach is used. For multiplexing of multiple streams, our approach can increase the number of supported streams (e.g., a 30% increase when around ten streams are multiplexed).     

GeoHbbTV: A framework for the development and evaluation of geographic interactive TV contents

Synchronizing TV contents with applications is a topic that has gained much interest during the last years. Reaching the viewers through various channels (TV, web, mobile devices, etc.) has shown to be a means for increasing the audience. Related to the above, the hybrid TV standard HbbTV (Hybrid Broadcast Broadband TV) synchronizes the broadcast of video and audio with applications that may be delivered through either the broadcast channel or a broadband network. Thus, HbbTV applications may be developed to provide contextual information for emitted TV shows and advertisements. This paper reports on the integration of the automatic generation of geographic focus of text content with interactive TV. In particular it describes a framework for the incorporation of geographic context to TV shows and its visualization through HbbTV. To achieve this, geographic named entities are first extracted from the available subtitles and next the spatial extension of those entities is used for the production of context maps. An evaluation strategy has been devised and used to test alternative prototype implementations for TV newscast in Spanish language. Finally, to go beyond the initial solution proposed, some challenges for future research are also discussed.     

基于AVS的嵌入式音视频同步方法

音视频同步是数字电视广播和多媒体通信等应用的关键技术。该文提出一种基于AVS并结合嵌入技术的音视频同步方法。将压缩音频数据嵌入AVS视频编码系统,保证传输或存储、接收端解码与播放过程中的音视频始终同步。实验结果表明,该方法实现了音视频完全同步,能减小用于同步的开销。     

Real-time communication for Kinect-based telerehabilitation

As chronic diseases and demographic changes alter the trends in world population, more pressure is put on health management to deal efficiently with a new scenario of longer life expectancy and chronic disabilities. Advances in telecommunication technology and miniaturization of sensors in combination with medical information technology provide a viable solution to reduce healthcare costs and deliver remote medical services through connected devices. Although remote consultation via video-conferencing has been well established, many of the chronic or long-term musculoskeletal conditions require pro-active management and therapy. There is a need to develop more advanced interactive telerehabilitation systems that support real-time remote delivery of physical therapy sessions into patients’ homes. In this paper, we introduce KinectRTC, an innovative framework that can be used for Kinect-based telerehabilitation with efficient real-time transmission of video, audio, depth, and skeletal data. By taking advantage of the Web Real-Time Communication (WebRTC) technology, the proposed framework is able to manage video and audio streams based on the state of the network and the available bandwidth to guarantee the real-time performance of the communication.     

A Multimodal Scheme for Program Segmentation and Representation in Broadcast Video Streams

With the advance of digital video recording and playback systems, the request for efficiently managing recorded TV video programs is evident so that users can readily locate and browse their favorite programs. In this paper, we propose a multimodal scheme to segment and represent TV video streams. The scheme aims to recover the temporal and structural characteristics of TV programs with visual, auditory, and textual information. In terms of visual cues, we develop a novel concept named program-oriented informative images (POIM) to identify the candidate points correlated with the boundaries of individual programs. For audio cues, a multiscale Kullback-Leibler (K-L) distance is proposed to locate audio scene changes (ASC), and accordingly ASC is aligned with video scene changes to represent candidate boundaries of programs. In addition, latent semantic analysis (LSA) is adopted to calculate the textual content similarity (TCS) between shots to model the inter-program similarity and intra-program dissimilarity in terms of speech content. Finally, we fuse the multimodal features of POIM, ASC, and TCS to detect the boundaries of programs including individual commercials (spots). Towards effective program guide and attracting content browsing, we propose a multimodal representation of individual programs by using POIM images, key frames, and textual keywords in a summarization manner. Extensive experiments are carried out over an open benchmarking dataset TRECVID 2005 corpus and promising results have been achieved. Compared with the electronic program guide (EPG), our solution provides a more generic approach to determine the exact boundaries of diverse TV programs even including dramatic spots.     

Receiver-initiated communication with ST-II

As part of its work on the Heidelberg Transport System (HeiTS) for multimedia communication, the IBM European Networking Center (ENC) has implemented the Experimental Internet Stream Protocol, Version 2 (ST-II). ST-II is a connection-oriented internetworking protocol for multiple-destination communication of real-time data such as digital audio and video. Although ST-II fits well into typical multimedia application scenarios, some functions are still missing No mechanisms are provided for receiver-initiated communication, allowing receivers to join streams, specify their quality of service (QOS) requirements, or initiate stream establishment and resource reservation. In this paper, we present some extensions to tile protocol that make it fulfill the receivers' needs move adequately. Our approach allows intermediate nodes to execute stream management functions on behalf of the sender. This way, the protocol scales better with the number of receivers per stream, as the origin does not need to track every target.     

多模态体育视频语义分析

以足球运动为例提出了一种体育视频语义结构,并提出相应的语义分析框架。视频被分解为纯视频流和音频流两种模态,每种模态均可依次提取和综合出低层内容和中层内容。视频流可根据低层（物理）内容分割为物理镜头,然后根据特定的中间层内容可以确定为语法镜头。音频也可以在物理特征的基础上形成有意义的中间层内容,如解说员兴奋时的声音。最后,根据视频流和音频流的中间层内容,按照足球比赛转播的规律,分析出比赛中的精彩事件,并选取相关的镜头作为反映此事件的序列组合。     

车载主机与智能手机互动系统设计

通过无线和有线结合的方式将智能手机与车载主机连接起来,智能手机的音频/视频通过 MHL 或者HDMI接口输出,经过转换后传输到车载主机,车载主机显示手机屏幕和播放多媒体声音。车载主机后台运行一个监控鼠标事件的服务器程序,捕获用户在车载主机屏幕上的触摸事件和坐标,并经过蓝牙或 WiFi 发送给手机客户端,客户端模拟对应的鼠标事件,实现用户在车载主机屏幕操作手机的应用程序,运行手机上的应用程序,实现车载主机与智能手机的互动。     

A robust method for estimating synchronization and delay of audio and video for communication services

One of the main contributions to the quality of experience in streaming services or in two-way communication of audio and video applications is synchronization. This has been shown in several studies and experiments but methods to measure synchronization are less frequent, especially for situations without internal access to the application and independent of platform and device. In this paper we present a method for measuring synchronization skewness as well as delay for audio and video. The solution incorporates audio and video reference streams, where audio and video frames are marked with frame numbers which are decoded on the receiver side to enable calculation of synchronization and delay. The method has been verified in a two-way communication application in a transparent network with and without inserting known delays, as well as in a network with 5 and 10 % packet loss levels. The method can be used for both streaming and two-way communication services, both with and without access to the internal structures, and enables measurements of applications running on e.g. smartphones, tablets, and laptops under various conditions.     

Audiovisual diarization of people in video content

Audio-Visual People Diarization (AVPD) is an original framework that simultaneously improves audio, video, and audiovisual diarization results. Following a literature review of people diarization for both audio and video content and their limitations, which includes our own contributions, we describe a proposed method for associating both audio and video information by using co-occurrence matrices and present experiments which were conducted on a corpus containing TV news, TV debates, and movies. Results show the effectiveness of the overall diarization system and confirm the gains audio information can bring to video indexing and vice versa.     

Digital TV in the convergent environment

The growth of digital TV heralds a seamless convergence of media, telecommunications, and information technology, offering viewers increasingly exciting, informative interactive programming. In countries such as the UK where digital TV services have become extensively available, the ground is prepared for a seamless convergence of media, telecommunications, and information technology interactivity is provided mainly by a combination of local storage (which is now becoming vast) and the Public Switched Telephone Network (PSTN), including mobile access. The industry is also investigating other return channels, such as those using satellites or the terrestrial TV antenna systems. Thus digital TV is a key element in the communications revolution. Where there is rapid growth in the number of homes with digital TV, there is likely to be a more favorable climate for the development of electronic commerce