Streaming of scalable h.264 videos over the Internet

To investigate the benefits of scalable codecs in the case of rate adaptation problem, a streaming system for scalable H.264 videos has been implemented. The system considers congestion level in the network and buffer status at the client during adaptation process. The rate adaptation algorithm is content adaptive. It selects an appropriate substream from the video file by taking into account the motion dynamics of video. The performance of the system has been tested under congestion-free and congestion scenarios. The performance results indicate that the system reacts to congestion properly and can be used for Internet video streaming where losses occur unpredictably.

Optimized scalable cache management for video streaming system

An important research issue in video streaming is how to efficiently utilize the network resources to provide clients instant access to multiple video objects. Caching strategy and transmission scheme are the two essential points inside the video streaming framework. Recent research efforts on them are not sufficient due to their inflexible support for scalable encoded video streams and heterogeneous requests from clients. In this paper, we propose an optimized caching strategy (OCS) and a scalable transmission scheme (STS) for scalable coded video streaming. By exploring the characteristics of video streaming workload and system design objectives, OCS and STS work efficiently to minimize both network bandwidth cost and user access latency. Firstly, we analyze the caching problem for the proxy-assisted video streaming system and derive a maneuverable caching scenario. Secondly, we develop an efficient transmission scheme for scalable coded videos. Thirdly, we formulate a multi-objective optimization model with closed-form expressions to obtain the optimized caching strategy. Finally, with designed algorithms, an excellent compromise between two competing objectives (minimizing the bandwidth cost and the access latency) is achieved. We start our evaluation by studying the optimized caching strategy for a single video object. Then we apply the strategy to multiple video objects and illustrate the tradeoff between the optimization objectives. Our evaluation results show that compared with other caching strategies, the proposed optimized scalable caching strategy can achieve a significant reduction in bandwidth cost with even a small proxy cache size. Meanwhile, the best performance (in terms of bandwidth cost) is obtained together with the proposed scalable batch-patching transmission scheme.

Periodic properties of user mobility and access-point popularity

Understanding user mobility and its effect on access points (APs) is important in designing location-aware systems and wireless networks. Although various studies of wireless networks have provided useful insights, it is hard to apply them to other situations. Here we present a general methodology for extracting mobility information from wireless network traces, and for classifying mobile users and APs. We used the Fourier transform to reveal important periods and chose the two strongest periods to serve as parameters to a classification system based on Bayes’ theory. Analysis of 1-month traces shows that while a daily pattern is common among both users and APs, a weekly pattern is common only for APs. Analysis of 1-year traces revealed that both user mobility and AP popularity depend on the academic calendar. By plotting the classes of APs on our campus map, we discovered that their periodic behavior depends on their proximity to other APs.

Dynamically adjusting MPEG4 video streams based on network bandwidth

In the paper, we try to find a method that can service more users in a video-on-demand (VoD) system, based on MPEG-4 object streams. The characteristics of object segmentation made on MPEG-4 videos can be utilized to reduce re-transmission of the same objects, and then the saved bandwidth can be used to service more users. However, some thresholds must be analyzed first to maintain the acceptable quality of services (QoS) requested by users, when reducing unnecessary object transmission on one side. Thus, according to the defined thresholds, we propose a dynamically adjusting algorithm to coordinate the object streams between the server and clients. The server not only allocates network bandwidth, but also adjusts ever-allocated QoS appropriately using a degrading and upgrading strategy, based on the current network status. Lastly, through the simulation, we found that our method has better performance than the other three methods owing to its flexibility to the network status.

P-chaining: a practical VoD service scheme autonomically handling interactive operations

Providing scalable VoD streaming services has recently become a hot issue, and many approaches have been proposed. Because video streaming services through the Internet are widely used, the need to support VCR operations also increases. However, there are few approaches to supporting VCR operations on the Internet. We propose a service scheme based on chaining, in which clients as well as the server provide streaming services. In the proposed scheme, services are provided by unicast and managed locally using node lists. In addition, our scheme can support frequent VCR operations without incurring significant overhead in the server workload. We have evaluated our scheme through simulation with real traces from a content distribution network (CDN) company and with various parameters. The results show that the proposed scheme reduces server workload significantly. The results also verify that frequent VCR operations can be served smoothly without causing too much overhead.

Caching collaboration and cache allocation in peer-to-peer video systems

Providing scalable video services in a peer-to-peer (P2P) environment is challenging. Since videos are typically large and require high communication bandwidth for delivery, many peers may be unwilling to cache them in whole to serve others. In this paper, we address two fundamental research problems in providing scalable P2P video services: (1) how a host can find enough video pieces, which may scatter among the whole system, to assemble a complete video; and (2) given a limited buffer size, what part of a video a host should cache and what existing data should be expunged to make necessary space. We address these problems with two new ideas: Cell caching collaboration and Controlled Inverse Proportional (CIP) cache allocation. The Cell concept allows cost-effective caching collaboration in a fully distributed environment and can dramatically reduce video lookup cost. On the other hand, CIP cache allocation challenges the conventional caching wisdom by caching unpopular videos in higher priority. Our approach allows the system to retain many copies of popular videos to avoid creating hot spots and at the same time, prevent unpopular videos from being quickly evicted from the system. We have implemented a Gnutella-like simulation network and use it as a testbed to evaluate the proposed technique. Our extensive study shows convincingly the performance advantage of the new scheme.

A scalable signature scheme for video authentication

This paper addresses the problem of ensuring the integrity of a digital video and presents a scalable signature scheme for video authentication based on cryptographic secret sharing. The proposed method detects spatial cropping and temporal jittering in a video, yet is robust against frame dropping in the streaming video scenario. In our scheme, the authentication signature is compact and independent of the size of the video. Given a video, we identify the key frames based on differential energy between the frames. Considering video frames as shares, we compute the corresponding secret at three hierarchical levels. The master secret is used as digital signature to authenticate the video. The proposed signature scheme is scalable to three hierarchical levels of signature computation based on the needs of different scenarios. We provide extensive experimental results to show the utility of our technique in three different scenarios—streaming video, video identification and face tampering.

Multicast video-on-demand service in an enterprise network with client-assisted patching

Multicast Video-on-Demand (VoD) systems are scalable and  cheap-to-operate. In such systems, a single stream is shared by a batch of common user requests. In this research, we propose multicast communication technique in an Enterprise Network where multimedia data are stored in distributed servers. We consider a novel patching scheme called Client-Assisted Patching where clients’ buffer of a multicast group can be used to patch the missing portion of the clients who will request the same movie shortly. This scheme significantly reduces the server load without requiring larger client cache space than conventional patching schemes. Clients can join an existing multicast session without waiting for the next available server stream which reduces service latency. Moreover, the system is more scalable and cost effective than similar existing systems. Our simulation experiment confirms all these claims.

Interacting with mobile services: an evaluation of camera-phones and visual tags

We present a study of using camera-phones and visual-tags to access mobile services. Firstly, a user-experience study is described in which participants were both observed learning to interact with a prototype mobile service and interviewed about their experiences. Secondly, a pointing-device task is presented in which quantitative data was gathered regarding the speed and accuracy with which participants aimed and clicked on visual-tags using camera-phones. We found that participants’ attitudes to visual-tag-based applications were broadly positive, although they had several important reservations about camera-phone technology more generally. Data from our pointing-device task demonstrated that novice users were able to aim and click on visual-tags quickly (well under 3 s per pointing-device trial on average) and accurately (almost all meeting our defined speed/accuracy tradeoff of 6% error-rate). Based on our findings, design lessons for camera-phone and visual-tag applications are presented.

---

Real time video frames allocation in mobile networks using cooperative pre-fetching

In this paper, we study the bandwidth allocation problem for serving video requests in a mobile real-time video player system. One of the main issues to maintain the quality of services (QoS) in mobile video playback is to ensure sufficient number of video frames available at the client side while a video is being played. However, due to the limited bandwidth of a mobile network and variable workload streaming video, this is not easy to achieve in practice. In addition, the communication link between mobile clients and a video server is subject to frequent errors and temporary disconnection. In this paper, we propose to use the notion of buffered bandwidth in addition to the network bandwidth to support real-time playback of videos at mobile clients. Based on this, we designed a bandwidth allocation scheme called Cooperative Pre-Fetching (CP) in which the amount of bandwidth to be allocated to serve a video request depends on the current buffer level of the video at the client relative to the target buffer level of the client. In determining the target buffer level, we consider the errors in communication to the client as well as the other clients who are concurrently served by the system. The buffered video frames at the clients are then used to minimize the impact of error in communications on the overall QoS of video playbacks in the whole system.

Toward cinematizing our daily lives

We introduce a cinematographic video production system to create movie-like attractive footage from our indoor daily life. Since the system is designed for ordinary users in non-studio environments, it is composed of standard hardware components, provides a simple interface, and works in near real-time of 5 ~ 6 frames/sec. The proposed system reconstructs a visual hull from acquired multiple videos and then generates final videos from the model by referring to the camera shots used in film-making. The proposed method utilizes “Reliability” to compensate for errors that may have occurred in non-studio environments and to produce the most natural scene from the reconstructed model. By using a virtual camera control system, even non-experts can easily convert the 3D model to movies that look as if they were created by experienced filmmakers.

Adaptive training of video sets for image recognition on mobile phones

We present an enhancement towards adaptive video training for PhoneGuide, a digital museum guidance system for ordinary camera-equipped mobile phones. It enables museum visitors to identify exhibits by capturing photos of them. In this article, a combined solution of object recognition and pervasive tracking is extended to a client–server-system for improving data acquisition and for supporting scale-invariant object recognition. A static as well as a dynamic training technique are presented that preprocess the collected object data differently and apply two types of neural networks (NN) for classification. Furthermore, the system enables a temporal adaptation for ensuring a continuous data acquisition to improve the recognition rate over time. A formal field experiment reveals current recognition rates and indicates the practicability of both methods under realistic conditions in a museum.

---

OCS: An effective caching scheme for video streaming on overlay networks

Video streaming is vital for many important applications such as distance learning, digital video libraries, and movie-on-demand. Since video streaming requires significant server and networking resources, caching has been used to reduce the demand on these resources. In this paper, we propose a novel collaboration scheme for video caching on overlay networks, called Overlay Caching Scheme (OCS), to further minimize service delays and loads placed on an overlay network for video streaming applications. OCS is not a centralized nor a hierarchical collaborative scheme. Despite its design simplicity, OCS effectively uses an aggregate storage space and capability of distributed overlay nodes to cache popular videos and serve nearby clients. Moreover, OCS is light-weight and adaptive to clients’ locations and request patterns. We also investigate other video caching techniques for overlay networks including both collaborative and non-collaborative ones. Compared with these techniques on topologies inspired from actual networks, OCS offers extremely low average service delays and approximately half the server load. OCS also offers smaller network load in most cases in our study.

A framework for development and evaluation of a dynamic subchannel allocation scheme in an OFDMA system

This paper presents a framework for allocating radio resources to the Access Points (APs) introducing an Access Point Controller (APC). Radio resources can be either time slots or subchannels. The APC assigns subchannels to the APs using a dynamic subchannel allocation scheme. The developed framework evaluates the dynamic subchannel allocation scheme for a downlink multicellular Orthogonal Frequency Division Multiple Access (OFDMA) system. In the considered system, each AP and the associated Mobile Terminals (MTs) are not operating on a frequency channel with fixed bandwidth, rather the channel bandwidth for each AP is dynamically adapted according to the traffic load. The subchannels assignment procedure is based on quality estimations due to the interference measurements and the current traffic load. The traffic load estimation is realized with the measurement of the utilization of the assigned radio resources. The reuse partitioning for the radio resources is done by estimating mutual Signal to Interference Ratio (SIR) of the APs. The developed dynamic subchannel allocation ensures Quality of Service (QoS), better traffic adaptability, and higher spectrum efficiency with less computational complexity.

Detecting repeats for video structuring

Television daily produces massive amounts of videos. Digital video is unfortunately an unstructured document in which it is very difficult to find any information. Television streams have however a strong and stable but hidden structure that we want to discover by detecting repeating objects in the video stream. This paper shows that television streams are actually highly redundant and that detecting repeats can be an effective way to detect the underlying structure of the video. A method for detecting these repetitions is presented here with an emphasis on the efficiency of the search in a large video corpus. Very good results are obtained both in terms of effectiveness (98% in recall and precision) as well as efficiency since one day of video is queried against a 3 weeks dataset in only 1 s.

Security and privacy for multimedia database management systems

This paper describes security and privacy issues for multimedia database management systems. Multimedia data includes text, images, audio and video. It describes access control for multimedia database management systems and describes security policies and security architectures for such systems. Privacy problems that result from multimedia data mining are also discussed.

Real time 3D face and facial feature tracking

Detecting and tracking human faces in video sequences is useful in a number of applications such as gesture recognition and human-machine interaction. In this paper, we show that online appearance models (holistic approaches) can be used for simultaneously tracking the head, the lips, the eyebrows, and the eyelids in monocular video sequences. Unlike previous approaches to eyelid tracking, we show that the online appearance models can be used for this purpose. Neither color information nor intensity edges are used by our proposed approach. More precisely, we show how the classical appearance-based trackers can be upgraded in order to deal with fast eyelid movements. The proposed eyelid tracking is made robust by avoiding eye feature extraction. Experiments on real videos show the usefulness of the proposed tracking schemes as well as their enhancement to our previous approach.

Connecting the family with awareness systems

Awareness systems have attracted significant research interest for their potential to support interpersonal relationships. Investigations of awareness systems for the domestic environment have suggested that such systems can help individuals stay in touch with dear friends or family and provide affective benefits to their users. Our research provides empirical evidence to refine and substantiate such suggestions. We report our experience with designing and evaluating the ASTRA awareness system, for connecting households and mobile family members. We introduce the concept of connectedness and its measurement through the Affective Benefits and Costs of communication questionnaire (ABC-Q). We inform results that testify the benefits of sharing experiences at the moment they happen without interrupting potential receivers. Finally, we document the role that lightweight, picture-based communication can play in the range of communication media available.

---

P2Cast: peer-to-peer patching for video on demand service

Providing video on demand (VoD) service over the Internet in a scalable way is a challenging problem. In this paper, we propose P2Cast—an architecture that uses a peer-to-peer approach to cooperatively stream video using patching techniques, while only relying on unicast connections among peers. We address the following two key technical issues in P2Cast: (1) constructing an application overlay appropriate for streaming; and (2) providing continuous stream playback (without glitches) in the face of disruption from an early departing client. Our simulation experiments show that P2Cast can serve many more clients than traditional client-server unicast service, and that it generally out-performs multicast-based patching if clients can cache more than 10% of a stream’s initial portion. We handle disruptions by delaying the start of playback and applying the shifted forwarding technique. The threshold in P2Cast, i.e., the length of time during which arriving clients form a single session, can serve as a “knob” to adjust the balance between the scalability and the clients’ viewing quality.

Online updating appearance generative mixture model for meanshift tracking

This paper proposes an appearance generative mixture model based on key frames for meanshift tracking. Meanshift tracking algorithm tracks an object by maximizing the similarity between the histogram in tracking window and a static histogram acquired at the beginning of tracking. The tracking therefore could fail if the appearance of the object varies substantially. In this paper, we assume the key appearances of the object can be acquired before tracking and the manifold of the object appearance can be approximated by piece-wise linear combination of these key appearances in histogram space. The generative process is described by a Bayesian graphical model. An Online EM algorithm is proposed to estimate the model parameters from the observed histogram in the tracking window and to update the appearance histogram. We applied this approach to track human head motion and to infer the head pose simultaneously in videos. Experiments verify that our online histogram generative model constrained by key appearance histograms alleviates the drifting problem often encountered in tracking with online updating, that the enhanced meanshift algorithm is capable of tracking object of varying appearances more robustly and accurately, and that our tracking algorithm can infer additional information such as the object poses. Electronic supplementary material  The online version of this article (doi:) contains supplementary material, which is available to authorized users.

---