MedSMan: a live multimedia stream querying system

Querying live media streams is a challenging problem that is becoming an essential requirement in a growing number of applications. Research in multimedia information systems has addressed and made good progress in dealing with archived data. Meanwhile, research in stream databases has received significant attention for querying alphanumeric symbolic streams. The lack of a data model capable of representing different multimedia data in a declarative way, hiding the media heterogeneity and providing reasonable abstractions for querying live multimedia streams poses the challenge of how to make the best use of data in video, audio and other media sources for various applications. In this paper we propose a system that enables directly capturing media streams from sensors and automatically generating more meaningful feature streams that can be queried by a data stream processor. The system provides an effective combination between extendible digital processing techniques and general data stream management research. Together with other query techniques developed in related data stream management streams, our system can be used in those application areas where multifarious live media senors are deployed for surveillance, disaster response, live conferencing, telepresence, etc.

MinGPU: a minimum GPU library for computer vision

In the field of computer vision, it is becoming increasingly popular to implement algorithms, in sections or in their entirety, on a graphics processing unit (GPU). This is due to the superior speed GPUs offer compared to CPUs. In this paper, we present a GPU library, MinGPU, which contains all of the necessary functions to convert an existing CPU code to GPU. We have created GPU implementations of several well known computer vision algorithms, including the homography transformation between two 3D views. We provide timing charts and show that our MinGPU implementation of homography transformations performs approximately 600 times faster than its C++ CPU implementation.

Distributed Video Coding using Turbo Trellis Coded Modulation

Distributed Video Coding (DVC) has been proposed for increasingly new application domains. This rise is apparently motivated by the very attractive features of its flexibility for building very low cost video encoders and the very high built-in error resilience when applied over noisy communication channels. Yet, the compression efficiency of DVC is notably lagging behind the state-of-the-art in video coding and compression, H.264/AVC in particular. In this context, a novel coding solution for DVC is presented in this paper, which promises to improve its rate-distortion (RD) performance towards the state-of-the-art. Here, Turbo Trellis Coded Modulation (TTCM), with its attractive coding gain in channel coding, is utilized and its resultant impact in both pixel domain and transform domain DVC framework is discussed herein. Simulations have shown a significant gain in the RD performance when compared with the state-of-the-art Turbo coding based DVC implementations.

EPCRTT-based smoothing and multiplexing of VBR video traffic

Applying video smoothing techniques to real-time video transmission can significantly reduce the peak rate and rate variability of compressed video streams. Moreover, statistical multiplexing of the smoothed traffic can substantially improve network utilization. In this paper we propose a new smoothing scheme, which exploits statistical multiplexing gain that can be obtained after smoothing of individual video streams. We present a new bandwidth allocation algorithm that allows for responsive interactivity. The local re-smoothing algorithm is carried out using an iterative process. In the proposed scheme the smoothed video streams are divided into fixed intervals and then a new transmission schedule for each interval is calculated. The problem of applying an optimal transmission schedule for aggregated smoothing video streams is shown to be NP-hard problem. Partitioning the whole stream into sections enables parallel processing of the smoothing algorithm in real-time before transmission. This approach allows partial transmission of the multiplexed stream while smoothing other intervals. The simulation results show a significant reduction in peak rate and rate variability of the aggregated stream, compared to the non-smoothing case. Therefore the proposed scheme allows us to increase the number of simultanusally-served video streams.

Fast adaptive termination mode selection for H.264 scalable video coding

Although scalable video coding can achieve coding efficiencies comparable with single layer video coding, its computational complexity is higher due to its additional inter-layer prediction process. This paper presents a fast adaptive termination algorithm for mode selection to increase its computation speed while attempting to maintain its coding efficiency. The developed algorithm consists of the following three main steps which are applied not only to the enhancement layer but also to the base layer: a prediction step based on neighboring macroblocks, a first round check step, and a second round check step or refinement if failure occurs during the first round check. Comparison results with the existing algorithms are provided. The results obtained on various video sequences show that the introduced algorithm achieves about one-third reduction in the computation speed while generating more or less the same video quality.

A Necessary and Sufficient Contractivity Condition for the Fractal Transform Operator

This paper revisits the concept of fractal image coding and the contractivity conditions of the fractal transform operator. All such existing conditions are only sufficient. This paper formulates a necessary and sufficient condition for the contractivity of the fractal transform operator associated to a fractal code. Furthermore, analytical results on the convergence of the fractal image decoding will be derived.

Multimodal support to group dynamics

The complexity of group dynamics occurring in small group interactions often hinders the performance of teams. The availability of rich multimodal information about what is going on during the meeting makes it possible to explore the possibility of providing support to dysfunctional teams from facilitation to training sessions addressing both the individuals and the group as a whole. A necessary step in this direction is that of capturing and understanding group dynamics. In this paper, we discuss a particular scenario, in which meeting participants receive multimedia feedback on their relational behaviour, as a first step towards increasing self-awareness. We describe the background and the motivation for a coding scheme for annotating meeting recordings partially inspired by the Bales’ Interaction Process Analysis. This coding scheme was aimed at identifying suitable observable behavioural sequences. The study is complemented with an experimental investigation on the acceptability of such a service.

---

Adaptive real-time image processing exploiting two dimensional reconfigurable architecture

Fine grained reconfigurable architectures, like Xilinx field programmable gate arrays (FPGAs) provide a high flexibility through runtime re-programming, called dynamic and partial reconfiguration. This feature allows for runtime adaptation of the system architecture and behavior configured on the FPGA. The exploitation of this feature enables to load video image processing algorithms on-demand in order to adapt the configuration in correspondence to the changing requirements of the application depending on the image content. For high resolution sensor images, this novel computing paradigm can provide a huge benefit in power reduction and performance gain for actual and future embedded electronic systems. This paper presents a two dimensional system approach exploiting dynamic and partial reconfiguration in order to adapt the system architecture to the actual requirements of image processing applications. The methodology of runtime reconfiguration can be exploited beneficially for highly adaptive multiprocessor systems. Such systems, different from the traditional static approach for multi- and many-core architectures have the advantage, for providing computational performance directly linked to the requirements of the application. The architecture presented in this paper allows for adapting the processing elements as well as the communication infrastructure which is a novel 2D switch-based Network-on-Chip. The presented approach follows and extends the actual trend in computer science of using many- and multi-core processors for bridging the gap between required computation performance for future application in the field of image processing.

Computing the Discrete Fourier Transform on a Hexagonal Lattice

The computation of the Discrete Fourier Transform for a general lattice in ℝ ^d can be reduced to the computation of the standard 1-dimensional Discrete Fourier Transform. We provide a mathematically rigorous but simple treatment of this procedure and apply it to the DFT on the hexagonal lattice.

Reconfigurable computing: design methodology and hardware tasks scheduling for real-time image processing

Technology evolution makes possible the integration of heterogeneous components as programmable elements (processors), hardware dedicated blocks, hierarchical memories and buses. Furthermore, an optimized reconfigurable logic core embedded within a System-on-Chip will associate the performances of dedicated architecture and the flexibility of programmable ones. In order to increase performances, some of the applications are carried out in hardware, using dynamically reconfigurable logic, rather than software, using programmable elements. This approach offers a suitable hardware support to design malleable systems able to adapt themselves to a specific application. This article makes a synthesis of the Ardoise project. The first objective of Ardoise project was to design and to produce a dynamically reconfigurable platform based on commercial FPGAs. The concept of dynamically reconfigurable architecture depends partially on new design methodologies elaboration as well as on the programming environment. The platform architecture was designed to be suitable for real-time image processing. The article outlines mainly the Ardoise tools aspect: development environment and real-time management of the hardware tasks. The proposed methodology is based on a dynamic management of tasks according to an application scenario written using C++ language.

Towards building large scale live media streaming framework for a U-city

This paper proposes camera and media stream management techniques at the middleware level for implementing a U-City (ubiquitous city). The study focuses on overcoming the difficulties associated with developing middleware capable of processing and streaming multimedia data from a large number of cameras by expanding the traditional media processing technology. The content of the study can be classified into two main categories: One is a camera array management technique that involves the middleware-level framework and protocol for managing the camera array. The other is the media stream management technique for effective delivery management and processing of the multimedia streams from the camera array.

A time-predictable VLIW processor and its compiler support

Time predictability is an important requirement for real-time embedded application domains such as automotive, air transportation, and multimedia processing. However, the architectural design of modern microprocessors mainly concentrates on improving the average-case performance, which can significantly compromise the time predictability and can make accurate worst-case performance analysis extremely difficult if not impossible. This paper studies the time predictability of VLIW (Very Long Instruction Word) processors and its compiler support. We analyze the impediments to time predictability for VLIW processors and propose compiler-based techniques to address these problems with minimal disturbance on the VLIW hardware design. The VLIW compiler is enhanced to support full if conversion, hyperblock scheduling, and intra-block nop insertion to enable efficient WCET (Worst Case Execution Time) analysis for VLIW processors. Our experiments indicate that the time-predictability of VLIW processor can be improved significantly.

Detecting repeats for video structuring

Television daily produces massive amounts of videos. Digital video is unfortunately an unstructured document in which it is very difficult to find any information. Television streams have however a strong and stable but hidden structure that we want to discover by detecting repeating objects in the video stream. This paper shows that television streams are actually highly redundant and that detecting repeats can be an effective way to detect the underlying structure of the video. A method for detecting these repetitions is presented here with an emphasis on the efficiency of the search in a large video corpus. Very good results are obtained both in terms of effectiveness (98% in recall and precision) as well as efficiency since one day of video is queried against a 3 weeks dataset in only 1 s.

Distributed video coding based on part intracoding and soft side information estimation

Recently, distributed source coding (DSC) has been proposed to implement source compression by exploiting source statistics at the decoder only, which enables low-complexity video coding. However, to date, the video codecs based on DSC have been unable to compress as efficiently as traditional predictive video codecs, such as H.264. So, new techniques have to be investigated to improve the performance of the distributed video coding scheme for practical applications. In this paper, I propose a novel distributed video coding scheme based on part intracoding and soft side information estimation. Firstly, at the encoder side, to improve the compression performance of distributed video coding system, we divide the video data into strongly correlative data encoded by Slepian–Wolf codec and weakly correlative data encoded by Intracoding codec. Secondly, at the decoder side, to improve the accuracy of side information estimation, a soft side information estimation method is proposed, which is more suitable for video coding due to the non-stationary feature of video data. Our experimental results show that the performance of our coding system is better than that of the traditional distributed video coding system while keeping the simple encoding property. Also the concept of soft side information is a new idea in distributed video coding and will significantly influence the side information estimation method.

Processing Sparse Panoramic Images via Space Variant Operators

The use of omni-directional cameras has become increasingly popular in vision systems for video surveillance and autonomous robot navigation. However, to date most of the research relating to omni-directional cameras has focussed on the design of the camera or the way in which to project the omni-directional image to a panoramic view rather than the processing of such images after capture. Typically images obtained from omni-directional cameras are transformed to sparse panoramic images that are interpolated to obtain a complete panoramic view prior to low level image processing. This interpolation presents a significant computational overhead with respect to real-time vision. We present an efficient design procedure for space variant feature extraction operators that can be applied to a sparse panoramic image and directly processes this sparse image. This paper highlights the reduction of the computational overheads of directly processing images arising from omni-directional cameras through efficient coding and storage, whilst retaining accuracy sufficient for application to real-time robot vision.

Real time data hiding by exploiting the IPCM macroblocks in H.264/AVC streams

A new method for data hiding in H.264/AVC streams is presented. The proposed method exploits the IPCM encoded macroblocks during the intra prediction stage in order to hide the desired data. It is a blind data hiding scheme, i.e. the message can be extracted directly from the encoded stream without the need of the original host video. Moreover, the method exhibits the useful property of reusing the compressed stream for hiding different data numerous times without considerably affecting either the bit-rate or the perceptual quality. This property allows data hiding directly in the compressed stream in real time. The method perfectly suits to covert communication and content authentication applications.

The 2007 IEEE CEC simulated car racing competition

This paper describes the simulated car racing competition that was arranged as part of the 2007 IEEE Congress on Evolutionary Computation. Both the game that was used as the domain for the competition, the controllers submitted as entries to the competition and its results are presented. With this paper, we hope to provide some insight into the efficacy of various computational intelligence methods on a well-defined game task, as well as an example of one way of running a competition. In the process, we provide a set of reference results for those who wish to use the simplerace game to benchmark their own algorithms. The paper is co-authored by the organizers and participants of the competition.

---

Accelerating the multiple reference frames compensation in the H.264 video coder

Using the multiple reference frames compensation in the H264 coder improves the coding efficiency for sequences which contain uncovered backgrounds, repetitive motions and highly textured areas. Unfortunately this technique requires excessive memory and computation resources. In this article, we proposed and implemented a technique based on Markov Random Fields Algorithm relying on robust moving pixel segmentation. By the introduction of this technique, we were able to decrease the number of reference frames from five to three while keeping similar video coding performances. The coding time decreased by 35% and the sequence quality was preserved. After the validation of our idea, we evaluated the processing time of the Markov algorithm on architectures intended for embedded multimedia applications. Both DSP and FPGA implementations were explored. We were able to process 50 frames(128 × 128)/s on the EP1S10 FPGA paltform and 35 frames(128 × 128)/s on the ADSP BF533.

Methods for quantitative usability requirements: a case study on the development of the user interface of a mobile phone

Quantitative usability requirements are a critical but challenging, and hence an often neglected aspect of a usability engineering process. A case study is described where quantitative usability requirements played a key role in the development of a new user interface of a mobile phone. Within the practical constraints of the project, existing methods for determining usability requirements and evaluating the extent to which these are met, could not be applied as such, therefore tailored methods had to be developed. These methods and their applications are discussed.

---