Evolution of 3D mobile games development

3D computer graphics have been an important feature in games development since it was first introduced in the early 80s and there is no doubt that 3D based content is often viewed as more attractive in games than the more abstract 2D graphics. Many games publishers are keen to leverage their success in the console market into the mobile phone platform. However, the resource constraints of mobile phones and the fragmented nature of the environment present considerable challenges for games developers. In this paper we consider some of the current constraints together with current and, probable, future developments both in the software and hardware of mobile phones. As part of this process we benchmark some of the latest and most prevalent software and hardware devices to ascertain both the quality of the graphics produced and the effects upon battery life. Whilst our test results highlight that the current market does indeed present challenges, our research into the future developments highlights the fact we are approaching greater standardization, which will be an important factor for the successful development of 3D mobile games.

---

Video frame rate up conversion under inconsistent camera motion

In this paper, we address the problem of video frame rate up-conversion (FRC) in the compressed domain. FRC is often recognized as video temporal interpolation. This problem is very challenging when targeted for video sequences with inconsistent camera and object motion, such as sports videos. A novel compressed domain motion compensation scheme is presented and applied in this paper, aiming at up-sampling frame rates in sports videos. MPEG-2 encoded motion vectors (MVs) are utilized as inputs in the proposed algorithm. The decoded MVs undergo a cumulative spatiotemporal interpolation. An iterative rejection scheme based on the dense motion vector field (MVF) and the generalized affine motion model is exploited to detect global camera motion. Subsequently, the foreground object separation is performed by additionally examining the temporal consistency of the output of iterative rejections. This consistency check process helps coalesce the resulting foreground blocks and weed out the unqualified blocks. Finally, different compensation strategies for the camera and object motions are applied to interpolate the new frames. Illustrative examples are provided to demonstrate the efficacy of the proposed approach. Experimental results are compared with the popular block and non-block based frame interpolation approaches.

An algorithm to estimate mean vehicle speed from MPEG Skycam video

With low computation cost, motion vectors can be readily extracted from MPEG video streams and processed to estimate vehicle motion speed. A statistical model is proposed to model vehicle speed and noise. In order to achieve high estimation accuracy and also study the limitations of the proposed algorithm, we quantitatively evaluated four parameters used in our algorithm: temporal filter window size T, video resolution R _v (CIF/QCIF), motion vector frame distance m, and video bit-rates. Our experiments showed that the mean vehicle speed can be estimated with high accuracy, up to 85 to 92% by proper spatial and temporal processing. The proposed algorithm is especially suitable for Skycam-based application, where the traditional tracking-based or virtual-loop-based approaches perform poorly because of their requirements of high-resolution images. Although extensive work has been done in extracting motion information directly from MPEG video data in compressed domain, to our best knowledge, this paper is the very first work in which stationary motion (speed) of moving objects can be estimated with high accuracy directly from MPEG motion vectors. Furthermore the proposed method is not limited to vehicle speed estimation by nature and it can be applied to other applications where the stationary motion assumption is satisfied.

MPEG-4 to H.264 transcoding with frame rate reduction

In this paper, a temporal resolution reduction transcoding method that transforms an MPEG-4 video bitstream into an H.264 video bitstream is proposed. The block modes and motion vectors in the MPEG-4 bitstream are utilized in the H.264 encoder for the block mode conversion and motion vector interpolation methods. Four types of motion vector interpolation methods are proposed in order to avoid the use of brute-force motion estimation in H.264. According to the experimental results, the proposed methods achieve a 3∼4 times improvement in the computational complexity compared to the cascade pixel-domain transcoding method, while the PSNR (peak signal to noise ratio) is degraded by 0.2∼0.9 dB depending on the bitrates.

Real time 3D face and facial feature tracking

Detecting and tracking human faces in video sequences is useful in a number of applications such as gesture recognition and human-machine interaction. In this paper, we show that online appearance models (holistic approaches) can be used for simultaneously tracking the head, the lips, the eyebrows, and the eyelids in monocular video sequences. Unlike previous approaches to eyelid tracking, we show that the online appearance models can be used for this purpose. Neither color information nor intensity edges are used by our proposed approach. More precisely, we show how the classical appearance-based trackers can be upgraded in order to deal with fast eyelid movements. The proposed eyelid tracking is made robust by avoiding eye feature extraction. Experiments on real videos show the usefulness of the proposed tracking schemes as well as their enhancement to our previous approach.

Stable Algebraic Surfaces for 3D Object Representation

Linear fitting techniques by implicit algebraic models usually suffer from global stability problems. Ridge regression regularization can be used to improve the stability of algebraic surface fits. In this paper a Euclidean Invariant 3D ridge regression matrix is developed and applied to a particular linear algebraic surface fitting method. Utilization of such a regularization in fitting process makes it possible to globally stabilize 3D object fits with surfaces of any degree. Robustness to noise and moderate levels of occlusion has also been enhanced significantly. Experimental results are presented to verify the improvements in global stability and robustness of the resulting fits.

Real time data hiding by exploiting the IPCM macroblocks in H.264/AVC streams

A new method for data hiding in H.264/AVC streams is presented. The proposed method exploits the IPCM encoded macroblocks during the intra prediction stage in order to hide the desired data. It is a blind data hiding scheme, i.e. the message can be extracted directly from the encoded stream without the need of the original host video. Moreover, the method exhibits the useful property of reusing the compressed stream for hiding different data numerous times without considerably affecting either the bit-rate or the perceptual quality. This property allows data hiding directly in the compressed stream in real time. The method perfectly suits to covert communication and content authentication applications.

A model for adapting 3D graphics based on scalable coding, real-time simplification and remote rendering

Most current multiplayer 3D games can only be played on dedicated platforms, requiring specifically designed content and communication over a predefined network. To overcome these limitations, the OLGA (On-Line GAming) consortium has devised a framework to develop distributive, multiplayer 3D games. Scalability at the level of content, platforms and networks is exploited to achieve the best trade-offs between complexity and quality.