3D C-string: a new spatio-temporal knowledge representation for video database systems

In video database systems, one of the most important methods for discriminating the videos is by using the objects and the perception of spatial and temporal relations that exist between objects in the desired videos. In this paper, we propose a new spatio-temporal knowledge representation called 3D C-string. The knowledge structure of 3D C-string, extended from the 2D C⁺-string, uses the projections of objects to represent spatial and temporal relations between the objects in a video. Moreover, it can keep track of the motions and size changes of the objects in a video. The string generation and video reconstruction algorithms for the 3D C-string representation of video objects are also developed. By introducing the concept of the template objects and nearest former objects, the string generated by the string generation algorithm is unique for a given video and the video reconstructed from a given 3D C-string is unique too. This approach can provide us an easy and efficient way to retrieve, visualize and manipulate video objects in video database systems. Finally, some experiments are performed to show the performance of the proposed algorithms.     

Spatio-temporal composition of distributed multimedia objects forvalue-added networks

The overall process necessary to perform spatial and temporal data composition for a distributed multimedia information system is addressed. With respect to delays introduced through the network, it is found that temporal composition can be most suitably achieved at the workstation. Spatial composition is most effectively performed in a hierarchical fashion as dictated by the availability of system resources. The subsequent composition methodology combines spatial and temporal composition as a network service. Database organizations and data distributions are also investigated, and spatial and temporal composition functions and their composition into the network architecture are discussed. The issue of mapping the composition process onto the network resources as a value-added service is also addressed     

A relational database for non-manipulative representation of solid objects

As the heart of any solid modelling system, much effort has been spent on formulating the data models which represent the shape of a polyhedral solid object within the computer in an accurate, unique and complete manner. Most of this effort has been devoted to make the models manipulateable in terms of computational efficiency, and preserve their semantic integrity. The price paid for achieving these goals is a proliferation of cross-reference data links which, coupled with multiple redundancy, renders the models inefficient for storage or transmission over communication channels. Moreover, the interrogation of these models by non-manipulative applications is usually unsupported functionally, requiring direct accesses to the data, which involves extensive pointer chasing and therefore high familiarity with the manipulative data structure.This paper presents an example relational model as a complementary logical scheme for viewing the shape database that facilitates compact storage and supports non-manipulative query operations through the projection, selection and join operators defined for the relational model, without requiring expert knowledge of the manipulative structure. The flexibility of the relational model, compared with that of the hierarchical, manipulative one, allows easy extensibility and the association of non-geometric attributes with each data item.     

Scene change detection techniques for video database systems

Scene change detection (SCD) is one of several fundamental problems in the design of a video database management system (VDBMS). It is the first step towards the automatic segmentation, annotation, and indexing of video data. SCD is also used in other aspects of VDBMS, e.g., hierarchical representation and efficient browsing of the video data. In this paper, we provide a taxonomy that classifies existing SCD algorithms into three categories: full-video-image-based, compressed-video-based, and model-based algorithms. The capabilities and limitations of the SCD algorithms are discussed in detail. The paper also proposes a set of criteria for measuring and comparing the performance of various SCD algorithms. We conclude by discussing some important research directions.     

Spatio-temporal effects of multimedia objects storage and delivery for video-on-demand systems

As the number of video streams to be supported by a digital video delivery system (DVDS) increases, an improved understanding of the necessity for reliable and cost-efficient support for a considerable number of video streams (in the magnitude of tens of thousands), and the dependency largely on software capabilities emerges. Even in the presence of an optimal hardware configuration, or model, and associated costs, using software to exploit the underlying hardware capabilities is of paramount importance. Although a number of DVDSs have become operational, their ability to deliver the required services mainly depends on the small number of streams supported and the hardware trade-offs. It is imperative that current software developments account for the eventual scalability of the number of video streams without commensurate increase in hardware. In this paper, we present strategies for the management of video streams in order to maintain and satisfy their space and time requirements. We use a DVDS architectural model with functionally dichotomized nodes: a single-node partition is responsible for data retrieval, while the remaining partition of nodes accepts user requests, determines object locations, and routes requests through the network that connects both partitions. We present a detailed analysis of the issues related to queuing I/O requests and data buffering. The discussion includes the requirements for arranging and scheduling I/O requests and data buffers, with the objective of guaranteeing the required data availability rates for continuous media display.     

Spatio-temporal querying in video databases

A video data model that supports spatio-temporal querying in videos is presented. The data model is focused on the semantic content of video streams. Objects, events, activities, and spatial properties of objects are main interests of the model. The data model enables the user to query fuzzy spatio-temporal relationships between video objects and also trajectories of moving objects. A prototype of the proposed model has been implemented.     

Communication-oriented representation of mathematical objects

This paper presents two standards for communicating mathematical objects between computer programs, for representing these objects on multimedia (i.e., the web) and our experimentation with those standards to develop input/output compliant processors that can be used as basic components to build an effective World Wide Web computational server. This article was submitted by the author in English.     

Shape-based retrieval of video objects

The increasing availability of object-based video content requires new technologies for automatically extracting and matching of the low level features of arbitrarily shaped video. This paper proposes methods for shape retrieval of arbitrarily shaped video objects. Our methods take into account not only the still shape features but also the shape deformations that may occur in an object's lifespan. We compute the shape similarity of video objects by comparing the similarity of their representative temporal instances. We also describe motion of a video object via describing the deformations in an object's shape. Experimental results show that our proposed methods offer very good retrieval performance and match closely with the human ranking.     

Replicating data objects in large distributed database systems: an axiomatic game theoretic mechanism design approach

Data object replication onto distributed servers can potentially alleviate bottlenecks, reduce network traffic, increase scalability, add robustness, and decrease user perceived access time. The decision of selecting data object and server pairs requires solving a constraint optimization problem that in general is NP-complete. In this paper, we abstract the distributed database system as an agent-based model, wherein agents continuously compete for allocation and reallocation of data objects. Each agent aims to replicate objects onto its server such that the communication cost is minimized. However, these agents do not have a global view of the system. Thereby, the optimization process becomes highly localized. Such localized optimization may severely affect the overall system performance. To cope with such localized optimization, we propose a “semi-distributed” axiomatic game theoretical mechanism. The mechanism’s control is unique in its decision making process, wherein all the heavy processing is done on the servers of the distributed system and the central body is only required to take a binary decision: (0) not to replicate or (1) to replicate. The cost model used by the agents in the mechanism for the purpose of identifying beneficial data objects is tailored made so that even though the agents take decisions based on their local knowledge domain, the effect is translated into a system-wide performance enhancement. The proposed mechanism is extensively compared against seven well-known conventional and three game theoretical replica allocation methods, namely, branch and bound, greedy, genetic, data-aware replication, tree inspired bottom-up procedure, tree inspired min-max procedure, Benders’ decomposition based procedure, game theoretical English auction, game theoretical Dutch auction, and game theoretical selfish replication procedure. The experimental setup incorporates GT-ITM, Inet network topology generators, Soccer World Cup 1998 access logs, and NASA Kennedy Space Center access logs to closely mimic the Web in its infrastructure and user access patterns. The experimental results reveal that the proposed technique despite its non-cooperative nature improves the solution quality and reduces the execution time compared to other techniques.     

Caching video objects: layers vs versions?

Because Internet access rates are highly heterogeneous, many video content providers today make available different versions of the videos, with each version encoded at a different rate. Multiple video versions, however, require more server storage and may also dramatically impact cache performance in a traditional cache or in a CDN server. An alternative to versions is layered encoding, which can also provide multiple quality levels. Layered encoding requires less server storage capacity and may be more suitable for caching; but it typically increases transmission bandwidth due to encoding overhead. In this paper we compare video streaming of multiple versions with that of multiple layers in a caching environment. We examine caching and distribution strategies that use both versions and layers. We consider two cases: the request distribution for the videos is known a priori; and adaptive caching, for which the request distribution is unknown. Our analytical and simulation results indicate that mixed distribution/caching strategies provide the best overall performance.A shorter version of this work has appeared in Proc. of IEEE International Conference on Multimedia and Expo (ICME), Vol. 2, pages 45–48, Lausanne, Switzerland, August 2002     

Spatio-temporal random fields: compressible representation and distributed estimation

Modern sensing technology allows us enhanced monitoring of dynamic activities in business, traffic, and home, just to name a few. The increasing amount of sensor measurements, however, brings us the challenge for efficient data analysis. This is especially true when sensing targets can interoperate—in such cases we need learning models that can capture the relations of sensors, possibly without collecting or exchanging all data. Generative graphical models namely the Markov random fields (MRF) fit this purpose, which can represent complex spatial and temporal relations among sensors, producing interpretable answers in terms of probability. The only drawback will be the cost for inference, storing and optimizing a very large number of parameters—not uncommon when we apply them for real-world applications. In this paper, we investigate how we can make discrete probabilistic graphical models practical for predicting sensor states in a spatio-temporal setting. A set of new ideas allows keeping the advantages of such models while achieving scalability. We first introduce a novel alternative to represent model parameters, which enables us to compress the parameter storage by removing uninformative parameters in a systematic way. For finding the best parameters via maximum likelihood estimation, we provide a separable optimization algorithm that can be performed independently in parallel in each graph node. We illustrate that the prediction quality of our suggested method is comparable to those of the standard MRF and a spatio-temporal k-nearest neighbor method, while using much less computational resources.     

Watermarking of MPEG-4 video objects

The recent finalization of MPEG-4 will make this standard very attractive for a large range of applications such as video editing, Internet video distribution, wireless video communications. Some of these applications are likely to get great benefit from watermarking technology, since it can enable a number of innovative services, such as conditional access policies, data annotation, data labeling, content authentication, to be implemented at a low price. One of the key points of the MPEG-4 standard is the possibility to access and manipulate objects within a video sequence. Thus object watermarking has to be achieved in such a way that, while a video object is transferred from a sequence to another, it is still possible to correctly access the data embedded within the object itself. The algorithm proposed in this paper embeds a watermark in each video object by imposing a particular relationship between some predefined pairs of quantized discrete cosine transform (DCT) coefficients in the luminance blocks of pseudo-randomly selected macroblocks (MBs). Watermarks are equally embedded into intra and inter MBs. Experimental results are presented validating the effectiveness of the proposed approach.     

硬盘录像中的多媒体数据库

介绍了硬盘录像的系统结构、多媒体数据库的主要功能,着重讨论了系统规范、数据入库、OLE机制、查询、数据库安全等几个关键问题。     

Process objects/masked composition: an object-oriented approach formodeling and control of discrete-event systems

An object oriented framework for modeling and supervisory control of discrete-event systems is described. Control and observation masks are encapsulated with process logic to form process objects, and a single type of interconnection operator called masked composition is used to build complex process objects out of simpler component process objects. The approach applies to both deterministic and nondeterministic plant models and supervisory design. In addition to the usual benefits of object-oriented design, such as software reusability, it yields conditions under which the existence of a nondeterministic supervisor implies existence of a deterministic supervisor     

iSeeBetter: Spatio-temporal video super-resolution using recurrent generative back-projection networks

Computational Visual Media - Recently, learning-based models have enhanced the performance of single-image super-resolution (SISR). However, applying SISR successively to each video frame leads to...     

CartoonModes: Cartoon stylization of video objects through modal analysis

We transform the motion and shape of a video object into cartoon style using a set of representative cartooning deformations which we relate to a modal analysis of the object. Results can be obtained at interactive rate and a user survey confirmed their plausibility.     

Meaning representation in video outcomes of inquiry project

As student assignments and assessment evolve to keep pace with the technological developments, students have the opportunity to express their knowledge in a wider range of forms. The current study aims to examine the meaning representation in videos produced by secondary school students as part of a non-written inquiry project. The content of the video outcomes in the inquiry project, student interviews, and reflection essays written by the six participating students who were of different academic achievement levels were analyzed to better understand their meaning representation using videos. Five types of literacies used by the students in their video outcomes for meaning representation were identified: text, action, narration, cinematography and acoustics. The findings also showed that there were three types of meaning representations with different characteristics: drama, documentary and photo story. The components identified to be required for meaning representation in video were attitude towards video representation, choice of inquiry topic, organizational skills, and data management. The findings informed us of the possibility of diversified use of literacies for meaning representation in different genres of videos. The findings also alerted us of the need to equip students with necessary attitude and skills for better meaning representation. Nevertheless, the need to support learners to present meanings well with videos is suggested.