A data model and a query languagefor multimedia documents databases

A multimedia application involves information that may be in a form of video, images, audio, text and graphics, need to be stored, retrieved and manipulated in large databases. In this paper, we propose an object-oriented database schema that supports multimedia documents and their temporal, spatial and logical structures. We present a document example and show how the schema can adress all the structures described. We also present a multimedia query specification language that can be used to describe a multimedia content portion to be retrieved from the database. The language provides means by which the user can specify the information on the media as well as the temoral and spatial relationships among these media.     

Immersive authoring of Tangible Augmented Reality content: A user study

Immersive authoring refers to the style of programming or developing content from within the targetexecutable environment. Immersive authoring is important for fields such as augmented reality (AR) in which interaction usability and user perception of the target content must be checked first hand, in situ. In addition, the interaction efficiency and usability of the authoring tools itself is equally important forease of authoring. In this paper, we propose design principles and describe an implementation of animmersive authoring system for AR. More importantly, we present a formal user study demonstrating its benefits and weaknesses. In particular, our results demonstrate that, compared to using the traditional 2D desktop development method, immersive authoring gained significant efficiency in specifying spatial arrangements and behavior tasks, a major component of AR content authoring. However, it was not so successful for abstract tasks such as logical programming. Based on this result, we suggest that a comprehensive AR authoring tool should include such immersive authoring functionality to help, particularly non-technical media artists, create effective contents based on the characteristics of the underlying media and interaction style.     

A Novel Approach Based on Multi-View Content Analysis and Semi-Supervised Enrichment for Movie Recommendation

Although many existing movie recommender systems have investigated recommendation based on information such as clicks and tags, much less efforts have been made to explore the multimedia content of movies, which has potential information for the elicitation of the user’s visual and musical preferences.In this paper, we explore the content from three media types (image, text, audio) and propose a novel multi-view semi-supervised movie recommendation method, which represents each media type as a view space for movies.The three views of movies are integrated to predict the rating values under the multi-view framework.Furthermore, our method considers the casual users who rate limited movies.The algorithm enriches the user profile with a semi-supervised way when there are only few rating histories.Experiments indicate that the multimedia content analysis reveals the user’s profile in a more comprehensive way.Different media types can be a complement to each other for movie recommendation.And the experimental results validate that our semi-supervised method can effectively enrich the user profile for recommendation with limited rating history.     

A holistic approach for personalization,relevance feedback & recommendation in enriched multimedia content

As the prerequisites of production houses, broadcasters, advertising agencies and online publishing companies for enriched multimedia content increase rapidly, the need of innovative methods for the effective creation of enriched multimedia content is undeniable. Stemming from this need, in this paper we focus on the design, development and evaluation of a framework consisting of personalization, relevance feedback and recommendation mechanisms, as a principal method for the creation of enriched multimedia content targeted to each user’s needs, preferences and interests. As the multimedia content proliferates along with its consumption by the users, more effective ways of presenting it to the viewers are demanded in order to facilitate them with the multimedia content search and selection and improve their Quality of Experience (QoE). The main contribution of the paper is the introduction of a holistic framework that offers personalized enriched multimedia content, by extending the recommendation process to the set of enrichments that accompany the video except from the video itself and by collecting explicit and implicit relevance feedback from the interactions of the user with both the video and its enrichments. We evaluate the proposed framework following a two-step approach. Firstly, we perform extended experiments by applying reasonably simulated user interactions, in order to calibrate its parameters that refer to multiple aspects of the enriched multimedia content, aiming at high performance in terms of QoE. Here, most importantly, we have shown that appropriately designing the enrichments and considering users’ interactions with them allows for achieving a better quality in inferring users’ profiles in many realistic cases. Secondly, we integrated our proposed recommender framework within the MECANEX streaming platform in order to perform user studies about its usability within a realistic environment of use.     

Semantic-based framework for personalised ambient media

The paper proposes a semantic-based metadata framework for personalised interaction with TV media in a connected home context. Our approach allows the current home media centres to go beyond the simple concept of electronic programme guides and to offer the users a personalised media experience in an ambient home environment. The user’s characteristics, preferences and context are used to personalise the user’s experience of viewing and interacting with multimedia content on different heterogeneous devices. The TV-Anytime specification provides the basis for the metadata framework for handling content from IP, digital broadcast, and Blu-ray disc sources.     

Topic based photo set retrieval using user annotated tags

In this paper, we introduce a media decision taking engine (MDTE), enabling the automatic selection and/or rating of multimedia content versions, based on the available context information. The presented approach is fully semantic-driven, which means that we not only semantically model the context information, but also the decision algorithms themselve, which are represented in N3 Rules, a rule language that extends RDF. The decision rules are based on a rating function, supporting the specification of weights and affinity parameters for each environment property. Finally, we show how the MDTE is integrated in a media delivery platform, using the provisions of the existing Web infrastructure.     

Web-Based Support for Cooperative Software Engineering

The Tatami project is building a system to support software engineering over the internet, exploiting recent advances in Web technology, interface design, and specification. Our effort to improve the usability of such systems has led us into algebraic semiotics, while our effort to develop better formal methods for distributed concurrent systems has led us into hidden algebra and fuzzy logic. This paper discusses the Tatami system design, especially its software architecture, and its user interface principles. New work in the latter area includes an extension of algebraic semiotics to dynamic multimedia interfaces, and integrating Gibsonian affordances with algebraic semiotics.     

多媒体产品的可用性设计指南

多媒体产品的设计应在重视产品功能性的同时重视其可用性。可用性体现了从用户角度感受到的产品质量,是决定产品成败的关键因素之一。为了提高多媒体产品的可用性质量,必须在产品的设计开发过程中运用可用性工程方法,其中包括使用多媒体的可用性设计指南,该文将综合介绍目前提出的多媒体产品的可用性设计指南。     

Multimedia Clip Generation From Documents for Browsing on Mobile Devices

Small displays on mobile handheld devices, such as personal digital assistants (PDAs) and cellular phones, are the bottlenecks for usability of most content browsing applications. Generally, conventional content such as documents and Web pages need to be modified for effective presentation on mobile devices. This paper proposes a novel visualization for documents, called multimedia thumbnails, which consists of text and image content converted into playable multimedia clips. A multimedia thumbnail utilizes visual and audio channels of small portable devices as well as both spatial and time dimensions to communicate text and image information of a single document. The proposed algorithm for generating multimedia thumbnails includes 1) a semantic document analysis step, where salient content from a source document is extracted; 2) an optimization step, where a subset of this extracted content is selected based on time, display, and application constraints; and 3) a composition step, where the selected visual and audible document content is combined into a multimedia thumbnail. Scalability of MMNails that allows generation of multimedia clips of various lengths is also described. A user study is presented that evaluates the effectiveness of the proposed multimedia thumbnail visualization.     

Exploring context and content links in social media: a latent space method

Social media networks contain both content and context-specific information. Most existing methods work with either of the two for the purpose of multimedia mining and retrieval. In reality, both content and context information are rich sources of information for mining, and the full power of mining and processing algorithms can be realized only with the use of a combination of the two. This paper proposes a new algorithm which mines both context and content links in social media networks to discover the underlying latent semantic space. This mapping of the multimedia objects into latent feature vectors enables the use of any off-the-shelf multimedia retrieval algorithms. Compared to the state-of-the-art latent methods in multimedia analysis, this algorithm effectively solves the problem of sparse context links by mining the geometric structure underlying the content links between multimedia objects. Specifically for multimedia annotation, we show that an effective algorithm can be developed to directly construct annotation models by simultaneously leveraging both context and content information based on latent structure between correlated semantic concepts. We conduct experiments on the Flickr data set, which contains user tags linked with images. We illustrate the advantages of our approach over the state-of-the-art multimedia retrieval techniques.     

Multimedia layout adaptation through grammatical specifications

Online multimedia presentations, such as news, need to be constantly updated. Increasing demands are also being made for accessing online multimedia documents from mobile devices such as PDAs. There is an urgent need for a sound but practical formalism that supports automatic adaptation to the change of media content, display environments, and user intention. This paper presents a visual language approach to the layout adaptation of multimedia objects. The underlying theory of our approach is a context-sensitive graph grammar formalism enriched with facilities for spatial representation and specification. The paper focuses on the issues and techniques for size adaptation and style adaptation in response to the change of device requirements and user interactions.Published online: 2 February 2005 Correspondence to: Kang Zhang     

Articulation of information requirements for personalised knowledge construction

The quality of information provision influences considerably knowledge construction driven by individual users’ needs. In the design of information systems for e-learning, personal information requirements should be incorporated to determine a selection of suitable learning content, instructive sequencing for learning content, and effective presentation of learning content. This is considered as an important part of instructional design for a personalised information package. The current research reveals that there is a lack of means by which individual users’ information requirements can be effectively incorporated to support personal knowledge construction. This paper presents a method which enables an articulation of users’ requirements based on the rooted learning theories and requirements engineering paradigms. The user’s information requirements can be systematically encapsulated in a user profile (i.e. user requirements space), and further transformed onto instructional design specifications (i.e. information space). These two spaces allow the discovering of information requirements patterns for self-maintaining and self-adapting personalisation that enhance experience in the knowledge construction process.     

Innovative User Interfaces in Automation Engineering by Application of Usability Engineering Methods Shown by the Example of a Three-Dimensional Plant Representation

This article is the first of 3 that focus on user-centered design and usability in the field of industrial process control. This article starts with an introduction to this special domain and sets out the need for user-centered design of innovative user interfaces. The MediaPlant research project is then introduced. This project aims to generate innovative user interface solutions for process control. The usability engineering methods presented are well known in relation to office application software and were applied in the course of the project to (a) evaluate the potential and application fields of innovative multimedia technology and 3-dimensional information presentation (described at the end of this article), (b) define innovative user interface building blocks for process control that can be applied across sectors (described in the second article, Komischke & Burmester [this issue]) and (c) develop human-centered user interfaces for controlling paper recycling plants (described in the third article, Epstein & Beu [this issue]).     

基于网络多媒体管理系统的研究

随着网络技术的发展,多媒体信息不断丰富,用户在进行网络数据库的查询时,文字信息已不能满足其需要,而多媒体信息由于界面生动,应用广泛,日益受到重视。多媒体资源管理系统是一个典型的基于Web的信息管理系统,为B/S结构。主要为用户提供基于流媒体的多媒体信息网上传输。本文利用NET和SQL相结合,设计出基于网络的多媒体管理系统,为个人网站和企业网站的进一步优化提供良好方案。     

Usability of geographic information – Factors identified from qualitative analysis of task-focused user interviews

Understanding user needs for geographic information and the factors which influence the usability of such information in diverse user contexts is an essential part of user centred development of information products. There is relatively little existing research focused on the design and usability of information products in general. This paper presents a research approach based on semi structured interviews with people working with geographic information on a day to day basis, to establish a reference base of qualitative data on user needs for geographic information with respect to context of use. From this reference data nine key categories of geographic information usability are identified and discussed in the context of limited existing research concerned with geographic information usability.     

JSD and the design of user interface software

Systems analysis and design methods (SADMs), such as Jackson system development (JSD), have been proposed as a means of improving software quality. This is contrasted with the role of improving software usability proposed for human factors (HF). The paper argues the need for an explicit user interface specification stage in system development to ensure the usability of proposed systems. It is suggested that JSD supplemented by the specialist knowledge of HF can support such a specification stage. For HF and JSD to be successfully integrated, contributions from both need to be appropriately timed and structured. A model of the system development process that incorporates HF contributions is proposed. A means by which the HF contributions, in the form of a user interface specification, can be integrated with the JSD specification is developed and illustrated through examples.     

The influence of design decisions on the usability of direct manipulation user interfaces

It is assumed that the usability of direct manipulation user interfaces is influenced by a number of design aspects. In this experimental study, the order of command specification and the type of function activation were manipulated in a 2 × 2 factorial design, in order to test hypotheses H1, that object-function specification contributes more to usability than function-object specification; and H2. that the type of function activation (clicking vs. dragging) will influence the usability of direct manipulation user interfaces. Sixty-four subjects, balanced by sex, without computer experience, were assigned randomly to the four experimental conditions. The dependent variables include performance data such as time, efficiency and error rates (logfile-recording), and subjective user rating of the user interface (questionnaire). Whereas HI had to be rejected in this general form, a more elaborated analysis showed significant differences between the factor levels in terms of performance time and syntactically correct actions. Furthermore, the results of the study demonstrated evidence for H2.     

Supporting user-adapted interface design: The USE-IT system

This paper describes USE-IT, a knowledge-based tool for automating the design of interactions at the physical level, so as to ensure accessibility of the target user interface by different user groups, including people with disabilities. To achieve this, USE-IT elicits, manipulates and interprets representations of design knowledge in order to reason about, select and decide upon lexical adaptation constituents of a user interface. Adaptation constituents are attributes of abstract interaction object classes. USE-IT generates a collection of adaptation rules (i.e. a lexical specification scenario), based on design constraints generated from three basic knowledge sources: (a) the user model, (b) the task schema, and (c) a set of platform constraints (i.e. interaction objects, attributes, device availability, etc.). A data structure called the adaptability model tree has been designed to (i) facilitate the development of plausible semantics of adaptation at the lexical level of interaction, (ii) allow unification of design constraints, and (iii) enable selection of maximally preferred design options. The output of USE-IT can be subsequently interpreted by the run-time libraries of a high-level user interface development toolkit, which provides the required implementation support for realizing the user-adapted interface on a target platform.     

HADT: Hypermedia application development tool for tourist applications

This paper describes an authoring tool prototype design for the development of hypermedia applications. The system is based on a generic object oriented data model suitable for hypermedia, that has been designed and implemented in our department. The model is extensible for integrated multimedia manipulation and provides an enhanced linking system. The design aims at a general purpose hypermedia application development tool which may well be used for the development of tourist applications. It is of special interest in the Greek tourist market since tourism contributes considerably to the national income of Greece. Current technology provides the hardware for manipulation of multimedia information and increases the quantity and quality of information flow between the user and the machine. Thus in a travel agency environment it would be desirable to have an interactive multimedia demonstration helping the user find places of interest in any context (physical beauty, archaeological interest, tourist infrastructure, possible itineraries etc.). Such a tool would be very effective compared with the static presentation of brochures. HADT has the potential to fulfil these needs when it is developed, and in this paper we explore such possibilities. Furthermore, research directions for future work are discussed.