Security and privacy for multimedia database management systems

This paper describes security and privacy issues for multimedia database management systems. Multimedia data includes text, images, audio and video. It describes access control for multimedia database management systems and describes security policies and security architectures for such systems. Privacy problems that result from multimedia data mining are also discussed.

Intelligent Network Provisioning for Dynamically Downloadable Applications in Beyond 3G Mobile Networks

Mobile communications beyond 3G will integrate different (but complementary) access technologies into a common platform to deliver value-added services and multimedia content in an optimum way. However, the numerous possible configurations of mobile networks complicated the dynamic deployment of mobile applications. Therefore, research is intensely seeking a service provisioning framework that is technology-independent, supports multiple wireless network technologies, and can interwork high-level service management tasks to network management operations. This paper presents an open value chain paradigm, a model for downloadable applications and a mediating platform for service provisioning in beyond 3G mobile settings. Furthermore, we introduce mechanisms that support a coupled interaction between service deployment and network configuration operations, focusing on the dynamic provisioning of QoS state to data path devices according to the requirements of dynamically downloadable mobile value-added services (VAS).

PECOLE: P2P multimedia collaborative environment

PECOLE (Peer-to-pEer COLlaborative Environment) is a fully decentralized multimedia collaborative environment that supports a wide range of collaborative multimedia applications, including chat, shared browsing, shared telepointer, multipoint-to-multipoint audio/video conferencing and multilingual collaboration. PECOLE can intelligently run on very constrained resources, is highly resilient, scalable and does not rely on dedicated servers. Instead, PECOLE is built upon a Peer-to-Peer (P2P) overlay network, using SUN’s JXTA framework and SWT technology. In this paper, we present the architecture and implementation of PECOLE with the performance results of the tests we conducted.

ONTRACK: Dynamically adapting music playback to support navigation

Listening to music on personal, digital devices whilst mobile is an enjoyable, everyday activity. We explore a scheme for exploiting this practice to immerse listeners in navigation cues. Our prototype, ONTRACK, continuously adapts audio, modifying the spatial balance and volume to lead listeners to their target destination. First we report on an initial lab-based evaluation that demonstrated the approach’s efficacy: users were able to complete tasks within a reasonable time and their subjective feedback was positive. Encouraged by these results we constructed a handheld prototype. Here, we discuss this implementation and the results of field-trials. These indicate that even with a low-fidelity realisation of the concept, users can quite effectively navigate complicated routes.

---

COMUNICAR: designing a multimedia, context-aware human-machine interface for cars

The European Union co-funded COMUNICAR (communication multimedia unit inside car) project designed and developed an integrated multimedia human–machine interface (HMI) able to manage a wide variety of driver information systems (from entertainment to safety). COMUNICAR proposed an innovative information provision paradigm, in which the on-vehicle HMI is able to tailor the delivery of the information in real time according to the actual driving context and the drivers workload. COMUNICAR adopted a user-centred design process involving an iterative development based on extensive user tests since the early phases of the project. This approach was particularly useful to define and improve the layout of the user interface and specify the rules that decide the scheduling and the modalities of the delivery of the information messages to the driver. This paper introduces the COMUNICAR concept and the user-centred flow of design. Then, a concrete case of user-test driven, iterative improvement of a systems functionality is presented. We also briefly describe two software tools that we have designed to enhance the development process in a user-centred perspective. Finally, the future evolution of the concept of smart and safe information scheduling is sketched and discussed.

---

Multimodal support to group dynamics

The complexity of group dynamics occurring in small group interactions often hinders the performance of teams. The availability of rich multimodal information about what is going on during the meeting makes it possible to explore the possibility of providing support to dysfunctional teams from facilitation to training sessions addressing both the individuals and the group as a whole. A necessary step in this direction is that of capturing and understanding group dynamics. In this paper, we discuss a particular scenario, in which meeting participants receive multimedia feedback on their relational behaviour, as a first step towards increasing self-awareness. We describe the background and the motivation for a coding scheme for annotating meeting recordings partially inspired by the Bales’ Interaction Process Analysis. This coding scheme was aimed at identifying suitable observable behavioural sequences. The study is complemented with an experimental investigation on the acceptability of such a service.

---

A general audio classifier based on human perception motivated model

The audio channel conveys rich clues for content-based multimedia indexing. Interesting audio analysis includes, besides widely known speech recognition and speaker identification problems, speech/music segmentation, speaker gender detection, special effect recognition such as gun shots or car pursuit, and so on. All these problems can be considered as an audio classification problem which needs to generate a label from low audio signal analysis. While most audio analysis techniques in the literature are problem specific, we propose in this paper a general framework for audio classification. The proposed technique uses a perceptually motivated model of the human perception of audio classes in the sense that it makes a judicious use of certain psychophysical results and relies on a neural network for classification. In order to assess the effectiveness of the proposed approach, large experiments on several audio classification problems have been carried out, including speech/music discrimination in Radio/TV programs, gender recognition on a subset of the switchboard database, highlights detection in sports videos, and musical genre recognition. The classification accuracies of the proposed technique are comparable to those obtained by problem specific techniques while offering the basis of a general approach for audio classification.

XFlavor: providing XML features in media representation

We present XFlavor, a framework for providing XML representation of multimedia data. XFlavor can be used to convert multimedia data back and forth between binary and XML representations. Compared to bitstreams, XML documents are easier to access and manipulate, and consequently, the development of multimedia processing software is greatly facilitated, as one generic XML parser can be used to read and write different types of data in XML form.

Situated play in a tangible interface and adaptive audio museum guide

This paper explores the design issues of situated play within a museum through the study of a museum guide prototype that integrates a tangible interface, audio display, and adaptive modeling. We discuss our use of design ethnography in order to situate our interaction and to investigate the liminal and engagement qualities of a museum visit. The paper provides an overview of our case study and analysis of our user evaluation. We discuss the implications including degrees of balance in the experience design of play in interaction; the challenge in developing a discovery-based information model, and the need for a better understanding of the contextual aspects of tangible user interfaces (TUIs). We conclude that learning effectiveness and functionality can be balanced productively with playful interaction through an adaptive audio and TUI if designers balance the engagement between play and the environment, and the space between imagination and interpretation that links the audio content to the artifacts.

---

The 2007 IEEE CEC simulated car racing competition

This paper describes the simulated car racing competition that was arranged as part of the 2007 IEEE Congress on Evolutionary Computation. Both the game that was used as the domain for the competition, the controllers submitted as entries to the competition and its results are presented. With this paper, we hope to provide some insight into the efficacy of various computational intelligence methods on a well-defined game task, as well as an example of one way of running a competition. In the process, we provide a set of reference results for those who wish to use the simplerace game to benchmark their own algorithms. The paper is co-authored by the organizers and participants of the competition.

---

MedSMan: a live multimedia stream querying system

Querying live media streams is a challenging problem that is becoming an essential requirement in a growing number of applications. Research in multimedia information systems has addressed and made good progress in dealing with archived data. Meanwhile, research in stream databases has received significant attention for querying alphanumeric symbolic streams. The lack of a data model capable of representing different multimedia data in a declarative way, hiding the media heterogeneity and providing reasonable abstractions for querying live multimedia streams poses the challenge of how to make the best use of data in video, audio and other media sources for various applications. In this paper we propose a system that enables directly capturing media streams from sensors and automatically generating more meaningful feature streams that can be queried by a data stream processor. The system provides an effective combination between extendible digital processing techniques and general data stream management research. Together with other query techniques developed in related data stream management streams, our system can be used in those application areas where multifarious live media senors are deployed for surveillance, disaster response, live conferencing, telepresence, etc.

Distributed multimedia information systems: an end-to-end perspective

Emerging Web-based applications require distributed multimedia information system (DMIS) infrastructures. Examples of such applications abound in the domains of medicine, entertainment, manufacturing, e-commerce, as well as military and critical national infrastructures. Development of DMIS for such applications need a broad range of technological solutions for organizing, storing, and delivering multimedia information in an integrated, secure and timely manner with guaranteed end-to-end (E2E) quality of presentation (QoP). DMIS are viewed as catalysts for new research in many areas, ranging from basic research to applied technology. This view is a result of the fact that no single monolithic end-to-end architecture for DMIS can meet the wide spectrum of characteristics and requirements of various Web-based multimedia applications. One size does not fit all in this medium of communication. Management of integrated end-to-end QoP and ensuring information security in DMIS, when viewed in conjunction with real world constraints and system-wide performance requirements, present formidable research and implementation challenges. These challenges encompass all the sub-system components of a DMIS. The ultimate objective of achieving a comprehensive end-to-end QoP management relies on the performance and allocation of resources of each of the DMIS sub-system components including networks, databases, and end-systems. In this paper, we elaborate on these challenges and present a high level distributed architecture aimed at providing the critical functionality for a DMIS.

Acoustic OFDM system and its extension

This paper presents a method of aerial acoustic communication in which data is modulated using OFDM (Orthogonal Frequency Division Multiplexing) and embedded in regular audio material without significantly degrading the quality of the original sound. It replaces the high frequency band of the audio signal with OFDM carriers, each of which is power-controlled according to the spectrum envelope of the original audio signal. The implemented system enables the transmission of short text messages from loudspeakers to mobile handheld devices at a distance of around 3 m. This paper also provides the subjective assessment results and the transmission performances.

A cross-modal approach for karaoke artifacts correction

Karaoke singing is a popular form of entertainment in several parts of the world. Since this genre of performance attracts amateurs, the singing often has artifacts related to scale, tempo, and synchrony. We have developed an approach to correct these artifacts using cross-modal multimedia streams information. We first perform adaptive sampling on the user’s rendition and then use the original singer’s rendition as well as the video caption highlighting information in order to correct the pitch, tempo and the loudness. A method of analogies has been employed to perform this correction. The basic idea is to manipulate the user’s rendition in a manner to make it as similar as possible to the original singing. A pre-processing step of noise removal due to feedback and huffing also helps improve the quality of the user’s audio. The results are described in the paper which shows the effectiveness of this multimedia approach.

Methods for quantitative usability requirements: a case study on the development of the user interface of a mobile phone

Quantitative usability requirements are a critical but challenging, and hence an often neglected aspect of a usability engineering process. A case study is described where quantitative usability requirements played a key role in the development of a new user interface of a mobile phone. Within the practical constraints of the project, existing methods for determining usability requirements and evaluating the extent to which these are met, could not be applied as such, therefore tailored methods had to be developed. These methods and their applications are discussed.

---

Protecting business intelligence and customer privacy while outsourcing data mining tasks

Nowadays data mining plays an important role in decision making. Since many organizations do not possess the in-house expertise of data mining, it is beneficial to outsource data mining tasks to external service providers. However, most organizations hesitate to do so due to the concern of loss of business intelligence and customer privacy. In this paper, we present a Bloom filter based solution to enable organizations to outsource their tasks of mining association rules, at the same time, protect their business intelligence and customer privacy. Our approach can achieve high precision in data mining by trading-off the storage requirement. This research was supported by the USA National Science Foundation Grants CCR-0310974 and IIS-0546027.

---

Audio indexing: primary components retrieval

This work addresses the soundtrack indexing of multimedia documents. Our purpose is to detect and locate sound unity to structure the audio dataflow in program broadcasts (reports). We present two audio classification tools that we have developed. The first one, a speech music classification tool, is based on three original features: entropy modulation, stationary segment duration (with a Forward–Backward Divergence algorithm) and number of segments. They are merged with the classical 4 Hz modulation energy. It is divided into two classifications (speech/non-speech and music/non-music) and provides more than 90% of accuracy for speech detection and 89% for music detection. The other system, a jingle identification tool, uses an Euclidean distance in the spectral domain to index the audio data flow. Results show that is efficient: among 132 jingles to recognize, we have detected 130. Systems are tested on TV and radio corpora (more than 10 h). They are simple, robust and can be improved on every corpus without training or adaptation.

Connecting the family with awareness systems

Awareness systems have attracted significant research interest for their potential to support interpersonal relationships. Investigations of awareness systems for the domestic environment have suggested that such systems can help individuals stay in touch with dear friends or family and provide affective benefits to their users. Our research provides empirical evidence to refine and substantiate such suggestions. We report our experience with designing and evaluating the ASTRA awareness system, for connecting households and mobile family members. We introduce the concept of connectedness and its measurement through the Affective Benefits and Costs of communication questionnaire (ABC-Q). We inform results that testify the benefits of sharing experiences at the moment they happen without interrupting potential receivers. Finally, we document the role that lightweight, picture-based communication can play in the range of communication media available.

---

Interacting with mobile services: an evaluation of camera-phones and visual tags

We present a study of using camera-phones and visual-tags to access mobile services. Firstly, a user-experience study is described in which participants were both observed learning to interact with a prototype mobile service and interviewed about their experiences. Secondly, a pointing-device task is presented in which quantitative data was gathered regarding the speed and accuracy with which participants aimed and clicked on visual-tags using camera-phones. We found that participants’ attitudes to visual-tag-based applications were broadly positive, although they had several important reservations about camera-phone technology more generally. Data from our pointing-device task demonstrated that novice users were able to aim and click on visual-tags quickly (well under 3 s per pointing-device trial on average) and accurately (almost all meeting our defined speed/accuracy tradeoff of 6% error-rate). Based on our findings, design lessons for camera-phone and visual-tag applications are presented.

---

Towards Environment-to-Environment (E2E) multimedia communication systems

We present an approach to connect multiple remote environments over web for natural interaction among people and objects. Focus of current communication and telepresence systems severely restrict user affordances in terms of movement, interaction, peripheral vision, spatio-semantic integrity and even information flow. These systems allow information transfer rather than experiential interaction. We propose Environment-to-Environment (E2E) as a new paradigm for communication which allows users to interact in natural manner using text, audio, and video by connecting environments. Each Environment is instrumented using as many different types of sensors as may be required to detect presence and activity of objects. This object position and activity information is used by a scalable event-based multimodal information system called EventServer to share the appropriate experiential information with other environments as well as to present incoming multimedia information on right displays and speakers. This paper describes the design principles for E2E communication, discusses system architecture, and gives our experience in implementing prototypes of such systems in telemedicine and office collaboration applications. We also discuss the research challenges and a road-map for creating more sophisticated E2E applications in near future.