A two-level queueing system for interactive browsing and searching of video content

This paper presents a two-level queueing system for dynamic summarization and interactive searching of video content. Video frames enter the queueing system; some insignificant and redundant frames are removed; the remaining frames are pulled out of the system as top-level key frames. Using an energy-minimization method, the first queue removes the video frames that constitute the gradual transitions of video shots. The second queue measures the content similarity of video frames and reduces redundant frames. In the queueing system, all key frames are linked in a directed-graph index structure, allowing video content to be accessed at any level-of-detail. Furthermore, this graph-based index structure enables interactive video content exploration, and the system is able to retrieve the video key frames that complement the video content already viewed by users. Experimental results on four full-length videos show that our queueing system performs much better than two existing methods on video key frame selection at different compression ratios. The evaluation on video content search shows that our interactive system is more effective than other systems on eight video searching tasks. Compared with the regular media player, our system reduces the average content searching time by half.     

Storage and retrieval methods to support fully interactive playout in a disk-array-based video server

One of the most important challenges in a videoon-demand (VOD) system is to support interactive browsing functions such as fast forward and fast backward. Typically, these functions impose additional resource requirements on the VOD system in terms of storage space, retrieval throughput, network bandwidth, etc. Moreover, prevalent video compression techniques such as MPEG impose additional constraints on the process since they introduce interframe dependencies. In this paper, we devise methods to support variable rate browsing for MPEG-like video streams and minimize the additional resources required. Specifically, we consider the storage and retrieval for video data in a diskarray-based video server and address the issue of distributing the retrieval requests across the disks evenly. The overall approach proposed in this paper for interactive browsing is composed of (1) a storage method, (2) sampling and placement methods, and (3) a playout method, in which the sampling and placement methods are two alternatives for video-segment selection. The segment-sampling scheme supports browsing at any desired speed while balancing the load on the disk array, as well as minimizing the variation on the number of video segments skipped between samplings. In contrast, the segment-placement scheme supports completely uniform segment sampling across the disk array for some specific speed-up rates. Several theoretical properties for the problem studied are derived. Finally, we describe experimental results on the visual effect of the proposed frame-skipping approach.     

An interactive assembly level debugging system

An interactive assembly level debugging system has been developed to facilitate program development on an INTEL 8080A/8085 based microcomputer. It has features such as decoding machine level instructions into the assembly language, relocating programs in memory, changing instructions interactively at assembly level etc. This paper deals with the design of the assembly level debugging system and the various facilities and features it provides. The debugging system requires only 4.5K bytes of RAM besides the memory requirements of the application program that has to be debugged.     

Development and validation of a behavioural video coding scheme for detecting mental workload in manual assembly

Abstract

Manual assembly in the future Industry 4.0 workplace will put high demands on operators’ cognitive processing. The development of mental workload (MWL) measures therefore looms large. Physiological gauges such as electroencephalography (EEG) show promising possibilities, but still lack sufficient reliability when applied in the field. This study presents an alternative measure with a substantial ecological validity. First, we developed a behavioural video coding scheme identifying 11 assembly behaviours potentially revealing MWL being too high. Subsequently, we explored its validity by analysing videos of 24 participants performing a high and a low complexity assembly. Results showed that five of the behaviours identified, such as freezing and the amount of part rotations, significantly differed in occurrence and/or duration between the two conditions. The study hereby proposes a novel and naturalistic method that could help practitioners to map and redesign critical assembly phases, and researchers to enrich validation of MWL-measures through measurement triangulation.

Practitioner summary: Current physiological mental workload (MWL) measures still lack sufficient reliability when applied in the field. Therefore, we identified several observable assembly behaviours that could reveal MWL being too high. The results propose a method to map MWL by observing specific assembly behaviours such as freezing and rotating parts.

Abbreviations: MWL: mental workload; EEG: electroencephalography; fNIRS: functional near infrared spectroscopy; AOI: area of interest; SMI: SensoMotoric Instruments, ETG: Eye-Tracking Glasses; FPS: frames per second; BORIS: Behavioral Observation Research Interactive Software; IRR: inter-rater reliability; SWAT: Subjective Workload Assessment Technique; NASA-TLX: National Aeronautics and Space Administration Task Load Index; EL: emotional load; DSSQ: Dundee Stress State Questionnaire; PHL: physical load; SBO: Strategisch Basis Onderzoek     

Incremental and interactive mining of web traversal patterns

Web mining involves the application of data mining techniques to large amounts of web-related data in order to improve web services. Web traversal pattern mining involves discovering users’ access patterns from web server access logs. This information can provide navigation suggestions for web users indicating appropriate actions that can be taken. However, web logs keep growing continuously, and some web logs may become out of date over time. The users’ behaviors may change as web logs are updated, or when the web site structure is changed. Additionally, it can be difficult to determine a perfect minimum support threshold during the data mining process to find interesting rules. Accordingly, we must constantly adjust the minimum support threshold until satisfactory data mining results can be found.The essence of incremental data mining and interactive data mining is the ability to use previous mining results in order to reduce unnecessary processes when web logs or web site structures are updated, or when the minimum support is changed. In this paper, we propose efficient incremental and interactive data mining algorithms to discover web traversal patterns that match users’ requirements. The experimental results show that our algorithms are more efficient than other comparable approaches.     

Volumetric video compression for interactive playback

We develop a volumetric video system which supports interactive browsing of compressed time-varying volumetric features (significant isosurfaces and interval volumes). Since the size of even one volumetric frame in a time-varying 3D data set is very large, transmission and on-line reconstruction are the main bottlenecks for interactive remote visualization of time-varying volume and surface data. We describe a compression scheme for encoding time-varying volumetric features in a unified way, which allows for on-line reconstruction and rendering. To increase the run-time decompression speed and compression ratio, we decompose the volume into small blocks and encode only the significant blocks that contribute to the isosurfaces and interval volumes. The results show that our compression scheme achieves high compression ratio with fast reconstruction, which is effective for interactive client-side rendering of time-varying volumetric features.     

VisionGo: Towards video retrieval with joint exploration of human and computer

This paper introduces an effective interactive video retrieval system named VisionGo. It jointly explores human and computer to accomplish video retrieval with high effectiveness and efficiency. It assists the interactive video retrieval process in different aspects: (1) it maximizes the interaction efficiency between human and computer by providing a user interface that supports highly effective user annotation and an intuitive visualization of retrieval results; (2) it employs a multiple feedback technique that assists users in choosing proper method to enhance relevance feedback performance; and (3) it facilitates users to assess the retrieval results of motion-related queries by using motion-icons instead of static keyframes. Experimental results based on over 160 h of news video shows demonstrate the effectiveness of the VisionGo system.     

Personal orchestra: a real-time audio/video system for interactive conducting

We present the first multimedia system to conduct a realistic electronic orchestra. Users can control tempo, dynamics, and instrument emphasis of the orchestra through natural conducting gestures with an infrared baton. Using gesture recognition and tempo adjustment algorithms, the system plays back an audio and video recording of an actual orchestra that follows the users conducting in real time. A major achievement of this system is its ability to vary playback speed in real time while avoiding audio artifacts such as pitch changes. The system has been deployed as an exhibit and has become a major attraction of a large Vienna-based music exhibition center.An erratum to this article can be found at     

Dynamic and interactive typography in digital art

The objective of this investigation was to produce a work that demonstrates the potential of typography as a subject matter for a contemporary artistic discourse by exploiting the dynamism and interactivity of digital typography.

This paper describes the process of the conceptual development and implementation of an interactive multimedia installation based on the history of writing and typography.

The main aspects of the methodology that inspired the building of an installation are emphasised and it is argued that in arts, technology is meant to serve a concept and poetics, not the opposite way round.     

Distributed video coding with adaptive two-step side information generation for smart and interactive media

Conventional video coding standards, such as MPEGx and H.26x, use a hybrid architecture of block-based motion compensation and discrete cosine transform (DCT) within the structure of a complex encoder and a simple decoder. Contrary to conventional video codecs, Wyner-Ziv (WZ) video coding, which is a practical application of distributed video coding (DVC) based on the Slepian-Wolf and WZ theorems, exploits the source correlation at the decoder, thereby allowing for the development of simpler encoders. However, the current WZ video coding algorithms cannot match the coding performance of conventional video coding. In order to improve the coding performance of transform-domain WZ video coding, an adaptive two-step side-information generation method is thus proposed for evaluation and analysis in this study. The proposed method uses decoded WZ frames in the down-sampled spatial resolution along with already decoded coefficients to successively improve rate-distortion (RD) performance as the decoding progresses. The experimental results show that the proposed method, compared to previous WZ video coding method, shows improved coding performance, particularly under critical conditions, such as cases with high motion content.     

Video scene retrieval with interactive genetic algorithm

This paper proposes a video scene retrieval algorithm based on emotion. First, abrupt/gradual shot boundaries are detected in the video clip of representing a specific story. Then, five video features such as “average color histogram,” “average brightness,” “average edge histogram,” “average shot duration,” and “gradual change rate” are extracted from each of the videos, and mapping through an interactive genetic algorithm is conducted between these features and the emotional space that a user has in mind. After the proposed algorithm selects the videos that contain the corresponding emotion from the initial population of videos, the feature vectors from them are regarded as chromosomes, and a genetic crossover is applied to those feature vectors. Next, new chromosomes after crossover and feature vectors in the database videos are compared based on a similarity function to obtain the most similar videos as solutions of the next generation. By iterating this process, a new population of videos that a user has in mind are retrieved. In order to show the validity of the proposed method, six example categories of “action,” “excitement,” “suspense,” “quietness,” “relaxation,” and “happiness” are used as emotions for experiments. This method of retrieval shows 70% of effectiveness on the average over 300 commercial videos.

Automatic personalized video abstraction for sports videos using metadata

Video abstraction is defined as creating a video abstract which includes only important information in the original video streams. There are two general types of video abstracts, namely the dynamic and static ones. The dynamic video abstract is a 3-dimensional representation created by temporally arranging important scenes while the static video abstract is a 2-dimensional representation created by spatially arranging only keyframes of important scenes. In this paper, we propose a unified method of automatically creating these two types of video abstracts considering the semantic content targeting especially on broadcasted sports videos. For both types of video abstracts, the proposed method firstly determines the significance of scenes. A play scene, which corresponds to a play, is considered as a scene unit of sports videos, and the significance of every play scene is determined based on the play ranks, the time the play occurred, and the number of replays. This information is extracted from the metadata, which describes the semantic content of videos and enables us to consider not only the types of plays but also their influence on the game. In addition, user’s preferences are considered to personalize the video abstracts. For dynamic video abstracts, we propose three approaches for selecting the play scenes of the highest significance: the basic criterion, the greedy criterion, and the play-cut criterion. For static video abstracts, we also propose an effective display style where a user can easily access target scenes from a list of keyframes by tracing the tree structures of sports games. We experimentally verified the effectiveness of our method by comparing our results with man-made video abstracts as well as by conducting questionnaires.

Disk placement for arbitrary-rate playback in an interactive video server

Multimedia data, especially continuous media including video and audio objects, represent a rich and natural stimulus for humans, but require large amount of storage capacity and real-time processing. In this paper, we describe how to organize video data efficiently on multiple disks in order to support arbitrary-rate playback requested by different users independently. Our approach is to segment and decluster video objects and to place the segments in multiple disks using a restricted round-robin scheme, called prime round-robin (PRR). Its placement scheme provides uniform load balance of disks for arbitrary retrieval rate as well as normal playback, since it eliminates hot spots. Moreover, it does not require any additional disk bandwidth to support VCR-like operations such as fast-forward and rewind. We have studied the various effects of placement and retrieval schemes in a storage server by simulation. The results show that PRR offers even disk accesses, and the failure in reading segment by deadline occurs only at the beginning of new operations. In addition, the number of users admitted is not decreased, regardless of arbitrary-rate playback requests.     

An interactive fuzzy programming approach for bi-objective straight and U-shaped assembly line balancing problem

The consideration of this paper is given to address the straight and U-shaped assembly line balancing problem. Although many attempts in the literature have been made to develop deterministic version of the assembly line model, the attention is not considerably given to those in uncertain environment. In this paper, a novel bi-objective fuzzy mixed-integer linear programming model (BOFMILP) is developed so that triangular fuzzy numbers (TFNs) are employed in order to represent uncertainty and vagueness associated with the task processing times in the real production systems. In this proposed model, two conflicting objectives (minimizing the number of stations as well as cycle time) are considered simultaneously with respect to set of constraints. For this purpose, an appropriate strategy in which new two-phase interactive fuzzy programming approach is proposed as a solution method to find an efficient compromise solution. Finally, validity of the proposed model as well as its solution approach are evaluated though numerical examples. In addition, a comparison study is conducted over some test problems in order to assess the performance of the proposed solution approach. The results demonstrate that our proposed interactive fuzzy approach not only can be applied in ALBPs but also is capable to handle any practical MOLP models. Moreover, in light of these results, the proposed model may constitute a framework aiming to assist the decision maker (DM) to deal with uncertainty in assembly line problem.     

Paradigms for the design of interactive drama

The essential purpose of drama, whether it is theatre, film or television, is to evoke subjective experience in its audience. The advent of interactive digital television opens up many new possibilities for the structure and presentation of drama and the way in which people may engage with it. However, the design of interactive, computer-based environments requires theoretical and practical bases that are largely new to the dramatic arts. This paper describes research which is exploring the convergence of ideas and design models from a range of traditional sources including HCl, psychology, the performing arts and media production for this new media form. It starts by reevaluating two traditional dramatic paradigms, Stanislavski's Realism and Brecht's objectivity, and contrasts them with Boal's spect-actor paradigm in which audiences become direct participants in the drama. Although existing technology cannot enable a faithful computer based implementation of the spect-actor paradigm, the model may be approachable in various ways and thus provides a useful goal for designers of interactive drama. The realities of designing and producing interactive drama are illustrated with a practical example. The paper concludes with a discussion of the implications of the work for the design of interactive drama.     

Assembly operation process planning by mapping a virtual assembly simulation to real operation

Virtual assembly has been widely used in product development. However, virtual operation and actual operation are different in time and space, the simulation of interactive virtual assembly cannot support the assembly operation's process planning directly. In this paper, the solution for assembly operation's process planning is developed based on interactive virtual assembly. According to the solution, interactive assembly operation is used to obtain the actions of operation sequence. The actions are mapped into the data of a real operation action to obtain real operation actions. Then assembly operation cards can be obtained. To support the assembly operation actions obtained through virtual assembly simulation, a product assembly model is proposed. An operation semantic model is used to replace the geometric constraint model of assembly, which contains several ordered geometric constraints and some engineering restriction conditions. To test the solution and the models, one process planning example of an automobile engine is introduced. The results verified the feasibility and the effectiveness of the methods.     

采用流媒体技术实现网络中的视频和音频传播

概述了流媒体技术的定义，原理以及发展现状，在以RealMedia为应用实例的基础上介绍了如何应用流媒体技术实现音频，视频信息在网络上的传播。     

Design and implementation of an interactive test bed

An interactive test bed for programs written in the system development language CLSD is described. It consists of a compiler and an interactive interpreter. The test facilities and the internal design of the test and bed are described in detail. Experience has demonstrated this to be a very valuable tool in program development.     

Files in an interactive environment

A well designed file system can significantly simplify the design of and increase the reliability of interactive programs. A summary of the specifications of a convenient file system that is substantially independent of the host operating system and some experiences using the system are described.     

A unified framework for web video topic discovery and visualization

Together with the explosive growth of web video in sharing sites like YouTube, automatic topic discovery and visualization have become increasingly important in helping to organize and navigate such large-scale videos. Previous work dealt with the topic discovery and visualization problem separately, and did not take fully into account of the distinctive characteristics of multi-modality and sparsity in web video features. This paper tries to solve web video topic discovery problem with visualization under a single framework, and proposes a Star-structured K-partite Graph based co-clustering and ranking framework, which consists of three stages: (1) firstly, represent the web videos and their multi-model features (e.g., keyword, near-duplicate keyframe, near-duplicate aural frame, etc.) as a Star-structured K-partite Graph; (2) secondly, group videos and their features simultaneously into clusters (topics) and organize the generated clusters as a linked cluster network; (3) finally, rank each type of nodes in the linked cluster network by “popularity” and visualize them as a novel interface to let user interactively browse topics in multi-level scales. Experiments on a YouTube benchmark dataset demonstrate the flexibility and effectiveness of our proposed framework.