Dynamic context capture and distributed video arrays for intelligent spaces

Intelligent environments can be viewed as systems where humans and machines (rooms) collaborate. Intelligent (or smart) environments need to extract and maintain an awareness of a wide range of events and human activities occurring in these spaces. This requirement is crucial for supporting efficient and effective interactions among humans as well as humans and intelligent spaces. Visual information plays an important role for developing accurate and useful representation of the static and dynamic states of an intelligent environment. Accurate and efficient capture, analysis, and summarization of the dynamic context requires the vision system to work at multiple levels of semantic abstractions in a robust manner. In this paper, we present details of a long-term and ongoing research project, where indoor intelligent spaces endowed with a range of useful functionalities are designed, built, and systematically evaluated. Some of the key functionalities include: intruder detection; multiple person tracking; body pose and posture analysis; person identification; human body modeling and movement analysis; and for integrated systems for intelligent meeting rooms, teleconferencing, or performance spaces. The paper includes an overall system architecture to support design and development of intelligent environments. Details of panoramic (omnidirectional) video camera arrays, calibration, video stream synchronization, and real-time capture/processing are discussed. Modules for multicamera-based multiperson tracking, event detection and event based servoing for selective attention, voxelization, streaming face recognition, are also discussed. The paper includes experimental studies to systematically evaluate performance of individual video analysis modules as well as to evaluate basic feasibility of an integrated system for dynamic context capture and event based servoing, and semantic information summarization.     

Visualization for situational awareness

We focus on theater-wide situational awareness covering a large geographic area containing many military units. The Joint Operations Visualization Environment (JOVE), developed at Sarnoff to assist military decision makers, optimizes the display of tactical data for information presentation. JOVE uses three rear-projected displays driven from an SGI Onyx2 Infinite Reality2 computer. Stereoscopic display is available. A joystick, graphical user interface, and speech commands provide interaction. JOVE delivers accurate navigation through the model over a range of viewer positions. This case study describes techniques used for effectively modeling and navigating geospatial and tactical data for situational awareness. The visualization technology could be applied to commercial air traffic control, emergency management response, and geographic information systems as well     

Distributed video coding with adaptive two-step side information generation for smart and interactive media

Conventional video coding standards, such as MPEGx and H.26x, use a hybrid architecture of block-based motion compensation and discrete cosine transform (DCT) within the structure of a complex encoder and a simple decoder. Contrary to conventional video codecs, Wyner-Ziv (WZ) video coding, which is a practical application of distributed video coding (DVC) based on the Slepian-Wolf and WZ theorems, exploits the source correlation at the decoder, thereby allowing for the development of simpler encoders. However, the current WZ video coding algorithms cannot match the coding performance of conventional video coding. In order to improve the coding performance of transform-domain WZ video coding, an adaptive two-step side-information generation method is thus proposed for evaluation and analysis in this study. The proposed method uses decoded WZ frames in the down-sampled spatial resolution along with already decoded coefficients to successively improve rate-distortion (RD) performance as the decoding progresses. The experimental results show that the proposed method, compared to previous WZ video coding method, shows improved coding performance, particularly under critical conditions, such as cases with high motion content.     

HMM based soccer video event detection using enhanced mid-level semantic

Highlight detection is a fundamental step in semantics based video retrieval and personalized sports video browsing. In this paper, an effective hidden Markov models (HMMs) based soccer video event detection method based on a hierarchical video analysis framework is proposed. Soccer video shots are classified into four coarse mid-level semantics: global, median, close-up and audience. Global and local motion information is utilized for the refinement of coarse mid-level semantics. Sequential soccer video is segmented into event clips. Both the temporal transitions of the mid-level semantics and the overall features of an event clip are fused using HMMs to determine the type of event. Highlight detection performance of dynamic Bayesian networks (DBN), conditional random fields (CRF) and the proposed HMM based approach are compared. The average F-score of our highlights (including goal, shoot, foul and placed kick) detection approach is 82.92%, which outperforms that of DBN and CRF by 9.85% and 11.12% respectively. The effects of number of hidden states, overall features, and the refinement of mid-level semantics on the event detection performance are also discussed.     

Volumetric video compression for interactive playback

We develop a volumetric video system which supports interactive browsing of compressed time-varying volumetric features (significant isosurfaces and interval volumes). Since the size of even one volumetric frame in a time-varying 3D data set is very large, transmission and on-line reconstruction are the main bottlenecks for interactive remote visualization of time-varying volume and surface data. We describe a compression scheme for encoding time-varying volumetric features in a unified way, which allows for on-line reconstruction and rendering. To increase the run-time decompression speed and compression ratio, we decompose the volume into small blocks and encode only the significant blocks that contribute to the isosurfaces and interval volumes. The results show that our compression scheme achieves high compression ratio with fast reconstruction, which is effective for interactive client-side rendering of time-varying volumetric features.     

Knowledge discovery using genetic algorithm for maritime situational awareness

Due to the large volume of data related to vessels, to manually pore through and to analyze the information in a bid to identify potential maritime threat is tedious, if at all possible. This study aims to enhance maritime situational awareness through the use of computational intelligence techniques in detecting anomalies. A knowledge discovery system based on genetic algorithm termed as GeMASS was proposed and investigated in this research. In the development of GeMASS, a machine learning approach was applied to discover knowledge that is applicable in characterizing maritime security threats. Such knowledge is often implicit in datasets and difficult to discover by human analysts. As the knowledge relevant to maritime security may vary from time to time, GeMASS was specified to learn from streaming data and to generate up-to-date knowledge in a dynamic fashion. Based on the knowledge discovered, the system functions to screen vessels for anomalies in real-time. Traditionally in maritime security studies, datasets that are applied as knowledge sources are related to vessels’ geographical and movement information. This study investigated a novel leverage of multiple data sources, including Automatic Identification System, classification societies, and port management and security systems for the enhancement of maritime security. A prototype of GeMASS was developed and employed as a vehicle to study and demonstrate the functions of the proposed methodology.     

Optimal and interactive keyframe selection for motion capture

Computational Visual Media - Motion capture is increasingly used in games and movies, but often requires editing before it can be used, for many reasons. The motion may need to be adjusted to...     

网络安全态势感知模型设计和实现

随着网络技术的发展,互联网技术得到突飞猛进的发展,但是在高速发展的背后,给很多网络黑客可乘之机,网络入侵行为逐渐泛滥化,给网络安全带来巨大的挑战.传统的网络维护技术早已不能满足当前技术水平.网络安全态势感知是一门应对网络入侵的新技术,给网络监管人员提供维护网络安全的途径,对当前信息技术的发展有重要的意义.本文将以网络安全态势感知模型的设计为基础,对其提供合理化的建议,以达到维护网络安全的作用.     

面向网络态势感知的多源异构日志传感器设计

探讨了一种基于网络态势感知的多源异构日志传感器的设计.给出了网络结构图和日志传感器的框架图,阐述了传感器设计的难点,相关的关键技术和设计方案.实验证明:该方案在数据采集和分析方面性能有显著的提高.     

Proxy servers for scalable interactive video support

Targeting scalability, load balance and resource efficiency in streaming multimedia applications, proxy servers facilitate faster real-time access to cached objects and reduce response time to document requests     

Intelligent download and cache management for interactive non-linear video

Since the invention of digital video significant progress has been made in reducing the amount of data needed to be transferred in the World Wide Web while improving viewing experience. However the paradigm of linear behavior has not changed at all. While the feature set of traditional digital video may be sufficient for some applications, there are several use cases where a significantly improved way of interacting with the content is highly desirable. It is possible to organize a video in an interactive and non-linear way. Additional information (for example high resolution images) can be added to any scene of the video. The non-linearity of the video flow and the implementation of additional content not found in traditional videos may lead to an increased download volume and/or a playback with many breaks for downloading missing elements. This paper describes a player framework for interactive non-linear videos. We developed the framework and its associated algorithms to simulate the playback of non-linear video content. It minimizes interruption when the sequence of scenes is directly influenced by interaction, while the traditional viewing experience is not altered. The evaluation showed that it is possible to reach shorter startup times using our strategies than using the strategies of other players. Furthermore, we demonstrated that a prefetching of elements does not lead to an increased download volume in every case. In contrast, it can even decrease the download volume if the right delete strategy is selected. It can be noted that the knowledge of the structure of interactive non-linear videos can be used to minimize startup times at the beginning of scenes while the download volume is not increased.     

An interactive video test for pharmaceutical chemist's assistants

CITO (Dutch acronym for National Institute for Educational Measurement) has developed a test based on the interactive possibilities of a videodisc. The test is used within the departments of pharmaceutical education in the senior secondary vocational schools for public health-care and welfare. The test measures the vocational qualifications of students at the end of their schooling. The interactive test is an adequate solution for testing theoretical knowledge and practical skills in a simulated real-life situation. There is an indication that open actions of the students discriminate better between “good” and “weak” students than other types of questions.     

Combining vision and computer graphics for video motion capture

We describe innovative methods for extracting three-dimensional motion of humans and animals from unrestricted monocular video, using a combination of new and established computer vision and computer graphics techniques. We identity features using image processing and active contours. Active contours become anchored to model segments in specified image frames and automatically pull the segments to feature positions in other frames. Adjustments are subject to joint limits and may use inverse kinematics. Occluded contour points are detected using object geometry and do not participate in feature tracking. Interactive adjustments are possible at any time in the process, allowing extraction of complicated movements regardless of background, camera movement, or feature clarity.     

4D video textures for interactive character appearance

4D Video Textures (4DVT) introduce a novel representation for rendering video‐realistic interactive character animation from a database of 4D actor performance captured in a multiple camera studio. 4D performance capture reconstructs dynamic shape and appearance over time but is limited to free‐viewpoint video replay of the same motion. Interactive animation from 4D performance capture has so far been limited to surface shape only. 4DVT is the final piece in the puzzle enabling video‐realistic interactive animation through two contributions: a layered view‐dependent texture map representation which supports efficient storage, transmission and rendering from multiple view video capture; and a rendering approach that combines multiple 4DVT sequences in a parametric motion space, maintaining video quality rendering of dynamic surface appearance whilst allowing high‐level interactive control of character motion and viewpoint. 4DVT is demonstrated for multiple characters and evaluated both quantitatively and through a user‐study which confirms that the visual quality of captured video is maintained. The 4DVT representation achieves >90% reduction in size and halves the rendering cost.     

Fusing uncertain knowledge and evidence for maritime situational awareness via Markov Logic Networks

The concepts of event and anomaly are important building blocks for developing a situational picture of the observed environment. We here relate these concepts to the JDL fusion model and demonstrate the power of Markov Logic Networks (MLNs) for encoding uncertain knowledge and compute inferences according to observed evidence. MLNs combine the expressive power of first-order logic and the probabilistic uncertainty management of Markov networks. Within this framework, different types of knowledge (e.g. a priori, contextual) with associated uncertainty can be fused together for situation assessment by expressing unobservable complex events as a logical combination of simpler evidences. We also develop a mechanism to evaluate the level of completion of complex events and show how, along with event probability, it could provide additional useful information to the operator. Examples are demonstrated on two maritime scenarios of rules for event and anomaly detection.     

Efficient support for interactive operations in multi-resolution video servers

In this paper, we present a placement algorithm that interleaves multi-resolution video streams on a disk array and enables a video server to efficiently support playback of these streams at different resolution levels. We then combine this placement algorithm with a scalable compression technique to efficiently support interactive scan operations (i.e., fast-forward and rewind). We present an analytical model for evaluating the impact of the scan operations on the performance of disk-arr ay-based servers. Our experiments demonstrate that: (1) employing our placement algorithm substantially reduces seek and rotational latency overhead during playback, and (2) exploiting the characteristics of video streams and human perceptual tolerances enables a server to support interactive scan operations without any additional overhead.     

Suspending nanoliter droplet arrays for cell capture and copper ion stimulation

In this paper, we developed a microdevice with raised cylinder arrays for capture of Chlorella vulgaris cells and kinetic analysis of stimulation. The cell solution was injected into the chip and held in the gaps between the opposite raised cylinders, forming 0.314 nL volume of suspending droplets. The number of the captured cells in each nanoliter suspending droplet could be controlled within five under our experiment conditions. When the stimulation reagent of copper ion (Cu²⁺) was injected into the chip and contacted with the suspending droplets, gradient concentration of Cu²⁺ stimulation to the captured C. vulgaris cells by free diffusion was formed. The bioabsorption kinetic process of C. vulgaris cells under continuous Cu²⁺ stimulation was investigated and the two-step bioabsorption process was revealed clearly. The Cu²⁺ toxicity accumulation effect on the cell was also studied by monitoring the fluorescence of cellular chlorophyll.     

Conceptual indexing and active retrieval of video for interactive learning environments

Selecting an instructive story from a video case base is an information retrieval problem, but standard indexing and retrieval techniques [1] were not developed with such applications in mind. The classical model assumes a passive retrieval system queried by interested and well-informed users. In educational situations, students cannot be expected to form appropriate queries or to identify their own ignorance. Systems that teach must, therefore, be active retrievers that formulate their own retrieval cues and reason about the appropriateness of intervention.

The Story Producer for InteractivE Learning (SPIEL) is an active retrieval system for recalling stories to tell to students who are learning social skills in a simulated environment [2, 3]. SPIEL is a component of the Guided Social Simulation (GuSS) architecture [4] used to build YELLO, a program that teaches account executives the fine points of selling Yellow Pages advertising. SPIEL uses structured, conceptual indices derived from research in case-based reasoning [5, 6]. SPIEL's manually-created indices are detailed representations of what stories are about, and they are needed to make precise assessments of stories' relevance.

SPIEL's opportunistic retrieval architecture operates in two phases. During the storage phase, the system uses its educational knowledge encapsulated in a library of “storytelling strategies” to determine, for each story, what an opportunity to tell that story would look like. During the retrieval phase, the system tries to recognize those opportunities while the student interacts with the simulation. This design is similar to “opportunistic memory” architectures proposed for opportunistic planning [7, 8].     

RFID数据流分布式层次复杂事件处理

当前RFID应用中的复杂事件处理通常采用集中式的体系结构,这样不能适用于分布式的应用,并且由于存在单点失效和网络拥塞而缺乏健壮性和可伸缩性。同时,很多高效的RFID复杂事件处理方法不能很好地支持层次复杂事件。提出了一种基于CORBA的分布式层次复杂事件处理模型以及高效的基于查询规划的分布式层次复杂事件处理方法。实验结果表明该方法在处理大规模的分布式RFID应用时是有效的。