SUN Database: Exploring a Large Collection of Scene Categories

Progress in scene understanding requires reasoning about the rich and diverse visual environments that make up our daily experience. To this end, we propose the Scene Understanding database, a nearly exhaustive collection of scenes categorized at the same level of specificity as human discourse. The database contains 908 distinct scene categories and 131,072 images. Given this data with both scene and object labels available, we perform in-depth analysis of co-occurrence statistics and the contextual relationship. To better understand this large scale taxonomy of scene categories, we perform two human experiments: we quantify human scene recognition accuracy, and we measure how typical each image is of its assigned scene category. Next, we perform computational experiments: scene recognition with global image features, indoor versus outdoor classification, and “scene detection,” in which we relax the assumption that one image depicts only one scene category. Finally, we relate human experiments to machine performance and explore the relationship between human and machine recognition errors and the relationship between image “typicality” and machine recognition accuracy.     

仿真视景数据库自动生成系统

本文介绍用于ＣＩＧ系统的视景光学成像仿真数据自动生成系统，从总体的角度阐述了整个系统的设计思想，总体结构，并详细介绍了系统的功能以及特点，阐明了其在仿真中的重要作用。     

Stereoscopic Scene Flow Computation for 3D Motion Understanding

Building upon recent developments in optical flow and stereo matching estimation, we propose a variational framework for the estimation of stereoscopic scene flow, i.e., the motion of points in the three-dimensional world from stereo image sequences. The proposed algorithm takes into account image pairs from two consecutive times and computes both depth and a 3D motion vector associated with each point in the image. In contrast to previous works, we partially decouple the depth estimation from the motion estimation, which has many practical advantages. The variational formulation is quite flexible and can handle both sparse or dense disparity maps. The proposed method is very efficient; with the depth map being computed on an FPGA, and the scene flow computed on the GPU, the proposed algorithm runs at frame rates of 20 frames per second on QVGA images (320×240 pixels). Furthermore, we present solutions to two important problems in scene flow estimation: violations of intensity consistency between input images, and the uncertainty measures for the scene flow result.     

Generating Dynamic Projection Images for Scene Representation and Understanding

This paper explores an interesting image projection produced by scanning dynamic scenes with a slit camera. Based on the concept of Anorthoscopic Perception, we investigate how a two-dimensionalDynamic Projection Imageof three-dimensional scenes is generated from consecutive 1-D snapshots taken through a slit, when the relative motion is homogeneous between the viewer and scenes. By moving the camera in the 3-D environment or rotating an object, we can obtain various dynamic projection images. These dynamic projection images contain major spatial and temporal information about 3-D scenes in a small amount of data. Consequently, the projection is suited for the memorization, registration, and indexing of image sequences. The generated images also directly show some of the motion properties in dynamic scenes. If a relative motion between the camera and a subject is planned properly, the dynamic projection image can even provide a texture image of the subject along with some expected photometry characteristics. Therefore, the dynamic projection can facilitate dynamic object recognition, 3-D structure acquisition, and image compression, all for a stable motion between the objects and camera. We outline various applications in vision, robotics, and multimedia and summarize the motion types and the camera setting for generating such dynamic projection images.     

Understanding the Adversary: Virus Writers and Beyond

In this column, I explain why understanding the cyber adversary - virus writers, for example - is important and explore the history of both public perceptions and reality, which might not always be the same. In a later column, I'll delve into recent developments, such as the reported increase in financial motivation on the part of spyware/adware manufacturers     

Beyond Tracking: Modelling Activity and Understanding Behaviour

In this work, we present a unified bottom-up and top-down automatic model selection based approach for modelling complex activities of multiple objects in cluttered scenes. An activity of multiple objects is represented based on discrete scene events and their behaviours are modelled by reasoning about the temporal and causal correlations among different events. This is significantly different from the majority of the existing techniques that are centred on object tracking followed by trajectory matching. In our approach, object-independent events are detected and classified by unsupervised clustering using Expectation-Maximisation (EM) and classified using automatic model selection based on Schwarz's Bayesian Information Criterion (BIC). Dynamic Probabilistic Networks (DPNs) are formulated for modelling the temporal and causal correlations among discrete events for robust and holistic scene-level behaviour interpretation. In particular, we developed a Dynamically Multi-Linked Hidden Markov Model (DML-HMM) based on the discovery of salient dynamic interlinks among multiple temporal processes corresponding to multiple event classes. A DML-HMM is built using BIC based factorisation resulting in its topology being intrinsically determined by the underlying causality and temporal order among events. Extensive experiments are conducted on modelling activities captured in different indoor and outdoor scenes. Our experimental results demonstrate that the performance of a DML-HMM on modelling group activities in a noisy and cluttered scene is superior compared to those of other comparable dynamic probabilistic networks including a Multi-Observation Hidden Markov Model (MOHMM), a Parallel Hidden Markov Model (PaHMM) and a Coupled Hidden Markov Model (CHMM). First online version published in February, 2006     

Using Human Perceptual Categories for Content-Based Retrieval from a Medical Image Database

It is often difficult to come up with a well-principled approach to the selection of low-level features for characterizing images for content-based retrieval. This is particularly true for medical imagery, where gross characterizations on the basis of color and other global properties do not work. An alternative for medical imagery consists of the “scattershot” approach that first extracts a large number of features from an image and then reduces the dimensionality of the feature space by applying a feature selection algorithm such as the Sequential Forward Selection method.This contribution presents a better alternative to initial feature extraction for medical imagery. The proposed new approach consists of (i) eliciting from the domain experts (physicians, in our case) the perceptual categories they use to recognize diseases in images; (ii) applying a suite of operators to the images to detect the presence or the absence of these perceptual categories; (iii) ascertaining the discriminatory power of the perceptual categories through statistical testing; and, finally, (iv) devising a retrieval algorithm using the perceptual categories. In this paper we will present our proposed approach for the domain of high-resolution computed tomography (HRCT) images of the lung. Our empirical evaluation shows that feature extraction based on physicians' perceptual categories achieves significantly higher retrieval precision than the traditional scattershot approach. Moreover, the use of perceptually based features gives the system the ability to provide an explanation for its retrieval decisions, thereby instilling more confidence in its users.     

数据库应用系统理解冲突测试研究

与传统软件相比，数据库应用系统更加庞大而复杂，因此对数据库应用系统的测试工作更加困难．在系统开发过程中，由于开发人员对数据库模式的理解不同产生一些错误．这些错误表现为各功能单独运行时结果正常，而多个功能一起运行时就会造成错误．针对这种现象，即数据库模式的理解冲突错误，本文给出了形式化的定义，并提出了一种基于功能一数据访问图的集成测试策略，使得在通常需要在系统测试和用户测试阶段才能发现的问题能在集成测试阶段发现和修正．     

Curriculum Model Adaptation with Synthetic and Real Data for Semantic Foggy Scene Understanding

International Journal of Computer Vision - This work addresses the problem of semantic scene understanding under fog. Although marked progress has been made in semantic scene understanding, it is...     

大规模场景的消隐技术

计算机图形学的快速发展使处理大规模场景中的数据变得日益重要,文章主要讨论了近几年最新发展的消隐算法。     

Beyond exchange models: Understanding the structure of B2B information systems

This paper provides a framework to characterize the structure of information systems that enable transactions between businesses. We propose that the structure of various inter-organizational information systems, including traditional EDI systems and more recent web based exchanges, can be characterized in terms of four underlying dimensions: control, relational support, integration, and technology dimensions. In our paper we explain and operationalize these dimensions as constructs, and examine how each of them might be affected by inter-organizational relationship or product specific antecedents. We argue that these dimensions are fundamental to more realistically representing an organizations involvement with and use of inter-organizational information systems. Further, we argue that specific product types and preexisting relationships between organizations will be strongly associated with the variables representing the dimensions of Inter-organizational information systems.     

Deep Affect Prediction in-the-Wild: Aff-Wild Database and Challenge,Deep Architectures,and Beyond

International Journal of Computer Vision - Automatic understanding of human affect using visual signals is of great importance in everyday human–machine interactions. Appraising human...     

Beyond ubiquitous computing: The Malaysian HoneyBee project for Innovative Digital Economy

In the proposed advanced computing environment, known as the HoneyBee Platform, various computing devices using single or multiple interfaces and technologies/standards need to communicate and cooperate efficiently with a certain level of security and safety measures. These computing devices may be supported by different types of operating systems with different features and levels of security support. In order to ensure that all operations within the environment can be carried out seamlessly in an ad-hoc manner, there is a need for a common mobile platform to be developed. The purpose of this long-term project is to investigate and implement a new functional layered model of the common mobile platform with secured and trusted ensemble computing architecture for an innovative Digital Economic Environment in the Malaysian context. This mobile platform includes a lightweight operating system to provide a common virtual environment, a middleware for providing basic functionalities of routing, resource and network management, as well as to provide security, privacy and a trusted environment. A generic application programming interface is provided for application developers to access underlying resources. The aim is for the developed platform to act as the building block for an ensemble environment, upon which higher level applications could be built. Considered as the most essential project in a series of related projects towards a more digital socio-economy in Malaysia, this article presents the design of the target computational platform as well as the conceptual framework for the HoneyBee project.     

aiTPR: Attribute Interaction-Tensor Product Representation for Image Caption

Neural Processing Letters - Region visual features enhance the generative capability of the machines based on features. However, they lack proper interaction-based attentional perceptions and end...     

Flow Curves: an Intuitive Interface for Coherent Scene Deformation

Effective composition in visual arts relies on the principle of movement, where the viewer's eye is directed along subjective curves to a center of interest. We call these curves subjective because they may span the edges and/or center‐lines of multiple objects, as well as contain missing portions which are automatically filled by our visual system. By carefully coordinating the shape of objects in a scene, skilled artists direct the viewer's attention via strong subjective curves. While traditional 2D sketching is a natural fit for this task, current 3D tools are object‐centric and do not accommodate coherent deformation of multiple shapes into smooth flows. We address this shortcoming with a new sketch‐based interface called Flow Curves which allows coordinating deformation across multiple objects. Core components of our method include an understanding of the principle of flow, algorithms to automatically identify subjective curve elements that may span multiple disconnected objects, and a deformation representation tailored to the view‐dependent nature of scene movement. As demonstrated in our video, sketching flow curves requires significantly less time than using traditional 3D editing workflows.     

Scene pathfinder: unsupervised clustering techniques for movie scenes extraction

The need for watching movies is in perpetual increase due to the widespread of the internet and the increasing popularity of the video on demand service. The important mass of movies stored in the Internet or in VOD servers need to be structured to accelerate the browsing operation. In this paper, we propose a new system called "The Scene Pathfinder" that aims at segmenting the movies into scenes to give users the opportunity to have a non- sequential access and to watch particular scenes of the movie. This helps them to judge quickly the movie and decide if they have to buy or to download it and avoiding waste of time and money. The proposed approach is multimodal. We use both of visual and auditory information to accomplish the segmentation. We base on the assumption that every movie scene is either action or non- action scene. Non-action scenes are generally characterized by static backgrounds and occur in the same place. For this reason, we base on the content information and on the Kohonen map to extract these kinds of scenes (shots agglomerations). Action scenes are characterized by high tempo and motion. For this reason, we base on tempo features and on the Fuzzy CMeans to classify shots and to localize the action zones. The two processes are complementary. Indeed, the over segmentation that may occur in the extraction of action scenes by basing on the content information is repaired by the Fuzzy clustering. Our system is tested on a varied database and obtained results show the merit of our approach and that our assumptions are well-founded.