Distributed intelligence for multi-camera visual surveillance

Latest advances in hardware technology and state of the art of computer vision and artificial intelligence research can be employed to develop autonomous and distributed monitoring systems. The paper proposes a multi-agent architecture for the understanding of scene dynamics merging the information streamed by multiple cameras. A typical application would be the monitoring of a secure site, or any visual surveillance application deploying a network of cameras. Modular software (the agents) within such architecture controls the different components of the system and incrementally builds a model of the scene by merging the information gathered over extended periods of time. The role of distributed artificial intelligence composed of separate and autonomous modules is justified by the need for scalable designs capable of co-operating to infer an optimal interpretation of the scene. Decentralizing intelligence means creating more robust and reliable sources of interpretation, but also allows easy maintenance and updating of the system. Results are presented to support the choice of a distributed architecture, and to prove that scene interpretation can be incrementally and efficiently built by modular software.     

Hierarchical database for a multi-camera surveillance system

This paper presents a framework for event detection and video content analysis for visual surveillance applications. The system is able to coordinate the tracking of objects between multiple camera views, which may be overlapping or non-overlapping. The key novelty of our approach is that we can automatically learn a semantic scene model for a surveillance region, and have defined data models to support the storage of tracking data with different layers of abstraction into a surveillance database. The surveillance database provides a mechanism to generate video content summaries of objects detected by the system across the entire surveillance region in terms of the semantic scene model. In addition, the surveillance database supports spatio-temporal queries, which can be applied for event detection and notification applications.     

Adaptive online camera coordination for multi-camera multi-target surveillance

Online camera selection is introduced as a result of the improved mobility of cameras and the increased scale of surveillance systems. Most existing camera assignment algorithms achieve an optimal observation under the assumption of the unlimited camera computational capacities. However, practical surveillance systems experience resource limitation and see a degradation in the system performance as the number of objects to be processed increases. To address this issue, we propose an adaptive camera assignment algorithm considering the limited camera computational capacities. In so doing, camera resources can be dynamically allocated to multiple objects according to their priorities and the current camera computational load. Experimental results illustrate that the proposed camera assignment algorithm is capable of maintaining a constant frame rate and achieving a substantially decreased object rejection rate in comparison with the algorithm presented by Bakhtari and Benhabib.     

A semantic approach to a framework for business domain software systems

In enterprise firms, enormous amounts of electronic documents are generated by business analysts and other business domain application users. Applications that use these documents are often driven by business logic that is hard-coded together with application logic. One approach to the separation of business logic from applications is to create and maintain business and information extraction rules in an external, user-friendly format. The drawback of such an externalization is that the business rules, usually, do not have machine interpretable semantics. This situation often leads to misinterpretation of domain analysis documents, which can inhibit the productivity of computer-assisted analytical work and the effectiveness of business solutions. This paper proposes an ontology and rule-based framework for the development of business domain applications, which includes semantic processing of externalized business rules and to some extent externalization of application logic. The creation of external information extraction rules by the business analyst is a cumbersome and time consuming task. In order to overcome this problem, the framework also includes a rule learning system to semi-automate the generation of information extraction rules from source documents with the help of manual annotations. The main idea behind the work presented in this paper is to re-engineer very large enterprise information systems to adapt to Semantic Web computing techniques. The work presented in this paper is inspired by an industrial project.     

A novel framework and concept-based semantic search Interface for abnormal crowd behaviour analysis in surveillance videos

Multimedia Tools and Applications - Monitoring continuously captured surveillance videos is a challenging and time consuming task. To assist this issue, a new framework is introduced that applies...     

A semantic framework for open processes

We propose a general methodology for analysing the behaviour of open systems modelled as coordinators, i.e., open terms of suitable process calculi. A coordinator is understood as a process with holes or placeholders where other coordinators and components (i.e., closed terms) can be plugged in, thus influencing its behaviour. The operational semantics of coordinators is given by means of a symbolic transition system, where states are coordinators and transitions are labeled by spatial/modal formulae expressing the potential interaction that plugged components may enable. Behavioural equivalences for coordinators, like strong and weak bisimilarities, can be straightforwardly defined over such a transition system. Different from other approaches based on universal closures, i.e., where two coordinators are considered equivalent when all their closed instances are equivalent, our semantics preserves the openness of the system during its evolution, thus allowing dynamic instantiation to be accounted for in the semantics. To further support the adequacy of the construction, we show that our symbolic equivalences provide correct approximations of their universally closed counterparts, coinciding with them over closed components. For process calculi in suitable formats, we show how tractable symbolic semantics can be defined constructively using unification.     

A framework for defining semantic metrics

Software metrics are measures of particular characteristics found in the software. Research in this area seeks to identify relationships between software characteristics and software engineering processes. Most software metrics are based on program structure and are determined statically. This article presents a framework by which semantic information can be quantified. By semantic information, we mean information concerning what occurs internally during execution as program states are created.     

A semantic framework for metamodel-based languages

In the model-based development context, metamodel-based languages are increasingly being defined and adopted either for general purposes or for specific domains of interest. However, meta-languages such as the MOF (Meta Object Facility)—combined with the OCL (Object Constraint Language) for expressing constraints—used to specify metamodels focus on structural and static semantics but have no built-in support for specifying behavioral semantics. This paper introduces a formal semantic framework for the definition of the semantics of metamodel-based languages. Using metamodelling principles, we propose several techniques, some based on the translational approach while others based on the weaving approach, all showing how the Abstract State Machine formal method can be integrated with current metamodel engineering environments to endow language metamodels with precise and executable semantics. We exemplify the use of our semantic framework by applying the proposed techniques to the OMG metamodelling framework for the behaviour specification of the Finite State Machines provided in terms of a metamodel.     

A semantic interoperability framework for software as a service systems in cloud computing environments

In cloud computing environments in software as a service (SaaS) level, interoperability refers to the ability of SaaS systems on one cloud provider to communicate with SaaS systems on another cloud provider. One of the most important barriers to the adoption of SaaS systems in cloud computing environments is interoperability. A common tactic for enabling interoperability is the use of an interoperability framework or model. During the past few years, in cloud SaaS level, various interoperability frameworks and models have been developed to provide interoperability between systems. The syntactic interoperability of SaaS systems have already been intensively researched. However, not enough consideration has been given to semantic interoperability issues. Achieving semantic interoperability is a challenge within the world of SaaS in cloud computing environments. Therefore, a semantic interoperability framework for SaaS systems in cloud computing environments is needed. We develop a semantic interoperability framework for cloud SaaS systems. The capabilities and value of service oriented architecture for semantic interoperability within cloud SaaS systems have been studied and demonstrated. This paper is accomplished through a number of steps (research methodology). It begins with a study on related works in the literature. Then, problem statement and research objectives are explained. In the next step, semantic interoperability requirements for SaaS systems in cloud computing environments that are needed to support are analyzed. The details of the proposed semantic interoperability framework for SaaS systems in cloud computing environments are presented. It includes the design of the proposed semantic interoperability framework. Finally, the evaluation methods of the semantic interoperability framework are elaborated. In order to evaluate the effectiveness of the proposed semantic interoperability framework for SaaS systems in cloud computing environments, extensive experimentation and statistical analysis have been performed. The experiments and statistical analysis specify that the proposed semantic interoperability framework for cloud SaaS systems is able to establish semantic interoperability between cloud SaaS systems in a more efficient way. It is concluded that using the proposed framework, there is a significant improvement in the effectiveness of semantic interoperability of SaaS systems in cloud computing environments.     

Information assimilation framework for event detection in multimedia surveillance systems

Most multimedia surveillance and monitoring systems nowadays utilize multiple types of sensors to detect events of interest as and when they occur in the environment. However, due to the asynchrony among and diversity of sensors, information assimilation – how to combine the information obtained from asynchronous and multifarious sources is an important and challenging research problem. In this paper, we propose a framework for information assimilation that addresses the issues – “when”, “what” and “how” to assimilate the information obtained from different media sources in order to detect events in multimedia surveillance systems. The proposed framework adopts a hierarchical probabilistic assimilation approach to detect atomic and compound events. To detect an event, our framework uses not only the media streams available at the current instant but it also utilizes their two important properties – first, accumulated past history of whether they have been providing concurring or contradictory evidences, and – second, the system designer’s confidence in them. The experimental results show the utility of the proposed framework.     

Performance analysis for automated gait extraction and recognition in multi-camera surveillance

Many studies have confirmed that gait analysis can be used as a new biometrics. In this research, gait analysis is deployed for people identification in multi-camera surveillance scenarios. We present a new method for viewpoint independent markerless gait analysis that does not require camera calibration and works with a wide range of walking directions. These properties make the proposed method particularly suitable for gait identification in real surveillance scenarios where people and their behaviour need to be tracked across a set of cameras. Tests on 300 synthetic and real video sequences, with subjects walking freely along different walking directions, have been performed. Since the choice of the cameras’ characteristics is a key-point for the development of a smart surveillance system, the performance of the proposed approach is measured with respect to different video properties: spatial resolution, frame-rate, data compression and image quality. The obtained results show that markerless gait analysis can be achieved without any knowledge of camera’s position and subject’s pose. The extracted gait parameters allow recognition of people walking from different views with a mean recognition rate of 92.2% and confirm that gait can be effectively used for subjects’ identification in a multi-camera surveillance scenario.     

A semantic framework for textual data enrichment

In this work we present a semantic framework suitable of being used as support tool for recommender systems. Our purpose is to use the semantic information provided by a set of integrated resources to enrich texts by conducting different NLP tasks: WSD, domain classification, semantic similarities and sentiment analysis. After obtaining the textual semantic enrichment we would be able to recommend similar content or even to rate texts according to different dimensions. First of all, we describe the main characteristics of the semantic integrated resources with an exhaustive evaluation. Next, we demonstrate the usefulness of our resource in different NLP tasks and campaigns. Moreover, we present a combination of different NLP approaches that provide enough knowledge for being used as support tool for recommender systems. Finally, we illustrate a case of study with information related to movies and TV series to demonstrate that our framework works properly.     

A framework for automatic semantic video annotation

Hypermedia composite templates define generic structures of nodes and links to be added to a document composition, providing spatio-temporal synchronization semantics. This paper presents EDITEC, a graphical editor for hypermedia composite templates. EDITEC templates are based on the XTemplate 3.0 language. The editor was designed for offering a user-friendly visual approach. It presents a new method that provides several options for representing iteration structures graphically, in order to specify a certain behavior to be applied to a set of generic document components. The editor provides a multi-view environment, giving the user a complete control of the composite template during the authoring process. Composite templates can be used in NCL documents for embedding spatio-temporal semantics into NCL contexts. NCL is the standard declarative language used for the production of interactive applications in the Brazilian digital TV system and ITU H.761 IPTV services. Hypermedia composite templates could also be used in other hypermedia authoring languages offering new types of compositions with predefined semantics.     

SETL: A programmable semantic extract-transform-load framework for semantic data warehouses

In order to create better decisions for business analytics, organizations increasingly use external structured, semi-structured, and unstructured data in addition to the (mostly structured) internal data. Current Extract-Transform-Load (ETL) tools are not suitable for this “open world scenario” because they do not consider semantic issues in the integration processing. Current ETL tools neither support processing semantic data nor create a semantic Data Warehouse (DW), a repository of semantically integrated data. This paper describes our programmable Semantic ETL (SETL) framework. SETL builds on Semantic Web (SW) standards and tools and supports developers by offering a number of powerful modules, classes, and methods for (dimensional and semantic) DW constructs and tasks. Thus it supports semantic data sources in addition to traditional data sources, semantic integration, and creating or publishing a semantic (multidimensional) DW in terms of a knowledge base. A comprehensive experimental evaluation comparing SETL to a solution made with traditional tools (requiring much more hand-coding) on a concrete use case, shows that SETL provides better programmer productivity, knowledge base quality, and performance.     

Moving people tracking with detection by latent semantic analysis for visual surveillance applications

The latent semantic analysis (LSA) has been widely used in the fields of computer vision and pattern recognition. Most of the existing works based on LSA focus on behavior recognition and motion classification. In the applications of visual surveillance, accurate tracking of the moving people in surveillance scenes, is regarded as one of the preliminary requirement for other tasks such as object recognition or segmentation. However, accurate tracking is extremely hard under challenging surveillance scenes where similarity among multiple objects or occlusion among multiple objects occurs. Usual temporal Markov chain based tracking algorithms suffer from the ‘tracking error accumulation problem’. The accumulated errors can finally make the tracking to drift from the target. To handle the problem of tracking drift, some authors have proposed the idea of using detection along with tracking as an effective solution. However, many of the critical issues still remain unsettled in these detection based tracking algorithms. In this paper, we propose a novel moving people tracking with detection based on (probabilistic) LSA. By employing a novel ‘twin-pipeline’ training framework to find the latent semantic topics of ‘moving people’, the proposed detection can effectively detect the interest points on moving people in different indoor and outdoor environments with camera motion. Since the detected interest points on different body parts can be used to locate the position of moving people more accurately, by combining the detection with incremental subspace learning based tracking, the proposed algorithms resolves the problem of tracking drift during each target appearance update process. In addition, due to the time independent processing mechanism of detection, the proposed method is also able to handle the error accumulation problem. The detection can calibrate the tracking errors during updating of each state of the tracking algorithm. Extensive, experiments on various surveillance environments using different benchmark datasets have proved the accuracy and robustness of the proposed tracking algorithm. Further, the experimental comparison results clearly show that the proposed tracking algorithm outperforms the well known tracking algorithms such as ISL, AMS and WSL algorithms. Furthermore, the speed performance of the proposed method is also satisfactory for realistic surveillance applications.     

A framework for real-time semantic social media analysis

This paper presents a framework for collecting and analysing large volume social media content. The real-time analytics framework comprises semantic annotation, Linked Open Data, semantic search, and dynamic result aggregation components. In addition, exploratory search and sense-making are supported through information visualisation interfaces, such as co-occurrence matrices, term clouds, treemaps, and choropleths. There is also an interactive semantic search interface (Prospector), where users can save, refine, and analyse the results of semantic search queries over time. Practical use of the framework is exemplified through three case studies: a general scenario analysing tweets from UK politicians and the public’s response to them in the run up to the 2015 UK general election, an investigation of attitudes towards climate change expressed by these politicians and the public, via their engagement with environmental topics, and an analysis of public tweets leading up to the UK’s referendum on leaving the EU (Brexit) in 2016. The paper also presents a brief evaluation and discussion of some of the key text analysis components, which are specifically adapted to the domain and task, and demonstrate scalability and efficiency of our toolkit in the case studies.