RUSHES—an annotation and retrieval engine for multimedia semantic units

Multimedia analysis and reuse of raw un-edited audio visual content known as rushes is gaining acceptance by a large number of research labs and companies. A set of research projects are considering multimedia indexing, annotation, search and retrieval in the context of European funded research, but only the FP6 project RUSHES is focusing on automatic semantic annotation, indexing and retrieval of raw and un-edited audio-visual content. Even professional content creators and providers as well as home-users are dealing with this type of content and therefore novel technologies for semantic search and retrieval are required. In this paper, we present a summary of the most relevant achievements of the RUSHES project, focusing on specific approaches for automatic annotation as well as the main features of the final RUSHES search engine.     

Interfacing human and computer with wireless body area sensor networks: the WiMoCA solution

Wireless Body Area Sensor Networks (WBASN) are an emerging technology enabling the design of natural human–computer interfaces (HCI). Automatic recognition of human motion, gestures, and activities is studied in several contexts. For example, mobile computing technology is being considered as a replacement of traditional input systems. Moreover, body posture and activity monitoring can be used for entertainment and health-care applications. However, until now, little work has been done to develop flexible and efficient WBASN solutions suitable for a wide range of applications. Their requirements pose new challenges for sensor network designs, such as optimizing traditional solutions for use as environmental monitoring-like applications and developing on-the-field stress tests. In this paper, we demonstrate the flexibility of a custom-designed WBASN called WiMoCA with respect to a wide range of posture and activity recognition applications by means of practical implementation and on-the-field testing. Nodes of the network mounted on different parts of the human body exploit tri-axial accelerometers to detect its movements. The advanced digital Micro-electro-mechanical system (MEMS) based inertial sensor has been chosen for WiMoCA because it demonstrated high flexibility of use in many different situations, providing the chance to exploit both static and dynamic acceleration components for different purposes. Furthermore, the sensibility and accuracy of the sensing element is perfectly adequate for monitoring human movement, while keeping cost low and size compact, thus meeting our requirements. We implemented three types of applications, stressing the WBASN in many aspects. In fact, they are characterized by different requirements in terms of accuracy, timeliness, and computation distributed on sensing nodes. For each application, we describe its implementation, and we discuss results about performance and power consumption.

Real-time scheduling for energy harvesting sensor nodes

Energy harvesting has recently emerged as a feasible option to increase the operating time of sensor networks. If each node of the network, however, is powered by a fluctuating energy source, common power management solutions have to be reconceived. This holds in particular if real-time responsiveness of a given application has to be guaranteed. Task scheduling at the single nodes should account for the properties of the energy source, capacity of the energy storage as well as deadlines of the single tasks. We show that conventional scheduling algorithms (like e.g. EDF) are not suitable for this scenario. Based on this motivation, we have constructed optimal scheduling algorithms that jointly handle constraints from both energy and time domain. Further we present an admittance test that decides for arbitrary task sets, whether they can be scheduled without deadline violations. To this end, we introduce the concept of energy variability characterization curves (EVCC) which nicely captures the dynamics of various energy sources. Simulation results show that our algorithms allow significant reductions of the battery size compared to Earliest Deadline First scheduling.

A low-power wireless video sensor node for distributed object detection

In this paper we propose MicrelEye, a wireless video node for cooperative distributed video processing applications that involve image classification. The node is equipped with a low-cost VGA CMOS image sensor, a reconfigurable processing engine (FPGA, Microcontroller, SRAM) and a Bluetooth 100-m transceiver. It has a size of few cubic centimeters and its typical power consumption is approximately ten times less than that of typical commercial DSP-based solutions. As regards classification, a highly optimized hardware-oriented support vector machine-like (SVM-like) algorithm called ERSVM is proposed and implemented. We describe our hardware and software architecture, its performance and power characteristics. The case study considered in this paper is people detection. The obtained results suggest that the present technology allows for the design of simple intelligent video nodes capable of performing classification tasks locally.

Shot scale distribution in art films

The scale of shot, i.e. the apparent distance of the camera from the main subject of a scene, is one of the main stylistic and narrative functions of audiovisual products, conveying meaning and inducing the viewer’s emotional state. The statistical distribution of different shot scales in a film may be an important identifier of an individual film, an individual author, and of various narrative and affective functions of a film. In order to understand at which level shot scale distribution (SSD) of a movie might become its fingerprint, it is necessary to produce automatic recognition of shot scale on a large movie corpus. In our work we propose an automatic framework for estimating the SSD of a movie by using inherent characteristics of shots containing information about camera distance, without the need to recover the 3D structure of the scene. In the experimental investigation, the comparison of obtained results with manual SSD annotations proves the validity of the framework. Experiments conducted on movies by Michelangelo Antonioni taken from different stylistic periods (1950–57, 1960–64, 1966–75, 1980–82) show a strong similarity in shot scale distributions within each period, thus opening interesting research lines regarding the possible aesthetic and cognitive sources of such a regularity.     

p53 expression, DNA content and cell proliferation in primary and synchronous metastatic lesions from ovarian surface epithelial-stromal tumours

The aim of this study was to investigate biological heterogeneity between primary and metastatic ovarian cancer lesions from individual patients as a means of elucidating steps in clinical progression. Cancer tissue from 61 untreated patients with ovarian surface epithelial-stromal tumours was examined. p53 expression detected immunocytochemically by the PAb1801 antibody, DNA content evaluated by flow cytometry, and cell proliferation evaluated as the [3H]thymidine labelling index were investigated in primary tumours and corresponding synchronous metastases. The frequency of p53 positivity was similar in primary (62%) and metastatic (66%) sites, with an agreement between the two lesions from the same patient in 97% of the cases. Similarly, aneuploidy frequency (80%) and DNA indices were superimposable in primary and metastatic lesions from the same patient, with a 94% agreement. The frequency of aneuploidy was higher in p53-positive than in p53-negative lesions. An overall poor agreement (rs = 0.44) was observed for proliferative activity of primary and metastatic lesions, due to a heterogeneous profile in omental with respect to primary tumours, which was mainly evident in p53-positive cancers. Conversely, cell proliferation of peritoneal, abdominal and pelvic lesions was qualitatively similar to that of the primary tumour in 88% of patients.     

Minimizing memory access energy in embedded systems by selective instruction compression

We propose a technique for reducing the energy spent in the memory-processor interface of an embedded system during the execution of firmware code. The method is based on the idea of compressing the most commonly executed instructions so as to reduce the energy dissipated during memory access. Instruction decompression is performed on-the-fly by a hardware block located between processor and memory: No changes to the processor architecture are required. Hence, our technique is well suited for systems employing IP cores whose internal architecture cannot be modified. We describe a number of decompression schemes and architectures that effectively trade off hardware complexity and static code size increase for memory energy and bandwidth reduction, as proved by the experimental data we have collected by executing several test programs on different design templates.     

Discrete-time battery models for system-level low-power design

For portable applications, long battery lifetime is the ultimate design goal. Therefore, the availability of battery and voltage converter models providing accurate estimates of battery lifetime is key for system-level low-power design frameworks. In this paper, we introduce a discrete-time model for the complete power supply subsystem that closely approximates the behavior of its circuit-level continuous-time counterpart. The model is abstract and efficient enough to enable event-driven simulation of digital systems described at a very high level of abstraction and that includes, among their components, also the power supply. The model gives the designer the possibility of estimating battery lifetime during system-level design exploration, as shown by the results we have collected on meaningful case studies. In addition, it is flexible and it can thus be employed for different battery chemistries     

Specification and analysis of power-managed systems

Dynamic power management encompasses several techniques for reducing energy dissipation in electronic systems by selective slowdown or shutdown of components. We present a theoretical framework for explaining and classifying different approaches to power management. Within this framework, we model power-manageable components, workloads, and controllers as discrete-event systems (DESs). The structure of these DESs is specified in terms of physical states (representing operation modes) and events (triggering state transitions), while system behavior is specified in terms of next-event and next-state functions. In particular, nondeterministic next-event and next-state functions are modeled by conditional probability distributions, according to generalized semi-Markov processes (GSMPs). The modeling framework provides a general denotational model for system specification and a rigorous execution semantics that enables event-driven simulation. We introduce a modeling framework, built on top of MathWork's Simulink, supporting the specification and execution of our model. In particular, we present templates for the Simulink simulator to execute GSMP models, and we describe how to use such templates for specifying, analyzing, and optimizing dynamic power-managed systems. Finally, we demonstrate the expressive power and versatility of the proposed approach by using the modeling framework and the simulator for the analysis of representative real-life case studies, including the Intel Xscale processor architecture, a multitasking real-time system, and a sensor network.