Algorithmic and architectural design for real-time and power-efficient Retinex image/video processing

This paper presents novel algorithmic and architectural solutions for real-time and power-efficient enhancement of images and video sequences. A programmable class of Retinex-like filters, based on the separation of the illumination and reflectance components, is proposed. The dynamic range of the input image is controlled by applying a suitable non-linear function to the illumination, while the details are enhanced by processing the reflectance. An innovative spatially recursive rational filter is used to estimate the illumination. Moreover, to improve the visual quality results of two-branch Retinex operators when applied to videos, a novel three-branch technique is proposed which exploits both spatial and temporal filtering. Real-time implementation is obtained by designing an Application Specific Instruction-set Processor (ASIP). Optimizations are addressed at algorithmic and architectural levels. The former involves arithmetic accuracy definition and linearization of non-linear operators; the latter includes customized instruction set, dedicated memory structure, adapted pipeline, bypasses, custom address generator, and special looping structures. The ASIP is synthesized in standard-cells CMOS technology and its performances are compared to known Digital signal processor (DSP) implementations of real-time Retinex filters. As a result of the comparison, the proposed algorithmic/architectural design outperforms state-of-art Retinex-like operators achieving the best trade-off between power consumption, flexibility, and visual quality.

A hierarchical pipelining architecture and FPGA implementation for lifting-based 2-D DWT

Numerous VLSI architectures for 2-D discrete wavelet transform (DWT) have been brought forward. While most of the designs displayed good performance through parallel processing, few of them addressed thoroughly how to sustain such high throughput computing which is crucial in real-time applications. Although the affordable data transfer bandwidth has been increased tremendously during the past decade, the pressure on data communication has not yet been relieved from stream-intensive applications. The design of 2-D DWT belongs to such cases. In this paper, we expose the performance gap between the computing core and the entire system, distinguishing them by quantitative approach with metrics of peak performance and mean-time performance. In order to narrow down the discrepancy without degrading either of the two criteria, on the one hand, we introduce a software-pipelining lifting-based computing kernel to remove data dependence for peak performance, on the other hand, we apply loop fusing technique and a hierarchical pipelining method to enhance data locality and boost the mean-time performance. The architecture has been implemented in Xilinx Virtex-II FPGA, taking advantage of Virtex-II’s embedded multipliers and block RAMs. We use Daubechies (9, 7) and LeGall (5, 3) filters (the default lossy and lossless filters in JPEG2000) for illustration whereas it is a general method for other DWT filters. The post-place and routing operation frequency for Daubechies (9, 7) is 138 MHz. Notably, the mean-time performance parameterized by image size and decomposition level achieves closely to peak performance.

---

Change-driven data flow image processing architecture for optical flow computation

Optical flow computation has been extensively used for motion estimation of objects in image sequences. The results obtained by most optical flow techniques are computationally intensive due to the large amount of data involved. A new change-based data flow pipelined architecture has been developed implementing the Horn and Schunk smoothness constraint; pixels of the image sequence that significantly change, fire the execution of the operations related to the image processing algorithm. This strategy reduces the data and, combined with the custom hardware implemented, it achieves a significant optical flow computation speed-up with no loss of accuracy. This paper presents the bases of the change-driven data flow image processing strategy, as well as the implementation of custom hardware developed using an Altera Stratix PCI development board.

Empirical evaluation of optimization algorithms when used in goal-oriented automated test data generation techniques

Software testing is an essential process in software development. Software testing is very costly, often consuming half the financial resources assigned to a project. The most laborious part of software testing is the generation of test-data. Currently, this process is principally a manual process. Hence, the automation of test-data generation can significantly cut the total cost of software testing and the software development cycle in general. A number of automated test-data generation approaches have already been explored. This paper highlights the goal-oriented approach as a promising approach to devise automated test-data generators. A range of optimization techniques can be used within these goal-oriented test-data generators, and their respective characteristics, when applied to these situations remain relatively unexplored. Therefore, in this paper, a comparative study about the effectiveness of the most commonly used optimization techniques is conducted.

A Technique for Adaptive Scheduling of Soft Real-Time Tasks

A number of multimedia and process control applications can take advantage from the ability to adapt soft real-time load to available computational capacity. This capability is required, for example, to react to changed operating conditions as well as to ensure graceful degradation of an application under transient overloads. In this paper, we illustrate a novel adaptive scheduling technique based on rate modulation of a set of periodic tasks in a range of admissible rates. By casting constraints on rate ranges in a linear programming formulation, several adaptation policies can be considered, along with additional constraints reflecting various application requirements. The paper investigates the effectiveness of rate modulation strategies both on simulated task sets and on real experiments. Partial support for this research has been provided by MURST, Italy (PRIN project ISIDE on “Dependable reactive computing systems for industrial applications” and special project “RoboCare” funded by L. 449/97), and by ASI, Agenzia Spaziale Italiana (contract I/R/134/00). Giuseppe Beccari received the Laurea degree in Electronic Engineering in 1993, and the Ph.D. in Information Technology in 1999, both from the University of Parma, Italy. In 1995 he was visiting scholar at the Technical University of Delft, Holland, and at the Laboratoire de Robotique de Paris, France. In 1999 he was employed by CSELT (Centro Studi E Laboratori Telecomunicazioni, currently TILAB, the Telecom Italia Group research center). In 2002 he moved to a spin off company involved in the EUROSAM/FSAF (Future Surface-to-Air Family self defense missile system) project. While his current professional duties focus more on software development and team coordination, dr. Beccari still enjoys investigating real-time scheduling issues and technology. Stefano Caselli received a Laurea degree in Electronic Engineering in 1982 and the Ph.D. degree in Computer and Electronic Engineering in 1987, both from the University of Bologna, Italy. In 1989-90 he has been visiting scholar at the University of Florida. From 1990 to 1999 he has held research fellow and associate professor positions at the University of Parma, Italy. He is now professor of Computer Engineering at the University of Parma, where he is also director of the Laboratory of Robotics and Intelligent Machines (RIMLab). His current research interests include development of autonomous and remotely operated robot systems, service robotics, and real-time systems. Francesco Zanichelli received a Laurea degree in Electronic Engineering in 1987 from the University of Bologna, Italy and the Ph.D. degree in Information Technologies in 1994 from the University of Parma, Italy. Since 1996 he has been an Assistant Professor with the Department of Information Engineering of the University of Parma where he is currently teaching Operating Systems, Information Systems and Multimedia Systems courses. His current research interests include distributed multimedia architectures and protocols, real-time systems, security and Quality of Service technologies for wireless networks, as well as service-oriented Grid middleware.     

Assessing IR-based traceability recovery tools through controlled experiments

We report the results of a controlled experiment and a replication performed with different subjects, in which we assessed the usefulness of an Information Retrieval-based traceability recovery tool during the traceability link identification process. The main result achieved in the two experiments is that the use of a traceability recovery tool significantly reduces the time spent by the software engineer with respect to manual tracing. Replication with different subjects allowed us to investigate if subjects’ experience and ability play any role in the traceability link identification process. In particular, we made some observations concerning the retrieval accuracy achieved by the software engineers with and without the tool support and with different levels of experience and ability.

Retrieval of Spatial Join Pattern Instances from Sensor Networks

We study the continuous evaluation of spatial join queries and extensions thereof, defined by interesting combinations of sensor readings (events) that co-occur in a spatial neighborhood. An example of such a pattern is “a high temperature reading in the vicinity of at least four high-pressure readings”. We devise protocols for ‘in-network’ evaluation of this class of queries, aiming at the minimization of power consumption. In addition, we develop cost models that suggest the appropriateness of each protocol, based on various factors, including selectivity of query elements, energy requirements for sensing, and network topology. Finally, we experimentally compare the effectiveness of the proposed solutions on an experimental platform that emulates real sensor networks.

Supporting high interoperability of components by adopting an agent-based approach

While integrating components into systems, we will be confronted with problems concerned with the interoperability of components due to the interaction mismatches at multiple levels, such as interaction behaviors between components and features imposed by architectural styles. In this paper, we studied the interoperability of components and explored the approach to supporting high interoperability of components involved in mismatching interactions. First, we formalized components involved in different architectural styles in the pi-calculus. Next, we studied the formal foundation of the interoperability of components for reasoning about the conditions under which two heterogeneous components are possible to interoperate and interconnect together properly. Then, we described a wrapper-based solution for integrating components into systems that impose mismatching assumptions about usage of the components. In the end, we presented an agent-based implementation for the solution, in which agents are used to wrap components and can automatically resolve multiple levels of interaction mismatches between components. We also gave a simple example to illustrate our approach.

Quadtree-based eigendecomposition for pose estimation in the presence of occlusion and background clutter

Eigendecomposition-based techniques are popular for a number of computer vision problems, e.g., object and pose estimation, because they are purely appearance based and they require few on-line computations. Unfortunately, they also typically require an unobstructed view of the object whose pose is being detected. The presence of occlusion and background clutter precludes the use of the normalizations that are typically applied and significantly alters the appearance of the object under detection. This work presents an algorithm that is based on applying eigendecomposition to a quadtree representation of the image dataset used to describe the appearance of an object. This allows decisions concerning the pose of an object to be based on only those portions of the image in which the algorithm has determined that the object is not occluded. The accuracy and computational efficiency of the proposed approach is evaluated on 16 different objects with up to 50% of the object being occluded and on images of ships in a dockyard.

An RTP/RTCP based approach for multimedia group and inter-stream synchronization

Most multimedia group and inter-stream synchronization techniques define or use proprietary protocols with new control messages. Many multimedia applications have been developed using RTP/RTCP as the standard for transmission of multimedia streams over IP networks. Instead of defining a new protocol, we propose the use of RTP/RTCP to provide synchronization. We take advantage of the feedback capabilities provided by RTCP and the ability to extend the protocol by extending and creating RTCP messages containing synchronization information. We have implemented our proposal and tested it in our University WAN. Our experiments have shown that network load resulting from synchronization is minimized and that asynchronies are within acceptable limits for multimedia applications.

Theory of relative defect proneness

In this study, we investigated the functional form of the size-defect relationship for software modules through replicated studies conducted on ten open-source products. We consistently observed a power-law relationship where defect proneness increases at a slower rate compared to size. Therefore, smaller modules are proportionally more defect prone. We externally validated the application of our results for two commercial systems. Given limited and fixed resources for code inspections, there would be an impressive improvement in the cost-effectiveness, as much as 341% in one of the systems, if a smallest-first strategy were preferred over a largest-first one. The consistent results obtained in this study led us to state a theory of relative defect proneness (RDP): In large-scale software systems, smaller modules will be proportionally more defect-prone compared to larger ones. We suggest that practitioners consider our results and give higher priority to smaller modules in their focused quality assurance efforts.

Semi-supervised discriminative classification with application to tumorous tissues segmentation of MR brain images

Due to the large data size of 3D MR brain images and the blurry boundary of the pathological tissues, tumor segmentation work is difficult. This paper introduces a discriminative classification algorithm for semi-automated segmentation of brain tumorous tissues. The classifier uses interactive hints to obtain models to classify normal and tumor tissues. A non-parametric Bayesian Gaussian random field in the semi-supervised mode is implemented. Our approach uses both labeled data and a subset of unlabeled data sampling from 2D/3D images for training the model. Fast algorithm is also developed. Experiments show that our approach produces satisfactory segmentation results comparing to the manually labeled results by experts.

A practical approach to testing GUI systems

GUI systems are becoming increasingly popular thanks to their ease of use when compared against traditional systems. However, GUI systems are often challenging to test due to their complexity and special features. Traditional testing methodologies are not designed to deal with the complexity of GUI systems; using these methodologies can result in increased time and expense. In our proposed strategy, a GUI system will be divided into two abstract tiers—the component tier and the system tier. On the component tier, a flow graph will be created for each GUI component. Each flow graph represents a set of relationships between the pre-conditions, event sequences and post-conditions for the corresponding component. On the system tier, the components are integrated to build up a viewpoint of the entire system. Tests on the system tier will interrogate the interactions between the components. This method for GUI testing is simple and practical; we will show the effectiveness of this approach by performing two empirical experiments and describing the results found.

Open media service architecture for advanced collaboration environments

Advanced collaboration environments are extensively utilized for distance learning, e-science, and other distributed global collaboration events. In such environments, high-quality and seamless media services play an important role in improving the quality of user experience to participants. In this paper, to support high-quality media-based services, we design open media service architecture for advanced collaboration environments, by combining the open interface for state-of-the-art media tools, the performance monitoring tools for devices and networks, and application-level adaptation schemes for media streaming. By implementing the proposed architecture on top of an open-source Access Grid (AG) collaboration toolkit, we verify that high-quality collaboration among several collaboration sites can be effectively realized over a multicast-enabled network testbed with improved media quality experience.

A Framework for Providing User Level Quality of Service Guarantees in Multi-Class Rate Adaptive Systems

The problem of channel sharing by rate adaptive streams belonging to various classes is considered. Rate adaptation provides the opportunity for accepting more connections by adapting the bandwidth of connections that are already in the system. However, bandwidth adaptation must be employed in a careful manner in order to ensure that (a) bandwidth is allocated to various classes in a fair manner (system perspective) and (b) bandwidth adaptation does not affect adversely the perceived user quality of the connection (user quality). The system perspective aspect has been studied earlier. This paper focuses on the equally important user perspective. It is proposed to quantify user Quality of Service (QoS) through measures capturing short and long-term bandwidth fluctuations that can be implemented with the mechanisms of traffic regulators, widely used in networking for the purpose of controlling the traffic entering or exiting a network node. Furthermore, it is indicated how to integrate the user perspective metrics with the optimal algorithms for system performance metrics developed earlier by the authors. Simulation results illustrate the effectiveness of the proposed framework.

Multiple-parameter coupling metrics for layered component-based software

Coupling represents the degree of interdependence between two software components. Understanding software dependency is directly related to improving software understandability, maintainability, and reusability. In this paper, we analyze the difference between component coupling and component dependency, introduce a two-parameter component coupling metric and a three-parameter component dependency metric. An important parameter in both these metrics is coupling distance, which represents the relevance of two coupled components. These metrics are applicable to layered component-based software. These metrics can be used to represent the dependencies induced by all types of software coupling. We show how to determine coupling and dependency of all scales of software components using these metrics. These metrics are then applied to Apache HTTP, an open-source web server. The study shows that coupling distance is related to the number of modifications of a component, which is an important indicator of component fault rate, stability and subsequently, component complexity.

Video JET: packet loss-resilient video joint encryption and transmission based on media-hash-embedded residual data

Media encryption technologies actively play the first line of defense in securing the access of multimedia data. Traditional cryptographic encryption can achieve provable security but is unfortunately sensitive to a single bit error, which will cause an unreliable packet to be dropped creating packet loss. In order to achieve robust media encryption, the requirement of error resilience can be achieved with error-resilient media transmission. This study proposes a video joint encryption and transmission (video JET) scheme by exploiting media hash-embedded residual data to achieve motion estimation and compensation for recovering lost packets, while maintaining format compliance and cryptographic provable security. Interestingly, since video block hash preserves the condensed content to facilitate search of similar blocks, motion estimation is implicitly performed through robust media hash matching – which is the unique characteristic of our method. We analyze and compare the performance of resilience to (bursty) packet loss between the proposed method and forward error correction (FEC), which has been extensively employed to protect video packets over error-prone networks. The feasibility of our packet loss-resilient video JET approach is further demonstrated through experimental results.

Periodicity and stability issues of a <Emphasis Type="Italic">chaotic</Emphasis> pattern recognition neural network

Traditional pattern recognition (PR) systems work with the model that the object to be recognized is characterized by a set of features, which are treated as the inputs. In this paper, we propose a new model for PR, namely one that involves chaotic neural networks (CNNs). To achieve this, we enhance the basic model proposed by Adachi (Neural Netw 10:83–98, 1997), referred to as Adachi’s Neural Network (AdNN), which though dynamic, is not chaotic. We demonstrate that by decreasing the multiplicity of the eigenvalues of the AdNN’s control system, we can effectively drive the system into chaos. We prove this result here by eigenvalue computations and the evaluation of the Lyapunov exponent. With this premise, we then show that such a Modified AdNN (M-AdNN) has the desirable property that it recognizes various input patterns. The way that this PR is achieved is by the system essentially sympathetically “resonating” with a finite periodicity whenever these samples (or their reasonable resemblances) are presented. In this paper, we analyze the M-AdNN for its periodicity, stability and the length of the transient phase of the retrieval process. The M-AdNN has been tested for Adachi’s dataset and for a real-life PR problem involving numerals. We believe that this research also opens a host of new research avenues. Research partially supported by the Natural Sciences and Engineering Research Council of Canada.

---