Metrics for MT evaluation: evaluating reordering

Translating between dissimilar languages requires an account of the use of divergent word orders when expressing the same semantic content. Reordering poses a serious problem for statistical machine translation systems and has generated a considerable body of research aimed at meeting its challenges. Direct evaluation of reordering requires automatic metrics that explicitly measure the quality of word order choices in translations. Current metrics, such as BLEU, only evaluate reordering indirectly. We analyse the ability of current metrics to capture reordering performance. We then introduce permutation distance metrics as a direct method for measuring word order similarity between translations and reference sentences. By correlating all metrics with a novel method for eliciting human judgements of reordering quality, we show that current metrics are largely influenced by lexical choice, and that they are not able to distinguish between different reordering scenarios. Also, we show that permutation distance metrics correlate very well with human judgements, and are impervious to lexical differences.     

Monte Carlo techniques for phrase-based translation

Recent advances in statistical machine translation have used approximate beam search for NP-complete inference within probabilistic translation models. We present an alternative approach of sampling from the posterior distribution defined by a translation model. We define a novel Gibbs sampler for sampling translations given a source sentence and show that it effectively explores this posterior distribution. In doing so we overcome the limitations of heuristic beam search and obtain theoretically sound solutions to inference problems such as finding the maximum probability translation and minimum risk training and decoding.     

Why Machines Cannot Feel

For a long time, emotions have been ignored in the attempt to model intelligent behavior. However, within the last years, evidence has come from neuroscience that emotions are an important facet of intelligent behavior being involved into cognitive problem solving, decision making, the establishment of social behavior, and even conscious experience. Also in research communities like software agents and robotics, an increasing number of researchers start to believe that computational models of emotions will be needed to design intelligent systems. Nevertheless, modeling emotions in technical terms poses many difficulties and has often been accounted as just not feasible. In this article, there are identified the main problems, which occur when attempting to implement emotions into machines. By pointing out these problems, it is aimed to avoid repeating mistakes committed when modeling computational models of emotions in order to speed up future development in this area. The identified issues are not derived from abstract reflections about this topic but from the actual attempt to implement emotions into a technical system based on neuroscientific research findings. It is argued that besides focusing on the cognitive aspects of emotions, a consideration of the bodily aspects of emotions—their grounding into a visceral body—is of crucial importance, especially when a system shall be able to learn correlations between environmental objects and events and their “emotional meaning”.     

A Biologically-Inspired Micro Aerial Vehicle

This paper introduces a novel framework for the design, modeling and control of a Micro Aerial Vehicle (MAV). The vehicle’s conceptual design is based on biologically-inspired principles and emulates a dragonfly (Odonata–Anisoptera). We have taken inspiration from the flight mechanism features of the dragonfly and have developed indigenous designs in creating a novel version of a Flapping Wing MAV (FWMAV). The MAV design incorporates a complex mechanical construction and a sophisticated multi-layered, hybrid, linear/non-linear controller to achieve extended flight times and improved agility compared to other rotary wing and FWMAV Vertical Take Off and Landing (VTOL) designs. The first MAV prototype will have a ballpark weight including sensor payload of around 30 g. The targeted lifting capability is about twice the weight. The MAV features state of the art sensing and instrumentation payload, which includes integrated high-power on-board processors, 6DoF inertial sensors, 3DoF compasses, GPS, embedded camera and long-range telemetry capability. A 3-layer control mechanism has been developed to harness the dynamics and attain complete navigational control of the MAV. The inner-layer is composed of a ‘quad hybrid-energy controller’ and two higher layers are at present, implementing a linear controller; the latter will be replaced eventually with a dynamic adaptive non-linear controller. The advantages of the proposed design compared to other similar ones include higher energy efficiency and extended flight endurance. The design features elastic storage and re-use of propulsion energy favoring energy conservation during flight. The design/modeling of the MAV and its kinematics & dynamics have been tested under simulation to achieve desired performance. The potential applications for such a high endurance vehicle are numerous, including air-deployable mass surveillance and reconnaissance in cluster and swarm formations. The efficacy of the design is demonstrated through a simulation environment. The dynamics are verified through simulations and a general linear controller coupled with an energy based non-linear controller is shown to operate the vehicle in a stable regime. In accordance with specified objectives a prototype is being developed for flight-testing and demonstration purposes.     

Direct Method Based Control System for an Autonomous Quadrotor

This paper proposes a real time control algorithm for autonomous operation of a quadrotor unmanned air vehicle. The quadrotor is a small agile vehicle, which as well as being a excellent test bed for advanced control techniques could also be suitable for internal surveillance, search and rescue and remote inspection. The proposed control scheme incorporates two key aspects of autonomy; trajectory planning and trajectory following. Using the differentially-flat dynamics property of the system, the trajectory optimization is posed as a non-linear constrained optimization within the output space in the virtual domain, not explicitly related to the time domain. A suitable parameterization using a virtual argument as opposed to time is applied, which ensures initial and terminal constraint satisfaction. The speed profile is optimized independently, followed by the mapping to the time domain achieved using a speed factor. Trajectory following is achieved with a standard multi-variable control technique and a digital switch is used to re-optimize the reference trajectory in the event of infeasibility or mission change. The paper includes simulations using a full dynamic model of the quadrotor demonstrating the suitability of the proposed control scheme.     

Spatially Varying Mixtures Incorporating Line Processes for Image Segmentation

Spatially varying mixture models are characterized by the dependence of their mixing proportions on location (contextual mixing proportions) and they have been widely used in image segmentation. In this work, Gauss-Markov random field (MRF) priors are employed along with spatially varying mixture models to ensure the preservation of region boundaries in image segmentation. To preserve region boundaries, two distinct models for a line process involved in the MRF prior are proposed. The first model considers edge preservation by imposing a Bernoulli prior on the normally distributed local differences of the contextual mixing proportions. It is a discrete line process model whose parameters are computed by variational inference. The second model imposes Gamma prior on the Student’s-t distributed local differences of the contextual mixing proportions. It is a continuous line process whose parameters are also automatically estimated by the Expectation-Maximization (EM) algorithm. The proposed models are numerically evaluated and two important issues in image segmentation by mixture models are also investigated and discussed: the constraints to be imposed on the contextual mixing proportions to be probability vectors and the MRF optimization strategy in the frameworks of the standard and variational EM algorithm.     

Ontology-driven web-based semantic similarity

Estimation of the degree of semantic similarity/distance between concepts is a very common problem in research areas such as natural language processing, knowledge acquisition, information retrieval or data mining. In the past, many similarity measures have been proposed, exploiting explicit knowledge—such as the structure of a taxonomy—or implicit knowledge—such as information distribution. In the former case, taxonomies and/or ontologies are used to introduce additional semantics; in the latter case, frequencies of term appearances in a corpus are considered. Classical measures based on those premises suffer from some problems: in the first case, their excessive dependency of the taxonomical/ontological structure; in the second case, the lack of semantics of a pure statistical analysis of occurrences and/or the ambiguity of estimating concept statistical distribution from term appearances. Measures based on Information Content (IC) of taxonomical concepts combine both approaches. However, they heavily depend on a properly pre-tagged and disambiguated corpus according to the ontological entities in order to compute accurate concept appearance probabilities. This limits the applicability of those measures to other ontologies –like specific domain ontologies- and massive corpus –like the Web-. In this paper, several of the presented issues are analyzed. Modifications of classical similarity measures are also proposed. They are based on a contextualized and scalable version of IC computation in the Web by exploiting taxonomical knowledge. The goal is to avoid the measures’ dependency on the corpus pre-processing to achieve reliable results and minimize language ambiguity. Our proposals are able to outperform classical approaches when using the Web for estimating concept probabilities.     

Queries with CASE expressions

In recent years more and more queries are generated automatically by query managers/builders with end-users providing only specific parameters through GUIs. Queries generated automatically can be quite different from queries written by humans. In particular, they contain non-declarative features, most notorious of which is the CASE expression. Current query optimizers are often ill-prepared for the new types of queries as they do not deal well with procedural ‘insertions’. In this paper, we discuss the inefficiencies of CASE expressions and present several new optimization techniques to address them. We also describe experimental evaluation of the prototype implemented in DB2 UDB V8.2.     

Second-Order Abstract Categorial Grammars as Hyperedge Replacement Grammars

Second-order abstract categorial grammars (de Groote in Association for computational linguistics, 39th annual meeting and 10th conference of the European chapter, proceedings of the conference, pp. 148–155, 2001) and hyperedge replacement grammars (Bauderon and Courcelle in Math Syst Theory 20:83–127, 1987; Habel and Kreowski in STACS 87: 4th Annual symposium on theoretical aspects of computer science. Lecture notes in computer science, vol 247, Springer, Berlin, pp 207–219, 1987) are two natural ways of generalizing “context-free” grammar formalisms for string and tree languages. It is known that the string generating power of both formalisms is equivalent to (non-erasing) multiple context-free grammars (Seki et al. in Theor Comput Sci 88:191–229, 1991) or linear context-free rewriting systems (Weir in Characterizing mildly context-sensitive grammar formalisms, University of Pennsylvania, 1988). In this paper, we give a simple, direct proof of the fact that second-order ACGs are simulated by hyperedge replacement grammars, which implies that the string and tree generating power of the former is included in that of the latter. The normal form for tree-generating hyperedge replacement grammars given by Engelfriet and Maneth (Graph transformation. Lecture notes in computer science, vol 1764. Springer, Berlin, pp 15–29, 2000) can then be used to show that the tree generating power of second-order ACGs is exactly the same as that of hyperedge replacement grammars.     

Automata for Epistemic Temporal Logic with Synchronous Communication

We suggest that developing automata theoretic foundations is relevant for knowledge theory, so that we study not only what is known by agents, but also the mechanisms by which such knowledge is arrived at. We define a class of epistemic automata, in which agents’ local states are annotated with abstract knowledge assertions about others. These are finite state agents who communicate synchronously with each other and information exchange is ‘perfect’. We show that the class of recognizable languages has good closure properties, leading to a Kleene-type theorem using what we call regular knowledge expressions. These automata model distributed causal knowledge in the following way: each agent in the system has a partial knowledge of the temporal evolution of the system, and every time agents synchronize, they update each other’s knowledge, resulting in a more up-to-date view of the system state. Hence we show that these automata can be used to solve the satisfiability problem for a natural epistemic temporal logic for local properties. Finally, we characterize the class of languages recognized by epistemic automata as the regular consistent languages studied in concurrency theory.