A task-performance evaluation of referring expressions in situated collaborative task dialogues

Appropriate evaluation of referring expressions is critical for the design of systems that can effectively collaborate with humans. A widely used method is to simply evaluate the degree to which an algorithm can reproduce the same expressions as those in previously collected corpora. Several researchers, however, have noted the need of a task-performance evaluation measuring the effectiveness of a referring expression in the achievement of a given task goal. This is particularly important in collaborative situated dialogues. Using referring expressions used by six pairs of Japanese speakers collaboratively solving Tangram puzzles, we conducted a task-performance evaluation of referring expressions with 36 human evaluators. Particularly we focused on the evaluation of demonstrative pronouns generated by a machine learning-based algorithm. Comparing the results of this task-performance evaluation with the results of a previously conducted corpus-matching evaluation (Spanger et al. in Lang Resour Eval, 2010b), we confirmed the limitation of a corpus-matching evaluation and discuss the need for a task-performance evaluation.     

Time-sensitive Arabic multiword expressions extraction from social networks

In this paper, we present a comprehensive approach for extracting and relating Arabic multiword expressions (MWE) from Social Networks. 15 million tweets were collected and processed to form our data set. Due to the complexity of processing Arabic and the lack of resources, we built an experimental system to extract and relate similar MWE using statistical methods. We introduce a new metrics for measuring valid MWE in Social Networks. We compare results obtained from our experimental system against semantic graph obtained from web knowledgebase.     

Learning from technical documents: the role of intermodal referring expressions

OBJECTIVE: We investigated the impact of two types of intermodal referring expressions on efficiency of instructions for use. BACKGROUND: User manuals for software or technical devices such as a video recording system frequently combine verbal instructions and illustrations. Much research has shown that the presence of an illustration has a beneficial effect on learning. The present study focuses on a factor that modulates this beneficial effect. The combination of text and an illustration can be effective only if the user integrates the information coming from these two media. This integration depends largely on the intermodal referential expressions, the function of which is to mark explicitly the relations between the text and the illustration. METHOD: In an experiment (N = 104), we compared the effectiveness of two intermodal referring expressions often used in procedural texts: indexes (numbers introduced in the illustrations and in the instructions to establish cross-references) and icons (visual representations of the components of the device, which are inserted in the verbal instructions). RESULTS: The icons condition led to the most efficient use of the device. CONCLUSION: This experiment shows that learning from multimedia documents depends on the possibility of effectively connecting the verbal instructions to the illustration. APPLICATION: Taking into account the ergonomic properties of the cross-media referring expressions should allow text designers to improve the effectiveness of technical documents.     

Arabic GramCheck: a grammar checker for Arabic

Arabic is a Semitic language that is rich in its morphology and syntax. The very numerous and complex grammar rules of the language may be confusing for the average user of a word processor. In this paper, we report our attempt at developing a grammar checker program for Modern Standard Arabic, called Arabic GramCheck. Arabic GramCheck can help the average user by checking his/her writing for certain common grammatical errors; it describes the problem for him/her and offers suggestions for improvement. The use of the Arabic grammatical checker can increase productivity and improve the quality of the text for anyone who writes Arabic. Arabic GramCheck has been successfully implemented using SICStus Prolog on an IBM PC. The current implementation covers a well‐formed subset of Arabic and focuses on people trying to write in a formal style. Successful tests have been performed using a set of Arabic sentences. It is concluded that the approach is promising by observing the results as compared to the output of a commercially available Arabic grammar checker. Copyright © 2005 John Wiley & Sons, Ltd.     

Extracting an Arabic Lexicon from Arabic Newspaper Text

We describe how to build a largecomprehensive, integrated Arabic lexicon byautomatic parsing of newspaper text. We havebuilt a parser system to read Arabic newspaperarticles, isolate the tokens from them, findthe part of speech, and the features for eachtoken. To achieve this goal we designed a setof algorithms, we generated several sets ofrules, and we developed a set of techniques,and a set of components to carry out thesetechniques. As each sentence is processed, newwords and features are added to the lexicon, sothat it grows continuously as the system runs.To test the system we have used 100 articles(80,444 words) from the Al-Raya newspaper.The system consists of several modules: thetokenizer module to isolate the tokens, the type findersystem to find the part of speech of eachtoken, the proper noun phrase parser module tomark the proper nouns and to discover someinformation about them and the feature findermodule to find the features of the words.     

Constructing Arabic Reading Comprehension Datasets: Arabic WikiReading and KaifLematha

Neural machine reading comprehension models have gained immense popularity over the last decade given the availability of large-scale English datasets. A key limiting factor for neural model development and investigations of the Arabic language is the limitation of the currently available datasets. Current available datasets are either too small to train deep neural models or created by the automatic translation of the available English datasets, where the exact answer may not be found in the corresponding text. In this paper, we propose two high quality and large-scale Arabic reading comprehension datasets: Arabic WikiReading and KaifLematha with around +100 K instances. We followed two different methodologies to construct our datasets. First, we employed crowdworkers to collect non-factoid questions from paragraphs on Wikipedia. Then, we constructed Arabic WikiReading following a distant supervision strategy, utilizing the Wikidata knowledge base as a ground truth. We carried out both quantitative and qualitative analyses to investigate the level of reasoning required to answer the questions in the proposed datasets. We evaluated competitive pre-trained language model that attained F1 scores of 81.77 and 68.61 for the Arabic WikiReading and KaifLematha datasets, respectively, but struggled to extract a precise answer for the KaifLematha dataset. Human performance reported an F1 score of 82.54 for the KaifLematha development set, which leaves ample room for improvement.

     

Recognition of Arabic characters

A statistical approach for the recognition of Arabic characters is introduced. As a first step, the character is segmented into primary and secondary parts (dots and zigzags). The secondary parts of the character are then isolated and identified separately, thereby reducing the number of classes from 28 to 18. The moments of the horizontal and vertical projections of the remaining primary characters are then calculated and normalized with respect to the zero-order moment. Simple measures of the shape are obtained from the normalized moments. A 9-D feature vector is obtained for each character. Classification is accomplished using quadratic discriminant functions. The approach was evaluated using isolated, handwritten, and printed characters from a database established for this purpose. The results indicate that the technique offers better classification rates in comparison with existing methods     

Morphological decomposition in Arabic ASR systems

In recent years, the use of morphological decomposition strategies for Arabic Automatic Speech Recognition (ASR) has become increasingly popular. Systems trained on morphologically decomposed data are often used in combination with standard word-based approaches, and they have been found to yield consistent performance improvements. The present article contributes to this ongoing research endeavour by exploring the use of the ‘Morphological Analysis and Disambiguation for Arabic’ (MADA) tools for this purpose. System integration issues concerning language modelling and dictionary construction, as well as the estimation of pronunciation probabilities, are discussed. In particular, a novel solution for morpheme-to-word conversion is presented which makes use of an N-gram Statistical Machine Translation (SMT) approach. System performance is investigated within a multi-pass adaptation/combination framework. All the systems described in this paper are evaluated on an Arabic large vocabulary speech recognition task which includes both Broadcast News and Broadcast Conversation test data. It is shown that the use of MADA-based systems, in combination with word-based systems, can reduce the Word Error Rates by up to 8.1% relative.     

Behavior analysis in Arabic social media

International Journal of Speech Technology - Social media has allowed all individuals, organizations, and businesses to share their opinions, ideas, and inclinations with others. These opinions...     

眼动-语言跨模态共指消解方法

跨模态共指消解是根据人员交互意图对自然图像中所指目标的定位任务,作为智能人机交互领域的关键技术之一,能够应用于抢险救灾、家庭服务或养老助残等场景。现有的目标指代方法一般采用单模态信息表现人类意图,例如语言或者眼动等,然而单一的模态用户输入只能够传达有限的交互信息,难以实现自然而智能的人机协同。本文针对这一问题,同时融合眼动和语言信息,建立了跨模态共指消解模型,利用多种模态信息的优势互补,实现人类意图所指目标的图像定位任务。设计了对比试验,验证了本文提出的眼动-语言跨模态的融合方法性能优于单模态的输入形式。     

Off-line Arabic character recognition

A personal computer-based Arabic character recognition system that performs three preprocessing stages sequentially, thinning, stroke segmentation, and sampling, is described. The eight-direction code used for stroke representation and classification, the character classification done at primary and secondary levels, and the contextual postprocessor used for error detection and correction are described. Experimental results obtained using samples of handwritten and typewritten Arabic words are presented     

Arabic handwritten digit recognition

In this paper, we fill a gap in the literature by studying the problem of Arabic handwritten digit recognition. The performances of different classification and feature extraction techniques on recognizing Arabic digits are going to be reported to serve as a benchmark for future work on the problem. The performance of well known classifiers and feature extraction techniques will be reported in addition to a novel feature extraction technique we present in this paper that gives a high accuracy and competes with the state-of-the-art techniques. A total of 54 different classifier/features combinations will be evaluated on Arabic digits in terms of accuracy and classification time. The results are analyzed and the problem of the digit ‘0’ is identified with a proposed method to solve it. Moreover, we propose a strategy to select and design an optimal two-stage system out of our study and, hence, we suggest a fast two-stage classification system for Arabic digits which achieves as high accuracy as the highest classifier/features combination but with much less recognition time.     

KAFD Arabic font database

Font recognition is useful for improving optical text recognition systems’ accuracy and time, and to restore the documents’ original formats. This paper addresses a need for Arabic font recognition research by introducing an Arabic font recognition database consisting of 40 fonts, 10 sizes (ranging from 8 to 24 points) and 4 styles (viz. normal, bold, italic, and bold–italic). The database is split into three sets (viz. training, validation, and testing). The database is freely available to researchers.¹ Moreover, we introduce a baseline font recognition system for benchmarking purposes, and report identification rates on our KAFD database and the Arabic Printed Text Image (APTI) database with 20 and 10 fonts, respectively. The best recognition rates are achieved using log-Gabor filters.     

REX-J: Japanese referring expression corpus of situated dialogs

Identifying objects in conversation is a fundamental human capability necessary to achieve efficient collaboration on any real world task. Hence the deepening of our understanding of human referential behaviour is indispensable for the creation of systems that collaborate with humans in a meaningful way. We present the construction of REX-J, a multi-modal Japanese corpus of referring expressions in situated dialogs, based on the collaborative task of solving the Tangram puzzle. This corpus contains 24 dialogs with over 4?h of recordings and over 1,400 referring expressions. We outline the characteristics of the collected data and point out the important differences from previous corpora. The corpus records extra-linguistic information during the interaction (e.g. the position of pieces, the actions on the pieces) in synchronization with the participants’ utterances. This in turn allows us to discuss the importance of creating a unified model of linguistic and extra-linguistic information from a new perspective. Demonstrating the potential uses of this corpus, we present the analysis of a specific type of referring expression (“action-mentioning expression”) as well as the results of research into the generation of demonstrative pronouns. Furthermore, we discuss some perspectives on potential uses of this corpus as well as our planned future work, underlining how it is a valuable addition to the existing databases in the community for the study and modeling of referring expressions in situated dialog.