Automatic generation of multiple choice questions using dependency-based semantic relations

In this paper, we present an unsupervised dependency-based approach to extract semantic relations to be applied in the context of automatic generation of multiple choice questions (MCQs). MCQs also known as multiple choice tests provide a popular solution for large-scale assessments as they make it much easier for test-takers to take tests and for examiners to interpret their results. Manual generation of MCQs is a very expensive and time-consuming task and yet they often need to be produced on a large scale and within short iterative cycles. We approach the problem of automated MCQ generation with the help of unsupervised relation extraction, a technique used in a number of related natural language processing problems. The goal of Unsupervised relation extraction is to identify the most important named entities and terminology in a document and then recognise semantic relations between them, without any prior knowledge as to the semantic types of the relations or their specific linguistic realisation. We use these techniques to process instructional texts and identify those facts (terminology, entities, and semantic relations between them) that are likely to be important for assessing test-takers’ familiarity with the instructional material. We investigate an approach to learn semantic relations between named entities by employing a dependency tree model. Our findings show that an optimised configuration of our MCQ generation system is capable of attaining high precision rates, which are much more important than recall in the automatic generation of MCQs. We also carried out a user-centric evaluation of the system, where subject domain experts evaluated automatically generated MCQ items in terms of readability, usefulness of semantic relations, relevance, acceptability of questions and distractors and overall MCQ usability. The results of this evaluation make it possible for us to draw conclusions about the utility of the approach in practical e-Learning applications.     

Comparison of examination methods based on multiple-choice questions and constructed-response questions using personal computers

The aim of the present research was to compare the use of multiple-choice questions (MCQs) as an examination method, to the examination based on constructed-response questions (CRQs). Despite that MCQs have an advantage concerning objectivity in the grading process and speed in production of results, they also introduce an error in the final formulation of the score. The error is traced to the probability of answering a question by chance or based on an instinctive feeling, which does not enable the ascertainment of the knowledge of the whole background included in the question. In the present study, both MCQ and CRQ tests were given to examinees, in the framework of a computer-based learning system. Avoiding the procedure of mixed scoring, e.g. both positive and negative markings, a set of pairs of MCQs was composed. The MCQs in each pair were similar concerning the same topic, but this similarity was not evident for an examinee that did not possess adequate knowledge on the topic addressed in the questions of the pair. The examination based on these “paired” MCQs, by using a suitable scoring rule, when made to the same sample of students, οn the same topics and with the same levels of difficulty, gave results that were statistically indistinguishable with the grades produced by an examination based on CRQs, while both the “paired” MCQ test results and the CRQ test results differed significantly from those obtained from a MCQ test using positive-only scoring rule.     

Ontology Integration: Approaches and Challenging Issues

In recent years, the decentralized development of ontologies has led to the generation of multiple ontologies of overlapping knowledge. This heterogeneity problem can be tackled by integrating existing ontologies to build a single coherent one. Ontology integration has been investigated during the last two decades, but it is still a challenging task. In this article, we provide a comprehensive survey of all ontology integration aspects. We discuss related notions and scrutinize existing techniques and literature approaches. We also detail the role of ontology matching in the ontology integration process. Indeed, the ontology community has adopted the splitting of the ontology integration problem into matching, merging and repairing sub-tasks, where matching is a necessary preceding step for merging, and repairing can be included in the matching process or performed separately. Ontology matching and merging systems have become quite proficient, however the trickiest part lies in the repairing step. We also focus on the case of a holistic integration of multiple heterogeneous ontologies, which needs further exploration. Finally, we investigate challenges, open issues, and future directions of the ontology integration and matching areas.     

Domain ontology graph model and its application in Chinese text classification

This paper proposes an ontology learning method which is used to generate a graphical ontology structure called ontology graph. The ontology graph defines the ontology and knowledge conceptualization model, and the ontology learning process defines the method of semiautomatic learning and generates ontology graphs from Chinese texts of different domains, the so-called domain ontology graph (DOG). Meanwhile, we also define two other ontological operations—document ontology graph generation and ontology graph-based text classification, which can be carried out with the generated DOG. This research focuses on Chinese text data, and furthermore, we conduct two experiments: the DOG generation and ontology graph-based text classification, with Chinese texts as the experimental data. The first experiment generates ten DOGs as the ontology graph instances to represent ten different domains of knowledge. The generated DOGs are then further used for the second experiment to provide performance evaluation. The ontology graph-based approach is able to achieve high text classification accuracy (with 92.3 % in f-measure) over other text classification approaches (such as 86.8 % in f-measure for tf–idf approach). The better performance in the comparative experiments reveals that the proposed ontology graph knowledge model, the ontology learning and generation process, and the ontological operations are feasible and effective.     

Representing and Reasoning About XML with Ontologies

The eXtensible Markup Language (XML) has reached a wide acceptance as the relevant standardization for representing and exchanging data on the Web. Unfortunately, XML covers the syntactic level but lacks semantics, and thus cannot be directly used for the Semantic Web. Currently, finding a way to utilize XML data for the Semantic Web is challenging research. As we have known that ontology can formally represent shared domain knowledge and enable semantics interoperability. Therefore, in this paper, we investigate how to represent and reason about XML with ontologies. Firstly, we give formalized representations of XML data sources, including Document Type Definitions (DTDs), XML Schemas, and XML documents. On this basis, we propose formal approaches for transforming the XML data sources into ontologies, and we also discuss the correctness of the transformations and provide several transformation examples. Furthermore, following the proposed approaches, we implement a prototype tool that can automatically transform XML into ontologies. Finally, we apply the transformed ontologies for reasoning about XML, so that some reasoning problems of XML may be checked by the existing ontology reasoners.     

Comparison of oral examination and electronic examination using paired multiple-choice questions

The aim of the present research was to compare the use of multiple-choice questions (MCQs) as an examination method against the oral examination (OE) method. MCQs are widely used and their importance seems likely to grow, due to their inherent suitability for electronic assessment. However, MCQs are influenced by the tendency of examinees to guess answers, warranting research concerning scoring rules different from the simple positive-grades-only scores rule. Alternatively, OE is used in tertiary education, since it enables the assessment of intellectual capabilities and personal traits to a level not found in most other examination formats. However, the significant resource requirements of OE, especially in structured forms, might excessively strain the resources of academic institutions. In the present study, an MCQ test was given to examinees, in the framework of a computer-based learning system. The same examinees took also an OE possessing elements of structure, with three examiners concurrently and independently grading each of the examinees. In the MCQs examination a set of pairs of MCQs was composed. The MCQs in each pair were similar concerning the same topic, but this similarity was not evident for an examinee that did not possess adequate knowledge on the topic addressed in the questions of the pair. The scoring of the paired questions avoided the procedure of mixed-scoring, i.e., both positive and negative markings, while at the same time a pair-wise bonus/penalty scoring rule was adopted. The results of the “paired” MCQs examination, when using the pair-wise scoring rule, were statistically indistinguishable with the grades produced by the OE, when made to the same sample of students, on the same topics and with the same levels of difficult. Both the results of the paired MCQs examination, when using the pair-wise scoring rule, and the OE results differed significantly from those obtained by scoring the same MCQs using a positive-grades-only scoring rule that ignored the pairing of MCQs.     

Constructing the virtual Jing-Hang Grand Canal with onto-draw

Constructing virtual 3D historical scenes from literature and records is a very challenging problem due to the difficulty in incorporating different types of domain knowledge into the modeling system. The domain knowledge comes from different experts, including: architects, historians, rendering artists, user interface designers and computer engineers. In this paper we investigate the problem of automatically generating drawings of ancient scenes by ontologies extracted from these domains. We introduce a framework called onto-draw to generate semantic models of desired scenes by constructing hierarchical ontology concept domains. Inconsistencies among them are resolved via an iterative refinement algorithm. We implement the onto-draw based ontology design approach and inconsistency removal technique in the virtual Jing-Hang Grand Canal construction project (Chen et al., 2010) and achieve encouraging results.     

A systematic review on time-constrained ontology evolution in predictive maintenance

With the modernization of industry and introduction of IoT, maintenance practices have been moving from reactive to proactive and predictive approaches. The identification of faults often relies on the analysis of real-time data provided by streams and unstructured sources. Ontologies have been applied to the maintenance field in order to add a semantic layer to the data and facilitate interoperability, and combined with other approaches for explainability and fault diagnosis, among others. In such a time-sensitive domain, it is important that ontologies go beyond static representations of the domain and allow not only for the incorporation of time related knowledge, but must also be able to adapt to new knowledge and evolve. This systematic review presents four research questions to provide a general understanding of the state of the art of the representation of time and ontology evolution in the predictive maintenance field. The results have shown that there are several ways of representing the evolution of knowledge that are fairly established and several specific evolutionary actions are discriminated and analyzed. Similarly, there is a diverse group of metrics that can be exploited to measure change and to establish evolutionary trends and even predict future stages of the ontology. Studies on the representation of time show us that it can be done either quantitative or qualitatively, with some approaches combining the two. Applications of these to the problem of ontology evolution are still in the open. Finally, results show that while applications of ontologies to the field of predictive maintenance are plenty, there are not many studies focusing on their evolution or in the effective application of their ability to reason with time constraints. The results obtained in this systematic review are particularly relevant for devising solutions that make use of the ontology’s potential for time representation and evolution in the predictive maintenance field.

     

语义集成:本体映射方法研究

本体是客观世界知识的表现形式,随着语义Web研究的深入,研究者们构建了越来越多的本体,如何实现本体之间的知识共享和重用,成为了语义Web发展的关键。文中对本体映射的方法进行了研究,系统阐述了本体及本体映射的定义、本体映射中的相似度计算和本体映射框架等。如何减少本体映射中的人工干预,实现本体的半自动化或自动化映射将是该领域的发展方向。     

DOLCE ergo SUMO: On foundational and domain models in the SmartWeb Integrated Ontology (SWIntO)

Increased availability of mobile computing, such as personal digital assistants (PDAs), creates the potential for constant and intelligent access to up-to-date, integrated and detailed information from the Web, regardless of one's actual geographical position. Intelligent question-answering requires the representation of knowledge from various domains, such as the navigational and discourse context of the user, potential user questions, the information provided by Web services and so on, for example in the form of ontologies. Within the context of the SmartWeb project, we have developed a number of domain-specific ontologies that are relevant for mobile and intelligent user interfaces to open-domain question-answering and information services on the Web. To integrate the various domain-specific ontologies, we have developed a foundational ontology, the SmartSUMO ontology, on the basis of the DOLCE and SUMO ontologies. This allows us to combine all the developed ontologies into a single SmartWeb Integrated Ontology (SWIntO) having a common modeling basis with conceptual clarity and the provision of ontology design patterns for modeling consistency. In this paper, we present SWIntO, describe the design choices we made in its construction, illustrate the use of the ontology through a number of applications, and discuss some of the lessons learned from our experiences.     

Repairing and reasoning with inconsistent and uncertain ontologies

With the development of semantic web, the quality and correctness of ontologies play more and more important roles in semantic representation and knowledge sharing. However, ontologies are often inconsistent and uncertain in real situations. Because of the difficulty in ensuring the quality of ontologies, there is an increasing need for dealing with the inconsistency and uncertainty in real-world applications of ontological reasoning and management. This paper adopts two methods to handle the inconsistent and uncertain ontologies: the first one is to repair the inconsistency, algorithms RIO and RIUO are proposed to compute the candidate repair set, the consistency of ontology could be recovered through deleting or modifying the axioms in candidate repair set; the second one is to develop a non-standard reasoning method to obtain meaningful answers, algorithms RMU and RMIU are proposed to perform query-specific reasoning methods for inconsistent and uncertain ontologies without changing the original ontologies. Finally the prototype system is constructed and the experiment results validate the usability and effectiveness of our approaches.     

Detection and resolution of semantic inconsistency and redundancy in an automatic ontology merging system

In recent years, researchers have been developing algorithms for the automatic mapping and merging of ontologies to meet the demands of interoperability between heterogeneous and distributed information systems. But, still state-of-the-art ontology mapping and merging systems is semi-automatic that reduces the burden of manual creation and maintenance of mappings, and need human intervention for their validation. The contribution presented in this paper makes human intervention one step more down by automatically identifying semantic inconsistencies in the early stages of ontology merging. We are detecting semantic heterogeneities that occur due to conflicts among the set of Generalized Concept Inclusions, Property Subsumption Criteria, and Constraint Satisfaction Mechanism in local heterogeneous ontologies, which become obstacles for the generation of semantically consistent global merged ontology. We present several algorithms to detect such semantic inconsistencies based on subsumption analysis of concepts and properties in local ontologies from the list of initial mappings. We provide ontological patterns for resolving these inconsistencies automatically. This results global merged ontology free from ??circulatory error in class/property hierarchy??, ??common class between disjoint classes/properties??, ??redundancy of subclass/subproperty of relations?? and other types of ??semantic inconsistency?? errors. Experiments on the real ontologies show that our algorithms save time and cost of traversing local ontologies, improve system??s performance by producing only consistent accurate mappings, and reduce the users?? dependability for ensuring the satisfiability of merged ontology.     

How ontologies are made: Studying the hidden social dynamics behind collaborative ontology engineering projects

Traditionally, evaluation methods in the field of semantic technologies have focused on the end result of ontology engineering efforts, mainly, on evaluating ontologies and their corresponding qualities and characteristics. This focus has led to the development of a whole arsenal of ontology-evaluation techniques that investigate the quality of ontologies as a product. In this paper, we aim to shed light on the process of ontology engineering construction by introducing and applying a set of measures to analyze hidden social dynamics. We argue that especially for ontologies which are constructed collaboratively, understanding the social processes that have led to their construction is critical not only in understanding but consequently also in evaluating the ontologies. With the work presented in this paper, we aim to expose the texture of collaborative ontology engineering processes that is otherwise left invisible. Using historical change-log data, we unveil qualitative differences and commonalities between different collaborative ontology engineering projects. Explaining and understanding these differences will help us to better comprehend the role and importance of social factors in collaborative ontology engineering projects. We hope that our analysis will spur a new line of evaluation techniques that view ontologies not as the static result of deliberations among domain experts, but as a dynamic, collaborative and iterative process that needs to be understood, evaluated and managed in itself. We believe that advances in this direction would help our community to expand the existing arsenal of ontology evaluation techniques towards more holistic approaches.     

Efficient retrieval of ontology fragments using an interval labeling scheme

Nowadays very large domain ontologies are being developed in life-science areas like Biomedicine, Agronomy, Astronomy, etc. Users and applications can benefit enormously from these ontologies in very different tasks, such as visualization, vocabulary homogenizing and data classification. However, due to their large size, they are often unmanageable for these applications. Instead, it is necessary to provide small and useful fragments of these ontologies so that the same tasks can be performed as if the whole ontology is being used. In this work we present a novel method for efficiently indexing and generating ontology fragments according to the user requirements. Moreover, the generated fragments preserve relevant inferences that can be made with the selected symbols in the original ontology. Such a method relies on an interval labeling scheme that efficiently manages the transitive relationships present in the ontologies. Additionally, we provide an interval’s algebra to compute some logical operations over the ontology concepts. We have evaluated the proposed method over several well-known biomedical ontologies. Results show very good performance and scalability, demonstrating the applicability of the proposed method in real scenarios.     

Where to publish and find ontologies? A survey of ontology libraries

One of the key promises of the Semantic Web is its potential to enable and facilitate data interoperability. The ability of data providers and application developers to share and reuse ontologies is a critical component of this data interoperability: if different applications and data sources use the same set of well defined terms for describing their domain and data, it will be much easier for them to “talk” to one another. Ontology libraries are the systems that collect ontologies from different sources and facilitate the tasks of finding, exploring, and using these ontologies. Thus ontology libraries can serve as a link in enabling diverse users and applications to discover, evaluate, use, and publish ontologies. In this paper, we provide a survey of the growing—and surprisingly diverse—landscape of ontology libraries. We highlight how the varying scope and intended use of the libraries affects their features, content, and potential exploitation in applications. From reviewing 11 ontology libraries, we identify a core set of questions that ontology practitioners and users should consider in choosing an ontology library for finding ontologies or publishing their own. We also discuss the research challenges that emerge from this survey, for the developers of ontology libraries to address.     

Frame-based ontological view for semantic integration

Semantic integration is crucial for successful collaboration between heterogeneous information systems. Traditional ontology-driven approaches rely on the availability of explicit ontologies. However, in most application domains, this prerequisite cannot be met. In order to address this issue, this paper investigates the theoretical foundation of ontologies and extends the traditional ontology concept to an ontological view concept. To explicitly and formally specify the ontological views, a Frame-based Ontological view Specification Language (FOSL) is proposed. This language is based on the frame knowledge representation paradigm and uses XML as the encoding. The ontological view driven semantic integration can be achieved based on the specifications. A proof-of-concept prototype environment has been implemented to achieve semantic integration based on ontological views specified with FOSL.     

Evaluating the Generation of Domain Ontologies in the Knowledge Puzzle Project

One of the goals of the knowledge puzzle project is to automatically generate a domain ontology from plain text documents and use this ontology as the domain model in computer-based education. This paper describes the generation procedure followed by TEXCOMON, the knowledge puzzle ontology learning tool, to extract concept maps from texts. It also explains how these concept maps are exported into a domain ontology. Data sources and techniques deployed by TEXCOMON for ontology learning from texts are briefly described herein. Then, the paper focuses on evaluating the generated domain ontology and advocates the use of a three-dimensional evaluation: structural, semantic, and comparative. Based on a set of metrics, structural evaluations consider ontologies as graphs. Semantic evaluations rely on human expert judgment, and finally, comparative evaluations are based on comparisons between the outputs of state-of-the-art tools and those of new tools such as TEXCOMON, using the very same set of documents in order to highlight the improvements of new techniques. Comparative evaluations performed in this study use the same corpus to contrast results from TEXCOMON with those of one of the most advanced tools for ontology generation from text. Results generated by such experiments show that TEXCOMON yields superior performance, especially regarding conceptual relation learning.     

A semantic similarity method based on information content exploiting multiple ontologies

The quantification of the semantic similarity between terms is an important research area that configures a valuable tool for text understanding. Among the different paradigms used by related works to compute semantic similarity, in recent years, information theoretic approaches have shown promising results by computing the information content (IC) of concepts from the knowledge provided by ontologies. These approaches, however, are hampered by the coverage offered by the single input ontology. In this paper, we propose extending IC-based similarity measures by considering multiple ontologies in an integrated way. Several strategies are proposed according to which ontology the evaluated terms belong. Our proposal has been evaluated by means of a widely used benchmark of medical terms and MeSH and SNOMED CT as ontologies. Results show an improvement in the similarity assessment accuracy when multiple ontologies are considered.