Towards real-time GOMS: a model of expert behaviour in a highly interactive task

Abstract

We present an analysis of an expert performing a highly interactive computer task. The analysis uses GOMS models, specifying the Goals, Operators, Methods, and Selection rules used by the expert. Two models are presented, one with function-level operators which perform high-level functions in the domain, and one with keystroke-level operators which describe hand movements. For a segment of behaviour in which the expert accomplished about 30 functions in about 30 s, the function-level model predicted the observed behaviour well, while the keystroke-level model predicted only about half of the observed hand movements. These results, including the discrepancy between the models, are discussed.     

Knowledge elicitation for validation of a neonatal ventilation expert system utilising modified Delphi and focus group techniques

Objective, methods & materials, resultsIt is well known that ventilation strategies for newborn infants may vary significantly between individual doctors. The aim of this study was to elicit knowledge of ventilation management to provide a baseline for evaluating the performance of an expert system for neonatal ventilation (FLORENCE). The modified Delphi method and focus group techniques were used to arrive at consensus management strategies on 40 hypothetical ventilation scenarios. The underlying cognitive processes of the experts were also explored further to assist in the development of the expert system. The strategies arrived at were used to provide a performance level which FLORENCE was tested against. The solutions were judged to be equivalent between FLORENCE and neonatologists in 29 of the 40 cases. In the remaining 11 scenarios; FLORENCE also provided adequate solutions.ConclusionsThe focus group technique was more effective than modified Delphi method in achieving consensus on ventilation management. This consensus on ventilation was used as the baseline to evaluate the performance of an expert system.     

Expert systems for legal decision making

Abstract: AI technology is being used to develop expert systems that solve complex problems in the legal area. Most of these systems employ rules to describe the strategies and procedures used by litigators to analyze legal issues. The tasks performed by these systems include interpreting the law, anticipating the legal consequences of proposed actions, predicting the effects of changes in legislation, as well as analyzing and managing cases. The special characteristics of the legal domain cause certain problems for expert system builders. We discuss some of these problems and describe LDS and SAL, two expert systems we have developed for case evaluation and settlement in the product liability area. SAL (System for Asbestos Litigation) evolved from the earlier and more general expert system, LDS (Legal Decisionmaking System). We also describe XPL, an explanation facility we developed for use by SAL and other expert system applications.     

STRATEGIC KNOWLEDGE IN AN EXPERT SYSTEM FOR AGRICULTURE

Abstract

Most of the expert systems applied in vegetal pathology treat the problem of selecting treatment in a conventional manner, by means of production rules that associate to each pathology the most adequate chemical product. This makes it difficult to generate useful explanations. In order to generate satisfactory explanations the knowledge of the system must be based on the strategies used by human experts. This article introduces our approach for the identification and representation of strategic knowledge in an expert system for plague control in greenhouses. We present an introduction to the application domain and make an analysis of the strategic knowledge implied. We distinguish between the underlying strategy and the practical strategy used by the expert for the solution of the problem. From this we propose a preliminary architecture based on strategic reasoning agents.     

Model-based traffic control

Urban traffic control is a difficult problem because of the complex interdependence of control decisions. Known techniques for achieving global control are based on parameter optimization techniques or heuristic expert systems. Optimization fails in severely congested traffic situations which require a change in global strategy. Heuristic expert systems require knowledge of all possible traffic situations, which is difficult to obtain, especially when construction and incidents cause frequent changes to the traffic network.In this paper, we show how the techniques of model-based diagnosis can be used to select coordinated control plans for networked systems of this kind Suitable local control strategies are those whose underlying assumptions are consistent with other control strategies, the state of the road network, and traffic flow. We describe a system which uses an assumption-based truth maintenance system (ATMS) to compute suitable strategies. The system has been tested both on synthetic examples and on simulations using actual data, and results are encouraging.     

Local multi-valued logics in modular expert systems

Abstract

Abstract. In this paper we describe an approach to the problem of dealing with uncertainty by means of finite multi-valued logics in modular expert systems, and the results obtained. The modularity of the systems allows us to address two main characteristics of human problem-solving: the adaptation of general knowledge to particular problems and the dependency of the management of uncertainty on the different subtasks being implemented in the modules of the system, i.e. different modules can have different local multiple-valued logics as part of their local deductive mechanisms. Although the results obtained are general, we use, throughout the paper, examples of a medical expert system that has been designed using a modular language called MILORD-II, that implements them showing the practical interest of the theoretical concepts involved.     

On generating mutants for AspectJ programs

ContextMutation analysis has been widely used in research studies to evaluate the effectiveness of test suites and testing techniques. Faulty versions (i.e., mutants) of a program are generated such that each mutant contains one seeded fault. The mutation score provides a measure of effectiveness.ObjectiveWe study three problems with the use of mutation analysis for testing AspectJ programs:

• •The manual identification and removal of equivalent mutants is difficult and time consuming. We calculate the percentage of equivalent mutants generated for benchmark AspectJ programs using available mutation tools.
• •The generated mutants need to cover the various fault types described in the literature on fault models for AspectJ programs. We measure the distribution of the mutants generated using available mutation tools with respect to the AspectJ fault types.
• •We measure the difficulty of killing the generated mutants.

We propose the use of simple analysis of the subject programs to prevent the generation of some equivalent mutants.MethodWe revised existing AspectJ fault models and presented a fault model that removes the problems in existing fault models, such as overlapping between fault types and missing fault types. We also defined three new fault types that occur due to incorrect data-flow interactions occurring in AspectJ programs. We used three mutation tools: AjMutator, Proteum/AJ, and MuJava on three AspectJ programs. To measure the difficulty of killing the mutants created using a mutation operator, we compared the average number of the mutants killed by 10 test suites that satisfy block coverage criterion.ResultsA high percentage of the mutants are equivalent. The mutation tools do not cover all the fault types. Only 4 out of 27 operators generated mutants that were easy to kill.ConclusionsOur analysis approach removed about 80% of the equivalent mutants. Higher order mutation is needed to cover all the fault types.     

An incremental solution for developing knowledge-based software: its application to an expert system for isokinetics interpretation

Many researchers working in the field of knowledge engineering (KE) are now concerned with identifying a model suitable for developing knowledge-based software and, especially, expert systems (ES). It is important to find a standard model that meets current needs and incorporates techniques successfully implemented in SE (object- or event-orientation, etc.), which are also of keen interest in KE.In this paper, we present an iterative and incremental solution for developing ES, according to which the system domain is derived naturally from the problem domain, thus surmounting the problems now involved in the transition from the conceptual model of the problem to the formal model of the system.As compared with conventional development models, this solution encompasses five main tools, which are:• Use cases with their respective actor interaction diagrams and activity flow diagrams in order to specify the expert system.• The concept dictionary, which allows knowledge engineers to define, bound and select the meaning of each concept used by experts.• The static conceptual model, which provides an overview (concepts and their relations) of the expert system (ES) modelled.• The control and process model, which models the knowledge and metaknowledge used by the expert to attain a goal.• An object-oriented metamodel, which outputs the formal knowledge model, providing an efficient, reusable, extendible and easy-to-implement ES architecture.To demonstrate the robustness of this solution, we describe how it was applied to an ES that interprets the graphs output by an isokinetics machine for a blind person. An isokinetics machine assesses the strength of the muscles of the leg, arm, etc.     

Student modelling using plan recognition

Abstract

We present a student modelling approach based on plan recognition methods. In some domains, like theorem proving, the student's activity can be seen as consisting of the formation of plans (the proofs) and the execution of actions (the proof steps). Starting from the student's inputs and the problem's search space, the method infers the most plausible plan according to a criterion of coherence. Recognising the student's plan can help predict his next actions and provide him with well-adapted assistance. This modelling technique is applied in an intelligent tutoring system (ITS) which coaches a student during geometry problem-solving. We describe the architecture of the system: the expert, a set of geometry rules together with a theorem prover which can solve problems in different ways and recognise the student's errors; the interface; and the pedagogical module. Finally, we describe the implemented system and its evaluation.     

Enhanced qualitative physical reasoning system: Qupras

There are many expert systems that use experimental knowledge for diagnostic analysis and design. However, there are two problems for systems using only experiential knowledge:

1. unexpected problems cannot be solved and
2. acquiring experiential knowledge from human experts is difficult.

To solve these problems, general principles or basic knowledge must be added to expert systems in addition to the experimental knowledge. In response, we previously proposed Qupras (Qualitative physical reasoning system) as a framework for basic knowledge. This system has two knowledge representations, one related to physical laws and the other to objects. By using this knowledge, Qupras reasons about the relations among physical objects, and predicts the next state of a physical phenomenon. Recently, we have improved some of Qupras’ features, and this pater desctibes the following main enhancements:

1. inheritance for representation of objects,
2. new primitive representations to describe discontinuous change, and
3. control features for effective reasoning.

     

The s-ethos system: A methodology for systematic flight analysis centered on human factors

The aeronautics community needs several alternative methods and tools to describe and analyze interactions between human operators and systems, according to some constraints (e.g., human factors, air safety, etc.). Hence, it needs to build models from the observation of real interactions, especially piloting, and to use extant theories from several fields: cognitive ergonomics and artificial intelligence, mainly. S-ETHOS sketches out a knowledge-based system that analyzes human pilot activities and provides feedback to improve air safety by giving measured appraisal of pilot error. The core of S-ETHOS is the ETHOS model that depicts the standard behavior based on the human pilot. S-ETHOS helps any air safety expert to simulate the pilot behavior during his mission and then will compare behavior between the simulation and real situations. It allows the air safety expert to know how the pilot assesses each situation. We implemented the ETHOS model according to an object-oriented approach, relying on a knowledge modeling language called OBJLOG II+. This model provides a first keystone to understanding how the human pilot captures and builds his environment through complex states. We will discuss the identified behaviors and potential deviations and associated situations.     

Adaptive search through constraint violations

Abstract

We describe HS, a production system that learns control knowledge through adaptive search. Unlike most other psychological models of skill acquisition, HS is a model of analytical, or knowledge-based, learning. HS encodes general domain knowledge in state constraints, patterns that describe those search states that are consistent with the principles of the problem domain. When HS encounters a search state that violates a state constraint it revises the production rule that generated that state. The appropriate revisions are computed by regressing the constraint through the action of the production rule. HS can learn to solve problems that it cannot solve without learning. We present a Blocks World example of a rule revision, empirical results from both initial learning experiments and transfer experiments in the domain of counting, and an informal analysis of the conditions under which this learning technique is likely to be useful.     

Using intentional fragments to bridge the gap between organizational and intentional levels

ContextBusiness process models provide a natural way to describe real-world processes to be supported by software-intensive systems. These models can be used to analyze processes in the system-as-is and describe potential improvements for the system-to-be. But, how well does a given business process model satisfy its business goals? How can different perspectives be integrated in order to describe an inter-organizational process?ObjectiveThe aim of the present paper is to link the local and the global perspectives of the inter-organizational business process defined in BPMN 2.0 (Business Process Model and Notation) to KAOS goal models (Keep All Objectives Satisfied). We maintain a separation of concerns between the intentional level captured by the goal model and the organizational level captured by the process model. The paper presents the concept of intentional fragment (a set of flow elements of the process with a common purpose) and assess its usefulness.MethodWe conducted empirical experiments where the proposed concepts – here the intentional fragments – are validated by users. Our method relies on an iterative improvement process led by users feedback.ResultsWe find that the concept of intentional fragment is useful for (1) analyzing the business process model (2) reasoning about the relations between the goal model and the business process model and (3) identifying new goals. In a previous work we focused on BPMN 2.0 collaboration models (local view). This paper extends the previous work by integrating the global view given by choreography models in the approach.ConclusionWe conclude that the notion of intentional fragment is a useful mean to relate business process models and goal models while dealing with their different nature (activity oriented vs goal oriented). Intentional fragments can also be used to analyze the process model and to infer new goals in an iterative manner.     

Performance regression testing and run-time verification of components in robotics systems

Abstract

Unintended changes in the utilization of resources like CPU and memory can lead to severe problems for the operation of robotics and intelligent systems. Nevertheless, systematic testing for such performance regressions has largely been ignored in this domain. We present a method to specify and execute performance tests for individual components of component-based robotics systems based on their middleware interfaces. The method includes an automatic analysis of each component revision against previous ones that reports potential changes to the resource utilization characteristics. This informs developers about the impact of their changes. We describe the design of the framework and present evaluation results for the automatic detection of performance changes based on tests for a variety of robotics components. Additionally, we demonstrate how performance tests can be used as a basis for learning resource utilization models of components. These models can be used to detect faults at system run-time, which provides an additional level of safety for the systems besides the offline testing.     

THE SYSTEMIC FORMALIZATION OF KNOWLEDGE INVOLVING UNCERTAINTY

Abstract

We have developed generic models useful for modeling human expertise involving uncertainty. These generic models are based on fundamental work carried out in the field of systems science, in which various researchers have developed formalisms for modeling systems in general. In this paper, we present our Systemic U-Knowledge Framework, which utilizes our models and new mathematical constructs we have developed for modeling uncertainty. We have validated the resulting formalism by modeling the expertise of an education expert, and have used the model as a basis for automated problem-solving.     

Value prediction in hls allocation problems using intellectual properties

A value approximation-based global search algorithm is suggested to solve resource-constrained allocation in high level synthesis problems. Value approximation is preferred, because it can start by using expert heuristics, can estimate the global structure of the search problem, and can optimize heuristics. We are concerned by those allocation problems that have hidden global structure that value approximation may unravel. The value approximation applied here computes the cost of the actual solution and estimates the cost of the solution that could be achieved upon performing a global search on the hidden structure starting from the actual solution. We transcribed the allocation problem into a special form of weighted CNF formulae to suit our approach. We also extended the formalism to pipeline operations. Comparisons are made with expert heuristics. Scaling of computation time and performance are compared.     

The effect of time pressure on expert system based training for emergency management

In many emergency situations, human operators are required to derive countermeasures based on contingency rules whilst under time pressure. In order to contribute to the human success in playing such a role, the present study intends to examine the effectiveness of using expert systems to train for the time-constrained decision domain. Emergency management of chemical spills was selected to exemplify the rule-based decision task. An Expert System in this domain was developed to serve as the training tool. Forty subjects participated in an experiment in which a computerized information board was used to capture subjects' rule-based performance under the manipulation of time pressure and training. The experiment results indicate that people adapt to time pressure by accelerating their processing of rules where the heuristic of cognitive availability was employed. The simplifying strategy was found to be the source of human error that resulted in undesired decision performance. The results also show that the decision behaviour of individuals who undergo the expert system training is directed to a normative and expeditious pattern, which leads to an improved level of decision accuracy. Implications of these findings are examined in the present study.     

Learning goal hierarchies from structured observations and expert annotations

We describe a relational learning by observation framework that automatically creates cognitive agent programs that model expert task performance in complex dynamic domains. Our framework uses observed behavior and goal annotations of an expert as the primary input, interprets them in the context of background knowledge, and returns an agent program that behaves similar to the expert. We map the problem of creating an agent program on to multiple learning problems that can be represented in a “supervised concept learning’’ setting. The acquired procedural knowledge is partitioned into a hierarchy of goals and represented with first order rules. Using an inductive logic programming (ILP) learning component allows our framework to naturally combine structured behavior observations, parametric and hierarchical goal annotations, and complex background knowledge. To deal with the large domains we consider, we have developed an efficient mechanism for storing and retrieving structured behavior data. We have tested our approach using artificially created examples and behavior observation traces generated by AI agents. We evaluate the learned rules by comparing them to hand-coded rules. Editor: Rui Camacho     

双目系统全视域采样的SVM标定模型分析

目的 为改善摄像机间接标定采样不全、模型表达模糊问题,实现小视场下检测视域完备采样,提出一种基于双目系统全视域采样的支持向量机（SVM）标定方法。方法 该方法利用六角晶格标定板靶点序号可读特点为基础,采集整个双目系统有效视域中检测点的视差坐标、世界坐标并建立完备的样本集。选取SVM对该样本集进行训练,将SVM算法得到的模型参数代入其决策函数中进行求解,获得公式化的标定模型。由于六角晶格标定板的四角和中心分布了5个互为非中心对称的多边形,可在标定板部分区域被采集的情况下获取标定板位姿信息,进而读取采集的各靶点序号。通过上下移动标定板,利用HALCON算子获取图像中各靶点的序号,建立双目视觉系统检测区域的完备样本集。最后,利用SVM算法训练样本获得标定模型,可以明确表达出标定模型的数学形式。结果 与传统采样建立的模型进行对比分析,实验结果表明该方法建立模型的标定误差减小了24.51%,降低了标定模型在传统方法未采样区域的标定误差,证明了该方法的可行性。结论 提出一种基于双目系统全视域采样的支持向量机标定方法,通过非中心对称的多边形确定标定板上靶点的序号,实现双目视觉系统检测视域的完备采样。实验结果表明该方法提高了摄像机间接标定的精度,具有良好的适用性和鲁棒性,适用于小视域内双目视觉系统的间接标定。