Causal knowledge elicitation based on elicitation failures

The paper presents an approach to causal knowledge elicitation supported by a tool directly used by the domain expert. This knowledge elicitation approach is characterized by trying to guess an interpretation of the knowledge entered by the expert. The tool (initially general), as it is used, self customizes its guessing capability, remembers failures in guessing (in order to avoid similar failures in the future) and when they occur elicits their explanations. Even in this case, elicitation is supported by guessing on the basis of previous similar failures. The resulting overall effect is that the tool digs up tenaciously causal knowledge from the expert's mind, playing in this way a cooperative role for model building     

Knowledge-based versus data-driven fuzzy habitat suitability models for river management

Aquatic habitat suitability models have increasingly received attention due to their wide management applications. Ecological expert knowledge has been frequently incorporated in such models to link environmental conditions to the quantitative habitat suitability of aquatic species. Since the formalisation of problem-specific human expert knowledge is often difficult and tedious, data-driven machine learning techniques may be helpful to extract knowledge from ecological datasets. In this paper, both expert knowledge-based and data-driven fuzzy habitat suitability models were developed and the performance of these models was compared. For the data-driven models, a hill-climbing optimisation algorithm was applied to derive ecological knowledge from the available data. Based on the available ecological expert knowledge and on biological samples from the Zwalm river basin (Belgium), habitat suitability models were generated for the mayfly Baetis rhodani (Pictet 1843). Data-driven models appeared to outperform expert knowledge-based models substantially, while a step-forward model selection procedure indicated that physical habitat variables adequately described the mayfly habitat suitability in the studied area. This study has important implications on the application of expert knowledge in ecological studies, especially if this knowledge is extrapolated to other areas. The results suggest that data-driven models can complement expert knowledge-based approaches and hence improve model reliability.     

Eliciting knowledge and transferring it effectively to aknowledge-based system

The knowledge acquisition bottleneck impeding the development of expert systems is being alleviated by the development of computer-based knowledge acquisition tools. These work directly with experts to elicit knowledge, and structure it appropriately to operate as a decision support tool within an expert system. However, the elicitation of expert knowledge and its effective transfer to a useful knowledge-based system is complex and involves diverse activities. The complete development of a decision support system using knowledge acquisition tools is illustrated. The example is simple enough to be completely analyzed but exhibits enough real-world characteristics to give significant insights into the processes and problems of knowledge engineering     

Some methodological aspects of tacit knowledge elicitation

Methodological aspects of tacit knowledge elicitation with the aim of building expert systems are examined. Important aspects of expertise exerting considerable influence on knowledge elicitation have been enumerated and briefly described. Problems of knowledge elicitation are discussed in the light of findings of neuropsychology, investigations on brain asymmetry in particular. Several hypotheses in the course of the authors' work on development of expert systems in areas of psychology, human physiology and medical diagnostics have been formulated. In the framework of these hypotheses, the following points are discussed: the role of visual stimulus in the process of knowledge elicitation, expert strategies of the visual information analysis, influence exerted by the expert experience and formulation of question posed to the expert on his/her choice of strategy, alternation of strategy prepotency; exponential decrease of effectiveness of each strategy application and special significance of the end-insight. Proceeding from the principal propositions of the hypotheses having been formulated, methodological rules have been advanced, which can be assumed as a basis in working-out the semi-structured interview.     

ENIGMA – Enhanced interactive general movement assessment

General movement assessment is an accurate clinical method for predicting severe neurological dysfunctions such as cerebral palsy in young infants. Development of a computer-based diagnosis support system based on the General Movement Assessment method is dependent on features being effectively elicited from a General Movement expert. We present ENIGMA, a software tool for General Movement knowledge elicitation and modeling.

Video and motion data were collected in 15 recordings containing both normal and abnormal general movements from the fidgety period of infant development. ENIGMA shows video in synchrony with different visualized features of recorded motion data. Movement patterns are modeled through an iterative and incremental process, where the General Movement expert is guiding the modeling process through comparing movement patterns observed in video with corresponding visual patterns observed in visualized features, and giving feedback to the knowledge engineer.

Three visualized features were developed for exploring the so-called fidgety movements. The interactive work procedure introduced by ENIGMA enabled explicit motion features to be defined based on unconscious expert knowledge. Normal fidgety movements were found to be partly characterized by periodic patterns.

Our results demonstrate that ENIGMA is a capable tool for General Movement expert knowledge elicitation. It facilitates the modeling process and provides a basis for detailed discussions. Clinical and technical concepts are communicated well through visual notions.     

Eliciting and modelling expert knowledge

This paper evaluates the usefulness of various psychological techniques that can be utilized to elicit and model expert knowledge for subsequent representation in rule-based expert systems. Interviewing, protocol analysis and multidimensional scaling are described and evaluated as complementary methods of knowledge elicitation. In addition ‘context-focusing’ and card-sorting are introduced as short-cut methods for the knowledge engineer's ‘tool box’.It is argued that expert knowledge about uncertainty can be represented as subjective probabilities and that these assessments can (and therefore should) be checked for consistency and coherence as a pre-condition for realism.Finally, the issue of whether it is possible to improve upon expert judgement is discussed and evidence is reviewed which shows that, in repetitive decision-making situations, statistical models of the expert can out-perform the expert on whom the models are based. Statistical modelling has a valid but limited application as a replacement for expert judgement.     

基于Multi Agent的图象理解

最近十多年来,具有专家系统外壳的图象理解软件的开发一直是一个重要的研究课题。然而,多数基于知识的图象理解系统或者局限于诸如分割之类的特定操作上,或者局限于像理解建筑物的航空图象这样特定的应用。由于这些系统的目标和知识结构的局限性,因而不能将它们推广到其它领域。此研究就是试图解决此问题：首先将Agent构造环境扩展为一个Muliti Agent的图象理解系统,而此系统是知识入口的开发工具,它为用户提供一个类似于专家系统外壳的界面。这样一方面能加速图象理解系统的开发,另一方面方便那些缺乏图象理解知识的人开发图象理解应用程序。     

Knowledge elicitation via sequential probabilistic inference for high-dimensional prediction

Prediction in a small-sized sample with a large number of covariates, the “small n, large p” problem, is challenging. This setting is encountered in multiple applications, such as in precision medicine, where obtaining additional data can be extremely costly or even impossible, and extensive research effort has recently been dedicated to finding principled solutions for accurate prediction. However, a valuable source of additional information, domain experts, has not yet been efficiently exploited. We formulate knowledge elicitation generally as a probabilistic inference process, where expert knowledge is sequentially queried to improve predictions. In the specific case of sparse linear regression, where we assume the expert has knowledge about the relevance of the covariates, or of values of the regression coefficients, we propose an algorithm and computational approximation for fast and efficient interaction, which sequentially identifies the most informative features on which to query expert knowledge. Evaluations of the proposed method in experiments with simulated and real users show improved prediction accuracy already with a small effort from the expert.     

Expert systems and behavioral decision research

This paper reviews a part of the literature on behavioral decision research (policy capturing, psychophysics of numerical judgments and cognitive illusions) and examines implication for knowledge elicitation in expert systems. The literature on policy capturing demonstrates that simple and compact numerical models of expert knowledge can be built, but that experts are poor in verbalizing the knowledge expressed in them. The psychophysical literature indicates that numerical encoding of expert knowledge may be difficult and biased, but that it has definitive advantages over qualitative elicitation schemes: Numerical encoding forces hard throught, encourages precision, and allows to access a substantial computational apparatus. The literature on cognitive illusions suggests that the expert knowledge one elicits may be an illusion. The review concludes by recommending to use numerical judgments and explicit models by experts where possible, and to decompose the elicitation task in order to avoid cognitive illusions.     

The use of knowledge acquisition in instructional design

Expert systems and knowledge based systems have emerged from “esoteric” laboratory research in Artificial Intelligence (AI) to become an important tool for approaching real world problems. Expert systems are distinctive in that they are designed to address problems in a similar manner and with similar results as a human expert. The basic structure of an expert system is comprised of three functionally separate components: (a) knowledge base, which contains a representation of domain related facts; (b) means of knowledge base use to solve a problem, inference mechanism; and (c) working memory, which records the input data and progress for each problem. Given the complexity and cost of expert system construction, it is imperative that system developers and researchers attend to research issues which are critical to knowledge engineering. These questions can be categorized according to the parts of an expert system: (a) knowledge representation; (b) knowledge utilization; and (c) knowledge acquisition. A knowledge acquisition procedure is presented which displays the relationship between subject matter expert expertise consisting of declarative knowledge, procedural knowledge, heuristics, formal rules, and meta-rules. The knowledge engineer uses one or a combination of elicitation methods to gather relevant data to eventually build the components of an expert system. Further explained are the acquisition methods: (a) structured interview; (b) verbal reports; (c) teaching the subject matter; (d) observation; and (e) automated knowledge acquisition tools. The paper concludes with a discussion of the future research issues concerned with using knowledge mapping and task analysis vs. knowledge acquisition techniques.     

Automated knowledge acquisition for the surface-mount PWB assembly domain

The development of knowledge-based (or expert) systems for the surface-mount printed wiring board (PWB) assembly domain requires the understanding and regulation of several complex tasks. While the knowledge base in an expert system serves as a storehouse of knowledge primitives, its design and development is a bottleneck in the expert system development life-cycle. Therefore the development of an automated knowledge acquisition (KA) facility (or KA tool) would facilitate the implementation of expert systems for any domain. This paper describes an automated KA tool that helps to elicit and store information in domain-specific knowledge bases for surface-mount PWB assembly. A salient feature of this research is the acquisition of uncertain information.     

Requirements Elicitation and Analysis of Multiagent Systems Using Activity Theory

Requirements elicitation and analysis is intended to gain knowledge about customers' needs and the environment of a software system. Requirements not only commonly deal with business processes and their data but also with the motivation behind these activities, the social structures that forge them, and previous design decisions. Recent studies show that the intentional and social concepts of agent-oriented software engineering (AOSE) could be used for the analysis of these aspects. Nevertheless, even having specialized modeling primitives for these requirements is not sufficient for their elicitation. Specialized analysis techniques are also required, but this is commonly overlooked by agent-oriented methodologies. This paper aims to provide the needed modeling primitives and support by means of the theoretical and methodological foundation of a social sciences framework, the activity theory, and its activity checklist. They inspire our Requirements Elicitation Guide (REG) for AOSE. The REG contains the expert knowledge that developers need to grasp information about their multiagent systems, human environments, and their mutual influences. This knowledge takes the form of requirements described as diagrams in a proper modeling language. The REG is applied in a process with the corresponding support tool. In this way, the REG guides requirements elicitation and increases the productivity with the use of templates for a wide range of requirements. These elements have been validated with several case studies. Two of them appear as part of this paper.      

Development of a methodology for optimizing elicited knowledge

In this paper a conceptual framework and an operational methodology is presented for describing the most appropriate knowledge elicitation method (protocol, interview, induction and repertory grid) for three classes of tasks (diagnosis, debugging and interpretation) and for experts with strengths in various factors of cognitive abilities. Using the dependent variables of: (1) total knowledge captured; (2) time to acquire knowledge; (3) knowledge quality; (4) efficiency of the knowledge elicitation method; and (5) importance of resulting data, experimental results indicate the various strengths of the four knowledge elicitation methods. The knowledge acquired is also significantly affected by the combined factors of expert's strengths in different cognitive factors and the method of knowledge elicitation used. Based on these findings, a Matching Index for combining tasks, knowledge elicitation methods and cognitive abilities of the expert is described. The outcome of this research provides theoretical and practical implications for Human Computer Interaction (HCI) and training of knowledge engineers.     

A web-based tool for eliciting probability distributions from experts

We present a web-based probability distribution elicitation tool: The MATCH Uncertainty Elicitation Tool. The Tool is designed to help elicit probability distributions about uncertain model parameters from experts, in situations where suitable data is either unavailable or sparse. The Tool is free to use, and offers five different techniques for eliciting univariate probability distributions. A key feature of the Tool is that users can log in from different sites and view and interact with the same graphical displays, so that expert elicitation sessions can be conducted remotely (in conjunction with tele- or videoconferencing). This will make probability elicitation easier in situations where it is difficult to interview experts in person. Even when conducting elicitation remotely, interviewers will be able to follow good elicitation practice, advise the experts, and provide instantaneous feedback and assistance.     

Parameterisation and evaluation of a Bayesian network for use in an ecological risk assessment

Catchment managers face considerable challenges in managing ecological assets. This task is made difficult by the variable and complex nature of ecological assets, and the considerable uncertainty involved in quantifying how various threats and hazards impact upon them. Bayesian approaches have the potential to address the modelling needs of environmental management. However, to date many Bayesian networks (Bn) developed for environmental management have been parameterised using knowledge elicitation only. Not only are these models highly qualitative, but the time and effort involved in elicitation of a complex Bn can often be overwhelming. Unfortunately in environmental applications, data alone are often too limited for parameterising a Bn. Consequently, there is growing interest in how to parameterise Bns using both data and elicited information. At present, there is little formal guidance on how to combine what can be learned from the data with what can be elicited. In a previous publication we proposed a detailed methodology for this process, focussing on parameterising and evaluating a Bn. In this paper, we further develop this methodology using a risk assessment case study, with the focus being on native fish communities in the Goulburn Catchment (Victoria, Australia).     

Habitat prediction and knowledge extraction for spawning European grayling (Thymallus thymallus L.) using a broad range of species distribution models

This study aims to apply seven data-driven methods (i.e. artificial neural networks [ANNs], classification and regression trees [CARTs], fuzzy habitat suitability models [FHSMs], generalized additive models [GAMs], generalized linear models [GLMs], random forests [RF] and support vector machines [SVMs]) to develop data-driven species distribution models (SDMs) for spawning European grayling (Thymallus thymallus), and to compare the predictive performance and the ecological relevance, quantified by the habitat information retrieved from these SDMs (i.e. variable importance and habitat suitability curves [HSCs]). The results suggest RF to yield the most accurate SDM, followed by SVM, CART, ANN, GAM, FHSM and GLM. However, inconsistencies between different performance measures were observed, indicating that different models may obtain a high score on a particular aspect and perform worse on other aspects. Despite their lower predictive ability, GAM, GLM and FHSM proved to be useful, since HSCs could be obtained and thus these techniques allow testing of ecological relevance and habitat suitability. Water depth and flow velocity appeared to be important variables for spawning grayling. The HSCs clearly indicate higher habitat suitability at a lower water depth, a low to medium flow velocity and a higher percentage of medium-sized gravel, whereas the models disagreed on the habitat suitability for the percentage of small-sized gravel. These findings demonstrate the applicability of data-driven SDMs for both habitat prediction and ecological knowledge extraction that are useful for management of a target species.     

Applying systems analysis techniques to knowledge engineering

Abstract: This paper deals with the issue of knowledge elicitation for expert systems. Specifically, it looks at the requirements of the knowledge elicitation process and the suitability of structured methods from systems analysis to carry out part of the elicitation task. The techniques of data flow analysis, entity-relationship analysis and entity-life cycle analysis are used to structure the data associated with the expert task. The methods proposed lay emphasis on the definition of limited data sets at the boundary of the explicit knowledge base and the identification of status attributes to model the control of activation of 'processes' within the knowledge base. Attention is also paid to the relationship between the resulting logical model, and two popular methods of knowledge representation, namely, Production Systems and Frames.     

An approach to expert control of interactive software systems

Expert problem-solving strategies in many domains require the use of detailed mathematical techniques coupled with experiential knowledge about how and when to use the appropriate techniques. In many of these domains, such techniques are made available to experts in large software packages. In attempting to build expert systems for these domains, we wish to make use of these packages, and are therefore faced with an important problem: how to integrate the existing software, and knowledge about its use, into a practical expert system. The expert knowledge is used, in dynamic selection and interpretation of appropriate programs and parameters, to reach a successful goal in the problem solving. We describe the framework of a hybrid expert system for representing problem-solving knowledge in these domains. This hybrid system may be characterized as consisting of a production system and mathematical methods. The software package is reorganized as necessary to map it into the mathematical-method representation of a hybrid system. This approach has evolved out of an effort to build an expert system for performing well-log analysis, ELAS (expert log analysis system).     

The bank of Scotland''s COMPASS—The future of bank lending?

This paper describes the background surrounding the need for risk analysis of commercial business within the domain of bank lending and of the development of an expert system for that task. Previous attempts at constructing expert systems in this area have either proved unsuccessful, software difficulties often being cited as the cause; or have stopped short of encapsulating all the relevant expertise. This paper considers the relevance of knowledge engineering to successful expert system construction. It reports on the development and structure of COMPASS, the Bank of Scotland's commercial lending adviser expert system which, by appropriate application of knowledge engineering, has succeeded in capturing and modelling the inherent risk of the Bank of Scotland's commercial lending process. The stages of its development are outlined; the knowledge elicitation process is described; knowledge articulation is examined from the perspective of the expert; the architecture of the system is explained; and the consultation procedure is described. In addition, reference is made to major attempts elsewhere to produce lending adviser expert systems; and the advantages; by-products, and long-term benefits of COMPASS are summarized.     

Laddering

Abstract: There is still a need for new knowledge elicitation techniques and tools. Laddering is a technique which has a long history in a wide range of disciplines and which has proved extremely useful in knowledge elicitation. There have, however, been few attempts to describe and survey the technique per se.
This paper describes the technique, its background, its use, analysis, and automation, with particular reference to knowledge elicitation. The advantages and disadvantages of the technique are described. It is concluded that laddering is a valuable technique which could be used in a wide range of settings.