Nancy E. Reed, Ph.D.
Experts at computer hardware fault diagnosis are good at the task with little formal training in the design and functionality of the equipment. These experts seem to develop knowledge that is largely separate from design knowledge, but is sufficient and powerful for doing diagnosis. The diagnosis of faults in complex computer systems is a difficult task. Experts at this task work on diagnosing many different systems or sub-systems. In addition, they are often required to diagnose new systems or sub-systems. The complexity and variety of equipment that they are faced with prevents them from having complete knowledge of the structure and function of each system. As a result they must focus on the diagnostically relevant portion of the available data in solving each problem.
The results of this investigation provide much information about the task of computer hardware diagnosis. Experts use inexact models of the components in the hardware in addition to strategies for acquiring data relevant to the current problem. In this manner, experts solve problems without resorting to complex causal reasoning. The implementation of some of the types of knowledge used by the experts in a diagnostic expert system is described. This program demonstrates the usefulness of implementing knowledge in a form similar to that used by experts. This type of reasoning can be more efficient than reasoning from first principles. The knowledge acquisition task for future problems can be much reduced by using the information about how experts structure the diagnostic knowledge. In addition, by using strategies based on expert knowledge, an expert system can have increased coverage and solve novel problems.
AAAI 88 Abstract
We introduce specialized strategies, an alternative level of reasoning, falling in generality between recognition-based reasoning and reasoning from first principles. These strategies are weak methods that are specific to a class of problems that occur in different domains. Specialized strategies are applicable not only to familiar problems in a domain, but also to problems that have not been anticipated. As a result they can provide both broad coverage currently given by ``causal'' reasoning and an efficiency close to that of ``shallow'' reasoning. The specialized strategies use inexact models of the components in the faulty system which contain only diagnostically relevant knowledge. Specialized strategies may be used in expert systems to increase efficiency, reduce brittleness, and decrease knowledge base construction effort compared to other common approaches. Examples are given from the domain of computer hardware diagnosis where two prototype expert systems were implemented.
A framework is presented for the creation of knowledge-based systems in the domain of pavement management. It uses problem solving methods in the main structuring role within the knowledge base. During the knowledge acquisition phase, the domain knowledge is identified and organized into levels of tasks. This organization enables a separation of generic knowledge which is applicable to many tasks and specific knowledge which describes a particular task application. Such a knowledge base separation reduces knowledge acquisition time and facilitates interaction with experts. The implementation of the above framework addresses knowledge-based applications for pavement management in Switzerland. The decentralized character of the country (independent decision-making at the Federal, Cantonal, and Communal levels) requires that the knowledge base be created with many generic structures which can be used directly (or, easily modified) at different application sites. A case study is given for pavement management in the Republic and Canton of Neuchâtel, based on priority ranking. It is concluded that: (a) the applied decomposition of the task into problem solving methods facilitates transferability and adaptability of the knowledge base through modularization and increased generality; (b) the analysis focuses on the content of the knowledge-base, and thus allows an evaluation of existing knowledge-based applications in the domain; and (c) the overall methodology can aid in achieving important goals such as development of generic and easily modifiable software, and reducing knowledge acquisition time in the domain.
ECAI 90 Abstract
We investigate the knowledge used by expert troubleshooters in the domain of computer hardware diagnosis. Due to the constraints imposed by the task, experts develop methods and models of the hardware that are specialized and powerful, yet general enough to be applied under many situations. The methods we discuss were discovered through the analysis of expert problem-solving behavior, including directed interviews, observation, and techniques of protocol analysis. The task investigated was the diagnosis of a complex piece of computer hardware. The environment required that the experts work on a large number of different pieces of hardware, for varying periods of time. Because of this constraint, the experts acquired proficiency at diagnosing new hardware in addition to becoming faster on familiar hardware. We examine their problem-solving behavior while they learn to diagnose a new piece of hardware.
The methods used by the experts include strong methods, here termed specialized strategies, and weak methods, such as the process of elimination and recognition. The combination of methods is both powerful and general. They are powerful because they perform effective diagnosis on a variety of hardware. They are efficient for the task because they require a minimum of data acquisition and time. While being specialized and efficient for the task, they are also general enough to apply to the majority of the problems encountered, including diagnosing new hardware. The methods work using inexact models of the system and components. These models contain only diagnostically relevant information.
As a test of this knowledge, a prototype expert system was implemented. The prototype serves as a computational model of the experts diagnostic methods and models. Its performance is used to investigate the adequacy and generality of this knowledge to perform diagnosis.
The acquisition of generative knowledge shows promise for greatly reducing the time and effort necessary for knowledge engineering. Examining expert problem-solving behavior on new objects allows us to see the generative behavior of the experts. The definition of generative knowledge highlights the distinction between this ``generic'' type of knowledge and the additional ``specific'' knowledge gained through experience which can make diagnosis even more efficient.
IJMMS 93 Abstract
We present an analysis of expert reasoning in the domain of computer hardware diagnosis. The methods used in the study include directed interviews, observation, and techniques of protocol analysis. The task investigated was the diagnosis of a complex piece of computer hardware. The initial symptoms are usually insufficient to determine the cause of a fault, requiring the acquisition of more data. Many thousands of pieces of data may be sought, but there is only enough time to obtain a few. We found that the experts use strategies to focus and obtain only the most relevant of this data. In addition, they use models of the hardware containing the diagnostically useful information. The strategies used are both powerful and efficient. Although they are specialized and save time and effort in performing the task, they are also general enough to apply to the majority of the problems encountered, including novel faults. As a test of these ideas, a prototype expert system was implemented. It is a model of problem-solving and serves as a test of the adequacy of the knowledge described to perform diagnosis.
AAAI SS 94 Abstract
We describe a manufacturing screening task, the diagnosis of defects on a computer board. Then we describe several strategies used by expert troubleshooters performing the task. These strategies use ``inexact models'' of the components and connections on the board. A prototype expert system has been implemented that uses the strategies and models. The strategies and models are robust because they are applicable to a wide range of problems, including problems not previously encountered.
Brittleness is a common problem with diagnostic systems. We propose the use of these types of strategies to increase the robustness of diagnostic systems.
AAAI 94 Abstract (http://www.ida.liu.se/~nanre/pubs/ai94abstract.html)
Conventional diagnostic methods assume that only a single defect is present. Cases with multiple defects can be difficult to diagnose because the defects can interact, meaning that the observable cues are not a sum of the cues for the component defects. Diagnostic methods that use cue-to-defect relationships fail when interactions between defects change the observable cues. Model-based methods can be used to diagnose multiple defects by simulating the results of interactions. Model-based methods, however, are limited to domains with accurate and complete models as well as enough available data to initialize the models.
This research develops a description and classification of the ways cues change when defects interact, based on physical interactions and example cases. Each type of cue may combine in a different way, so each has a separate description. Also developed is a formula for quantifying the amount of abnormal data explained by a set of defects, termed explanation points. The descriptions and formula are used in a domain-independent computational diagnostic model that can diagnose multiple defects, even when cues are altered or missing due to interactions between the defects.
In the medical domain investigated in this research (diagnosis of congenital heart defects), we found that cues combine with one another in a small number of ways: all cues may appear, the values of the cues may be added, or dominant cues may mask any other cues present. Cues of each type combine in one of these basic ways, or use a combination of a few of the basic ways, based on characteristics of the cues, case, or domain. The diagnostic model is tested by constructing a program with a knowledge base in Pediatric Cardiology and testing it on cases of single and multiple defects from hospital files. This program correctly diagnoses cases with multiple interacting defects for which current methods are not applicable or fail.
M. Computing 96 Abstract
This report describes two sections of an introductory computing class in which M was used as the programming language. One course was offered using conventionally scheduled lectures and laboratories and the second in an independent (autotutorial) mode. Comparisons of the students' selection of each type of class and their performance in the course are presented. Results show that the independent study students performed as well as those in the lecture course. The report recommends that MTA consider offering Internet-based courses using a model similar to the one described in this study.
AAAI SS 96 Abstract
Diagnosing multiple defects continues to be a difficult problem in many domains, especially medical domains. When multiple defects might be present, the number of potential solutions to each problem is greatly increased. If the defects interact, meaning that the cues observable for a combination of defects are not a sum of the cues observable for the component defects, the task is even more difficult. In particular, when defects interact, expected abnormal cues may be combined, missing, or altered, and new abnormal cues may appear.
The Fallot diagnostic model uses a classification and description of how cues combine with one another to diagnose multiple interacting defects. We describe results obtained on hospital cases using a knowledge base for diagnosing congenital heart defects. Fallot correctly diagnoses cases with multiple interacting defects for which other methods are not applicable or fail.
Applied AI 96 Abstract
We describe several strategies used by expert troubleshooters performing a manufacturing screening task, the diagnosis of defects on a computer board. These strategies use ``inexact models'' of the components and connections on the board. A prototype expert system has been implemented that uses the strategies and models. The strategies and models are robust because they are applicable to a wide range of problems, including problems not previously encountered. The system saves useful data acquired during problem-solving to assist in future problems.
We also describe how the above strategies and models can be used in a sensor-based system that acquires information about the board through a vision camera and other sensing devices. This will further increase the productivity of human troubleshooters.
Technical Report CSE-96-19 (http://www.ida.liu.se/~nanre/pubs/cse-96-19.html)
AAAI 96 Abstract (http://www.ida.liu.se/~nanre/pubs/ai96abstract.html)
Technical Report CSE-96-20 (http://www.ida.liu.se/~nanre/pubs/cse-96-20.html)
Horizon Paper (http://www.ida.liu.se/~nanre/pubs/distance-learning97.html)
AIMJ 97 Abstract
This paper describes a computational model developed for the diagnosis of multiple defects. If multiple defects interact , meaning that the cues observable for multiple defects are not a sum of the cues observable for the component defects, diagnosis is particularly difficult. We developed a description and classification of the ways cues change when defects interact. A computational model (named Fallot) was implemented and a knowledge-base was constructed for the diagnosis of congenital heart defects. On each case, Fallot performs recognition-based reasoning followed by solution construction and evaluation with the cue combination methods. Fallot was tested on cases from hospital files and correctly diagnoses cases with multiple interacting defects for which conventional methods are not applicable or fail.
AAAI 98 Abstract
When multiple defects (also called diseases or faults) are present, there is a possibility of interactions between the defects. When defects interact, the cues (data obtainable) for a combination of defects is not a simple sum of the cues observable for the component defects. Expected cues may be missing, altered, or new cues may appear. Each of these alterations of cues makes diagnosis more difficult, as the correct defect combination may not even be considered (triggered) by a diagnostic system. We present an algorithm for heuristic solution construction that integrates multiple types of information about the case. Solutions are evaluated based on how many of the abnormal cues are accounted for, with a method that combines cues that may be altered due to interactions between defects. The method can account for cues that combine with one another in three basic ways, set union, additively and ordered dominance (some values mask other values) or with a combination of those basic ways.
For the solution space of one task, diagnosing congenital heart defects, we considered seven major defects and found the solution space (exhaustive) was reduced by approximately 50% because some of the defects could not physically occur together. Experimental results on cases from hospital files demonstrate the effectiveness of the heuristic solution construction algorithm to generate the correct solution early which reduced the number of solutions explored (compared to an exhaustive search) even further on most cases. With the computational power of current workstations, even cases requiring exploration of this entire solution space required less than 4 minutes of CPU time per case.