On recognizing ignorance

Abstract

How an expert system reasons about its own ability to deal with a particular problem is studied. Ideally, an expert system ought to rapidly recognize that a particular problem is beyond its abilities and defer to another, perhaps human, expert. This capability is extremely important in domains where expert systems may control life critical processes such as air traffic control, medicine, strategic defense, and manned space exploration. The methods used by knowledge engineers to infuse an expert system with knowledge of its own limitations is surveyed. A computability theory is employed to analyze the general problem of meta-knowledge and to give insight into the efficacy of specific solutions.