Search NASA⌕ Search

SEARCH · Search NASA

Results for “evidentiary reasoning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

Proactive Intelligence for Nuclear Nonproliferation

The project described in this paper leverages predictive models for proliferation detection in order to assess the complementary questions of capability and intent as they relate to the potential for nuclear weapon development. The ability to proactively assess the likelihood of a state to engage in nuclear power acquisition and development for non-peaceful purposes is one of the greatest challenges for analysts and policy makers working on proliferation detection and deterrence. Of further difficulty is determining whether a state is at risk to provide indirect support for proliferation via the relationship between industrial input/output and the legal framework of trade. In general, it is possible to gather evidence about precursor activities to the achieved nuclear potential of a state that function as indicators of the state's intent to acquire and develop capabilities to support nuclear weapons. Reasoning with these indicators to predict intent and capability to proliferate is of utmost importance to facilitate nuclear safeguards, e.g. through proactive implementation of countermeasures. Such a predictive reasoning task is difficult to perform without computational aid. While the need for a proactive and multi-perspective approach to proliferation detection is widely recognized, there is a lamentable lack of computational tools applied directly to the task. Applications of predictive modeling to the domain of nuclear nonproliferation are limited to physical/chemical properties of nuclear materials, such as nuclear weapons simulations and stockpile stewardship. The aim of this project is to address this gap by leveraging methods and data from different mission areas in support of proliferation detection and prevention in innovative ways. More specifically, the approach implemented in this project combines methods in information analysis and probabilistic evidentiary reasoning with expert knowledge from discipline areas germane to proliferation detection, and evidence extracted from relevant data sources, to assess alternative hypotheses about specific proliferation detection problems.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Technosocial Predictive Analytics for Illicit Nuclear Trafficking

Illicit nuclear trafficking networks are a national security threat. These networks can directly lead to nuclear proliferation, as state or non-state actors attempt to identify and acquire nuclear weapons-related expertise, technologies, components, and materials. The ability to characterize and anticipate the key nodes, transit routes, and exchange mechanisms associated with these networks is essential to influence, disrupt, interdict or destroy the function of the networks and their processes. The complexities inherent to the characterization and anticipation of illicit nuclear trafficking networks requires that a variety of modeling and knowledge technologies be jointly harnessed to construct an effective analytical and decision making workflow in which specific case studies can be built in reasonable time and with realistic effort. In this paper, we explore a solution to this challenge that integrates evidentiary and dynamic modeling with knowledge management and analytical gaming, and demonstrate its application to a geopolitical region at risk.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Expert evaluation of LLM world models: A high-T c superconductivity case study

Large Language Models (LLMs) show great promise as a powerful tool for scientific literature exploration. However, their effectiveness in providing scientifically accurate and comprehensive answers to complex questions within specialized domains remains an active area of research. Using the field of high-temperature cuprates as an exemplar, we evaluate the ability of LLM systems to understand the literature at the level of an expert. We construct an expert-curated database of 1,726 scientific papers that covers the history of the field, and a set of 67 expert-formulated questions that probe deep understanding of the literature. We then evaluate six different LLM-based systems for answering these questions, including both commercially available closed models and a custom retrieval-augmented generation (RAG) system capable of retrieving images alongside text. Experts then evaluate the answers of these systems against a rubric that assesses balanced perspectives, factual comprehensiveness, succinctness, and evidentiary support. Among the six systems, two using RAG on curated literature outperformed existing closed models across key metrics, particularly in providing comprehensive and well-supported answers. We discuss promising aspects of LLM performances as well as critical short-comings of all the models. The set of expert-formulated questions and the rubric will be valuable for assessing expert level performance of LLM based reasoning systems.

36 MATERIALS SCIENCE↗