The Chekhov Taxonomy: A Framework for AI Safety Evaluation Research
Explore the source record for details and available documents.
Odom, Peter Wesley [Sandia National Laboratories (↗
SEARCH · Search NASA
Results for “AI safety”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Safety in Artificial Intelligence: Challenges and Opportunities for the U.S. National Labs and Beyond
This report discusses the importance of the critical and underexplored topic of artificial intelligence (AI) safety, as highlighted during the “Strategy Alignment on AI Safety” workshop convened at Lawrence Livermore National Laboratory (LLNL) in April 2024. Through a summary of keynote talks, panel discussions, and breakout sessions, world-leading AI safety experts from academic, industry, national labs, and government agencies clearly agree on the need for and importance of large-scale investments for research and capabilities in AI safety. With the field innovating at unprecedented rates, there is increasing urgency to develop novel evaluation methodologies that allow full considerations of risks/threats of AI technologies in different domains. Quantitative metrics and effective methodologies that can evaluate and audit the “safeness” of how a given AI technology is trained, deployed, or regulated are, at best, nascent for certain scenarios or, more commonly, nonexistent. This maturation gap presents the possibility of serious threats to national security, and further inaction may have serious consequences. Additionally, the gap between the public’s and research community’s perceptions of AI risks/rewards is significant. While numerous voices from the AI community have expressed concern that the risks could be so high that future AI systems could inflict extinction-level damage to humanity if deployed incorrectly, the public largely is aware only of risk in low-impact scenarios. This discrepancy highlights the crucial need for researchers to articulate to governmental bodies what, why, and when various AI risks matter as part of motivating funding requests. Thus, the call to action for this community is to pursue AI safety as a “Big Science” project on a scale comparable to the Manhattan Project. High risks and high payoffs are on the table, but safe AI is a fast-moving target, and large-scale investments are needed to guide development of this technology in a responsible way. We highlight the need for a multilayered solution combining the development of new methods and algorithmic approaches to mitigate threats with an active participation of the government(s) in setting high industry standards and regulations based on state-of-the-art technology. The U.S. Department of Energy (DOE) national laboratories have served as leading institutions for scientific innovation in the U.S. for more than 70 years. Drawing on their expertise in the AI community and their history of safeguarding critical and sensitive information, and as we look to the future, national labs are the best choice for evaluating and safeguarding AI technologies.
97 MATHEMATICS AND COMPUTING↗
Enhancing Operational Safety via Agentic Dialogue Hazard Identification Analysis
Operational safety in high-stakes domains such as industrial process control, autonomous, and safety-critical systems demand reliable hazard identification. While large language models (LLMs) have shown promise in automating safety analysis tasks, single-turn, monolithic inference is brittle: it lacks the self-correction, deliberation, and contextual refinement that safety engineers apply iteratively. In this paper, we introduce HAZDIAL, a framework that investigates whether structured agentic dialogue (multi-agent, multi-turn interactions) improves the quality of NLP-based hazard identification over single-pass baselines. We systematically compare two dialogue modalities: adversarial debate and constructive discussion, and propose an genetic algorithm-based agentic interaction optimization. We evaluate all configurations against a curated golden dataset using standard classification metrics (accuracy, precision, recall, F1) and a novel dialogue metrics. This work advances the intersection of dialogue systems, multi-agent reasoning, and AI safety, providing empirical evidence for dialogue-driven hazard analysis.
Das, Sanjay [ORNL] (ORCID:0009000542591915)↗
Digital Documented Safety Analysis: Evaluating the Use of AI for Rapid Generation of Nuclear Safety Basis Documents
Presentation for INL's AI/ML Symposium. Topic: Evaluation of AI-generated documents for nuclear permitting/licensing.
42 - ENGINEERING↗
Evaluation of AI-Enabled Digital Documented Safety Analysis: A Case Study
Safety basis documentation development and review under U.S. Department of Energy (DOE) authorization have emerged as critical constraint throttling deployment of advanced nuclear reactors, with traditional processes demanding extraordinary resource investment that delays the delivery of these technologies. Traditional Documented Safety Analysis (DSA) processes rely on static documents with limited traceability [U.S. DOE]. The regulatory review and engagement processes are similarly constrained, often requiring significant effort and extensive manual verification. The scale of this challenge is exemplified by the U.S. Nuclear Regulatory Commission (NRC) review of the NuScale application, which required over 250,000 staff hours and the evaluation of approximately two million pages of documentation [Bergman 2021]. The volume and complexity of information within nuclear licensing applications or authorization reviews demands innovative approaches to document generation and data management.
22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
TruePAL – An AI Assistant for First Responder Safety
This paper presents the development of an AI assistant, Trusted and Explainable Artificial Intelligence for Saving Lives (TruePAL), to provide real-time warning of risks of potential crashes to the first responders. The TruePAL system employs an AI and deep learning technology for saving first responders and roadside crews lives in and around active traffic. A deep neural network (DNN) and a Non-Axiomatic Reasoning System (NARS) are implemented as an AI system. A mobile app with AI interface is developed to perform verbal communication with the first responders. The TruePAL team has developed an explainable AI approach by opening up the DNN blackbox to extract the activation filters of various features and parts of the targeted objects. The combination of DNN and NARS makes the TruePAL system explainable to the users. TruePAL ingests on-board cameras, radar, and other sensor signals, analyzes the environment and traffic patterns to generate timely warning to drivers and roadside crews to avoid crashes. The TruePAL team, in collaboration with the Miami/Dade Police Dept., has designed five use cases and multiple sub-scenarios in a CARLA driving simulator to test the capability of TruePAL in timely warning to the first responder drivers in potential crash scenarios. We have successfully demonstrated its capability of timely warning in over a dozen scenarios based on the use cases. The preliminary test simulation results show that TruePAL could provide the drivers and crew members advanced warning before a crash occurs.
Chow, Edward↗
Draft Feasibility Assessment for Use of AI in Preparing Transportation Safety Analysis Reports
Preparing transportation safety analysis reports for microreactors is time and labor intensive, requiring extensive cross referencing to Federal regulations, previously approved documents, and expert review comments across structural, thermal, criticality, shielding, containment, and security. These burdens are magnified by the novelty of microreactor technologies and the evolving regulatory landscape, as well as current workforce constraints. Generative AI and supporting machine learning tools present an opportunity to accelerate drafting timelines, lift generalized writing burdens, and systematically enforce regulatory adherence through retrieval augmented generation and other knowledge retrieval and mapping methods. This draft report presents a preliminary feasibility assessment of the use of AI to expedite the preparation of microreactor transportation safety analysis reports and proposes an initial methodology for doing so.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Evaluation of AI-enabled Digital Documented Safety Analysis
The National Reactor Innovation Center (NRIC) is leading a transformative initiative to accelerate advanced reactor deployment by fundamentally reimagining how nuclear safety basis documentation is developed, reviewed, and maintained. Traditional Documented Safety Analysis (DSA) processes for DOE-authorized facilities rely on static, document-centric workflows that consume significant time and resources, exemplified by recent major licensing efforts requiring hundreds of thousands of staff hours and millions of pages of documentation review. These conventional approaches create barriers to the rapid, cost-effective deployment of advanced reactors that America's future energy needs demand. NRIC's DOE Authorization Digital Transformation Project addresses these challenges through an innovative framework that integrates artificial intelligence (AI), digital engineering, and systems-based data management into a cohesive digital ecosystem. This white paper presents NRIC's methodology for evaluating AI-enabled document generation capabilities within this broader digital infrastructure, using the Demonstration of Microreactor Experiments (DOME) facility as a pilot case study. The evaluation will assess an AI tool's ability to generate a Preliminary Documented Safety Analysis (PDSA) through progressive integration stages—from standalone document processing to full digital thread connectivity—while maintaining rigorous verification, validation, and regulatory acceptance standards. By establishing dynamic, traceable connections between design data and safety documentation, NRIC's approach has the potential to reduce both document development time and regulatory review cycles by as much as 50%, while simultaneously improving accuracy, consistency, and traceability. This initiative represents a critical step toward establishing reusable digital infrastructure that reactor developers can leverage to accelerate their path from concept to commercial operation, directly supporting NRIC's mission to demonstrate and deploy advanced nuclear energy technologies.
22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
The role of AI in detecting and mitigating human errors in safety-critical industries: A review
For safety-critical industries, human error (HE) presents continual risks to system productivity, reliability and safety. Artificial intelligence (AI) and machine learning (ML) methods have emerged as promising approaches to understand, categorize and mitigate the risk of HE in safety-critical industries. Furthermore, this review offers an examination of the current landscape regarding the utilization of AI/ML with regards to HE in safety-critical industries, categorizing literature into descriptive modeling, predictive modeling, prescriptive modeling, and generative modeling techniques. Additionally, the review aims to provide insights regarding themes in literature, challenges, and future research directions. Findings of the review suggest that AI/ML methods can prove useful in addressing the HE problem across safety-critical industries.
42 ENGINEERING↗
Genesis Mission-Enabled Secure AI to Fortify Energy Process Safety (Genesis-SAFE)
Argonne National Laboratory is supporting the U.S. Department of Transportation’s (USDOT’s) Bureau of Transportation Statistics (BTS) with collaborative research on development and application of privacy preserving AI frameworks that leverage unmatched AI expertise and secure computing resources made available through the U.S. Genesis Mission1 . This research advances U.S. energy security goals by supporting a safe offshore energy industry with secure, domain-specific AI tools to analyze confidential industry datasets collected by BTS to rapidly improve identification of hazards, precursors, and systemic safety risks in high-risk operational environments. The staged, security-first approach begins with development and testing of Argonne’s Genesis Mission-enabled Secure AI to Fortify Energy Process Safety (Genesis-SAFE) framework within Argonne’s accredited secure computing enclave (ABLE) leveraging Argonne’s AI scientific assistant substrate (AISAC). Methods to build synthetic datasets were developed together with BTS for use in preparing synthetic datasets that can be used to validate data containment, governance, and security controls in the ABLE environment. Future research directions would focus on applying the Genesis-SAFE framework to CIPSEA-protected datasets entirely within ABLE to support confidentiality-preserving analysis of safety risks, trends, and contributing factors.
Kim, Hyekyung [Argonne National Laboratory (ANL), ↗
An MBSE Approach for Developing an Autonomous Rover Platform
The proliferation of increasingly autonomous systems calls for new ways to address how safety is assured. As these systems become more advanced and complex, it becomes more important to model and prototype autonomous functions at the systems level and the functions that assure they are operating safely and as expected. To that effect, researchers at the National Aeronautics and Space Administration (NASA) 's Robust Software Engineering (RSE) group are working on prototyping a Research Autonomous Vehicle, commonly referred to as R-RAV. The R-RAV is an autonomous rover platform designed to act as a case study for assured autonomy research. Moreover, an overarching goal is for the R-RAV to serve as a training ground for other mission projects. In this paper, we will detail how we have used a Model-Based Systems Engineering (MBSE) approach to model a prototype of the R-RAV and test and verify its different functionalities.
MBSE↗
Leveraging artificial intelligence and advanced food processing techniques for enhanced food safety, quality, and security: a comprehensive review
Artificial intelligence is emerging as a transformative force in addressing the multifaceted challenges of food safety, food quality, and food security. This review synthesizes advancements in AI-driven technologies, such as machine learning, deep learning, natural language processing, and computer vision, and their applications across the food supply chain, based on a comprehensive analysis of literature published from 1990 to 2024. AI enhances food safety through real-time contamination detection, predictive risk modeling, and compliance monitoring, reducing public health risks. It improves food quality by automating defect detection, optimizing shelf-life predictions, and ensuring consistency in taste, texture, and appearance. Furthermore, AI addresses food security by enabling resource-efficient agriculture, yield forecasting, and supply chain optimization to ensure the availability and accessibility of nutritious food resources. This review also highlights the integration of AI with advanced food processing techniques such as high-pressure processing, ultraviolet treatment, pulsed electric fields, cold plasma, and irradiation, which ensure microbial safety, extend shelf life, and enhance product quality. Additionally, the integration of AI with emerging technologies such as the Internet of Things, blockchain, and AI-powered sensors enables proactive risk management, predictive analytics, and automated quality control. By examining these innovations' potential to enhance transparency, efficiency, and decision-making within food systems, this review identifies current research gaps and proposes strategies to address barriers such as data limitations, model generalizability, and ethical concerns. These insights underscore the critical role of AI in advancing safer, higher-quality, and more secure food systems, guiding future research and fostering sustainable food systems that benefit public health and consumer trust.
AI↗
Artificial Intelligence in Aviation Safety Applications - Exploring Myths and Truths of AI and ML
Artificial Intelligence (AI) and machine learning (ML) are gaining increased attention as ways to leverage the world's data to solve problems. Although AI and ML offer much potential, there are often misconceptions about the application of such techniques.Panel speakers will present machine learning approaches they have developed on a variety of aviation data, including digital flight data, safety reporting data, and voice communications data. They will discuss the purpose of the application, the data used, and the lessons learned in the development and deployment of their solutions. The panel will also discuss common pitfalls in developing an AI solution, the dangers of the current hype around AI, tips for gaining value from a ML solution, how to determine whether a ML approach is appropriate for a problem, and more.
Reeves, Scott (Capt.)↗
AI & Machine Learning: A Primer for Nuclear Safety Professionals
Explore the source record for details and available documents.
Computer science↗
Transfer-AE: A novel autoencoder-based impact detection model for structural digital twin
Accurately detecting the location and intensity of impacts is crucial for ensuring structural safety. Currently, AI-based structural impact detection methods are widely used for their excellent detection accuracy. However, their generalization capability is limited by the scenarios present in the training data. Many complex and dangerous impact scenarios are difficult to conduct real-world experiments on to collect sufficient samples. To capture all impact scenarios and fully leverage the advantages of AI-based detection technologies, advanced methods involve combining real-world structural monitoring data with corresponding numerical models to construct digital twins. These methods continuously refine the created numerical models with limited real-world data and provide diverse impact scenarios through numerical model simulations. However, there are inevitable differences between digital models and physical models that are challenging to correct through mechanical means. This discrepancy in data distribution between the two models significantly hinders the application of digital twin technology in impact/event identification tasks. To address this challenge, this study proposes a novel model based on autoencoders, named Transfer-AE. Transfer-AE encodes the common features of digital twins in the latent space to bridge the uncertainty gap at a macro scale between numerical models and physical models and synchronously fits the magnitude and location of the impact load in the decoder. This enables consistent detection results for the same impact event, whether the sample comes from the numerical model or the physical model. Transfer-AE includes two operating modes: Mode 1 has a fixed computational complexity with stable inference speed, but the training cost and difficulty increase with data distribution. Mode 2's computational complexity increases with data distribution, but it has a fixed training cost and speed. In both cases involving the geodesic dome structure simulating a deep space habitat and the IASC-ASCE benchmark structure, Transfer-AE demonstrated the best performance in impact localization and quantification tasks compared to mainstream domain-adaptive transfer models.
Chengjia Han↗
Artificial Neural Networks and AI in high Assurance Applications: Gaps and Techniques
In recent years, capabilties of Deep Neural Networks (DNN) and Artificial Intelligence (AI) systems have grown tremendously. They are now applied in many areas ranging from game playing, social media, science, to robotics, automotive, and aerospace applications.Based upon requirements for safety of DNN and AI in high assurance automotive and aerospace applications, I will discuss the necessity to ensure that AI technqiues for the analysis of Earth observation data and reasoning are working correctly and reliably.In this talk I will present modern techniques for the verification and validation (V&V) of DNN and other AI components as well as approaches for interpretable AI. I will discuss how these techniques can help to ensure quality of the AI results, improve confidence in their application, and facilitate human-AI interaction and collaboration.
Johann Schumann↗
REFSafE: A RAG-Enabled Framework for Predictive Risk Analysis and Automated Safety Report Generation in Mission-Critical Environments
Operational safety in mission-critical environments requires AI systems that are accurate, interpretable, and resistant to hallucination. We present an agentic Retrieval-Augmented Generation (RAG) framework, REFSafe, for grounded hazard analysis and automated safety report generation. The system integrates Large Language Models (LLMs) with structured operational data, historical incident repositories, policy documents, and external authoritative sources. Through iterative agentic reasoning, the framework retrieves, verifies, and synthesizes evidence prior to generation, enforcing citation-backed outputs with explicit source attribution (documents, links, and prior events) to ensure traceability and trust. To mitigate hallucinations and unsupported claims, all risk assessments and forecasts are constrained to retrieved evidence, with confidence signals derived from retrieval relevance and source consistency. A transparent pipeline enables subject matter experts (SMEs) to validate predictions, and provide structured feedback, forming a continuous performance calibration loop. Preliminary deployment demonstrates improved reliability in hazard detection and safety/vulnerability report generation. This work advances trustworthy, evidence-grounded AI for predictive safety intelligence in mission-critical operations.
Das, Sanjay [ORNL] (ORCID:0009000542591915)↗