Search NASA⌕ Search

SEARCH · Search NASA

Results for “adversarial evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Parameters, Properties, and Process: Conditional Neural Generation of Realistic SEM Imagery Toward ML-Assisted Advanced Manufacturing

Abstract The research and development cycle of advanced manufacturing processes traditionally requires a large investment of time and resources. Experiments can be expensive and are hence conducted on relatively small scales. This poses problems for typically data-hungry machine learning tools which could otherwise expedite the development cycle. We build upon prior work by applying conditional generative adversarial networks (GANs) to scanning electron microscope (SEM) imagery from an emerging advanced manufacturing process, shear-assisted processing and extrusion (ShAPE). We generate realistic images conditioned on temper and either experimental parameters or material properties. In doing so, we are able to integrate machine learning into the development cycle, by allowing a user to immediately visualize the microstructure that would arise from particular process parameters or properties. This work forms a technical backbone for a fundamentally new approach for understanding manufacturing processes in the absence of first-principle models. By characterizing microstructure from a topological perspective, we are able to evaluate our models’ ability to capture the breadth and diversity of experimental scanning electron microscope (SEM) samples. Our method is successful in capturing the visual and general microstructural features arising from the considered process, with analysis highlighting directions to further improve the topological realism of our synthetic imagery.

36 MATERIALS SCIENCE↗

THE ATTRACTIVENESS OF MATERIALS IN ADVANCED NUCLEAR FUEL CYCLES FOR VARIOUS PROLIFERATION AND THEFT SCENARIOS

We must anticipate that the day is approaching when details of nuclear weapons design and fabrication will become common knowledge. On that day we must be particularly certain that all special nuclear materials (SNM) are adequately accounted for and protected and that we have a clear understanding of the utility of nuclear materials to potential adversaries. To this end, this paper examines the attractiveness of materials mixtures containing SNM and alternate nuclear materials associated with the plutonium-uranium reduction extraction (Purex), uranium extraction (UREX), coextraction (COEX), thorium extraction (THOREX), and PYROX (an electrochemical refining method) reprocessing schemes. This paper provides a set of figures of merit for evaluating material attractiveness that covers a broad range of proliferant state and subnational group capabilities. The primary conclusion of this paper is that all fissile material must be rigorously safeguarded to detect diversion by a state and must be provided the highest levels of physical protection to prevent theft by subnational groups; no 'silver bullet' fuel cycle has been found that will permit the relaxation of current international safeguards or national physical security protection levels. The work reported herein has been performed at the request of the U.S. Department of Energy (DOE) and is based on the calculation of 'attractiveness levels' that are expressed in terms consistent with, but normally reserved for, the nuclear materials in DOE nuclear facilities. The methodology and findings are presented. Additionally, how these attractiveness levels relate to proliferation resistance and physical security is discussed.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Cyber Security Research Frameworks For Coevolutionary Network Defense

Several architectures have been created for developing and testing systems used in network security, but most are meant to provide a platform for running cyber security experiments as opposed to automating experiment processes. In the first paper, we propose a framework termed Distributed Cyber Security Automation Framework for Experiments (DCAFE) that enables experiment automation and control in a distributed environment. Predictive analysis of adversaries is another thorny issue in cyber security. Game theory can be used to mathematically analyze adversary models, but its scalability limitations restrict its use. Computational game theory allows us to scale classical game theory to larger, more complex systems. In the second paper, we propose a framework termed Coevolutionary Agent-based Network Defense Lightweight Event System (CANDLES) that can coevolve attacker and defender agent strategies and capabilities and evaluate potential solutions with a custom network defense simulation. The third paper is a continuation of the CANDLES project in which we rewrote key parts of the framework. Attackers and defenders have been redesigned to evolve pure strategy, and a new network security simulation is devised which specifies network architecture and adds a temporal aspect. We also add a hill climber algorithm to evaluate the search space and justify the use of a coevolutionary algorithm.

97 MATHEMATICS AND COMPUTING↗

Security Self-Assessment Toolkit for Nuclear Materials Facilities: Focus on Insider Threat Mitigation

Theft or sabotage of weapons-usable nuclear materials is a global concern. To minimize this threat, establishing and maintaining an effective nuclear security regime is required to protect against criminal or other negligent acts. Use of a formalized insider threat mitigation program is one such security measure. Individuals who have or held authorized access to an organization's critical assets, such as nuclear materials, are considered "insiders." Insider threats, or insider adversaries, are motivated individuals who possess access, authority, and knowledge to conduct a malicious act or facilitate that of an external party. To thwart insider threats (both intentional and unintentional), organizations can formalize an enterprise-wide approach to identify and mitigate the unique risks presented by insiders. This report provides an approach to evaluate an insider threat mitigation program at facilities with nuclear materials. Formal program evaluations serve many purposes and can be designed using several different methods and techniques. This report presents a self-assessment approach to program evaluation whereby an organization can assess its strengths, identify key gaps, and set priorities for ongoing improvement efforts to mitigate insider threats. Results of the self-assessment can provide critical information to contribute to the continuous improvement of an organization’s insider threat mitigation program within eight specific domain areas.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

CO 2 storage site characterization using ensemble-based approaches with deep generative models

Estimating spatially distributed properties such as permeability from available sparse measurements is a great challenge in efficient subsurface CO 2 storage operations. In this paper, a deep generative model that can accurately capture complex subsurface structure is tested with an ensemble-based inversion method for accurate and accelerated characterization of CO 2 storage sites. We chose Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) for its realistic reservoir property representation and Ensemble Smoother with Multiple Data Assimilation (ES-MDA) for its robust data fitting and uncertainty quantification capability. WGAN-GP are trained to generate high-dimensional permeability fields from a low-dimensional latent space and ES-MDA then updates the latent variables by assimilating available measurements. Several subsurface site characterization examples including Gaussian, channelized, and fractured reservoirs are used to evaluate the accuracy and computational efficiency of the proposed method and the main features of the unknown permeability fields are characterized accurately with reliable uncertainty quantification. Furthermore, the estimation performance is compared with a widely-used variational, i.e., optimization-based, inversion approach, and the proposed approach outperforms the variational inversion method in several benchmark cases. We explain such superior performance by visualizing the objective function in the latent space: because of nonlinear and aggressive dimension reduction via generative modeling, the objective function surface becomes extremely complex while the ensemble approximation can smooth out the multi-modal surface during the minimization. This suggests that the ensemble-based approach works well over the variational approach when combined with deep generative models at the cost of forward model runs unless convergence-ensuring modifications are implemented in the variational inversion.

42 ENGINEERING↗

BioADAPT-MRC: adversarial learning-based domain adaptation improves biomedical machine reading comprehension task

ABSTRACT Motivation Biomedical machine reading comprehension (biomedical-MRC) aims to comprehend complex biomedical narratives and assist healthcare professionals in retrieving information from them. The high performance of modern neural network-based MRC systems depends on high-quality, large-scale, human-annotated training datasets. In the biomedical domain, a crucial challenge in creating such datasets is the requirement for domain knowledge, inducing the scarcity of labeled data and the need for transfer learning from the labeled general-purpose (source) domain to the biomedical (target) domain. However, there is a discrepancy in marginal distributions between the general-purpose and biomedical domains due to the variances in topics. Therefore, direct-transferring of learned representations from a model trained on a general-purpose domain to the biomedical domain can hurt the model’s performance. Results We present an adversarial learning-based domain adaptation framework for the biomedical machine reading comprehension task (BioADAPT-MRC), a neural network-based method to address the discrepancies in the marginal distributions between the general and biomedical domain datasets. BioADAPT-MRC relaxes the need for generating pseudo labels for training a well-performing biomedical-MRC model. We extensively evaluate the performance of BioADAPT-MRC by comparing it with the best existing methods on three widely used benchmark biomedical-MRC datasets—BioASQ-7b, BioASQ-8b and BioASQ-9b. Our results suggest that without using any synthetic or human-annotated data from the biomedical domain, BioADAPT-MRC can achieve state-of-the-art performance on these datasets. Availability and implementation BioADAPT-MRC is freely available as an open-source project at https://github.com/mmahbub/BioADAPT-MRC. Supplementary information Supplementary data are available at Bioinformatics online.

60 APPLIED LIFE SCIENCES↗

Assessment of Russian VSTOL technology evaluating the YAK-38 'FORGER' and YAK-141 'FREESTYLE'

The dissolution of the Former Soviet Union (FSU) created new relationships between the world superpowers. Overnight, the Commonwealth of Independent States (CIS), formed from the remnants of the FSU, began the difficult transformation to a free market society. Military hardware that had once been highly classified and the basis for our own defense planning was now openly marketed at airshows around the world. 'Test' flights were available for potential customers and cooperative partnerships were explored between former adversaries. This environment permitted a visit to the Yakovlev Design Bureau, (YAK) for a vertical/short takeoff and landing (VSTOL) technology assessment. Yakovlev is the FSU's sole Design Bureau with experience in VSTOL aircraft and has developed two flying examples, the YAK-38 'FORGER' and YAK-141 'FREESTYLE'. This article reviews the performance of the YAK-38 'FORGER' and the YAK-141 'FREESTYLE'.

Nalls, Art↗

Robust Explanations using Diverse Adversarially Trained Ensembles, Multi-Modal Contrastive Learning, and Attribution-based Confidence Metrics

The primary objective of this project is to strengthen the trustworthiness of AI systems by designing algorithms that make their internal decision-making processes more understandable to human users. This involves creating clear, interpretable explanations for AI decisions and developing metrics to assess these explanations' validity and reliability. Significant progress has been achieved through (i) developing symbolic explanations, (ii) generating meaningful interpretive insights, (iii) establishing accuracy and confidence metrics, and (iv) devising methods to evaluate the knowledge boundaries of AI models. To date, the research findings have been shared in peer-reviewed publications, with accompanying scientific and technical information (STI) detailed below.

97 MATHEMATICS AND COMPUTING↗

Deconstructing the Assessment of Anomaly-based Intrusion Detectors for Critical Applications

Anomaly detection is a key strategy for cyber intrusion detection because it is conceptually capable of detecting novel attacks. This makes it an appealing defensive technique for environments such as the nation's critical infrastructures that is currently facing increased cyber adversarial activity. When considering deployment within the purview of such critical infrastructures it is imperative that the technology is well understood and reliable, where its performance is benchmarked on the results of principled assessments. This paper works towards such an imperative by analyzing the current state of anomaly detector assessments with a view toward mission critical deployments. We compile a framework of key evaluation constructs that identify how and where current assessment methods may fall short in providing sufficient insight into detector performance characteristics. Within the context of three case studies from literature, we show how error factors that influence the performance of detectors interact with different phases of a canonical evaluation strategy to compromise the integrity of the final results.

Viswanathan, Arun↗

A Communication Channel Density Estimating Generative Adversarial Network

Autoencoder-based communication systems use neural network channel models to backwardly propagate message reconstruction error gradients across an approximation of the physical communication channel. In this work, we develop and test a new generative adversarial network (GAN) architecture for the purpose of training a stochastic channel approximating neural network. In previous research, investigators have focused on additive white Gaussian noise (AWGN) channels and/or simplified Rayleigh fading channels, both of which are linear and have well defined analytic solutions. Given that training a neural network is computationally expensive, channel approximation networks— and more generally the autoencoder systems—should be evaluated in communication environments that are traditionally difficult. To that end, our investigation focuses on channels that contain a combination of non-linear amplifier distortion, pulse shape filtering, intersymbol interference, frequency-dependent group delay, multipath, and non-Gaussian statistics. Each of our models are trained without any prior knowledge of the channel. We show that the trained models have learned to generalize over an arbitrary amplifier drive level and constellation alphabet. We demonstrate the versatility of our GAN architecture by comparing the marginal probability density function of several channel simulations with that of their corresponding neural network approximations

Smith, Aaron↗

Probability of Encounter Between Ports of Entry (PrEP): Aversion Force-Field “Early Validation” Report

To enhance detection and interdiction of radiological or nuclear material trafficking between primary POEs into the US, The Department of Homeland Security is faced with highly variable smuggling pathways through expansive geographic regions. The goal of this project is to test an approach to estimating likely pathways, as well as the probability of encounter along any given route. Our goal is to test the feasibility of a novel approach based on assessing the adversary’s perception of discovery likelihood over a broad geographic region, and using that result to estimate an “aversion force field” over the same region. The aversion force field is derived from analogies based on the physics of electric charges. Combining the aversion force with the intention of an adversary to reach their destination, generates a route that is consistent with both the adversary’s intention and their aversion to deterrent measures. Phase I is exploratory. Its goal to demonstrate the feasibility of the approach. This report describes progress in an essential part of the approach, specifically, to implement and couple two principle model components: (1) the Bayesian network model of an adversary’s perception of discovery likelihood; and (2) the physics-inspired aversion field model. The objective is to show that the field produced from the two components is plausible and consistent with the assumptions and the data. Figure 1 illustrates the workflow of this phase to the present stage. After collecting preliminary data and implementing each of the two model components, they were separately evaluated. Following this step, the output from the Bayesian network was input to the aversion field solver, and the combined results evaluated as reported here.

97 MATHEMATICS AND COMPUTING↗

Complete Evaluation on Advanced Reactor Machine Learning Subversion Attacks (Final)

Navigating through the world of Artificial Intelligence (AI) in nuclear reactors and their Instrumentation and Control (I&C) systems demands a careful, deliberate journey. AI’s capability to manage massive datasets and streamline control systems has indeed carved out a significant role in various sectors, including nuclear energy. However, while AI, and particularly Large Language Models (LLMs), bring a lot to the table in terms of operational efficiency and anomaly detection, they also expose the sector to a new breed of cybersecurity threats, like Inference Attacks, Adversarial Attacks, and Trojan Attacks. This guide is designed to be a straightforward manual, diving deep into the intertwining worlds of AI and cybersecurity within nuclear reactors, and tailoring insights for three crucial audiences: I&C Vendors/Developers, Nuclear Regulators, and Nuclear Reactor Operators and Cyber Defense Teams. (1) Section 2, directed at I&C Vendors/Developers, will provide a clear and focused look at several cybersecurity attacks, offering practical recommendations and detailed scenarios related to AI cybersecurity. This section isn’t just about identifying problems but also about giving solid, usable solutions. (2) Section 3, meant for Nuclear Regulators, gets straight to the point about regulations, policy suggestions, and guidelines that are needed to lay down a robust, secure, and ethical foundation for the application of AI in nuclear operations. The focus is on making sure that everything adheres to international standards and laws while being practicable and clear-cut. (3) Section 4, aimed at Nuclear Reactor Operators and Cyber Defense Teams, offers an exhaustive exploration and technical reports, with clear recommendations and scenario analyses vital to protect operational environments and guarantee the secure application of AI in nuclear reactor operations. The goal is simple: as we step into an era where AI becomes a fundamental element of our technological and energy infrastructures, this guide is here to act as a clear, direct handbook, ensuring that AI is implemented within the nuclear sector in a manner that is secure, responsible, and practical. It’s about striking a balance – optimizing the undeniable benefits offered by AI while securing and shielding against potential cyber threats as we move through this new and complex landscape.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

MultiLoad-GAN: A GAN-Based Synthetic Load Group Generation Method Considering Spatial-Temporal Correlations

This paper presents a deep-learning framework, Multi-load Generative Adversarial Network (MultiLoad-GAN), for generating a group of synthetic load profiles (SLPs) simultaneously. The main contribution of MultiLoad-GAN is the capture of spatial-temporal correlations among a group of loads that are served by the same distribution transformer. This enables the generation of a large amount of correlated SLPs required for microgrid and distribution system studies. Here, the novelty and uniqueness of the MultiLoad-GAN framework are three-fold. First, to the best of our knowledge, this is the first method for generating a group of load profiles bearing realistic spatial- temporal correlations simultaneously. Second, two complementary realisticness metrics for evaluating generated load profiles are developed: computing statistics based on domain knowledge and comparing high-level features via a deep-learning classifier. Third, to tackle data scarcity, a novel iterative data augmentation mechanism is developed to generate training samples for enhancing the training of both the classifier and the MultiLoad-GAN model. Simulation results show that MultiLoad- GAN can generate more realistic load profiles than existing approaches, especially in group level characteristics. With little finetuning, MultiLoad-GAN can be readily extended to generate a group of load or PV profiles for a feeder or a service area.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Cyber-physical Architecture For Automated Responses (cyphar) In Adversarial Ot Environments

The ability to react to a malicious attack starts with high fidelity recognition, and with that, an agile response to the attack. The current operational technology (OT) systems for a critical infrastructure may include an intrusion detection system (IDS), but the ability to adapt to an intrusion is a human initiated response. Orchestrators, which are coming of age in the financial sector and allow for levels of automated response, are not prevalent in the OT space. To evolve to such responses in the OT space, a tradeoff analysis is first needed, The tradeoff analysis, which evaluates the mitigation benefits of responses versus the physical affects that result, inform an automated response decision. The intent of this paper is to provide a formulation of a tradeoff analysis and its use in advancing automated, agile responses.

Rieger, CraigG.↗

Transactional Knowledge Graph Generation To Model Adversarial Activities

A Knowledge Graph (KG) is a formal and structured representation of facts, relationships, and semantic descriptions of a set of entities. Traditionally, KGs are used to describe metadata about entities and to provide additional context to target application results. Many real-world domains also involve temporal interactions between entities in addition to the metadata data. Modeling these attributed transactions is a critical requirement when using KGs in complex real-world applications. Modeling adversarial activities is one such application that develops methodology and tools to produce realistic large-scale background activity graphs that include embedded Weapons of Mass Destruction (WMD) activity patterns. We present a novel platform for constructing a transactional knowledge graph from a diverse set of sources. We present the core components and architecture of the framework, and a use case for generating a background knowledge graph and WMD activity template to evaluate network alignment and subgraph matching algorithms.

Purohit, Sumit↗

Cyber Friendly Fire

Cyber friendly fire (FF) is a new concept that has been brought to the attention of Department of Defense (DoD) stakeholders through two workshops that were planned and conducted by the Air Force Research Laboratory (AFRL) and research conducted for AFRL by the Pacific Northwest National Laboratory. With this previous work in mind, we offer a definition of cyber FF as intentional offensive or defensive cyber/electronic actions intended to protect cyber systems against enemy forces or to attack enemy cyber systems, which unintentionally harms the mission effectiveness of friendly or neutral forces. Just as with combat friendly fire, a fundamental need in avoiding cyber FF is to maintain situation awareness (SA). We suggest that cyber SA concerns knowledge of a system's topology (connectedness and relationships of the nodes in a system), and critical knowledge elements such as the characteristics and vulnerabilities of the components that comprise the system (and that populate the nodes), the nature of the activities or work performed, and the available defensive (and offensive) countermeasures that may be applied to thwart network attacks. A training implication is to raise awareness and understanding of these critical knowledge units; an approach to decision aids and/or visualizations is to focus on supporting these critical knowledge units. To study cyber FF, we developed an unclassified security test range comprising a combination of virtual and physical devices that present a closed network for testing, simulation, and evaluation. This network offers services found on a production network without the associated costs of a real production network. Containing enough detail to appear realistic, this virtual and physical environment can be customized to represent different configurations. For our purposes, the test range was configured to appear as an Internet-connected Managed Service Provider (MSP) offering specialized web applications to the general public. The network is essentially divided into a production component that hosts the web and network services, and a user component that hosts thirty employee workstations and other end devices. The organization's network is separated from the Internet by a Cisco ASA network security device that both firewalls and detects intrusions. Business sensitive information is stored in various servers. This includes data comprising thousands of internal documents, such as finance and technical designs, email messages for the organization's employees including the CEO, CFO, and CIO, the organization's source code, and Personally Identifiable client data. Release of any of this information to unauthorized parties would have a significant, detrimental impact on the organization's reputation, which would harm earnings. The valuable information stored in these servers pose obvious points of interest for an adversary. We constructed several scenarios around this environment to support studies in cyber SA and cyber FF that may be run in the test range. We describe mitigation strategies to combat cyber FF including both training concepts and suggestions for decision aids and visualization approaches. Finally, we discuss possible future research directions.

99 GENERAL AND MISCELLANEOUS↗

Anomaly Detection and Mitigation for Wide-Area Damping Control using Machine Learning

In an interconnected multi-area power system, wide-area measurement based damping controllers are used to damp out inter-area oscillations, which jeopardize grid stability and constrain the power flows below to their transmission capacity. The effect of wide-area damping control (WADC) significantly depends on both power and cyber systems. At the cyber system layer, an adversary can inflict the WADC process by compromising either measurement signals, control signals or both. Stealthy and coordinated cyber-attacks may bypass the conventional cybersecurity measures to disrupt the seamless operation of WADC. This paper proposes an anomaly detection (AD) algorithm using supervised Machine Learning and a model-based logic for mitigation. The proposed AD algorithm considers measurement signals (input of WADC) and control signals (output of WADC) as input to evaluate the type of activity such as normal, perturbation (small or large signal faults), attack and perturbation-and-attack. Upon anomaly detection, the mitigation module tunes the WADC signal and sets the control status mode as either wide-area mode or local mode. The proposed anomaly detection and mitigation (ADM) module works inline with the WADC at the control center for attack detection on both measurement and control signals and eliminates the need for ADMs at the geographically distributed actuators. Here, we consider coordinated and primitive data-integrity attack vectors such as pulse, ramp, relay-trip and replay attacks. The performance of the proposed ADM algorithms was evaluated under these attack vector scenarios on a testbed environment for 2-area 4-machine power system. The ADM module shows effective performance with 96:5% accuracy to detect anomalies.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

3D reconstruction and neural rendering for adversarial machine learning

While evasion attacks on computer vision systems have been widely studied, creating attacks that remain effective under significant changes in viewpoint continues to be challenging. Traditional approaches often rely on affine transformations of images, but these approaches degrade at larger perspective shifts and often produce unrealistic or ineffective perturbations. Recent methods use differentiable renderers to improve viewpoint robustness, but they typically depend on manually constructed 3D models. We introduce a semi-automated pipeline that generates physically printable and perspective-invariant adversarial patches using only a small set of 2D images. Our method integrates 3D reconstruction, neural rendering, adversarial patch optimization, and an object detection victim model into a unified workflow. We use 2D Gaussian Splatting for high fidelity mesh reconstruction and FlexPara for surface parameterization that produces texture maps suitable for patch editing. Together, these components form a fully differentiable pipeline in PyTorch3D that links texture modification to model outputs, enabling efficient optimization of patches that remain effective across many viewpoints. The complete process, from image capture to patch printing and physical evaluation, can be completed within a few hours. We demonstrate the effectiveness of the resulting patches through attacks on the YOLOv8 object detection model and discuss remaining challenges and opportunities for improving robustness and scalability.

Singhvi, Vivaan [ORNL] (ORCID:0009000586288221)↗