Search NASA⌕ Search

SEARCH · Search NASA

Results for “adversarial evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

SoK: What does it Mean to Benchmark Database Forensics?

Relational Database Management Systems are the backbone of modern enterprises and public-sector services, and are thus frequent targets of security incidents, insider threats, and thorough regulatory audits. Consequently, databases have become key sources of digital evidence, requiring investigators to reconstruct past activity from audit logs, transaction logs, and backups. Although benchmarking frameworks such as those developed by the Transaction Processing Performance Council (TPC) are widely used to evaluate database performance, they do not capture forensic requirements such as evidentiary completeness, tamper-evidence, chain of custody, or regulatory compliance under GDPR and CCPA. This survey examines the emerging domain of forensic database benchmarking. We gathered prior research on database forensics, secure logging, and tamper-evident data structures; we analyze modern forensic-ready features in commercial and open-source systems (SQL Server Ledger, Oracle Blockchain Tables, PostgreSQL pgAudit, Db2 Audit, Aurora Database Activity Streams, Oracle Real Application Security and IBM Guardium) and assess why existing benchmarks are insufficient. We propose forensic workloads, metrics, and methodologies that incorporate adversarial stressors, deleted-record recovery, and backup analysis. We also identify open research problems and call for a community-driven forensic benchmark suite. The result is an idea for evaluating not only database performance but also forensic soundness, bridging the gap between system engineering, compliance, and digital investigations.

Lenard, Ben↗

Regulatory Models and the Environment: Practice, Pitfalls, and Prospects

Computational models support environmental regulatory activities by providing the regulator an ability to evaluate available knowledge, assess alternative regulations, and provide a framework to assess compliance. But all models face inherent uncertainties, because human and natural systems are always more complex and heterogeneous than can be captured in a model. Here we provide a summary discussion of the activities, findings, and recommendations of the National Research Council's Committee on Regulatory Environmental Models, a committee funded by the US Environmental Protection Agency to provide guidance on the use of computational models in the regulatory process. Modeling is a difficult enterprise even outside of the potentially adversarial regulatory environment. The demands grow when the regulatory requirements for accountability, transparency, public accessibility, and technical rigor are added to the challenges. Moreover, models cannot be validated (declared true) but instead should be evaluated with regard to their suitability as tools to address a specific question. The committee concluded that these characteristics make evaluation of a regulatory model more complex than simply comparing measurement data with model results. Evaluation also must balance the need for a model to be accurate with the need for a model to be reproducible, transparent, and useful for the regulatory decision at hand. Meeting these needs requires model evaluation to be applied over the"life cycle" of a regulatory model with an approach that includes different forms of peer review, uncertainty analysis, and extrapolation methods than for non-regulatory models.

54 ENVIRONMENTAL SCIENCES↗

Within a Stone's Throw: Proximal Geolocation of Internet Users via Covert Wireless Signaling

While Internet users may often believe they have anonymity online, a culmination of technologies and recent research may allow an adversary to precisely locate an online user s geophysical location. In many cases, such as peer-to-peer applications, an adversary can easily use a target s IP address to quickly obtain the general geographical location of the target. Recent research has scoped this general area to a 690m (0.43 mile) radius circle. In this work, we show how an adversary can exploit Internet communication for geophysical location by embedding covert signals in communication with a target on a remote wireless local area network. We evaluated the approach in two common real-world settings: a residential neighborhood and an apartment building. In the neighborhood case, we used a single-blind trial in which an observer located a target network to within three houses in less than 40 minutes. Directional antennas may have allowed even more precise geolocation. This approach had only a 0.38% false positive rate, despite 24,000 observed unrelated packets and many unrelated networks. This low rate allowed the observer to exclude false locations and continue searching for the target. Our results enable law enforcement or copyright holders to quickly locate online Internet users without requiring time-consuming subpoenas to Internet Service Providers. Other privacy use cases include rapidly locating individuals based on their online speech or interests. We hope to raise awareness of these issues and to spur discussion on privacy and geolocating techniques.

geolocation↗

ARM-IRL: Adaptive Resilience Metric Quantification Using Inverse Reinforcement Learning

The resilience of safety-critical systems is gaining importance due to the rise in cyber and physical threats, especially within critical infrastructure. Traditional static resilience metrics may not capture dynamic system states, leading to inaccurate assessments and ineffective responses to cyber threats. This work aims to develop a data-driven, adaptive method for resilience metric learning. We propose a data-driven approach using inverse reinforcement learning (IRL) to learn a single, adaptive resilience metric. The method infers a reward function from expert control actions. Unlike previous approaches using static weights or fuzzy logic, this work applies adversarial inverse reinforcement learning (AIRL), training a generator and discriminator in parallel to learn the reward structure and derive an optimal policy. The proposed approach is evaluated on multiple scenarios: optimal communication network rerouting, power distribution network reconfiguration, and cyber–physical restoration of critical loads using the IEEE 123-bus system. The adaptive, learned resilience metric enables faster critical load restoration in comparison to conventional RL approaches.

97 MATHEMATICS AND COMPUTING↗

Evaluating Direct and Indirect Influence on EV Charging Stations Across the US

The adoption of new technology for electric vehicles (EV) and mobility applications can bring underappreciated vulnerabilities to the power grid. One area of potential fraud and adversarial influence is through the business ecosystem of startups that own and deploy EV technology. Yet, there are no models or analyses that map the network of organizations and people that have direct and indirect influence over technologies currently deployed in the grid. To fill this gap, we develop a multilayer network model to measure direct and indirect influence on EV charging stations. First, we create and adversarial socio-technical network (ASTN) model via a data fusion pipeline for different US regions of interest (ROI). Then, we develop an integrated ASTN for Chicago, Los Angeles, New York, and Philadelphia. We rank EV charging companies direct influence within each geographic region as well as indirect influence via social network analysis. While some companies have strong direct and indirect influence (i.e., ChargePoint) others show a mismatch between their influence over charging stations and their position within the social network. For example, Tesla has strong direct influence on stations and weak indirect influence over competitors. In contrast, 7Charge has weak direct influence over stations, but strong indirect influence over competitors.

29 - ENERGY PLANNING, POLICY AND ECONOMY↗

Understanding Impacts of Data Integrity Attacks on Transactive Control Systems

With the rapid growth of internet-connected smart devices capable of exchanging energy price information and adaptively controlling energy consumption of connected loads, the role of transactive control is expected to become more prominent in the modern grid. Transactive control systems integrate the wholesale and retail energy markets, and enable active participation of end users, thereby playing a key role in managing the rising number of distributed assets in the smart grid. The use of internet for the communication of data between the controllers at the building, distribution, and transmission levels makes the system susceptible to cyber-attacks. A skilled adversary can potentially manipulate the exchanged data with the intention of inflicting damage to the system. In this paper, four categories of metrics are put forth for evaluating the performance of the system when under data integrity attacks. A model for scaling-based data integrity attack has been described, and co-simulations have been performed on a 240 bus WECC transmission system model with detailed modeling of distribution systems at specific buses. Our results show that scaling-based data integrity attacks can have non-trivial impacts on the operational, financial, and comfort aspects of the transactive control system.

Smart grid, transactive control system, Buildings-↗

Synthetic Scientific Image Generation with VAE, GAN, and Diffusion Model Architectures

Generative AI (genAI) has emerged as a powerful tool for synthesizing diverse and complex image data, offering new possibilities for scientific imaging applications. This review presents a comprehensive comparative analysis of leading generative architectures, ranging from Variational Autoencoders (VAEs) to Generative Adversarial Networks (GANs) on through to Diffusion Models, in the context of scientific image synthesis. We examine each model's foundational principles, recent architectural advancements, and practical trade-offs. Our evaluation, conducted on domain-specific datasets including microCT scans of rocks and composite fibers, as well as high-resolution images of plant roots, integrates both quantitative metrics (SSIM, LPIPS, FID, CLIPScore) and expert-driven qualitative assessments. Results show that GANs, particularly StyleGAN, produce images with high perceptual quality and structural coherence. Diffusion-based models for inpainting and image variation, such as DALL-E 2, delivered high realism and semantic alignment but generally struggled in balancing visual fidelity with scientific accuracy. Importantly, our findings reveal limitations of standard quantitative metrics in capturing scientific relevance, underscoring the need for domain-expert validation. We conclude by discussing key challenges such as model interpretability, computational cost, and verification protocols, and discuss future directions where generative AI can drive innovation in data augmentation, simulation, and hypothesis generation in scientific research.

Generative Adversarial Networks↗

Device Feasibility Analysis of Multi-level FeFETs for Neuromorphic Computing

As an emerging non-volatile memory device technology, Ferroelectric Field-Effect Transistors (FeFETs) can enable low-power, adaptive intelligent system design. However, device dimension and operating voltage dependent reliability issues of scaled FeFETs can ultimately lead to degraded performance in solving machine learning tasks. In this article, detailed experimental characterization of FeFET devices of different dimensions have been carried out to explicitly evaluate the non-ideal behavior in device conductance programming properties like number of programming states, cycle-to-cycle (C2C) variations, device-to-device (D2D) variations, and state retention. A hardware-aware software simulation approach has been adopted to capture the adversarial effects of the non-idealities on recognition accuracy through algorithm-level performance assessment by including them in NeuroSim, a popular neural network hardware simulator, to execute a neural network model considering all other hardware constraints. With the added non-idealities, significant accuracy degradation has been observed compared to the ideal scenarios where D2D variations play the most critical role. Thereafter, feasibility of a variation-aware training method has been evaluated to tackle the accuracy drop.

42 ENGINEERING↗

ESM data downscaling: a comparison of super-resolution deep learning models

Abstract Climate projections at fine spatial resolutions are required to conduct accurate risk assessment for critical infrastructure and design adaptation planning. Generating these projections using advanced Earth system models (ESM) requires significant computational resources. To address this issue, various statistical downscaling techniques have been introduced to generate fine-resolution data from coarse-resolution simulations. In this study, we evaluate and compare five deep learning-based downscaling techniques, namely, super-resolution convolutional neural networks, fast super-resolution convolutional neural network ESM, efficient sub-pixel convolutional neural network, enhanced deep residual network (EDRN), and super-resolution generative adversarial network (SRGAN). These techniques are applied to a dataset generated by the Energy Exascale Earth System Model (E3SM), focusing on key surface variables such as surface temperature, shortwave heat flux, and longwave heat flux. Models are trained and validated using paired fine-resolution (0.25 $$^{\circ }$$ ∘ ) and coarse-resolution (1 $$^{\circ }$$ ∘ ) monthly data obtained from a 9-year simulation. Next, blind testing is performed using monthly data obtained from two different years outside of the training and validation set. To evaluate the efficiency of each technique, different statistical metrics are used, including mean squared error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS). The results show that EDRN outperforms other algorithms in terms of PSNR, SSIM, and MSE, but struggles to capture fine-scale features in the data. In contrast, SRGAN, a generative model that uses perceptual loss, excels in capturing fine details at boundaries and internal structures, resulting in lower LPIPS than other methods.

Pawar, Nikhil M. (ORCID:0000000211613289)↗

Video performance for high security applications.

The complexity of physical protection systems has increased to address modern threats to national security and emerging commercial technologies. A key element of modern physical protection systems is the data presented to the human operator used for rapid determination of the cause of an alarm, whether false (e.g., caused by an animal, debris, etc.) or real (e.g., a human adversary). Alarm assessment, the human validation of a sensor alarm, primarily relies on imaging technologies and video systems. Developing measures of effectiveness (MOE) that drive the design or evaluation of a video system or technology becomes a challenge, given the subjectivity of the application (e.g., alarm assessment). Sandia National Laboratories has conducted empirical analysis using field test data and mathematical models such as binomial distribution and Johnson target transfer functions to develop MOEs for video system technologies. Depending on the technology, the task of the security operator and the distance to the target, the Probability of Assessment (PAs) can be determined as a function of a variety of conditions or assumptions. PAs used as an MOE allows the systems engineer to conduct trade studies, make informed design decisions, or evaluate new higher-risk technologies. This paper outlines general video system design trade-offs, discusses ways video can be used to increase system performance and lists MOEs for video systems used in subjective applications such as alarm assessment.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Advanced Reactor Designs Security Analysis, Risk, and Recommendations: Risks, Consequences, and Possible by-Design Mitigation Approaches Associated with Select Advanced Reactors

Next-generation advanced reactors (ARs) incorporate enhanced safety systems, have smaller source terms, and feature compact modular designs, which should lessen their collective risk profiles. However, to fully evaluate risk, security needs to be a part of the equation. Without taking security into consideration, safety systems and components in the new ARs may be vulnerable to sabotage. These base attributes, coupled with enhanced security features specific to AR design through sound engineering and security-by-design (SeBD), should provide developers and operators with lower inherent security risk profiles. Building security early into the AR design may remove or passively secure potential critical targets from an adversary’s reach , thereby increasing overall safety and security. An integrated approach and diverse design team that includes engineering, operations, and security experts are fundamental to building security into the design without sacrificing fundamental operational efficiencies and principles. The objective of this project was to evaluate the security and safety interfaces for five classes of reactors, identify potential security vulnerabilities of structures, systems, and components (SSC), and underscore the need to consider security alongside safety in the design o f these concepts. The five reactor classes evaluated in this project and presented in this report are molten-salt reactors (MSR), high temperature gas reactors (HTGR), sodium-fast reactors (SFR), advanced light-water reactors (ALWR), and microreactors. These designs were selected because they reflect the concepts that are closest to market deployment and have received significant resource investments from the public and private sector. This project assesses the inherent security risks posed by common classes of ARs, provides a methodology and framework to assess security along with safety, and offers an analysis of potential mitigation strategies that could be incorporated. For each AR technology, the SSCs that relate to radionuclide source safety functions are discussed to understand the SSC contribution to safety and relative importance in the protective strategy for the design. The assumptions that went into evaluating each reactor concept originated from generic publicly available nonproprietary information and should not directly be used to qualify an absolute risk profile nor to rank specific AR designs. Instead, the purpose of the analysis is to understand and compare the generic inherent security risks of different AR technologies.

98 - NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL↗

XSub: Explanation-Driven Adversarial Attack against Blackbox Classifiers via Feature Substitution

Despite its significant benefits in enhancing the transparency and trustworthiness of artificial intelligence (AI) systems, explainable AI (XAI) can unintentionally provide adversaries with insights into blackbox models, increasing their vulnerability to various attacks. In this paper, we develop a novel explanation-driven adversarial attack against blackbox classifiers based on feature substitution, called XSub. The key idea of XSub is to strategically replace important features (identified via XAI) in the original sample with corresponding important features of a different label, thereby increasing the likelihood of the model misclassifying the perturbed sample. XSub only requires a minimal number of queries and can be easily extended to launch backdoor attacks in case the attacker has access to the model's training data. Our evaluation shows that XSub is not only effective and stealthy but also low-cost, showcasing its feasibility across a wide range of AI applications.

adversarial attack↗

SURVEILLANCE DETECTION FOR TRANSPORTATION OPERATIONS PERSONNEL TO PREVENT HIJACKING, THEFT, SABOTAGE, AND MALICIOUS SECURITY EVENTS DURING TRANSPORTING NUCLEAR MATERIAL

The secure transportation of high-consequence materials, including nuclear and radiological assets, is a critical global priority in the face of escalating terrorism, security threats, and violent protests targeting these operations. Effective surveillance detection—the ability to identify, assess, and respond to potential threats across a continuum of scenarios—is paramount in addressing these challenges. This paper outlines a phased, multi-tiered training program designed to strengthen the surveillance detection capabilities of organizations responsible for nuclear material transport. The proposed training program adopts a progressive approach, gradually increasing in technical complexity to provide participants with comprehensive knowledge and tools for implementing robust security strategies. It targets a wide spectrum of stakeholders, including competent authorities, regulators, inspectors, shippers, carriers, law enforcement, and emergency response personnel, equipping them to plan, evaluate, and safeguard nuclear material transportation effectively. Each phase of the program emphasizes distinct elements of the surveillance detection continuum and transport security, focusing on critical topics such as threat identification, adversary task timelines, protective methodologies, and attack mitigation strategies. The training framework is anchored in technical exchanges and scenario-driven courses that reflect real-world complexities and challenges. By addressing the surveillance detection continuum comprehensively—from early threat assessment to active countermeasures—the program reinforces global efforts to secure nuclear assets. It aligns with international security objectives and fosters a strong security culture within participating organizations, ensuring personnel are prepared to counter potential threats and maintain the safe, secure movement of these materials. Ultimately, this initiative aims to enhance preparedness, security, and response capabilities, supporting the global mission to safeguard high-consequence materials against evolving threats.

Zineddin, Dr. Z. [ORNL] (ORCID:0009000848740725)↗

Using GANs to predict milling stability from limited data

Milling is a key manufacturing process that requires the selection of operating parameters that provide efficient performance. However, the presence of chatter, a self-excited vibration causing poor surface finish and potential damage to the machine and cutting tool, makes it challenging to select the appropriate parameters. To predict chatter, stability maps are commonly used, but their generation requires expensive data, making it difficult to employ these maps in industry. Therefore, there is a pressing need for an approach that can accurately predict stability maps using limited experimental data. This study introduces the new Encoder GAN (EGAN) approach based on Generative Adversarial Networks (GANs) that predicts stability maps using limited experimental data. The approach consists of the encoder, generator, and discriminator subnetworks and uses the trained encoder and generator to predict the target stability map. This versatile method can be applied to various tool setups and can accurately predict stability maps with limited experimental data (five to 10 cutting tests) even when there is little information available for unknown parameters. In conclusion, the study evaluates the proposed approach using both numerical data and experiments and demonstrates its superior performance compared to state-of-the-art benchmarks.

42 ENGINEERING↗

Evaluating generative networks using Gaussian mixtures of image features

We develop a measure for evaluating the performance of generative networks given two sets of images. A popular performance measure currently used to do this is the Fréchet Inception Distance (FID). However, FID assumes that images featurized using the penultimate layer of Inception follow a Gaussian distribution. This assumption allows FID to be easily computed, since FID uses the 2-Wasserstein distance of two Gaussian distributions fitted to the featurized images. However, we show that Inception features of the ImageNet dataset are not Gaussian; in particular, each marginal is not Gaussian. To remedy this problem, we model the featurized images using Gaussian mixture models (GMMs) and compute the 2-Wasserstein distance restricted to GMMs. We define a performance measure, which we call WaM, on two sets of images by using inception (or another classifier) to featurize the images, estimate two GMMs, and use the restricted 2-Wasserstein distance to compare the GMMs. We experimentally show the advantages of WaM over FID, including how FID is more sensitive than WaM to image perturbations. By modelling the non-Gaussian features obtained from inception as GMMs and using a GMM metric, we can more accurately evaluate generative network performance.

machine learning, genrative adversarial networks↗

Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural Gradient Descent

Despite the great achievements of the modern deep neural networks (DNNs), the vulnerability/robustness of state-of-the-art DNNs raises security concerns in many application domains requiring high reliability. Various adversarial attacks are proposed to sabotage the learning performance of DNN models. Among those, the black-box adversarial attack methods have received special attentions owing to their practicality and simplicity. Black-box attacks usually prefer less queries in order to maintain stealthy and low costs. However, most of the current black-box attack methods adopt the first-order gradient descent method, which may come with certain deficiencies such as relatively slow convergence and high sensitivity to hyper-parameter settings. In this paper, we propose a zeroth-order natural gradient descent (ZO-NGD) method to design the adversarial attacks, which incorporates the zeroth-order gradient estimation technique catering to the black-box attack scenario and the second-order natural gradient descent to achieve higher query efficiency. The empirical evaluations on image classification datasets demonstrate that ZO-NGD can obtain significantly lower model query complexities compared with state-of-the-art attack methods.

Zhao, Pu↗

DTS: Building custom, intelligent schedulers

DTS is a decision-theoretic scheduler, built on top of a flexible toolkit -- this paper focuses on how the toolkit might be reused in future NASA mission schedulers. The toolkit includes a user-customizable scheduling interface, and a 'Just-For-You' optimization engine. The customizable interface is built on two metaphors: objects and dynamic graphs. Objects help to structure problem specifications and related data, while dynamic graphs simplify the specification of graphical schedule editors (such as Gantt charts). The interface can be used with any 'back-end' scheduler, through dynamically-loaded code, interprocess communication, or a shared database. The 'Just-For-You' optimization engine includes user-specific utility functions, automatically compiled heuristic evaluations, and a postprocessing facility for enforcing scheduling policies. The optimization engine is based on BPS, the Bayesian Problem-Solver (1,2), which introduced a similar approach to solving single-agent and adversarial graph search problems.

Hansson, Othar↗

Geometrical-Based Generative Adversarial Network to Enhance Digital Rock Image Quality

X-ray microcomputed tomography (micro-CT) is a common tool for the study of porous media structures and properties. High-quality micro-CT data are required to accurately capture pore structures. Acquiring high-quality micro-CT data, however, is not always possible, owing to application limitations and experimental constraints. Therefore, we propose a geometrical-based generative adversarial network (GAN) to rapidly restore noisy micro-CT images to their clean counterparts. The training data and related ground-truth (GT) data are scanned for 7 min and 9.5 h, respectively. To evaluate the performance of the geometrical-based GAN, a 6003 voxel image that has never been used for training is reconstructed and compared with the corresponding GT image. Histogram matching and linear normalization are implemented to adjust the histogram of the reconstructed image to that of the GT image. A watershed-based segmentation method is then applied to delineate pore and solid phases. Lastly, we measure the Minkowski functionals and petrophysical properties, including absolute permeability, pore size distribution, drainage capillary pressure-saturation curve, and imbibition relative permeability, to estimate the physical accuracy of the denoised image. The results show that the proposed geometrical-based GAN can accurately restore noisy micro-CT data. By reducing the scanning time from 9.5 h to 7 min, the expenditure of collecting micro-CT can be decreased significantly. This is particularly important for applications where time-lapse images of a dynamic process are required, high-throughput imaging is necessary for real-time data analysis, or where the quantification of large sample volumes is required.

58 GEOSCIENCES↗