Search NASA⌕ Search

SEARCH · Search NASA

Results for “synthetic data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Identifying microbial drivers in biological phenotypes with a Bayesian network regression model

Abstract In Bayesian Network Regression models, networks are considered the predictors of continuous responses. These models have been successfully used in brain research to identify regions in the brain that are associated with specific human traits, yet their potential to elucidate microbial drivers in biological phenotypes for microbiome research remains unknown. In particular, microbial networks are challenging due to their high dimension and high sparsity compared to brain networks. Furthermore, unlike in brain connectome research, in microbiome research, it is usually expected that the presence of microbes has an effect on the response (main effects), not just the interactions. Here, we develop the first thorough investigation of whether Bayesian Network Regression models are suitable for microbial datasets on a variety of synthetic and real data under diverse biological scenarios. We test whether the Bayesian Network Regression model that accounts only for interaction effects (edges in the network) is able to identify key drivers (microbes) in phenotypic variability. We show that this model is indeed able to identify influential nodes and edges in the microbial networks that drive changes in the phenotype for most biological settings, but we also identify scenarios where this method performs poorly which allows us to provide practical advice for domain scientists aiming to apply these tools to their datasets. BNR models provide a framework for microbiome researchers to identify connections between microbes and measured phenotypes. We allow the use of this statistical model by providing an easy‐to‐use implementation which is publicly available Julia package at https://github.com/solislemuslab/BayesianNetworkRegression.jl .

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Validation of the ERO2.0 code using W7-X and JET experiments and predictions for ITER operation

Abstract The paper provides an overview of recent modelling of global material erosion and deposition in the fusion devices Wendelstein 7-X (W7-X), JET and ITER using the Monte-Carlo code ERO2.0. For validating the modelling tool in a three-dimensional environment, W7-X simulations are performed to describe carbon erosion from the graphite test divertor units, which were equipped in operational phase OP 1.2 and analysed post-mortem. Synthetic spectroscopy of carbon line emission is compared with experimental results from the divertor spectrometer measurement system, showing a good agreement in the e-folding lengths in the radial intensity profiles of carbon. In the case of metallic wall materials, earlier modelling of the Be/W environment in JET and ITER is revisited and extended with an updated set of sputtering and reflection data, as well as including the mixing model for describing the Be/W dynamics in the divertor. Motivated by recent H/D/T isotope experiments in JET, limited and diverted configuration pulses are modelled, showing the expected trend of both Be and W erosion increasing with isotope mass. For the JET diverted configuration pulses, it is shown that Be migrates predominantly to the upper part of the inner divertor where it initially leads to strong W erosion. With longer exposure time, the growth of a Be deposited layer leads to a reduction of W erosion in that region. A similar trend is observed in simulations of the ITER baseline Q = 10 scenario, however with a more symmetric Be migration pattern leading to deposition also on the outer divertor.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Multifrontal Non-negative Matrix Factorization

Non-negative matrix factorization (Nmf) is an important tool in high-performance large scale data analytics with applications ranging from community detection, recommender system, feature detection and linear and non-linear unmixing. While traditional Nmf works well when the data set is relatively dense, however, it may not extract sufficient structure when the data is extremely sparse. Specifically, traditional Nmf fails to exploit the structured sparsity of the large and sparse data sets resulting in dense factors. We propose a new algorithm for performing Nmf on sparse data that we call multifrontal Nmf (Mf-Nmf) since it borrows several ideas from the multifrontal method for unconstrained factorization (e.g. LU and QR). We also present an efficient shared memory parallel implementation of Mf-Nmf and discuss its performance and scalability. We conduct several experiments on synthetic and realworld datasets and demonstrate the usefulness of the algorithm by comparing it against standard baselines. We obtain a speedup of 1.2x to 19.5x on 24 cores with an average speed up of 10.3x across all the real world datasets.

Sao, Piyush↗
DOE OSTI2022

Synthetic data generation for machine learning model training for energy theft scenarios using cosimulation

Abstract Technical and non‐technical losses in distribution circuits result in significant economic costs to power utilities. One type of non‐technical loss is energy theft by various means including illegal tapping of feeders, bypassing the meter, and billing fraud. These losses are usually hard to detect, and can remain undetected for long periods of time. Machine learning models have been proven effective in detecting these conditions, but rely on the availability of large, good‐quality training data sets. The problem is exacerbated by the imbalanced nature of data related to these conditions—energy theft, though costly, is very rare. The available data sets generally have very few samples of theft with most of the data pertaining to normal operation. Such data sets are generally not suitable to train machine learning models. In this paper, an overview of energy theft detection techniques, the challenges with their data needs, and the limitations of current techniques to bridge such data limitations is presented. A co‐simulation framework is proposed to generate reliable training data for machine learning algorithms for theft detection. An example scenario is presented and a machine learning model is built to detect certain kinds of energy theft.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

A Sparse Tensor Benchmark Suite for CPUs and GPUs

Tensor computations present significant performance chal- lenges that impact a wide spectrum of applications ranging from machine learning, healthcare analytics, social network analysis, data mining to quantum chemistry and signal processing. Efforts to improve the perfor- mance of tensor computations include exploring data layout, execution scheduling, and parallelism in common tensor kernels. This work presents a benchmark suite for arbitrary-order sparse tensor kernels using state- of-the-art tensor formats: coordinate (COO) and hierarchical coordinate (HiCOO) on CPUs and GPUs. It presents a set of reference tensor kernel implementations that are compatible with real-world tensors and power law tensors extended from synthetic graph generation techniques. We also propose Roofline performance models for these kernels to provide insights of computer platforms from sparse tensor view. This benchmark suite along with the synthetic tensor generator is publicly available.

Li, Jiajia↗
DOE OSTI2022

Joint physics-based and data-driven time-lapse seismic inversion: Mitigating data scarcity

In carbon capture and sequestration (CCS), developing rapid and effective imaging techniques is crucial for real-time monitoring of the spatial and temporal dynamics of CO 2 propagation during/after injection. With continuing improvements in computational power and data storage, data-driven techniques based on machine learning (ML) have been effectively applied to seismic inverse problems. In particular, ML helps alleviate the ill-posedness and high computational cost of full-waveform inversion (FWI). However, such data-driven inversion techniques require massive high-quality training data sets to ensure prediction accuracy, which hinders their application to time-lapse monitoring of CO 2 sequestration. We propose an efficient “hybrid” time-lapse workflow that combines physics-based FWI and data-driven ML inversion. The scarcity of the available training data is addressed by developing a new data-generation technique with physics constraints. The method is vali dated on a synthetic CO 2 -sequestration model based on the Kimberlina storage reservoir in California. The proposed approach is shown to synthesize a large volume of high-quality, physically realistic training data, which is critically important in accurately characterizing the CO 2 movement in the reservoir. In conclusion, the developed hybrid methodology can also simultaneously predict the variations in velocity and saturation and achieve high spatial resolution in the presence of realistic noise in the data.

58 GEOSCIENCES↗
DOE OSTI2021

Crystallography companion agent for high-throughput materials discovery

The discovery of new structural and functional materials is driven by phase identification, often using X-ray diffraction (XRD). Automation has accelerated the rate of XRD measurements, greatly outpacing XRD analysis techniques that remain manual, time-consuming, error-prone and impossible to scale. With the advent of autonomous robotic scientists or self-driving laboratories, contemporary techniques prohibit the integration of XRD. Here, we describe a computer program for the autonomous characterization of XRD data, driven by artificial intelligence (AI), for the discovery of new materials. Starting from structural databases, we train an ensemble model using a physically accurate synthetic dataset, which outputs probabilistic classifications—rather than absolutes—to overcome the overconfidence in traditional neural networks. This AI agent behaves as a companion to the researcher, improving accuracy and offering substantial time savings. It is demonstrated on a diverse set of organic and inorganic materials characterization challenges. This method is directly applicable to inverse design approaches and robotic discovery systems, and can be immediately considered for other forms of characterization such as spectroscopy and the pair distribution function.

36 MATERIALS SCIENCE↗
DOE OSTI2023

EV charging site day-ahead load prediction in a synthetic environment for RL based grid-informed charging

Ensuring grid health in the face of increasing demand for power is an emerging challenge especially due to transportation electrification. A free market approach to influencing electric vehicle (EV) load through grid-informed hourly dynamic pricing is introduced in this work. The setting of charging price is done by a reinforcement learning (RL) agent that learns the complicated dynamics by interacting with a synthetic environment. This synthetic environment is a combination of distribution feeder simulation, EV charger user behavior dynamics, and EV charging simulation. A key module in this synthetic environment involves obtaining the day-ahead charging profile of EV charging stations based on real-world past data. The day-ahead prediction is also useful in other traditional optimizations related to EV charge scheduling. The proposed approach involves using EV charging data from two different past time horizons – one to determine the shape of the daily profile and the other to determine a scaling value to capture actual energy consumption. Real-world charging data over many years from the ACN charging network has been used to demonstrate the ability to predict the day-ahead profile with only charge session data. Both Python and MATLAB have been used for data cleaning, processing, analysis, and prediction.

Suryanarayana, Harish↗
DOE OSTI2021

bnl/pub-Maffettone_2020_08

The application of the XCA package as first demonstrated in aXiv:2008.00283. ABSTRACT: The discovery of new structural and functional materials is driven by phase identification, often using X-ray diffraction (XRD). Automation has accelerated the rate of XRD measurements, greatly outpacing XRD analysis techniques that remain manual, time consuming, error prone, and impossible to scale. With the advent of autonomous robotic scientists or self-driving labs, contemporary techniques prohibit the integration of XRD. Here, we describe a computer program for the autonomous characterization of XRD data, driven by artificial intelligence (AI), for the discovery of new materials. Starting from structural databases, we train an ensemble model using a physically accurate synthetic dataset, which output probabilistic classifications --- rather than absolutes --- to overcome the overconfidence in traditional neural networks. This AI agent behaves as a companion to the researcher, improving accuracy and offering unprecedented time savings, and is demonstrated on a diverse set of organic and inorganic materials challenges. This innovation is directly applicable to inverse design approaches, robotic discovery systems, and can be immediately considered for other forms of characterization such as spectroscopy and the pair distribution function.

Maffettone, PhillipM [Brookhaven National Lab. (BN↗
DOE OSTI2021

Hyperparameter Setting for a Marked Multidimensional Hawkes Process with Dissimilar Decays

We provide further details for using Lim, et al.'s marked multidimensional Hawkes processes with dissimilar decays. We first describe what makes these different from other Hawkes processes, then describe each model hyperparameter and how to initialize it informed by the input data and any prior biases. We derived tighter bounds than Lim, et al. for faster convergence of rejection sampling. The resulting hyperparameters and bounds have been helpful against both synthetic and real-world datasets.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

AlloSHP: deconvoluting single homeologous polymorphism for phylogenetic analysis of allopolyploids

Background The genomic and evolutionary study of allopolyploid organisms involves multiple copies of homeologous chromosomes, making their assembly, annotation, and phylogenetic analysis challenging. Bioinformatics tools and protocols have been developed to study polyploid genomes, but sometimes require the assembly of their genomes, or at least the genes, limiting their use. Results We have developed AlloSHP, a command-line tool for detecting and extracting single homeologous polymorphisms (SHPs) from the subgenomes of allopolyploid species. This tool integrates three main algorithms, WGA, VCF2ALIGNMENT and VCF2SYNTENY, and allows the detection of SHPs for the study of diploid-polyploid complexes with available diploid progenitor genomes, without assembling and annotating the genomes of the allopolyploids under study. AlloSHP has been validated on three diploid-polyploid plant complexes, Brachypodium, Brassica, and Triticum-Aegilops, and a set of synthetic hybrid yeasts and their progenitors of the genus Saccharomyces. The results and congruent phylogenies obtained from the four datasets demonstrate the potential of AlloSHP for the evolutionary analysis of allopolyploids with a wide range of ploidy and genome sizes. Conclusions AlloSHP combines the strategies of simultaneous mapping against multiple reference genomes and syntenic alignment of these genomes to call SHPs, using as input data a single VCF file and the reference genomes of the known or closest extant diploid progenitor species. This novel approach provides a valuable tool for the evolutionary study of allopolyploid species, both at the interspecific and intraspecific levels, allowing the simultaneous analysis of a large number of accessions and avoiding the complex process of assembling polyploid genomes.

Allopolyploids↗
DOE OSTI2026

Microbial spies and bloggers: programming cells to convert environmental information into discernible signals

Microbes regulate their dynamic behaviors using the chemical and physical characteristics of their environment. The ability of microbes to continuously convert this physicochemical information into biochemical information and to use organic matter in the environment as a power source makes these organisms attractive as chassis for building sensors. However, most biosensors have severe limitations when considering applications in hard-to-image settings like soils, sediments, and wastewater. Emerging technologies at the interface of biomolecular design, microbiome engineering, and synthetic biology offer new tools to program cells and communities as biosensors for these settings. Here, in this review, we describe innovations in biosensor outputs that are enabling new applications in complex environments, including reporters that are read out using electrochemical, gas chromatography, hyperspectral imaging, and next-generation sequencing methods. We also discuss computational advances that are accelerating the diversification of sensing components by mining metagenomics data for new transcriptional regulators and by designing allosteric protein switches that directly regulate reporter outputs using analytes. We highlight emerging opportunities for programming undomesticated microbes in communities to function as distributed sensors in the environment. Finally, we discuss the need for responsible biosensor development and to modernize regulatory frameworks to support evidence-based assessment of environmental biosensors.

analyte↗
DOE OSTI2022

Intracardiac Electrical Imaging using the 12-lead ECG: A Machine Learning Approach using Synthetic Data

Current state-of-the-art techniques for non-invasive imaging of cardiac electrical phenomena require voltage recordings from dozens of different torso locations and anatomical models built from expensive medical diagnostic imaging procedures. Here this study aimed to assess if recent machine learning advances could alternatively reconstruct electroanatomical maps at clinically relevant resolutions using only the standard 12-lead electrocardiogram (ECG) as input. To that end, a computational study was conducted to generate a dataset of over 16000 detailed cardiac simulations, which was then used to train neural network (NN) architectures designed to exploit both spatial and temporal correlations in the ECG signal. Analysis over a validation set showed average errors in activation map reconstruction below 1.7 msec over 75 intracardiac locations. Furthermore, phenotypical patterns of activation and the morphology of the activation potential were correctly reconstructed. The approach offers opportunities to stratify patients non-invasively, both retrospectively and prospectively, using metrics otherwise only available through invasive clinical procedures.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

A robust estimator of mutual information for deep learning interpretability

Abstract We develop the use of mutual information (MI), a well-established metric in information theory, to interpret the inner workings of deep learning (DL) models. To accurately estimate MI from a finite number of samples, we present GMM-MI (pronounced ‘Jimmie’), an algorithm based on Gaussian mixture models that can be applied to both discrete and continuous settings. GMM-MI is computationally efficient, robust to the choice of hyperparameters and provides the uncertainty on the MI estimate due to the finite sample size. We extensively validate GMM-MI on toy data for which the ground truth MI is known, comparing its performance against established MI estimators. We then demonstrate the use of our MI estimator in the context of representation learning, working with synthetic data and physical datasets describing highly non-linear processes. We train DL models to encode high-dimensional data within a meaningful compressed (latent) representation, and use GMM-MI to quantify both the level of disentanglement between the latent variables, and their association with relevant physical quantities, thus unlocking the interpretability of the latent representation. We make GMM-MI publicly available in this GitHub repository.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Bayesian event categorization matrix approach for explosion monitoring

Current efforts to correctly categorize natural events from suspected explosion sources with data that is collected by ground- or space-based sensors presents historical challenges that remain unaddressed by the Event Categorization Matrix (ECM) model. Smaller historical events (lower yield explosions) may have data available from fewer measurement techniques than are available today, and therefore, a historical event record can lack a complete set of discriminants. The covariance structures can also differ between such observations of event (source-type) categories. Both obstacles are problematic for the classic ECM model. Our work addresses this gap and presents a Bayesian update to the previous ECM model, termed the Bayesian Event Categorization Matrix model, which can be trained on partial observations and does not rely on a pooled covariance structure. We further augment the ECM model with Bayesian Decision Theory so that false negative or false positive rates of an event categorization can be reduced in an intuitive manner. To demonstrate improved categorization rates for the Bayesian Event Categorization Matrix model, we compare an array of Bayesian and classic models with multiple performance metrics using Monte Carlo experiments. We use both synthetic and real data. Our Bayesian models show consistent gains in overall accuracy and lower false negative rates relative to the classic ECM model. Here, we propose future avenues to improve Bayesian Event Categorization Matrix models’ decision making and predictive capability.

58 GEOSCIENCES↗
DOE OSTI2020

Detecting anomalous packets in network transfers: investigations using PCA, autoencoder and isolation forest in TCP

Large-scale scientific workflows rely heavily on high-performance file transfers. These transfers require strict quality parameters such as guaranteed bandwidth, no packet loss or data duplication. To have successful file transfers, methods such as predetermined thresholds and statistical analysis need to be done to determine abnormal patterns. Network administrators routinely monitor and analyze network data for diagnosing and alleviating these, making decisions based on their experience. However, as networks grow and become complex, monitoring large data files and quickly processing them, makes it improbable to identify errors and rectify these. Abnormal file transfers have been classified by simply setting alert thresholds, via tools such as PerfSonar and TCP statistics (Tstat). This paper investigates the feasibility of unsupervised feature extraction methods for identifying network anomaly patterns with three unsupervised classification methods—principal component analysis, autoencoder and isolation forest. Here, we collect file transfer statistics from two experiment sets—synthetic iPerf generated traffic and 1000 Genome workflow runs, with synthetically introduced anomalies. Our results show that while PCA and a simple autoencoder finds it difficult to detect clusters, the tree-variant isolation forest is able to identify anomalous packets by breaking down TCP traces into tree classes early.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Stochastic economic dispatch of wind power under uncertainty using clustering-based extreme scenarios

Operation of power systems with high penetrations of renewable energy sources requires tools for robust decision making under uncertainty. Stochastic economic dispatch and stochastic unit commitment are effective techniques for planning and operation under uncertainty, whose effectiveness depends on the cardinality and quality of the scenario set. Here, this article proposes a machine learning method using -means clustering for capturing relevant physical information from a large population of analog scenarios. Extreme scenario samples drawn from the clusters are used in a two-stage stochastic economic dispatch computation. The effectiveness of the proposed approach is assessed on a synthetic 200-bus system with a geographic footprint over Illinois, USA for four months from each season of WIND Toolkit data. The combination of -means clustering with importance sampling is shown to reduce the total operational cost by over 43% compared to sampling from populations based on heuristic clustering-based methods. Additionally, the variability in the mean cost is about 56% lower than the variability using Monte Carlo sampling. Moreover, the operational cost with the presented approach is shown to be close to the cost calculated based on a hindsight exact wind profile, signifying a highly accurate quantification of wind uncertainty by the presented -means clustering based sampling method.

17 WIND ENERGY↗
DOE OSTI2025

Summary of the 5th IAEA technical meeting on fusion data processing, validation and analysis (FDPVA)

The purpose of the 5th International Atomic Energy Agency technical meeting on fusion data processing, validation and analysis (FDPVA) (Ghent University, Ghent, Belgium, 12–15 June 2023) was to provide a platform during which a set of topics relevant to FDPVA were discussed with the view of meeting the needs of next step fusion devices such as ITER. The validation and analysis of experimental data obtained from diagnostics used to characterize fusion plasmas are crucial for a knowledge-based understanding of the physical processes governing the dynamics of these plasmas. This paper presents the recent progress and achievements in the domain of plasma diagnostics data analysis and synthetic diagnostics reported at the meeting, including concept description of new devices; fusion databases; integrated data analysis; inverse problems; uncertainty propagation, verification and validation; probabilistic methods and machine learning. The relevant results underline trends observed in the current major fusion confinement devices.

fusion databases↗