SEARCH · Search NASA

Results for “synthetic data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Fast increased fidelity samplers for approximate Bayesian Gaussian process regression

Gaussian processes (GPs) are common components in Bayesian non-parametric models having a rich methodological literature and strong theoretical grounding. The use of exact GPs in Bayesian models is limited to problems containing several thousand observations due to their prohibitive computational demands. We develop a posterior sampling algorithm using H-matrix approximations that scales at O(n log 2 n). We show that this approximation’s Kullback-Leibler divergence to the true posterior can be made arbitrarily small. Though multidimensional GPs could be used with our algorithm, d-dimensional surfaces are modeled as tensor products of univariate GPs to minimize the cost of matrix construction and maximize computational efficiency. We illustrate the performance of this fast increased fidelity approximate GP, FIFA-GP, using both simulated and non-synthetic data sets

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Inferring microbial co-occurrence networks from amplicon data: a systematic evaluation

Microbes commonly organize into communities consisting of hundreds of species involved in complex interactions with each other. 16S ribosomal RNA (16S rRNA) amplicon profiling provides snapshots that reveal the phylogenies and abundance profiles of these microbial communities. These snapshots, when collected from multiple samples, can reveal the co-occurrence of microbes, providing a glimpse into the network of associations in these communities. However, the inference of networks from 16S data involves numerous steps, each requiring specific tools and parameter choices. Moreover, the extent to which these steps affect the final network is still unclear. In this study, we perform a meticulous analysis of each step of a pipeline that can convert 16S sequencing data into a network of microbial associations. Through this process, we map how different choices of algorithms and parameters affect the co-occurrence network and identify the steps that contribute substantially to the variance. We further determine the tools and parameters that generate robust co-occurrence networks and develop consensus network algorithms based on benchmarks with mock and synthetic data sets. The Microbial Co-occurrence Network Explorer, or MiCoNE (available at https://github.com/segrelab/MiCoNE) follows these default tools and parameters and can help explore the outcome of these combinations of choices on the inferred networks. We envisage that this pipeline could be used for integrating multiple data sets and generating comparative analyses and consensus networks that can guide our understanding of microbial community assembly in different biomes.

16S rRNA↗

DOE OSTI2022

Emulation Modeling for Development of Cyber-Defense Capabilities for Satellite Systems

The objective of this project was to develop a novel capability to generate synthetic data sets for the purpose of training Machine Learning (ML) algorithms for the detection of malicious activities on satellite systems. The approach experimented with was to a) generate sparse data sets using emulation modeling and b) enlarge the sparse data using Generative Adversarial Networks (GANs). We based our emulation modeling on the Open Source NASA Operational Simulator for Small Satellites (NOS3) developed by the Katherine Johnson Independent Verification and Validation (IV&V) program in West Virginia. Significant new capabilities on NOS3 had to be developed for our data set generation needs. To expand these data sets for the purpose of training ML, we experimented with a) Extreme Learning Machines (ELMs) and b) Wasserstein-GANs (WGAN-GP).

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

DOE OSTI2023

NRAP-Open-IAM Multisegmented Wellbore Reduced-Order Model: Improvement and Quality Assurance

The multisegmented wellbore model (MSW) semi-analytically estimates the amount of CO 2 and brine leakage from a leaking legacy well by segmenting it into intervals to simulate site-specific stratigraphic and hydrogeologic properties. The model is a component of the National Risk Assessment Partnership Open-Source Integrated Assessment Model (NRAP-Open-IAM), which was developed to perform risk assessment for geologic CO 2 storage. The new wellbore leakage model, which uses deep learning networks for a caprock segment, was developed to enhance the analytical MSW. The model was trained and validated using a synthetic data set of Subsurface Transport Over Multiple Phases (STOMP) multiphase flow simulations from various geological, well attribute, and operational conditions to ensure its quality. The results demonstrate that the model is more accurate than the existing model in predicting the transport of two-phase fluids (brine and injected CO 2 ) through the well. This report provides a detailed explanation of the model development and quality assurance.

58 GEOSCIENCES↗

DOE OSTI2021

First M87 Event Horizon Telescope Results. VII. Polarization of the Ring

In 2017 April, the Event Horizon Telescope (EHT) observed the near-horizon region around the supermassive black hole at the core of the M87 galaxy. These 1.3 mm wavelength observations revealed a compact asymmetric ring-like source morphology. This structure originates from synchrotron emission produced by relativistic plasma located in the immediate vicinity of the black hole. Here we present the corresponding linear-polarimetric EHT images of the center of M87. We find that only a part of the ring is significantly polarized. The resolved fractional linear polarization has a maximum located in the southwest part of the ring, where it rises to the level of ~15%. The polarization position angles are arranged in a nearly azimuthal pattern. We perform quantitative measurements of relevant polarimetric properties of the compact emission and find evidence for the temporal evolution of the polarized source structure over one week of EHT observations. The details of the polarimetric data reduction and calibration methodology are provided. We carry out the data analysis using multiple independent imaging and modeling techniques, each of which is validated against a suite of synthetic data sets. The gross polarimetric structure and its apparent evolution with time are insensitive to the method used to reconstruct the image. These polarimetric images carry information about the structure of the magnetic fields responsible for the synchrotron emission. Their physical interpretation is discussed in an accompanying publication.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2024

First Sagittarius A* Event Horizon Telescope Results. VII. Polarization of the Ring

Abstract The Event Horizon Telescope observed the horizon-scale synchrotron emission region around the Galactic center supermassive black hole, Sagittarius A* (Sgr A*), in 2017. These observations revealed a bright, thick ring morphology with a diameter of 51.8 ± 2.3 μas and modest azimuthal brightness asymmetry, consistent with the expected appearance of a black hole with mass M ≈ 4 × 106 M ⊙. From these observations, we present the first resolved linear and circular polarimetric images of Sgr A*. The linear polarization images demonstrate that the emission ring is highly polarized, exhibiting a prominent spiral electric vector polarization angle pattern with a peak fractional polarization of ∼40% in the western portion of the ring. The circular polarization images feature a modestly (∼5%–10%) polarized dipole structure along the emission ring, with negative circular polarization in the western region and positive circular polarization in the eastern region, although our methods exhibit stronger disagreement than for linear polarization. We analyze the data using multiple independent imaging and modeling methods, each of which is validated using a standardized suite of synthetic data sets. While the detailed spatial distribution of the linear polarization along the ring remains uncertain owing to the intrinsic variability of the source, the spiraling polarization structure is robust to methodological choices. The degree and orientation of the linear polarization provide stringent constraints for the black hole and its surrounding magnetic fields, which we discuss in an accompanying publication.

79 ASTRONOMY AND ASTROPHYSICS↗

NASA NTRS1998

MODIS Science Team Member Semi-annual Report

This paper presents a semi-annual report of the MODerate resolution imaging Spectroradiometer (MODIS) Science Team Members. The most important activities undertaken during this reporting period are the following: 1) Versions 2.1 and 2.2 surface reflectance L2/L3 DAAC/SDST delivery; 2) Version 2.0 1km and 250m VI product delivery (assist Arizona); 3) Version 2.1 surface reflectance L2 testing; 4) Land Synthetic data set generator improvements; 5) QA; 6) Surface reflectance error budget generation (SWAMP request); 7) SCF Hardware; 8) Aerosol transport modeling; 9) Aerosol optical depth retrieval from AVHRR data; 10) Aerosol characteristics retrieval from SeaWIFS/AVHRR fusioned data; 11) Validation activities; 12) Aerosol climatology; and 13) 6S code. The report includes summaries of the topics above.

Vermote, Eric↗

NASA NTRS1997

Synthetic Data for Testing TRMM Radar Algorithms

Test data are required to test algorithms for the TRMM Precipitation Radar. These data are needed to test the design of the computer codes under development for the operational phase of the mission, and also to test and evaluate alternative or improved precipitation retrieval algorithms. Over a number of years we have developed and used a 3-dimensional radar model for simulating spaceborne precipitation radars. We have adapted this code to produce data files as close as possible to the TRMM file specifications. In this paper, we will describe the model as it is currently implemented, and show some samples of the synthetic data sets.

Jones, Jeffrey A.↗

NASA NTRS2003

Comment on 'Modeling of Convective-Stratiform Precipitation Processes: Sensitivity to Partitioning Methods' by Matthias Steiner

Despite the obvious notion that the presence of hail or graupel is a good indication of convection, the model results show this does not provide an objective benchmark partly due to the unrealistic presence of small amounts of hail or graupel throughout the anvil in the model but mainly because of the significant amounts of hail or graupel, especially in the tropical TOGA COARE simulation, in the transition zone. Without use of a "transition" category, it is open to debate as how this region should best be defined, as stratiform or as convective. So, the presence of significant hail or graupel contents in this zone significantly degrades its use an objective benchmark for convection. The separation algorithm comparison was done in the context of a cloud-resolving model. These models are widely used and serve a variety of purposes especially with regard to retrieving information that cannot be directly measured by providing synthetic data sets that are consistent and complete. Separation algorithms are regularly applied in these models. However, as with any modeling system, these types 'of models are constantly being improved to overcome any known deficiencies and make them more accurate representations of observed systems. The presence of hail and graupel in the anvil and the bias towards heavy rainfall rates are two such examples of areas that need improvement. Since, both of these can effect the perceived performance of the separation algorithms, the Lang et al. (2003) study did not want to overstate the relative performance of any specific algorithms.

Lang, Steve↗

NASA NTRS2004

Simulating Remote Sensing Systems

The Application Research Toolbox (ART) is a collection of computer programs that implement algorithms and mathematical models for simulating remote sensing systems. The ART is intended to be especially useful for performing design-tradeoff studies and statistical analyses to support the rational development of design requirements for multispectral imaging systems. Among other things, the ART affords a capability to synthesize coarser-spatial-resolution image-data products. The ART also provides for simulations of image-degradation effects, including point-spread functions, misregistration of spectral images, and noise. The ART can utilize real or synthetic data sets, along with sensor specifications, to create simulated data sets. In one example of a particular application, simulated imagery of a coarse resolution system was created using high-resolution imagery from another system in order to perform a radiometric cross-comparison. In the case of a proposed sensor system, the simulated data can be used to conduct trade studies and statistical analyses to ensure that the sensor system will satisfy the requirements of potential scientific, academic, and commercial user communities.

Zanoni, Vicki↗

NASA NTRS2013

Generalized Ocean Color Inversion Model for Retrieving Marine Inherent Optical Properties

Ocean color measured from satellites provides daily, global estimates of marine inherent optical properties (IOPs). Semi-analytical algorithms (SAAs) provide one mechanism for inverting the color of the water observed by the satellite into IOPs. While numerous SAAs exist, most are similarly constructed and few are appropriately parameterized for all water masses for all seasons. To initiate community-wide discussion of these limitations, NASA organized two workshops that deconstructed SAAs to identify similarities and uniqueness and to progress toward consensus on a unified SAA. This effort resulted in the development of the generalized IOP (GIOP) model software that allows for the construction of different SAAs at runtime by selection from an assortment of model parameterizations. As such, GIOP permits isolation and evaluation of specific modeling assumptions, construction of SAAs, development of regionally tuned SAAs, and execution of ensemble inversion modeling. Working groups associated with the workshops proposed a preliminary default configuration for GIOP (GIOP-DC), with alternative model parameterizations and features defined for subsequent evaluation. In this paper, we: (1) describe the theoretical basis of GIOP; (2) present GIOP-DC and verify its comparable performance to other popular SAAs using both in situ and synthetic data sets; and, (3) quantify the sensitivities of their output to their parameterization. We use the latter to develop a hierarchical sensitivity of SAAs to various model parameterizations, to identify components of SAAs that merit focus in future research, and to provide material for discussion on algorithm uncertainties and future ensemble applications.

Werdell, P. Jeremy↗

DOE OSTI2020

Building Highly Detailed Synthetic Electric Grid Data Sets for Combined Transmission and Distribution Systems

Not Available

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2020

Generation of Synthetic Data for a Radiation Detection Algorithm Competition

This paper details the generation of synthetic radiation data using large-scale Monte Carlo transport models to evaluate radiation search detection algorithms. Modular 3-D Monte Carlo models spanning multiple city blocks were constructed, loosely based on downtown Knoxville, TN, containing buildings composed of multiple materials (brick, granite, and concrete), sidewalks, a four-lane road, side streets, parking lots, and grassy fields. Background and simulated source detector response calculations from these models were used to create synthetic list mode data sets for a 2"×4"×16" NaI(Tl) detector moving through a city street at a constant speed. For the background simulations, major isotopes were computed individually so that background composition and variability could be computed efficiently outside Monte Carlo. The source detector response included six simulated sources placed at 15 source locations. Detector response was developed to be periodic through the city street so that a detector path could begin at one end of the model and wrap around to the other. This framework allowed for the creation of diverse data sets, each with its own unique background and simulated source detector response. Synthetic data allows for high-quality labels, which are useful in developing data-driven radiation detection algorithms. This methodology was used to create synthetic data sets which were released as part of a public data competition to spur the development of new radiation detection algorithms for radiological search applications.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2023

The Alcock–Paczynski effect from Lyman- α forest correlations: analysis validation with synthetic data

The three-dimensional distribution of the Ly α forest has been extensively used to constrain cosmology through measurements of the baryon acoustic oscillations (BAO) scale. However, more cosmological information could be extracted from the full shapes of the Ly α forest correlations through the Alcock–Paczynski (AP) effect. In this work, we prepare for a cosmological analysis of the full shape of the Ly α forest correlations by studying synthetic data of the extended Baryon Oscillation Spectroscopic Survey (eBOSS). We use a set of 100 eBOSS synthetic data sets in order to validate such an analysis. These mocks undergo the same analysis process as the real data. We perform a full-shape analysis on the mean of the correlation functions measured from the 100 eBOSS realizations, and find that our model of the Ly α correlations performs well on current data sets. We show that we are able to obtain an unbiased full-shape measurement of D M /D H (z eff ), where D M is the transverse comoving distance, D H is the Hubble distance, and z eff is the effective redshift of the measurement. We test the fit over a range of scales, and decide to use a minimum separation of r min = 25 h –1 Mpc. Here, we also study and discuss the impact of the main contaminants affecting Ly α forest correlations, and give recommendations on how to perform such analysis with real data. While the final eBOSS Ly α BAO analysis measured D M /D H (z eff = 2.33) with 4 per cent statistical precision, a full-shape fit of the same correlations could provide an $\sim 2~{{\ \rm per\ cent}}$ measurement.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

Validation of the DESI 2024 Lyα forest BAO analysis using synthetic datasets

The first year of data from the Dark Energy Spectroscopic Instrument (DESI) contains the largest set of Lyman-α (Lyα) forest spectra ever observed. This data, collected in the DESI Data Release 1 (DR1) sample, has been used to measure the Baryon Acoustic Oscillation (BAO) feature at redshift z = 2.33. In this work, we use a set of 150 synthetic realizations of DESI DR1 to validate the DESI 2024 Lyα forest BAO measurement presented in [1]. The synthetic data sets are based on Gaussian random fields using the log-normal approximation. We produce realistic synthetic DESI spectra that include all major contaminants affecting the Lyα forest. The synthetic data sets span a redshift range 1.8 < z < 3.8, and are analyzed using the same framework and pipeline used for the DESI 2024 Lyα forest BAO measurement. To measure BAO, we use both the Lyα auto-correlation and its cross-correlation with quasar positions. We use the mean of correlation functions from the set of DESI DR1 realizations to show that our model is able to recover unbiased measurements of the BAO position. We also fit each mock individually and study the population of BAO fits in order to validate BAO uncertainties and test our method for estimating the covariance matrix of the Lyα forest correlation functions. Finally, we discuss the implications of our results and identify the needs for the next generation of Lyα forest synthetic data sets, with the top priority being to simulate the effect of BAO broadening due to non-linear evolution.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

Unsupervised Machine Learning for Exploratory Data Analysis of Exoplanet Transmission Spectra

Abstract Transit spectroscopy is a powerful tool for decoding the chemical compositions of the atmospheres of extrasolar planets. In this paper, we focus on unsupervised techniques for analyzing spectral data from transiting exoplanets. After cleaning and validating the data, we demonstrate methods for: (i) initial exploratory data analysis, based on summary statistics (estimates of location and variability); (ii) exploring and quantifying the existing correlations in the data; (iii) preprocessing and linearly transforming the data to its principal components; (iv) dimensionality reduction and manifold learning; (v) clustering and anomaly detection; and (vi) visualization and interpretation of the data. To illustrate the proposed unsupervised methodology, we use a well-known public benchmark data set of synthetic transit spectra. We show that there is a high degree of correlation in the spectral data, which calls for appropriate low-dimensional representations. We explore a number of different techniques for such dimensionality reduction and identify several suitable options in terms of summary statistics, principal components, etc. We uncover interesting structures in the principal component basis, namely well-defined branches corresponding to different chemical regimes of the underlying atmospheres. We demonstrate that those branches can be successfully recovered with a K-means clustering algorithm in a fully unsupervised fashion. We advocate for lower-dimensional representations of the spectroscopic data in terms of the main principal components, in order to reveal the existing structure in the data and quickly characterize the chemical class of a planet.

Matchev, Konstantin T. (ORCID:0000000341829096)↗

NASA NTRS2022

Quantifying Errors in 3D CME Parameters Derived from Synthetic Data Using White-Light Reconstruction Techniques

Current efforts in space weather forecasting of CMEs have been focused on predicting their arrival time and magnetic structure. To make these predictions, methods have been developed to derive the true CME speed, size, position, and mass, among others. Difficulties in determining the input parameters for CME forecasting models arise from the lack of direct measurements of the coronal magnetic fields and uncertainties in estimating the CME 3D geometric and kinematic parameters after eruption. White-light coronagraph images are usually employed by a variety of CME reconstruction techniques that assume more or less complex geometries. This is the first study from our International Space Science Institute (ISSI) team “Understanding Our Capabilities in Observing and Modeling Coronal Mass Ejections”, in which we explore how subjectivity affects the 3D CME parameters that are obtained from the Graduated Cylindrical Shell (GCS) reconstruction technique, which is widely used in CME research. To be able to quantify such uncertainties, the “true” values that are being fitted should be known, which are impossible to derive from observational data. We have designed two different synthetic scenarios where the “true” geometric parameters are known in order to quantify such uncertainties for the first time. We explore this by using two sets of synthetic data: 1) Using the ray-tracing option from the GCS model software itself, and 2) Using 3D magnetohydrodynamic (MHD) simulation data from the Magnetohydrodynamic Algorithm outside a Sphere code. Our experiment includes different viewing configurations using single and multiple viewpoints. CME reconstructions using a single viewpoint had the largest errors and error ranges overall for both synthetic GCS and simulated MHD white-light data. As the number of viewpoints increased from one to two, the errors decreased by approximately 4° in latitude, 22° in longitude, 14° in tilt, and 10° in half-angle. Our results quantitatively show the critical need for at least two viewpoints to be able to reduce the uncertainty in deriving CME parameters. We did not find a significant decrease in errors when going from two to three viewpoints for our specific hypothetical three spacecraft scenario using synthetic GCS white-light data. As we expected, considering all configurations and numbers of viewpoints, the mean absolute errors in the measured CME parameters are generally significantly higher in the case of the simulated MHD white-light data compared to those from the synthetic white-light images generated by the GCS model. We found the following CME parameter error bars as a starting point for quantifying the minimum error in CME parameters from white-light reconstructions: Δθ (latitude)=6° +2° -3° , Δϕ (longitude)=11° +18° -6° , Δγ (tilt)=25° +8° -7° , Δx (half-angle)=10° +12° -6° , Δh (height)=0.6 +1.2 -0.4 R ⨀ , and Δκ (ratio)=0.1 +0.03 -0.02 .

Coronal mass ejections↗

NASA NTRS2022

Quantifying Errors in 3D CME Parameters Derived From Synthetic Data Using White-Light Reconstruction Techniques

Coronal mass ejections↗