Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distributed data generation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Anomaly detection in Hyper Suprime-Cam galaxy images with generative adversarial networks

ABSTRACT The problem of anomaly detection in astronomical surveys is becoming increasingly important as data sets grow in size. We present the results of an unsupervised anomaly detection method using a Wasserstein generative adversarial network (WGAN) on nearly one million optical galaxy images in the Hyper Suprime-Cam (HSC) survey. The WGAN learns to generate realistic HSC-like galaxies that follow the distribution of the data set; anomalous images are defined based on a poor reconstruction by the generator and outlying features learned by the discriminator. We find that the discriminator is more attuned to potentially interesting anomalies compared to the generator, and compared to a simpler autoencoder-based anomaly detection approach, so we use the discriminator-selected images to construct a high-anomaly sample of ∼13 000 objects. We propose a new approach to further characterize these anomalous images: we use a convolutional autoencoder to reduce the dimensionality of the residual differences between the real and WGAN-reconstructed images and perform UMAP clustering on these. We report detected anomalies of interest including galaxy mergers, tidal features, and extreme star-forming galaxies. A follow-up spectroscopic analysis of one of these anomalies is detailed in the Appendix; we find that it is an unusual system most likely to be a metal-poor dwarf galaxy with an extremely blue, higher-metallicity H ii region. We have released a catalogue with the WGAN anomaly scores; the code and catalogue are available at https://github.com/kstoreyf/anomalies-GAN-HSC; and our interactive visualization tool for exploring the clustered data is at https://weirdgalaxi.es.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2008

Hail Size Distribution Mapping

A 3-D weather radar visualization software program was developed and implemented as part of an experimental Launch Pad 39 Hail Monitor System. 3DRadPlot, a radar plotting program, is one of several software modules that form building blocks of the hail data processing and analysis system (the complete software processing system under development). The spatial and temporal mapping algorithms were originally developed through research at the University of Central Florida, funded by NASA s Tropical Rainfall Measurement Mission (TRMM), where the goal was to merge National Weather Service (NWS) Next-Generation Weather Radar (NEXRAD) volume reflectivity data with drop size distribution data acquired from a cluster of raindrop disdrometers. In this current work, we adapted these algorithms to process data from a cluster of hail disdrometers positioned around Launch Pads 39A or 39B, along with the corresponding NWS radar data. Radar data from all NWS NEXRAD sites is archived at the National Climatic Data Center (NCDC). That data can be readily accessed at . 3DRadPlot plots Level III reflectivity data at four scan elevations (this software is available at Open Channel Software, ). By using spatial and temporal interpolation/extrapolation based on hydrometeor fall dynamics, we can merge the hail disdrometer array data coupled with local Weather Surveillance Radar-1988, Doppler (WSR-88D) radial velocity and reflectivity data into a 4-D (3-D space and time) picture of hail size distributions. Hail flux maps can then be generated and used for damage prediction and assessment over specific surfaces corresponding to structures within the disdrometer array volume. Immediately following a hail storm, specific damage areas and degree of damage can be identified for inspection crews.

Source record↗
DOE OSTI2021

Adversarial sampling of unknown and high-dimensional conditional distributions

Many engineering problems require the prediction of realization-to-realization variability or a refined description of modeled quantities. In that case, it is necessary to sample elements from unknown high-dimensional spaces with possibly millions of degrees of freedom. While there exist methods able to sample elements from probability density functions (PDF) with known shapes, several approximations need to be made when the distribution is unknown. In this paper the sampling method, as well as the inference of the underlying distribution, are both handled with a data-driven method known as generative adversarial networks (GAN), which trains two competing neural networks to produce a network that can effectively generate samples from the training set distribution. In practice, it is often necessary to draw samples from conditional distributions. When the conditional variables are continuous, only one (if any) data point corresponding to a particular value of a conditioning variable may be available, which is not sufficient to estimate the conditional distribution. This work handles this problem using an a priori estimation of the conditional moments of a PDF. Herein, two approaches, stochastic estimation, and an external neural network are compared for computing these moments; however, any preferred method can be used. The algorithm is demonstrated in the case of the deconvolution of a filtered turbulent flow field. It is shown that all the versions of the proposed algorithm effectively sample the target conditional distribution with minimal impact on the quality of the samples compared to state-of-the-art methods. Additionally, the procedure can be used as a metric for the diversity of samples generated by a conditional GAN (cGAN) conditioned with continuous variables.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
NASA NTRS1975

The effects of correlation on goodness of fit

Autocorrelated normal random variates were generated via computer and the effects of various levels of correlation on goodness of fit problems were studied. The results are useful in determining the distribution or estimating the parameters of populations that correlate observations such as wind speeds and temperature. The model used to generate the autocorrelated data is an autoregressive process of order 1. The Kolmogorov-Smirnov and chi-square statistics are used in the analysis. It was observed in the simulation that high positive correlations tend to shift the sample mean away from the population mean and negative correlations tend to shift the sample mean towards the population mean. In many cases, it was observed that positive and negative correlations tend to decrease the standard deviation. However, since this did not occur in all cases, no definite conclusion can be made regarding the standard deviation. Since the autoregressive process is a linear transformation, it is not surprising that normality was preserved. However, a possible extension of this problem could be to generate non-normal data and observe how the distribution is affected by correlation. Another extension might utilize another model such as autoregressive of order k or a moving average process of order k.

Kitchens, L. J.↗
DOE OSTI2026

Decentralised Reinforcement Learning for Dynamic Cyberattack Response in Microgrid Networks

Microgrids rely on communication networks for reliable operation, which makes them inherently vulnerable to cyberattacks. Such attacks can destabilise system dynamics and drive states away from their nominal operating trajectories. Although several physics-informed and machine learning-based strategies have been developed to counter these threats, the rapidly evolving cyber landscape enables adversaries to bypass static defences or rules-based mitigation approaches. This paper proposes a dynamic, online-trained and fully decentralised reinforcement learning (RL)-based cyberattack response framework to protect microgrids from evolving cyberattacks. The proposed framework deploys multiple deep Q-networks (DQNs), each associated with a distributed energy resource (DER), to enable localised and adaptive attack mitigation. In this framework, each DQN processes local voltage and frequency measurements—combined with intrusion detection system (IDS) alerts—as observations and rewards to guide decision-making. Extensive simulation studies demonstrate the robustness of the proposed framework under diverse attack scenarios and varying IDS-induced detection delays. Comparative analysis highlights its superiority over existing static or preexisting rules-based mitigation approaches. Finally, we present an analysis that shows the framework's scalability to real-life microgrids with more interacting agents.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Selection of a Pair of Experiments to Optimally Reduce Uncertainty in Targeted Nuclear Data

We propose a novel process to select a pair of differential and integral experiments that best reduce uncertainties in targeted 239 ⁢Pu nuclear data while compressing the current nuclear data pipeline from 20 to 3 years. 239⁢ Pu nuclear data are poorly understood for neutrons in the intermediate energy range due to sparsity and uncertainty in historical experiments. New experiments targeting this range will enable better understanding of these nuclear data, but choosing the ideal experiments to conduct is challenging. Beginning with a prior distribution represented by samples of nuclear data generated from theory, generalized least squares adjustments are made to incorporate data from historical experiments. To quantify potential uncertainty reduction obtainable from a pair of candidate experiments, we compute the D-optimality criterion of the posterior covariance of intermediate energy range nuclear data compared to the equivalent covariance after additional adjustment to the pair of candidate experiments. Repeating the process for each of many candidate pairs facilitates the final selection. Results support 63⁢ Cu total cross section measurements for differential experiments and alumina and alumina/graphite configurations for integral experiments. This analysis enables choosing differential and integral experiments to be executed concurrently while shortening decision times relative to the current nuclear data pipeline.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Data Privacy for the Grid: Toward a Data Privacy Standard for Inverter-Based and Distributed Energy Resources

The traditional approach to planning the distribution grid has focused on reliability in the context of gradual and reasonably predictable load growth. Forecasts of load growth, combined with asset management practices, were used by system planners to identify upgrades to the system to maintain or improve reliability. The decisions, typically based within load flow analysis tools, included considerations about contingency scenarios and corporate forecasts (i.e., top-down predictions at a summary level of what would happen in a particular area that could impact load growth and behavior). As a result, today, this traditional approach no longer fits all purposes.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Historical Emissions - shipping emission perturbation data

Version of previously released anthropogenic SO2 emissions data re-generated with altered spatial distributions for shipping emissions for use in the Emissions-MIP model inter-comparison project. Also included are 30% and 60% global reductions in shipping SO2 emissions.

Ahsan, Hamza↗
DOE OSTI2022

A Probabilistic Autoencoder for Type Ia Supernova Spectral Time Series

We construct a physically parameterized probabilistic autoencoder (PAE) to learn the intrinsic diversity of Type Ia supernovae (SNe Ia) from a sparse set of spectral time series. The PAE is a two-stage generative model, composed of an autoencoder that is interpreted probabilistically after training using a normalizing flow. We demonstrate that the PAE learns a low-dimensional latent space that captures the nonlinear range of features that exists within the population and can accurately model the spectral evolution of SNe Ia across the full range of wavelength and observation times directly from the data. By introducing a correlation penalty term and multistage training setup alongside our physically parameterized network, we show that intrinsic and extrinsic modes of variability can be separated during training, removing the need for the additional models to perform magnitude standardization. We then use our PAE in a number of downstream tasks on SNe Ia for increasingly precise cosmological analyses, including the automatic detection of SN outliers, the generation of samples consistent with the data distribution, and solving the inverse problem in the presence of noisy and incomplete data to constrain cosmological distance measurements. We find that the optimal number of intrinsic model parameters appears to be three, in line with previous studies, and show that we can standardize our test sample of SNe Ia with an rms of 0.091 ± 0.010 mag, which corresponds to 0.074 ± 0.010 mag if peculiar velocity contributions are removed.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Portable Software Environment for Ultrahigh-Resolution ELM Development on GPUs

This paper presents our endeavors in developing the large-scale, ultra-high-resolution E3SM Land Model (uELM), specifically designed for exascale computers furnished with accelerators such as Nvidia GPUs. The uELM is a sophisticated code that substantially relies on High-Performance Computing (HPC) environments, necessitating particular machine and software configurations. To facilitate community-based uELM developments employing GPUs, we have created a portable, standalone software environment preconfigured with uELM input datasets, simulation cases, and source code. This environment, utilizing Docker, encompasses all essential code, libraries, and system software for uELM development on GPUs. It also features a functional unit test framework and an offline model testbed for comprehensive numerical experiments. From a technical perspective, the paper discusses GPU-ready container generations, uELM code management, and input data distribution across computational platforms. Lastly, the paper demonstrates the use of environment for functional unit testing, end-to-end simulation on CPUs and GPUs, and collaborative code development.

E3SM Land Model↗
NASA NTRS1992

NASA Surface-Modeling and Grid-Generation (SM/GG) activities

A NASA Steering Committee was formed to carry out the recommendations from the NASA Workshop on Future Directions in Surface Modeling and Grid Generation. Its function is to communicate and coordinate within NASA the acquisition and distribution of geometry/grid generation software/data, establish geometry data exchange standards, and interface with other government, university, and industry efforts. Two speakers present the committee's activities in viewgraph format.

Choo, Yung K.↗
NASA NTRS2003

Development of a Flexible Framework of Common Hypersonic Navier-Strokes Meshes for the Space Shuttle Orbiter

A flexible framework for the development of block structured volume grids for hypersonic Navier-Stokes flow simulations was developed for analysis of the Shuttle Orbiter Columbia. The development of the flexible framework, resulted in an ability to quickly generate meshes to directly correlate solutions contributed by participating groups on a common surface mesh, providing confidence for the extension of the envelope of solutions and damage scenarios. The framework draws on the experience of NASA Langely and NASA Ames Research Centers in structured grid generation, and consists of a grid generation process that is implemented through a division of responsibilities. The nominal division of labor consisted of NASA Johnson Space Center coordinating the damage scenarios to be analyzed by the Aerothermodynamics Columbia Accident Investigation (CAI) team, Ames developing the surface grids that described the computational volume about the orbiter, and Langely improving grid quality of Ames generated data and constructing the final volume grids. Distributing the work among the participants in the Aerothermodynamic CIA team resulted in significantly less time required to construct complete meshes than possible by any individual participant. The approach demonstrated that the One-NASA grid generation team could sustain the demand for new meshes to explore new damage scenarios within a aggressive timeline.

Alter, Stephen J.↗
NASA NTRS2009

Multi-Mission Automated Task Invocation Subsystem

Multi-Mission Automated Task Invocation Subsystem (MATIS) is software that establishes a distributed data-processing framework for automated generation of instrument data products from a spacecraft mission. Each mission may set up a set of MATIS servers for processing its data products. MATIS embodies lessons learned in experience with prior instrument- data-product-generation software. MATIS is an event-driven workflow manager that interprets project-specific, user-defined rules for managing processes. It executes programs in response to specific events under specific conditions according to the rules. Because requirements of different missions are too diverse to be satisfied by one program, MATIS accommodates plug-in programs. MATIS is flexible in that users can control such processing parameters as how many pipelines to run and on which computing machines to run them. MATIS has a fail-safe capability. At each step, MATIS captures and retains pertinent information needed to complete the step and start the next step. In the event of a restart, this information is retrieved so that processing can be resumed appropriately. At this writing, it is planned to develop a graphical user interface (GUI) for monitoring and controlling a product generation engine in MATIS. The GUI would enable users to schedule multiple processes and manage the data products produced in the processes. Although MATIS was initially designed for instrument data product generation,

Cheng, Cecilia S.↗
NASA NTRS2009

On-Orbit Calibration and Performance of Aqua MODIS Reflective Solar Bands

Aqua MODIS has successfully operated on-orbit for more than 6 years since its launch in May 2002, continuously making global observations and improving studies of changes in the Earth's climate and environment. 20 of the 36 MODIS spectral bands, covering wavelengths from 0.41 to 2.2 microns, are the reflective solar bands (RSB). They are calibrated on-orbit using an on-board solar diffuser (SD) and a solar diffuser stability monitor (SDSM). In addition, regularly scheduled lunar observations are made to track the RSB calibration stability. This paper presents Aqua MODIS RSB on-orbit calibration and characterization activities, methodologies, and performance. Included in this study are characterizations of detector signal-to-noise ratio (SNR), short-term stability, and long-term response change. Spectral wavelength dependent degradation of the SD bidirectional reflectance factor (BRF) and scan mirror reflectance, which also varies with angle of incidence (AOI), are examined. On-orbit results show that Aqua MODIS onboard calibrators have performed well, enabling accurate calibration coefficients to be derived and updated for the Level 1B (L1B) production and assuring high quality science data products to be continuously generated and distributed. Since launch, the short-term response, on a scan-by-scan basis, has remained extremely stable for most RSB detectors. With the exception of band 6, there have been no new RSB noisy or inoperable detectors. Like its predecessor, Terra MODIS, launched in December 1999, the Aqua MODIS visible (VIS) spectral bands have experienced relatively large changes, with an annual response decrease (mirror side 1) of 3.6% for band 8 at 0.412 microns, 2.3% for band 9 at 0.443 microns, 1.6% for band 3 at 0.469 microns, and 1.2% for band 10 at 0.488 microns. For other RSB bands with wavelengths greater than 0.5 microns, the annual response changes are typically less than 0.5%. In general, Aqua MODIS optics degradation is smaller than Terra MODIS and the mirror side differences are much smaller. Overall, Aqua MODIS RSB on-orbit performance is better than Terra MODIS.

Xiong, Xiaoxiong↗
NASA NTRS1989

The discrimination of sea ice types using SAR backscatter statistics

X-band (HH) synthetic aperture radar (SAR) data of sea ice collected during the Marginal Ice Zone Experiment in March and April of 1987 was statistically analyzed with respect to discriminating open water, first-year ice, multiyear ice, and Odden. Odden are large expanses of nilas ice that rapidly form in the Greenland Sea and transform into pancake ice. A first-order statistical analysis indicated that mean versus variance can segment out open water and first-year ice, and skewness versus modified skewness can segment the Odden and multilayer categories. In additions to first-order statistics, a model has been generated for the distribution function of the SAR ice data. Segmentation of ice types was also attempted using textural measurements. In this case, the general co-occurency matrix was evaluated. The textural method did not generate better results than the first-order statistical approach.

Shuchman, Robert A.↗
NASA NTRS1993

EOSDIS - Its role in the EOS program and its importance to the scientific community

In order to manage the global data sets required to understand the earth as a system, the EOS Data and Information System (EOSDIS) will collect and store satellite, aircraft, and in situ measurements and their resultant data products, and will distribute the data conveniently. EOSDIS will also provide product generation and science computing facilities to support the development, processing, and validation of standard EOS science data products. The overall architecture of EOSDIS, and how the Distributed Active Archive Centers fit into that structure, are shown. EOSDIS will enable users to query data bases nationally, make use of keywords and other mnemonic identifiers, and see graphic images of subsets of available data prior to ordering full (or selected pieces of) data sets for use in their 'home' environment.

Price, Robert D.↗