Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distributed data generation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41

Annual Technology Baseline: The 2024 Electricity Update

Consistent cost and performance data for various electricity generation technologies can be difficult to find and may change frequently for certain technologies. With the Annual Technology Baseline (ATB), the National Renewable Energy Laboratory annually provides an organized and centralized set of such cost and performance data. The ATB uses the best information from the Department of Energy national laboratories' energy analysts. The ATB has been reviewed by experts and it includes the following electricity generation and storage technologies: land-based wind, offshore wind, distributed wind, utility-scale solar photovoltaics (PV), commercial-scale solar PV, residential-scale solar PV, concentrating solar power, geothermal power, hydropower, utility-scale battery storage, commercial battery storage, residential battery storage, pumped storage hydropower, nuclear, coal, and natural gas. EIA data for conventional biopower are included for reference. This webinar presentation introduces the 2024 update to the ATB Electricity data and documentation.

battery storage↗

Air Traffic Sector Configuration Change Frequency

Several techniques for partitioning airspace have been developed in the literature. The question of whether a region of airspace created by such methods can be used with other days of traffic, and the number of times a different partition is needed during the day is examined in this paper. Both these aspects are examined for the Fort Worth Center airspace sectors. A Mixed Integer Linear Programming method is used with actual air traffic data of ten high-volume low-weather-delay days for creating sectors. Nine solutions were obtained for each two-hour period of the day by partitioning the center airspace into two through 18 sectors in steps of two sectors. Actual track-data were played back with the generated partitions for creating histograms of the traffic-counts. The best partition for each two-hour period was then identified based on the nine traffic-count distributions. Numbers of sectors in such partitions were analyzed to determine the number of times a different configuration is needed during the day. One to three partitions were selected for the 24-hour period, and traffic data from ten days were played back to test if the traffic-counts stayed below the threshold values associated with these partitions. Results show that these partitions are robust and can be used for longer durations than they were designed for

Chatterji, Gano Broto↗

We Know Where We Don't Know: 3D Bayesian CNNs for Credible Geometric Uncertainty

Deep learning has been successfully applied to the segmentation of 3D Computed Tomography (CT) scans. Establishing the credibility of these segmentations requires uncertainty quantification (UQ) to identify untrustworthy predictions. Recent UQ architectures include Monte Carlo dropout networks (MCDNs), which approximate deep Gaussian processes, and Bayesian neural networks (BNNs), which learn the distribution of the weight space. BNNs are advantageous over MCDNs for UQ but are thought to be computationally infeasible in high dimension, and neither architecture has produced interpretable geometric uncertainty maps. We propose a novel 3D Bayesian convolutional neural network (BCNN), the first deep learning method which generates statistically credible geometric uncertainty maps and scales for application to 3D data. We present experimental results on CT scans of graphite electrodes and laser-welded metals and show that our BCNN outperforms an MCDN in recent uncertainty metrics. The geometric uncertainty maps generated by our BCNN capture distributions of sigmoid values that are interpretable as confidence intervals, critical for applications that rely on deep learning for high-consequence decisions.

97 MATHEMATICS AND COMPUTING↗

Science Data Processing for the Advanced Microwave Scanning Radiometer: Earth Observing System

The National Aeronautics and Space Administration established the framework for the Science Investigator-led Processing Systems (SIPS) to enable the Earth science data products to be produced by personnel directly associated with the instrument science team and knowledgeable of the science algorithms. One of the first instantiations implemented for NASA was the Advanced Microwave Scanning Radiometer - Earth Observing System (AMSR-E) SIPS. The AMSR-E SIPS is a decentralized, geographically distributed ground data processing system composed of two primary components located in California and Alabama. Initial science data processing is conducted at Remote Sensing Systems (RSS) in Santa Rosa, California. RSS ingests antenna temperature orbit data sets from JAXA and converts them to calibrated, resampled, geolocated brightness temperatures. The brightness temperatures are sent to the Global Hydrology and Climate Center in Huntsville, Alabama, which generates the geophysical science data products (e.g., water vapor, sea surface temperature, sea ice extent, etc.) suitable for climate research and applications usage. These science products are subsequently sent to the National Snow and Ice Data Center Distributed Active Archive Center in Boulder, Colorado for archival and dissemination to the at-large science community. This paper describes the organization, coordination, and production techniques employed by the AMSR-E SIPS in implementing, automating and operating the distributed data processing system.

Goodman, H. Michael↗

Distributed Grid Control of Flexible Loads and DERs for Optimized Provision of Synthetic Regulating Reserves

Over the course of this project, we have successfully de-risked our distributed microgrid control architecture by tightly integrating its associated control algorithms into a unified software library, installing the software library on several industrial-grade target hardware platforms, and validating the performance of the resulting microgrid controller in a real-life microgrid. Upon completion of the project, we demonstrated that our distributed control architecture is resilient against (i) failures in control devices, (ii) unreliable communication links, (iii) delays in transmitted data, and (iv) imperfect knowledge of the number of (and state of) generation and load assets in the microgrid. In this final report, we present results from all the milestones that were accomplished over the course of this project.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Annual Technology Baseline: The 2022 Electricity Update

Consistent cost and performance data for various electricity generation technologies can be difficult to find and may change frequently for certain technologies. With the Annual Technology Baseline (ATB), the National Renewable Energy Laboratory annually provides an organized and centralized set of such cost and performance data. The ATB uses the best information from the Department of Energy national laboratories' renewable energy analysts. The ATB has been reviewed by experts and it includes the following electricity generation and storage technologies: land-based wind, offshore wind, distributed wind, utility-scale solar photovoltaics (PV), commercial-scale solar PV, residential-scale solar PV, concentrating solar power, geothermal power, hydropower, utility-scale battery storage, commercial battery storage, residential battery storage, pumped storage hydropower, coal, and natural gas. EIA data for nuclear and conventional biopower are included for reference. This webinar presentation introduces the 2022 update to the ATB Electricity data and documentation.

capacity factor↗

Annual Technology Baseline: The 2023 Electricity Update

Consistent cost and performance data for various electricity generation technologies can be difficult to find and may change frequently for certain technologies. With the Annual Technology Baseline (ATB), the National Renewable Energy Laboratory annually provides an organized and centralized set of such cost and performance data. The ATB uses the best information from the Department of Energy national laboratories' renewable energy analysts. The ATB has been reviewed by experts and it includes the following electricity generation and storage technologies: land-based wind, offshore wind, distributed wind, utility-scale solar photovoltaics (PV), commercial-scale solar PV, residential-scale solar PV, concentrating solar power, geothermal power, hydropower, utility-scale battery storage, commercial battery storage, residential battery storage, pumped storage hydropower, coal, and natural gas. EIA data for nuclear and conventional biopower are included for reference. This webinar presentation introduces the 2023 update to the ATB Electricity data and documentation.

capacity factor↗

DiffLense: a conditional diffusion model for super-resolution of gravitational lensing data

Abstract Gravitational lensing data is frequently collected at low resolution due to instrumental limitations and observing conditions. Machine learning-based super-resolution techniques offer a method to enhance the resolution of these images, enabling more precise measurements of lensing effects and a better understanding of the matter distribution in the lensing system. This enhancement can significantly improve our knowledge of the distribution of mass within the lensing galaxy and its environment, as well as the properties of the background source being lensed. Traditional super-resolution techniques typically learn a mapping function from lower-resolution to higher-resolution samples. However, these methods are often constrained by their dependence on optimizing a fixed distance function, which can result in the loss of intricate details crucial for astrophysical analysis. In this work, we introduce DiffLense , a novel super-resolution pipeline based on a conditional diffusion model specifically designed to enhance the resolution of gravitational lensing images obtained from the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP). Our approach adopts a generative model, leveraging the detailed structural information present in Hubble space telescope (HST) counterparts. The diffusion model, trained to generate HST data, is conditioned on HSC data pre-processed with denoising techniques and thresholding to significantly reduce noise and background interference. This process leads to a more distinct and less overlapping conditional distribution during the model’s training phase. We demonstrate that DiffLense outperforms existing state-of-the-art single-image super-resolution techniques, particularly in retaining the fine details necessary for astrophysical analyses.

Computer Science↗

Standardized Sky Partitioning for the Next Generation Astronomy and Space Science Archives

The Johns Hopkins University and Space Telescope Science Institute are working together on this project to develop a library of standard software for data archives that will benefit the wider astronomical community. The ultimate goal was to develop and distribute a software library aimed at providing a common system for partitioning and indexing the sky in manageable sized regions and provide complex queries on the objects stored in this system. Whilst ongoing maintenance work will continue the primary goal has been completed. Most of the next generation sky surveys in the different wavelengths like 2MASS, GALEX, SDSS, GSC-II, DPOSS and FIRST have agreed on this common set of utilities. In this final report, we summarize work on the work elements assigned to the STScI project team.

Lal, Nand↗

Communication systems and methods for authenticating data packets within network flow

A communication system includes a first quantum key distribution device and an intermediary device. The first quantum key distribution device is configured to be coupled to a second quantum key distribution device over a quantum channel and to generate a shared key with the second quantum key distribution device based on a quantum state transmitted along the quantum channel. The intermediary device is disposed along a communication pathway within a network between a sender device and a receiver device. The intermediary device is communicatively connected to the first quantum key distribution device and configured to utilize the shared key to authenticate one or more data packets communicated from the sender device along the communication pathway by examining the one or more data packets for a presence of an information pattern that is associated with the shared key.

Bush, Stephen Francis↗

An Improved Analysis of Forest Carbon Dynamics using Data Assimilation

There are two broad approaches to quantifying landscape C dynamics - by measuring changes in C stocks over time, or by measuring fluxes of C directly. However, these data may be patchy, and have gaps or biases. An alternative approach to generating C budgets has been to use process-based models, constructed to simulate the key processes involved in C exchange. However, the process of model building is arguably subjective, and parameters may be poorly defined. This paper demonstrates why data assimilation (DA) techniques - which combine stock and flux observations with a dynamic model - improve estimates of, and provide insights into, ecosystem carbon (C) exchanges. We use an ensemble Kalman filter (EnKF) to link a series of measurements with a simple box model of C transformations. Measurements were collected at a young ponderosa pine stand in central Oregon over a 3-year period, and include eddy flux and soil C02 efflux data, litterfall collections, stem surveys, root and soil cores, and leaf area index data. The simple C model is a mass balance model with nine unknown parameters, tracking changes in C storage among five pools; foliar, wood and fine root pools in vegetation, and also fresh litter and soil organic matter (SOM) plus coarse woody debris pools. We nested the EnKF within an optimization routine to generate estimates from the data of the unknown parameters and the five initial conditions for the pools. The efficacy of the DA process can be judged by comparing the probability distributions of estimates produced with the EnKF analysis vs. those produced with reduced data or model alone. Using the model alone, estimated net ecosystem exchange of C (NEE)= -251 f 197g Cm-2 over the 3 years, compared with an estimate of -419 f 29gCm-2 when all observations were assimilated into the model. The uncertainty on daily measurements of NEE via eddy fluxes was estimated at 0.5gCm-2 day-1, but the uncertainty on assimilated estimates averaged 0.47 g Cm-2 day-1, and only exceeded 0.5gC m-2 day-1 on days where neither eddy flux nor soil efflux data were available. In generating C budgets, the assimilation process reduced the uncertainties associated with using data or model alone and the forecasts of NEE were statistically unbiased estimates. The results of the analysis emphasize the importance of time series as constraints. Occasional, rare measurements of stocks have limited use in constraining the estimates of other components of the C cycle. Long time series are particularly crucial for improving the analysis of pools with long time constants, such as SOM, woody biomass, and woody debris. Long-running forest stem surveys, and tree ring data, offer a rich resource that could be assimilated to provide an important constraint on C cycling of slow pools. For extending estimates of NEE across regions, DA can play a further important role, by assimilating remote-sensing data into the analysis of C cycles. We show, via sensitivity analysis, how assimilating an estimate of photosynthesis - which might be provided indirectly by remotely sensed data - improves the analysis of NEE.

Williams, Mathew↗

Control systems and data management for high-power laser facilities

The next generation of high-power lasers enables repetition of experiments at orders of magnitude higher frequency than what was possible using the prior generation. Facilities requiring human intervention between laser repetitions need to adapt in order to keep pace with the new laser technology. A distributed networked control system can enable laboratory-wide automation and feedback control loops. These higher-repetition-rate experiments will create enormous quantities of data. A consistent approach to managing data can increase data accessibility, reduce repetitive data-software development and mitigate poorly organized metadata. An opportunity arises to share knowledge of improvements to control and data infrastructure currently being undertaken. We compare platforms and approaches to state-of-the-art control systems and data management at high-power laser facilities, and we illustrate these topics with case studies from our community

47 OTHER INSTRUMENTATION↗

Determination of phytoplankton chlorophyll concentrations in the Chesapeake Bay with aircraft remote sensing

Remote sensing measurements of the distribution of phytoplankton chlorophyll concentrations in Chesapeake Bay during 1989 are described. It is shown that remote sensing from light aircraft can complement and extend measurements made from traditional platforms and provide data of improved temporal and spatial resolution, leading to a better understanding of phytoplankton dynamics in the estuary. The developments of the winter-spring diatom bloom in the polyhaline to mesohaline regions of the estuary and of the late-spring and summer dinoflagellate blooms in oligohaline and mesohaline regions are traced. The study presents the local chlorophyll algorithm developed using the NASA Ocean Data Acquisition System data and in situ chlorophyll data, interpolated maps of chlorophyll concentration generated by applying the algorithm to aircraft radiance data, ancillary in situ data on nutrients, turbidity, streamflow, and light availability, and an interpretation of phytoplankton dynamics in terms of the chlorophyll distribution in Chesapeake Bay during 1989.

Harding, Lawrence W., Jr.↗

Effect of macroscopic surface defects on dynamic damage: An experimental and numerical study

This study examines the impact of macroscopic surface defects on the dynamic ductile damage behavior of polycrystalline metals using plate-impact experiments. Defects of various shapes (flat, round, and point) were manufactured on the free or impact surfaces of annealed copper specimens. The experiments were diagnosed with photon Doppler velocimetry measurements and soft recovery techniques. The experimental results revealed that defect shape and location significantly affect velocity–time profiles and void distribution. In order to understand the dynamics of shock propagation and corresponding ductile damage evolution within the target specimens, the impact experiments were modeled with a calibrated Tepla model, a dynamic ductile damage model for polycrystalline metals [Nguyen et al., Int. J. Solids Struct. 329, 113833 (2026)]. Overall, our resulting simulated velocity showed good agreement with measured velocity, and our simulated porosity distributions qualitatively matched experimental data. Based on our simulation results, defects on the free surface were found to distort rarefaction waves and therefore the corresponding spall planes. On the other hand, defects on the impact surface generated a delayed shock when the flyer plate and the defected target area were in contact, leading to distortion of the spall plane. The distortion of the spall plane resulted in a non-uniform distribution of voids within the defected specimens. Lastly, we examined the role of local work hardening due to the defect manufacturing process on the velocity and porosity distribution, using Tepla simulations with a simplified representation of local hardening. This investigation highlights the importance of defect geometry, location, and local hardening associated with defect generation in dynamic ductile damage processes.

36 MATERIALS SCIENCE↗

Visualizing 2D Probability Distributions from Satellite Image-Derived Data

Creating maps of biophysical and geophysical variables using Earth Observing System (EOS) satellite image data is an important component of Earth science. These 2D maps have a single value at every location and standard techniques are used to visualize them. Current tools fall short, however, when it is necessary to describe a distribution of values at each location. Distributions may represent a frequency of occurrence over time, frequency of occurrence from multiple runs of an ensemble forecast or possible values from an uncertainty model. 'Distribution data sets' are described, then a case study is presented to visualize such 2D distributions. Distribution data sets are different from multivariate data sets in the sense that the values are for a single variable instead of multiple variables. Our case study data consists of multiple realizations of percent forest cover, generated using a geostatistical technique that combines ground measurements and satellite imagery to model uncertainty about forest cover. We present several approaches for analyzing and visualizing such data sets. The first is a pixel-wise analysis of the probability density functions for the 2D image while the second is an analysis of features identified within the image. Such pixel-wise and feature-wise views will give Earth scientists a more complete understanding of distribution data sets.

Kao, David↗

Rucio beyond ATLAS: experiences from Belle II, CMS, DUNE, EISCAT3D, LIGO/VIRGO, SKA, XENON

For many scientific projects, data management is an increasingly complicated challenge. The number of data-intensive instruments generating unprecedented volumes of data is growing and their accompanying workflows are becoming more complex. Their storage and computing resources are heterogeneous and are distributed at numerous geographical locations belonging to different administrative domains and organisations. These locations do not necessarily coincide with the places where data is produced nor where data is stored, analysed by researchers, or archived for safe long-term storage. To fulfil these needs, the data management system Rucio has been developed to allow the high-energy physics experiment ATLAS at LHC to manage its large volumes of data in an efficient and scalable way. But ATLAS is not alone, and several diverse scientific projects have started evaluating, adopting, and adapting the Rucio system for their own needs. As the Rucio community has grown, many improvements have been introduced, customisations have been added, and many bugs have been fixed. Additionally, new dataflows have been investigated and operational experiences have been documented. In this article we collect and compare the common successes, pitfalls, and oddities that arose in the evaluation efforts of multiple diverse experiments, and compare them with the ATLAS experience. This includes the high-energy physics experiments Belle II and CMS, the neutrino experiment DUNE, the scattering radar experiment EISCAT3D, the gravitational wave observatories LIGO and VIRGO, the SKA radio telescope, and the dark matter search experiment XENON.

Lassnig, Mario↗

pop-cosmos : redshifts and physical properties of KiDS-1000 galaxies

ABSTRACT Principled Bayesian inference of galaxy properties has not previously been performed for wide-area weak-lensing surveys with millions of sources. We address this gap by applying the pop-cosmos generative model to perform spectral energy distribution (SED) fitting for 4 million KiDS (Kilo-Degree Survey)-1000 galaxies. Calibrated on deep COSMOS2020 photometric data, pop-cosmos specifies a physically motivated prior over the galaxy population up to $z \simeq 6$ in stellar population synthesis (SPS) parameter space. Using the Speculator SPS emulator with GPU (graphics processing unit)-accelerated Markov Chain Monte Carlo sampling, we perform full posterior inference at 8.2 GPU seconds per galaxy, obtaining joint constraints on galaxy redshifts and physical properties. We validate photometric redshifts against $\sim \!185\,\!000$ KiDS galaxies cross-matched to Dark Energy Spectroscopic Instrument Data Release 1 spectroscopic samples, achieving low bias ($2\times 10^{-3}$), scatter ($\sigma _{\mathrm{MAD}}=0.03$), and outlier fraction (3.2 per cent) for the Bright Galaxy Survey, with comparable performance (bias $3\times 10^{-2}$, $\sigma _{\mathrm{MAD}}=0.05$, 1.0 per cent outliers) for luminous red galaxies (LRGs). Within the LRG sample, we identify massive, dusty, star-forming contaminants at $z \simeq 0.4$ satisfying standard colour selections for quenched populations. We infer trends in stellar mass, star formation, metallicity, and dust across five tomographic redshift bins consistent with established scaling relations. Using specific star formation rate constraints, we identify $\sim$7 per cent of KiDS-1000 galaxies as quenched, versus 37 per cent implied by conservative colour cuts. This enables the construction of weak-lensing samples defined by physical properties while mitigating intrinsic alignment systematics and preserving statistical power. Our analysis validates pop-cosmos out of sample, establishing it as a scalable approach for galaxy evolution and cosmological analyses with photometric surveys.

Halder, Anik [Institute of Astronomy and Kavli Ins↗

pnnl/pakman

PaKman: A Scalable Algorithm for Generating Genomic Contigs on Distributed Memory Machines. PaKman presents a fully distributed method that tackles assembly of large genomes through the combinationof a novel data-structure (PaK-Graph) and algorithmic strategies to simplify communication and I/O footprint during the assembly process.

Ghosh, Priyanka↗