SEARCH · Search NASA

Results for “Distributed data generation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Experiential findings for sustainable software ecosystems to support experimental and observational science

In the search for a sustainable approach for software ecosystems that supports experimental and observational science (EOS) across Oak Ridge National Laboratory (ORNL), we conducted a survey to understand the current and future landscape of EOS software and data. This paper describes the survey design we used to identify significant areas of interest, gaps, and potential opportunities, followed by a discussion on the obtained responses. The survey formulates questions about project demographics, technical approach, and skills required for the present and the next five years. Here, the study was conducted among 38 ORNL participants between June and July of 2021 and followed the required guidelines for human subjects training. We plan to use the collected information to help guide a vision for sustainable, community-based, and reusable scientific software ecosystems that need to adapt effectively to: (i) the evolving landscape of heterogeneous hardware in the next generation of instruments and computing (e.g. edge, distributed, accelerators), and (ii) data management requirements for data-driven science using artificial intelligence.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1995

Short-range precipitation forecasts using assimilation of simulated satellite water vapor profiles and column cloud liquid water amounts

These observing system simulation experiments investigate the assimilation of satellite-observed water vapor and cloud liquid water data in the initialization of a limited-area primitive equations model with the goal of improving short-range precipitation forecasts. The assimilation procedure presented includes two aspects: specification of an initial cloud liquid water vertical distribution and diabatic initialization. The satellite data is simulated for the next generation of polar-orbiting satellite instruments, the Advanced Microwave Sounding Unit (AMSU) and the High-Resolution Infrared Sounder (HIRS), which are scheduled to be launched on the NOAA-K satellite in the mid-1990s. Based on cloud-top height and total column cloud liquid water amounts simulated for satellite data a diagnostic method is used to specify an initial cloud water vertical distribution and to modify the initial moisture distribution in cloudy areas. Using a diabatic initialization procedure, the associated latent heating profiles are directly assimilated into the numerical model. The initial heating is estimated by time averaging the latent heat release from convective and large-scale condensation during the early forecast stage after insertion of satellite-observed temperature, water vapor, and cloud water formation. The assimilation of satellite-observed moisture and cloud water, together withy three-mode diabatic initialization, significantly alleviates the model precipitation spinup problem, especially in the first 3 h of the forecast. Experimental forecasts indicate that the impact of satellite-observed temperature and water vapor profiles and cloud water alone in the initialization procedure shortens the spinup time for precipitation rates by 1-2 h and for regeneration of the areal coverage by 3 h. The diabatic initialization further reduces the precipitation spinup time (compared to adiabatic initialization) by 1 h.

Wu, Xiaohua↗

DOE OSTI2023

Forced Power Systems Oscillations Due to Cyberattacks: Threats, Detection and Partial Mitigation

Forced oscillations in power systems can be caused by misconfigured controllers at generator stations. They can also be caused by cyberattacks against the exciters or governors. This paper explores the effects of forced oscillations from cyberattacks on generator excitation and governor systems and the effectiveness of a novel control system for a static var compensator in mitigating those oscillations to enhance transmission system resilience. A brief overview of oscillations, especially forced oscillations, within power systems is presented, along with an overview of cyberattacks on power systems. This paper also examines and implements FACTS devices to partially mitigate the forced oscillations created by cyberattacks by reducing the magnitude of the oscillations caused by the attack. The proposed approach is more effective against attacks targeting exciters.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2021

GPU-based Image Compression for Efficient Compositing in Distributed Rendering Applications

Visualizations of large-scale data sets are often created on graphics clusters that distribute the rendering task amongst many processes. When using real-time GPU-based graphics algorithms, the most time-consuming aspect of distributed rendering is typically the com-positing phase - combining all partial images from each rendering process into the final visualization. Compo siting requires image data to be copied off the GPU and sent over a network to other processes. While compression has been utilized in existing distributed rendering compositors to reduce the data being sent over the network, this compression tends to occur after the raw images are transferred from the GPU to main memory. In this paper, we present work that leverages OpenGL / CUDA interoperability to compress raw images on the GPU prior to transferring the data to main memory. This approach can significantly reduce the device-to-host data transfer time, thus enabling more efficient compositing of images generated by distributed rendering applications.

Lipinksi, Riley↗

DOE OSTI2020

Real-time simulator and controller of power system using distributed data streaming server

Systems and methods for simulating and controlling a power system in real time, using a controller, are provided. The controller includes a simulation layer to simulate an operation of the power system, a disturbance generation layer to provide data to the simulation layer to disturb the simulated operation of the power system, an application layer to display the simulated operation of the power system and generate a control signal to control, based on the simulated operation of the power system, at least one element of the power system, and a distributed data streaming server (D2S2) to allow interoperability among the simulation layer, the disturbance layer, and the application layer.

Li, Fangxing↗

DOE OSTI2023

Panorama 360 (Final Report)

This is the final technical report for the DOE-funded Panorama 360 project. Panorama 360 provided a resource for the collection, analysis, and sharing of performance data about end-to-end scientific workflows executing on DOE facilities. The work focused on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: 1. A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); 2. A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; 3. A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and 4. Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Panorama 360 (Final Report)

This final technical report from the lead institution, USC grant #DE-SC0012636, serves as the final technical report for collaborative institution UNC-CH grant #DE-SC0012390. The goal was to develop a repository and associated capabilities for data collection, ingestion, and analysis for a broad class of DOE applications that span experimental and simulation science workflows. In particular, this work focuses on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: (1) A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); (2) A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; (3) A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and (4) Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1989

A correction algorithm for particle size distribution measurements made with the forward-scattering spectrometer probe

A correction algorithm for evaluating the particle size distribution measurements of atmospheric aerosols obtained with a forward-scattering spectrometer probe (FSSP) is examined. A model based on Poisson statistics is employed to calculate the average diameter and rms width of the particle size distribution. The dead time and coincidence errors in the measured number density are estimated. The model generated data are compared with a Monte Carlo simulation of the FSSP operation. It is observed that the correlation between the actual and measured size distribution is nonlinear. It is noted that the algorithm permits more accurate calculation of the average diameter and rms width of the distribution compared to uncorrected measured quantities.

Lock, James A.↗

NASA NTRS1986

Fault latency in the memory - An experimental study on VAX 11/780

Fault latency is the time between the physical occurrence of a fault and its corruption of data, causing an error. The measure of this time is difficult to obtain because the time of occurrence of a fault and the exact moment of generation of an error are not known. This paper describes an experiment to accurately study the fault latency in the memory subsystem. The experiment employs real memory data from a VAX 11/780 at the University of Illinois. Fault latency distributions are generated for s-a-0 and s-a-1 permanent fault models. Results show that the mean fault latency of a s-a-0 fault is nearly 5 times that of the s-a-1 fault. Large variations in fault latency are found for different regions in memory. An analysis of a variance model to quantify the relative influence of various workload measures on the evaluated latency is also given.

Chillarege, Ram↗

NASA NTRS2016

Evolution of the JPSS Ground Project Calibration and Validation System

The Joint Polar Satellite System (JPSS) is the National Oceanic and Atmospheric Administration's (NOAA) next-generation operational Earth observation Program that acquires and distributes global environmental data from multiple polar-orbiting satellites. The JPSS Program plays a critical role to NOAA's mission to understand and predict changes in weather, climate, oceans, coasts, and space environments, which supports the Nation's economy and protection of lives and property. The National Aeronautics and Space Administration (NASA) is acquiring and implementing the JPSS, comprised of flight and ground systems, on behalf of NOAA. The JPSS satellites are planned to fly in the afternoon orbit and will provide operational continuity of satellite-based observations and products for NOAA Polar-orbiting Operational Environmental Satellites (POES) and the Suomi National Polar-orbiting Partnership (SNPP) satellite. To support the JPSS Calibration and Validation (CalVal) node Government Resource for Algorithm Verification, Independent Test, and Evaluation (GRAVITE) services facilitate: Algorithm Integration and Checkout, Algorithm and Product Operational Tuning, Instrument Calibration, Product Validation, Algorithm Investigation, and Data Quality Support and Monitoring. GRAVITE is a mature, deployed system that currently supports the SNPP Mission and has been in operations since SNPP launch. This paper discusses the major re-architecture for Block 2.0 that incorporates SNPP lessons learned, architecture of the system, and demonstrates how GRAVITE has evolved as a system with increased performance. It is now a robust, stable, reliable, maintainable, scalable, and secure system that supports development, test, and production strings, replaces proprietary and custom software, uses open source software, and is compliant with NASA and NOAA standards.

Next-Generation↗

DOE OSTI2020

Stochastic Simulation of Daily Suspended Sediment Concentration Using Multivariate Copulas

Estimation of daily suspended sediment concentration (SSC) is required for water resources and environment management. In this paper, a copula-based stochastic method was proposed for daily SSC simulation. Here, the multivariate copula function, constructed based on a bivariate copula and two bivariate conditional probability distributions, was used to model the temporal and cross dependence structures in daily SSCs. Then, the daily SSCs were generated by sampling from the multivariate conditional distribution. As a result, synthetic long-term SSCs data beyond the limited observation period can be provided for water resources managers, which plays a critical role in accurately estimating frequency and magnitude of extreme SSCs events. The proposed method was under rigorous examination by applying to a case study at Pingshan station in the Jinsha River Basin, China. Results showed that the generated daily SSC sequences not only had a high degree of accuracy in preserving the statistical characteristics of the daily SSC observations, but also captured both the temporal correlation and the cross-correlation between the daily streamflow and daily SSC. Specifically, the average daily relative error values corresponding to mean, standard deviation, skewness, lag-1 temporal correlation, and cross correlation were 0.87%, 4.24%, 7.52%, 0.51% and 2.02%, respectively. The multivariate copula framework proposed here can accurately and efficiently generate long-term daily SSC data for water resources management such as frequency analysis and risk assessment of extreme SSC events.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2020

Better together: Elements of successful scientific software development in a distributed collaborative community

Many scientific disciplines rely on computational methods for data analysis, model generation, and prediction. Implementing these methods is often accomplished by researchers with domain expertise but without formal training in software engineering or computer science. This arrangement has led to underappreciation of sustainability and maintainability of scientific software tools developed in academic environments. Some software tools have avoided this fate, including the scientific library Rosetta. We use this software and its community as a case study to show how modern software development can be accomplished successfully, irrespective of subject area. Rosetta is one of the largest software suites for macromolecular modeling, with 3.1 million lines of code and many state-of-the-art applications. Since the mid 1990s, the software has been developed collaboratively by the RosettaCommons, a community of academics from over 60 institutions worldwide with diverse backgrounds including chemistry, biology, physiology, physics, engineering, mathematics, and computer science. Developing this software suite has provided us with more than two decades of experience in how to effectively develop advanced scientific software in a global community with hundreds of contributors. Here we illustrate the functioning of this development community by addressing technical aspects (like version control, testing, and maintenance), community-building strategies, diversity efforts, software dissemination, and user support. We demonstrate how modern computational research can thrive in a distributed collaborative community. The practices described here are independent of subject area and can be readily adopted by other software development communities

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2011

Estimated Accuracy of Three Common Trajectory Statistical Methods

Three well-known trajectory statistical methods (TSMs), namely concentration field (CF), concentration weighted trajectory (CWT), and potential source contribution function (PSCF) methods were tested using known sources and artificially generated data sets to determine the ability of TSMs to reproduce spatial distribution of the sources. In the works by other authors, the accuracy of the trajectory statistical methods was estimated for particular species and at specified receptor locations. We have obtained a more general statistical estimation of the accuracy of source reconstruction and have found optimum conditions to reconstruct source distributions of atmospheric trace substances. Only virtual pollutants of the primary type were considered. In real world experiments, TSMs are intended for application to a priori unknown sources. Therefore, the accuracy of TSMs has to be tested with all possible spatial distributions of sources. An ensemble of geographical distributions of virtual sources was generated. Spearman s rank order correlation coefficient between spatial distributions of the known virtual and the reconstructed sources was taken to be a quantitative measure of the accuracy. Statistical estimates of the mean correlation coefficient and a range of the most probable values of correlation coefficients were obtained. All the TSMs that were considered here showed similar close results. The maximum of the ratio of the mean correlation to the width of the correlation interval containing the most probable correlation values determines the optimum conditions for reconstruction. An optimal geographical domain roughly coincides with the area supplying most of the substance to the receptor. The optimal domain s size is dependent on the substance decay time. Under optimum reconstruction conditions, the mean correlation coefficients can reach 0.70 0.75. The boundaries of the interval with the most probable correlation values are 0.6 0.9 for the decay time of 240 h and 0.5 0.95 for the decay time of 12 h. The best results of source reconstruction can be expected for the trace substances with a decay time on the order of several days. Although the methods considered in this paper do not guarantee high accuracy they are computationally simple and fast. Using the TSMs in optimum conditions and taking into account the range of uncertainties, one can obtain a first hint on potential source areas.

Kabashnikov, Vitaliy P.↗

DOE OSTI2026

Uncertainty guided online ensemble for non-stationary data streams in fusion science

Machine Learning (ML) is poised to play a pivotal role in the development and operation of next-generation fusion devices. Fusion data shows non-stationary behavior with distribution drifts, resulted by both experimental evolution and machine wear-and-tear. ML models assume stationary distribution and fail to maintain performance when encountered with such non-stationary data streams. Online learning techniques have been leveraged in other domains, however it has been largely unexplored for fusion applications. In this paper, we investigate online learning for continuous adaptation to drifting data streams in the prediction of Toroidal Field (TF) coils deflection at the DIII-D fusion facility. We further address the short-term performance degradation inherent to standard online learning, which arises because ground truth is unavailable at prediction time. To mitigate this issue, we propose an uncertainty-guided online ensemble framework. The method leverages the Deep Gaussian Process Approximation (DGPA) for calibrated uncertainty estimation and uses these uncertainty measures to guide a meta-algorithm that aggregates predictions from learners trained over different historical horizons. Our results show that online learning reduces prediction error by 80% compared to a static model. The online ensemble and the proposed uncertainty-guided ensemble further reduce error by approximately 6%, and 10% respectively, relative to standard single-model online learning, while also providing calibrated uncertainty estimates to support operational decision-making.

AI↗

NASA NTRS1979

Feasibility of automated dropsize distributions from holographic data using digital image processing techniques

An automated technique for measuring particle diameters and their spatial coordinates from holographic reconstructions is being developed. Preliminary tests on actual cold-flow holograms of impinging jets indicate that a suitable discriminant algorithm consists of a Fourier-Gaussian noise filter and a contour thresholding technique. This process identifies circular as well as noncircular objects. The desired objects (in this case, circular or possibly ellipsoidal) are then selected automatically from the above set and stored with their parametric representations. From this data, dropsize distributions as a function of spatial coordinates can be generated and combustion effects due to hardware and/or physical variables studied.

Feinstein, S. P.↗

NASA NTRS1983

2-165 keV observations of active galaxies and the diffuse background

HEAO 1 spectral observations of 12 active galaxies in the 12-165 keV and 2-50 keV ranges are reported. The spectra of these galaxies in the 2-165 keV range are well represented by a single power law model; within experimental uncertainties a narrow dispersion in power law index attributable to the individual galaxies is observed, while the 2-165 keV luminosities of these galaxies ranged from 3 x 10 to the 43rd to 3 x 10 to the 45th ergs/s. An apparent universality of the spectral form is found which can be interpreted as due to a common electron distribution with a temperature of tens of keV in the Compton scattering region or as a common nonthermal power-law distribution generating the observed flux through synchrotron-Compton processes. The data indicate that relativistic particles are likely to be responsible for the X-rays from cores of active galaxies through synchroton-Compton processes. In addition, it is noted that only weak number evolution, if any at all, is present in active galaxies.

Rothschild, R. E.↗

NASA NTRS1990

The Xpress Transfer Protocol (XTP): A tutorial (short version)

The Xpress Transfer Protocol (XTP) is a reliable, light weight transfer layer protocol. Current transport layer protocols such as DoD's Transmission Control Protocol (TCP) and ISO's Transport Protocol (TP) were not designed for the next generation of high speed, interconnected reliable networks such as fiber distributed data interface (FDDI) and the gigabit/second wide area networks. Unlike all previous transport layer protocols, XTP is being designed to be implemented in hardware as a VLSI chip set. By streamlining the protocol, combining the transport and network layers, and utilizing the increased speed and parallelization possible with a VLSI implementation, XTP will be able to provide the end-to-end data transmission rates demanded in the high speed networks without compromising reliability and functionality. This tutorial briefly describes the operation of the XTP protocol and in particular, its error, flow and rate control; inter-networking addressing mechanisms; and multicast support features, as defined in the XTP Protocol Definition Revision 3.4.

Sanders, Robert M.↗

NASA NTRS1990

The Xpress Transfer Protocol (XTP): A tutorial (expanded version)

The Xpress Transfer Protocol (XTP) is a reliable, real-time, light weight transfer layer protocol. Current transport layer protocols such as DoD's Transmission Control Protocol (TCP) and ISO's Transport Protocol (TP) were not designed for the next generation of high speed, interconnected reliable networks such as fiber distributed data interface (FDDI) and the gigabit/second wide area networks. Unlike all previous transport layer protocols, XTP is being designed to be implemented in hardware as a VLSI chip set. By streamlining the protocol, combining the transport and network layers and utilizing the increased speed and parallelization possible with a VLSI implementation, XTP will be able to provide the end-to-end data transmission rates demanded in high speed networks without compromising reliability and functionality. This paper describes the operation of the XTP protocol and in particular, its error, flow and rate control; inter-networking addressing mechanisms; and multicast support features, as defined in the XTP Protocol Definition Revision 3.4.

Sanders, Robert M.↗