Search NASA⌕ Search

SEARCH · Search NASA

Results for “data distribution”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Overview of the distributed image processing infrastructure to produce the Legacy Survey of Space and Time

The Vera C. Rubin Observatory is preparing to execute the most ambitious astronomical survey ever attempted, the Legacy Survey of Space and Time (LSST). Currently the final phase of construction is under way in the Chilean Andes, with the Observatory’s ten-year science mission scheduled to begin in 2025. Rubin’s 8.4-meter telescope will nightly scan the southern hemisphere collecting imagery in the wavelength range 320–1050 nm covering the entire observable sky every 4 nights using a 3.2 gigapixel camera, the largest imaging device ever built for astronomy. Automated detection and classification of celestial objects will be performed by sophisticated algorithms on high-resolution images to progressively produce an astronomical catalog eventually composed of 20 billion galaxies and 17 billion stars and their associated physical properties. In this article we present an overview of the system currently being constructed to perform data distribution as well as the annual campaigns which reprocess the entire image dataset collected since the beginning of the survey. These processing campaigns will utilize computing and storage resources provided by three Rubin data facilities (one in the US and two in Europe). Each year a Data Release will be produced and disseminated to science collaborations for use in studies comprising four main science pillars: probing dark matter and dark energy, taking inventory of solar system objects, exploring the transient optical sky and mapping the Milky Way. Also presented is the method by which we leverage some of the common tools and best practices used for management of large-scale distributed data processing projects in the high energy physics and astronomy communities. We also demonstrate how these tools and practices are utilized within the Rubin project in order to overcome the specific challenges faced by the Observatory.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Question-answering system extracts information on injection drug use from clinical notes

Background. Injection drug use (IDU) can increase mortality and morbidity. Therefore, identifying IDU early and initiating harm reduction interventions can benefit individuals at risk. However, extracting IDU behaviors from patients’ electronic health records (EHR) is difficult because there is no other structured data available, such as International Classification of Disease (ICD) codes, and IDU is most often documented in unstructured free-text clinical notes. Although natural language processing can efficiently extract this information from unstructured data, there are no validated tools. Methods. Here, to address this gap in clinical information, we design a question-answering (QA) framework to extract information on IDU from clinical notes for use in clinical operations. Our framework involves two main steps: (1) generating a gold-standard QA dataset and (2) developing and testing the QA model. We use 2323 clinical notes of 1145 patients curated from the US Department of Veterans Affairs (VA) Corporate Data Warehouse to construct the gold-standard dataset for developing and evaluating the QA model. We also demonstrate the QA model’s ability to extract IDU-related information from temporally out-of-distribution data. Results. Here, we show that for a strict match between gold-standard and predicted answers, the QA model achieves a 51.65% F1 score. For a relaxed match between the gold-standard and predicted answers, the QA model obtains a 78.03% F1 score, along with 85.38% Precision and 79.02% Recall scores. Moreover, the QA model demonstrates consistent performance when subjected to temporally out-of-distribution data. Conclusions. Our study introduces a QA framework designed to extract IDU information from clinical notes, aiming to enhance the accurate and efficient detection of people who inject drugs, extract relevant information, and ultimately facilitate informed patient care.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

SILIA: software implementation of a multi-channel, multi-frequency lock-in amplifier for spectroscopy and imaging applications

In this work, we describe a software implementation of a multi-channel, multi-frequency Lock-in Amplifier (SILIA) to extract modulated signals from noisy data distributed over multiple channels of arbitrary number and size. This software implementation emulates the functionality of a multi-channel, multi-frequency lock-in amplifier in a post-processing step following data acquisition. Unlike most traditional lock-in amplifiers, SILIA can work with any number of input channels and is especially useful to analyze data distributed over many channels. We demonstrate the versatility and performance for extracting weak signals in spectroscopy and fluorescence microscopy. We also discuss more general applications and exhibit a method to automatically estimate error from a lock-in result.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Data-Driven Distribution System Coordinated PV Inverter Control Using Deep Reinforcement Learning

The deployment of distributed solar photovoltaic (PV) systems has increased consistently over the past decades. High penetrations of PVs could cause a series of adverse grid impacts, such as voltage violations. The recent development of smart inverter technologies rises the incentives of developing PV control solutions that regulate the inverter output power and seeking the optimization on system operational objectives. This paper proposes a data-driven control solution based on deep reinforcement learning (DRL) to optimize PV inverters for voltage regulation. The proposed solution can minimize PV real power curtailment while maintaining network voltage at an acceptable range. Comparison results between the proposed DRL control algorithms with deep deterministic policy gradient (DDPG) and volt-var control on a real feeder in west Colorado highlight the advantage of the proposed framework in controlling the system voltage while minimizing the PV real power curtailment.

deep reinforcement learning↗
DOE OSTI2025

Influence of Alkyne Precursor Structure on Carbon Nanotube Chiral Distribution: Data-Dense Analysis Across Multiple Catalyst Types

Carbon nanotubes (CNTs) are a desirable material in the field of optoelectronics and semiconductors due to electronic properties (e.g., bandgap) that are dependent upon their chirality, defined by their diameter and lattice angle. Unfortunately, industrial-scale syntheses have yet to realize growth of a single desired chirality and instead rely on postsynthetic separation techniques to refine a chiral mixture, which increases process complexity and cost. Here, we studied the influence of precursor structure on chiral distribution, using a series of terminal alkyne precursors (acetylene, methylacetylene, vinylacetylene, 1-butyne, two enantiomers of 3-butyn-2-ol and a racemic mixture thereof) to grow CNTs across five transition-metal catalysts (Fe, FeMo, and three proportions of CoMo). Multiwavelength Raman spectroscopy on 5,145 spots (5 catalysts, 7 precursors, 3 lasers, and 49 distinct substrate locations on each) determined that acetylene grew the smallest diameter CNTs, while vinylacetylene produced fewer subnanometer CNTs. Though precursor structure did not dictate a uniform chiral shift, it was shown to broaden or narrow chiral distribution, while catalyst structure played a dominant role. In conclusion, this is consistent with metal-precursor binding occurring through unsaturated bonds in the hydrocarbons via the alkyne polymerization mechanism.

Carbon nanotubes↗
DOE OSTI2020

Data-Centric Communication Framework for Multicast IEC 61850 Routable GOOSE Messages over the WAN in Modern Power Systems

In this paper, a data-centric communication framework is proposed for multicast routable generic object-oriented substation event (GOOSE) messages (MRGM) over the wide area network (WAN) for effective substation-to-substation (SS2SS) and substation to control center (SS2CC) communications. In this structure, the IEC 61850 GOOSE message is transmitted over the WAN using the data distribution service (DDS) as a fast, reliable, and secure data-centric communication middleware. The main feature of this framework is its multicast capability, where several authorized subscribers can receive a published message simultaneously. This can significantly improve the system monitoring and control of the protection systems in modern smart grids, where intelligent schemes can be applied. The effectiveness of the proposed platform, in terms of total end-to-end delay between participants, is evaluated through experimental results obtained from the actual hardware-based test setup developed at the Florida International University (FIU) smart grid testbed. The results demonstrate that the latency between sending and receiving a GOOSE message among participants is within its maximum time span defined by the IEC 61850-90-5 working group for communications over the WAN.

42 ENGINEERING↗
DOE OSTI2022

Behavioral and Population Data-Driven Distribution System Load Modeling

Distribution system residential load modeling and analysis for different geographic areas within a utility or an independent system operator territory are critical for enabling small-scale, aggregated distributed energy resources to participate in grid services under Federal Energy Regulatory Commission Order No. 2222 [1]. In this study, we develop a methodology of modeling residential load profiles in different geographic areas with a focus on human behavior impact. First, we construct a behavior-based load profile model leveraging state-of-the-art appliance models. We simulate human activity and occupancy using Markov chain Monte Carlo methods calibrated with the American Time Use Survey data set. Second, we link our model with cleaned Current Population Survey data from the U.S. Census Bureau. Finally, we populate two sets of 500 households using California and Texas census data, respectively, to perform an initial analysis of the load in different geographic areas with various group features (e.g., different income levels). To distinguish the effect of population behavior differences on aggregated load, we simulate load profiles for both sets assuming fixed physical household parameters and weather data. Analysis shows that average daily load profiles vary significantly by income and income dependency varies by locality.

American Time Use Survey↗
DOE OSTI2025

Visualization of Noisy and Less Noisy Computational Basis States in Quantum Computing

Quantum computing technology holds substantial promise as a reliable computational paradigm. However, current noisy intermediate scale quantum (NISQ) systems, are significantly impacted by noise originating from hardware inconsistencies. This noise causes errors and lowers output fidelity. So we must find which basis states cause errors. However, there are two main challenges in analyzing noise corresponding to basis states. First, the noise distribution data is high dimensional in nature, thereby making its analysis challenging. Second, although functional box plots have been used in the state of the art research to understand such a high dimensional data, they suffer from clutter and occlusion issues because of overplotting. In this study, we introduce an innovative visualization pipeline to address the aforementioned challenges to provide a clear depiction of noisy and less-noisy basis states. Specifically, our proposed visualization pipeline comprises three stages namely, low dimensional embedding, clustering, and violin plot visualization, to reduce visual clutter and effectively analyze high-dimensional noise distribution data. Our analysis uses quantum machine learning (QML) circuits as case study for drawing a distinction between noisy and less noisy basis states.

Senapati, Priyabrata [Kent State University]↗
DOE OSTI2025

EMPDF : inferring the Milky Way mass with data-driven distribution function in phase space

We introduce the emPDF (empirical distribution function), a novel dynamical modelling method that infers the gravitational potential from kinematic tracers with optimal statistical efficiency under the minimal assumption of steady state. emPDF determines the best-fitting potential by maximizing the similarity between instantaneous kinematics and the time-averaged phase-space distribution function (DF), which is empirically constructed from observation upon the theoretical foundation of oPDF (Han et al. 2016). This approach eliminates the need for presumed functional forms of DFs or orbit libraries required by conventional DF- or orbit-based methods. emPDF stands out for its flexibility, efficiency, and capability in handling observational effects, making it preferable to the popular Jeans equation or other minimal assumption methods, especially for the Milky Way (MW) outer halo where tracers often have limited sample size and poor data quality. We apply emPDF to infer the MW mass profile using Gaia DR3 data of satellite galaxies and globular clusters, obtaining enclosed masses of M (,r) = 26±8, 46±8, 90±13⁠, and 149±40 x 10 10 M ⊙ at r = 30, 50, 100⁠, and 200 kpc, respectively. These are consistent with the updated constraints from simulation-informed DF fitting (Li et al. 2020). While the simulation-informed DF offers superior precision owing to the additional information extracted from simulations, emPDF is independent of such supplementary knowledge and applicable to general tracer populations. emPDF is currently implemented for tracers with complete 6D kinematics within spherical potentials, but it can potentially be extended to address more general problems.

Astrophysics of Galaxies (astro-ph.GA)↗
DOE OSTI2022

Müllerian mimicry and the coloration patterns of sympatric coral snakes

Abstract Coral snakes in the genus Micrurus are venomous, aposematic organisms that signal danger to predators through vivid coloration. Previous studies found that they serve as models to several harmless species of Batesian mimics. However, the extent to which Micrurus species engage in Müllerian mimicry remains poorly understood. We integrate detailed morphological and geographical distribution data to investigate if coral snakes are Müllerian mimics. We found that coloration is spatially structured and that Micrurus species tend to be more similar where they co-occur. Though long supposed, we demonstrate for the first time that coral snakes might indeed be Müllerian mimics as they show some convergence in coloration patterns. Additionally, we found that the length of red-coloured rings in Micrurus is conserved, even at large geographic scales. This finding suggests that bright red rings may be under more substantial stabilizing selection than other aspects of coloration and probably function as a generalized signal for deterring predators.

Evolutionary Biology↗
DOE OSTI2022

Cross-Layered Distributed Data-Driven Framework for Enhanced Smart Grid Cyber-Physical Security

Smart Grid (SG) research and development has drawn much attention from academia, industry and government due to the great impact it will have on society, economics and the environment. Securing the SG is a considerably significant challenge due the increased dependency on communication networks to assist in physical process control, exposing them to various cyber-threats. In addition to attacks that change measurement values using False Data Injection (FDI) techniques, attacks on the communication network may disrupt the power system's real-time operation by intercepting messages, or by flooding the communication channels with unnecessary data. Addressing these attacks requires a cross-layer approach. In this paper a cross-layered strategy is presented, called Cross-Layer Ensemble CorrDet with Adaptive Statistics(CECD-AS), which integrates the detection of faulty SG measurement data as well as inconsistent network inter-arrival times and transmission delays for more reliable and accurate anomaly detection and attack interpretation. Numerical results show that CECD-AS can detect multiple False Data Injections, Denial of Service (DoS) and Man In The Middle (MITM) attacks with a high F1-score compared to current approaches that only use SG measurement data for detection such as the traditional physics-based State Estimation, Ensemble CorrDet with Adaptive Statistics strategy and other machine learning classification-based detection schemes.

cyber-physical security↗
DOE OSTI2020

A Survey of Singular Value Decomposition Methods for Distributed Tall/Skinny Data

The Singular Value Decomposition (SVD) is one of the most important matrix factorizations, enjoying a wide variety of applications across numerous application domains. In statistics and data analysis, the common applications of SVD inclue Principal Components Analysis (PCA) and regression. Usually these applications arise on data that has far more rows than columns, so-called "tall/skinny" matrices. In the big data analytics context, this may take the form of hundreds of millions to billions of rows with only a few hundred columns. There is a need, therefore, for fast, accurate, and scalable tall/skinny SVD implementations which can fully utilize modern computing resources. To that end, we present a survey of three different algorithms for computing the SVD for these kinds of tall/skinny data layouts using MPI for communication. We contextualize these with common big data analytics techniques. Finally, we present both CPU and GPU timing results from the Summit supercomputer, and discuss possible alternative approaches.

Schmidt, Drew↗