Search NASA⌕ Search

SEARCH · Search NASA

Results for “large scale network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 919 records · Page 51
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Risk-Aware Measurement Synchronization and Recovery for DSSE With Heterogeneous Data Sources

Power distribution systems are increasingly integrating heterogeneous sensors with varying data reporting rates and types, which pose challenges to achieving observability at the desired temporal resolution of distribution system state estimation (DSSE). Multisensor failures caused by extreme events exacerbate these issues, introducing substantial uncertainties into DSSE. This article proposes a novel solution to these challenges by ensuring high-resolution system observability despite heterogeneous data sources and multisensor failures. First, a deep learning architecture combining long short-term memory (LSTM) and graph convolutional network (GCN) is employed to synchronize meters with different reporting rates, aiming to achieve system observability. A random-walk-model-based approach is introduced to generate pseudo-measurements while properly characterizing their uncertainties under multisensor failures. Finally, a disaster-risk-informed observability metric (RiOM) is defined to quantify the uncertainty associated with state estimation results. The proposed framework offers deeper insights into the system observability on the fly compared with conventional analysis. The effectiveness of the framework is demonstrated on an IEEE standard test case and a large-scale real-world distribution feeder in mid-Minnesota in the U.S.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Multiplicative Shot-Noise: A New Route to Stability of Plastic Networks

Fluctuations of synaptic weights, among many other physical, biological, and ecological quantities, are driven by coincident events of two “parent” processes. Here we propose a multiplicative shot-noise model that can capture the behaviors of a broad range of such natural phenomena, and analytically derive an approximation that accurately predicts its statistics. We apply our results to study the effects of a multiplicative synaptic plasticity rule that was recently extracted from measurements in physiological conditions. Using mean-field theory analysis and network simulations, we investigate how this rule shapes the connectivity and dynamics of recurrent spiking neural networks. The multiplicative plasticity rule is shown to support efficient learning of input stimuli, and it gives a stable, unimodal synaptic-weight distribution with a large fraction of strong synapses. The strong synapses remain stable over long times but do not “run away.” Our results suggest that the multiplicative shot-noise offers a new route to understand the tradeoff between flexibility and stability in neural circuits and other dynamic networks.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Non-invasive authentication of mail packages using nuclear quadrupole resonance spectroscopy

The international postal network is one of the most widely used methods for correspondence throughout the world. Most postal traffic across the globe consists of legitimate interpersonal, business-consumer, and business-business communications. However, the global postal system is also utilized for criminal activity. In particular, it is often utilized to ship and distribute contraband, including illegal psychoactive drugs such as fentanyl and heroin, to consumers. Existing technological solutions are capable of identifying synthetic opioids and other illegal drugs within packages, but are accompanied by several disadvantages that make them unsuitable for large-scale authentication of international mail traffic. This paper presents a novel method for non-invasive authentication of mail packages that overcomes these challenges. The approach uses nuclear quadrupole resonance (NQR) spectroscopy to detect and quantify the presence of known active pharmaceutical ingredients (APIs) within the package. It has been experimentally demonstrated using a bench top prototype. Test results from a variety of package types demonstrate the effectiveness of the proposed authentication approach.

42 ENGINEERING↗
DOE OSTI2025

Scalable edge clustering of dynamic graphs via weighted line graphs

Timestamped relational datasets consisting of records (or connections) between pairs of entities are ubiquitous in network science. For applications like peer-to-peer communication, email, various social network interactions, and computer network security, it is useful to organize these records into groups based on how and when they are occurring. Weighted line graphs offer a natural way to model how records are related in such datasets but for large real-world graph topologies, building and utilizing the line graph is prohibitively expensive. Here, we present the framework to cluster the edges of a dynamic graph via the associated line graph that contains two major contributions. The first is a method to work with the line graph implicitly and the second is a distributed scale implementation of an agglomerative hierarchical graph clustering algorithm. We outline a novel hierarchical dynamic graph edge clustering approach that efficiently breaks massive relational datasets into small sets of edges containing events at various timescales. This is in stark contrast to traditional graph clustering algorithms that prioritize highly connected (clique-like) community structures. Our approach relies on constructing a sufficient subgraph of a weighted line graph and applying a hierarchical agglomerative clustering. This approach is related to scalable techniques from spatial clustering, nonlinear-dimension reduction, topological data analysis, and draws particular inspiration from HDBSCAN. As an edge clustering, this method yields an overlapping node clustering. Our algorithm is parallelizable and we demonstrate efficient clustering of a billion-scale, real-world dynamic graph into small edge sets that correlate in topology and time. The entire clustering process for a graph with tens of billions of edges takes just a few minutes of run time on 256 nodes of a distributed compute environment. We argue how the output of the edge clustering is useful for a multitude of data visualization and powerful machine learning tasks, both involving the original massive dynamic graph data and metadata associated with the nodes and edges. Finally, we describe how this approach can be extended to dynamic hypergraphs and dynamic graphs/hypergraphs with unstructured data living on vertices and edges.

Data Analysis↗
DOE OSTI2020

Streaming Data Reorganization at Scale with DeltaFS Indexed Massive Directories

We report complex storage stacks providing data compression, indexing, and analytics help leverage the massive amounts of data generated today to derive insights. It is challenging to perform this computation, however, while fully utilizing the underlying storage media. This is because, while storage servers with large core counts are widely available, single-core performance and memory bandwidth per core grow slower than the core count per die. Computational storage offers a promising solution to this problem by utilizing dedicated compute resources along the storage processing path. We present DeltaFS Indexed Massive Directories (IMDs), a new approach to computational storage. DeltaFS IMDs harvest available (i.e., not dedicated) compute, memory, and network resources on the compute nodes of an application to perform computation on data. We demonstrate the efficiency of DeltaFS IMDs by using them to dynamically reorganize the output of a real-world simulation application across 131,072 CPU cores. DeltaFS IMDs speed up reads by 1,740x while only slightly slowing down the writing of data during simulation I/O for in situ data processing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Mechanical Ventilator Milano (MVM): A Novel Mechanical Ventilator Designed for Mass Scale Production in Response to the COVID-19 Pandemics

We present here the design of the Mechanical Ventilator Milano (MVM), a novel mechanical ventilator designed for mass scale production in response to the COVID-19 pandemics, to compensate for the dramatic shortage of such ventilators in many countries. This ventilator is an electro-mechanical equivalent of the old, reliable Manley Ventilator. Our design is optimized to permit large sale production in short time and at a limited cost, relying on off-the-shelf components, readily available worldwide from hardware suppliers. Operation of the MVM requires only a source of compressed oxygen (or compressed medical air) and electrical power. The MVM control and monitoring unit can be connected and networked via WiFi so that no additional electrical connections are necessary other than the connection to the electrical power. At this stage the MVM is not a certified medical device. Construction of the first prototypes is starting with a team of engineers, scientists and computing experts. The purpose of this paper is to disseminate the conceptual design of the MVM broadly and to solicit feedback from the scientific and medical community to speed the process of review, improvement and possible implementation.

Galbiati, C.↗
DOE OSTI2024

Robust training of machine learning interatomic potentials with dimensionality reduction and stratified sampling

Abstract Machine learning interatomic potentials (MLIPs) enable accurate simulations of materials at scales beyond that accessible by ab initio methods and play an increasingly important role in the study and design of materials. However, MLIPs are only as accurate and robust as the data on which they are trained. Here, we present DImensionality-Reduced Encoded Clusters with sTratified (DIRECT) sampling as an approach to select a robust training set of structures from a large and complex configuration space. By applying DIRECT sampling on the Materials Project relaxation trajectories dataset with over one million structures and 89 elements, we develop an improved materials 3-body graph network (M3GNet) universal potential that extrapolates more reliably to unseen structures. We further show that molecular dynamics (MD) simulations with the M3GNet universal potential can be used instead of expensive ab initio MD to rapidly create a large configuration space for target systems. We combined this scheme with DIRECT sampling to develop a reliable moment tensor potential for titanium hydrides without the need for iterative augmentation of training structures. This work paves the way for robust high-throughput development of MLIPs across any compositional complexity.

Qi, Ji (ORCID:0000000158089931)↗
DOE OSTI2024

Direct numerical simulation of open-channel flow over a heterogeneous particle bed at low relative submergence

In this article, this study investigates turbulent open-channel flows over beds of irregularly arranged particles, using direct numerical simulations at a friction Reynolds number of Re T = 300⁠. Two distinct cases are examined: a polydisperse bed (P800) composed of multiple layers of randomly distributed spheres of varying sizes, and a monodisperse bed (M1015) formed by a random distribution of uniform sized spheres, with a bottommost single layer of varied-sized particles to introduce realistic randomness. Our investigation unveils a rich network of low- and high-speed streaks within the flow field, exhibiting distinctive behaviors in different bed configurations. The P800 case presents a poorly organized flow pattern induced by the varied particle sizes and arrangements, while the M1015 case shows a more regular flow pattern, marked by larger streaks. We also observe that total wall shear stress is substantially influenced by surface roughness-induced drag, extending beyond the effects documented in existing studies of open-channel flows. The present study reveals intricate secondary flow patterns over irregular particle beds. Large-scale circulations are discerned around particle crests in the P800 case and localized circulations with increased turbulence in the M1015 case. Furthermore, analysis of Reynolds stress tensor components indicates that roughness disrupts coherent turbulent eddies, consequently mitigating peak stress. We quantify correlations between drag force and local fluid velocity fluctuations. Notably, a larger deviation in drag is observed in the P800 case compared to M1015, accentuating the influence of particle size and distribution on fluid–particle interactions.

42 ENGINEERING↗
DOE OSTI2021

The genotype-phenotype landscape of an allosteric protein

Allostery is a fundamental biophysical mechanism that underlies cellular sensing, signaling, and metabolism. Yet a quantitative understanding of allosteric genotype-phenotype relationships remains elusive. Here, we report the large-scale measurement of the genotype-phenotype landscape for an allosteric protein: the lac repressor from Escherichia coli , LacI. Using a method that combines long-read and short-read DNA sequencing, we quantitatively measure the dose-response curves for nearly 10 5 variants of the LacI genetic sensor. The resulting data provide a quantitative map of the effect of amino acid substitutions on LacI allostery and reveal systematic sequence-structure-function relationships. We find that in many cases, allosteric phenotypes can be quantitatively predicted with additive or neural-network models, but unpredictable changes also occur. For example, we were surprised to discover a new band-stop phenotype that challenges conventional models of allostery and that emerges from combinations of nearly silent amino acid substitutions.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Identifying precursors of daily to seasonal hydrological extremes over the USA using deep learning techniques and climate model ensembles

Focal Area(s): We focus on two areas of crosscutting interest for DOE: 1) predictability of extreme precipitation and drought in the USA and 2) the integration of climate models with new AI tools, such as convolutional neural networks (CNN) and methods to understand their output (e.g. layer-wise relevance propagation; LRP). This project fits into focus area 3 of this call for white paper using AI to gain insight from complex data, including explainable AI tools. Science Challenge: Predicting hydrological extremes is important due to their impacts on people, agriculture and infrastructure. This prediction is difficult due to the infrequent occurrence of extremes and their complexity. However, extreme events can be related to more predictable conditions in the ocean, such as El Nino, long-term soil moisture or large scale modes of climate variability, such as the North Atlantic Oscillation (NAO).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Determining the biogeochemical transformations of organic matter composition in rivers using molecular signatures

Inland waters are hotspots for biogeochemical activity, but the environmental and biological factors that govern the transformation of organic matter (OM) flowing through them are still poorly constrained. Here we evaluate data from a crowdsourced sampling campaign led by the Worldwide Hydrobiogeochemistry Observation Network for Dynamic River Systems (WHONDRS) consortium to investigate broad continental-scale trends in OM composition compared to localized events that influence biogeochemical transformations. Samples from two different OM compartments, sediments and surface water, were collected from 97 streams throughout the Northern Hemisphere and analyzed to identify differences in biogeochemical processes involved in OM transformations. By using dimensional reduction techniques, we identified that putative biogeochemical transformations and microbial respiration rates vary across sediment and surface water along river continua independent of latitude (18°N–68°N). In contrast, we reveal small- and large-scale patterns in OM composition related to local (sediment vs. water column) and reach (stream order, latitude) characteristics. These patterns lay the foundation to modeling the linkage between ecological processes and biogeochemical signals. We further showed how spatial, physical, and biogeochemical factors influence the reactivity of the two OM pools in local reaches yet find emergent broad-scale patterns between OM concentrations and stream order. OM processing will likely change as hydrologic flow regimes shift and vertical mixing occurs on different spatial and temporal scales. As our planet continues to warm and the timing and magnitude of surface and subsurface flows shift, understanding changes in OM cycling across hydrologic systems is critical, given the unknown broad-scale responses and consequences for riverine OM.

rivers↗
DOE OSTI2024

Detecting Anomalous Images in Astronomical Datasets

Abstract Environmental and instrumental conditions can cause anomalies in astronomical images, which can potentially bias all kinds of measurements if not excluded. Detection of the anomalous images is usually done by human eyes, which is slow and sometimes not accurate. This is an important issue in weak lensing studies, particularly in the era of large-scale galaxy surveys, in which image qualities are crucial for the success of galaxy shape measurements. In this work we present two automatic methods for detecting anomalous images in astronomical data sets. The anomalous features can be divided into two types: one is associated with the source images, and the other appears on the background. Our first method, called the entropy method, utilizes the randomness of the orientation distribution of the source shapes and the background gradients to quantify the likelihood of an exposure being anomalous. Our second method involves training a neural network (autoencoder) to detect anomalies. We evaluate the effectiveness of the entropy method on the Canada–France–Hawaii Telescope Lensing Survey (CFHTLenS) and Dark Energy Camera Legacy Survey (DECaLS DR3) data. In CFHTLenS, with 1171 exposures, the entropy method outperforms human inspection by detecting 12 of the 13 anomalous exposures found during human inspection and uncovering 10 new ones. In DECaLS DR3, with 17112 exposures, the entropy method detects a significant number of anomalous exposures while keeping a low false-positive rate. We find that although the neural network performs relatively well in detecting source anomalies, its current performance is not as good as the entropy method.

Astronomy & Astrophysics↗
DOE OSTI2023

Simulation study of the effects of polymer network dynamics and mesh confinement on the diffusion and structural relaxation of penetrants

The diffusion of small molecular penetrants through polymeric materials represents an important fundamental problem, relevant to the design of materials for applications such as coatings and membranes. Polymer networks hold promise in these applications because dramatic differences in molecular diffusion can result from subtle changes in the network structure. In this paper, we use molecular simulation to understand the role that cross-linked network polymers have in governing the molecular motion of penetrants. By considering the local, activated alpha relaxation time of the penetrant and its long-time diffusive dynamics, we can determine the relative importance of activated glassy dynamics on penetrants at the segmental scale vs entropic mesh confinement on penetrant diffusion. We vary several parameters, such as the cross-linking density, temperature, and penetrant size, to show that cross-links primarily affect molecular diffusion through the modification of the matrix glass transition, with local penetrant hopping at least partially coupled to the segmental relaxation of the polymer network. This coupling is very sensitive to the local activated segmental dynamics of the surrounding matrix, and we also show that penetrant transport is affected by dynamic heterogeneity at low temperatures. To contrast, only at high temperatures and for large penetrants or when the dynamic heterogeneity effect is weak, does the effect of mesh confinement become significant, even though penetrant diffusion more broadly empirically follows similar trends as established models of mesh confinement-based transport.

Chemistry↗
DOE OSTI2023

Machine Learning Surrogates of a Fuel Matrix Degradation Process Model for Performance Assessment of a Nuclear Waste Repository

Spent nuclear fuel repository simulations are currently not able to incorporate detailed fuel matrix degradation (FMD) process models due to their computational cost, especially when large numbers of waste packages breach. The current paper uses machine learning to develop artificial neural network and k-nearest neighbor regression surrogate models that approximate the detailed FMD process model while being computationally much faster to evaluate. Further, using fuel cask temperature, dose rate, and the environmental concentrations of CO 3 2- , O 2 , Fe 2+ , and H 2 as inputs, these surrogates show good agreement with the FMD process model predictions of the UO 2 degradation rate for conditions within the range of the training data. A demonstration in a full-scale shale repository reference case simulation shows that the incorporation of the surrogate models captures local and temporal environmental effects on fuel degradation rates while retaining good computational efficiency.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2022

The weathered bedrock vadose zone: A hidden control on water availability in the western United States (Final Technical Report)

This report summarizes the research completed during the project entitled "The weathered bedrock vadose zone: A hidden control on water availability in the western United States" (Award: DE-SC0018039). The report covers work completed between 9/15/17 and 9/14/20. The objective of the exploratory project work was to evaluate the overarching hypothesis that the subsurface underlying actively eroding landscapes is systematically structured as a result of a balance between weathering and erosion and that this structure leads to dynamic moisture storage within weathered rock. To accomplish project objectives, we characterized hillslope weathering profile structure and conducted subsurface moisture monitoring across four study sites with comparable bedrock types but differing erosion rate and hydroclimate: The Angelo Coast Range Reserve (ACRR), Sagehorn-Russell Ranch (SRR), Antelope Valley Ranch (AVR), and the LBNL Watershed Function Scientific Focus Area (SFA). New characterization and monitoring infrastructure at the AVR and SFA sites was established through this project and enables long-term monitoring of hydrologic and carbon fluxes at the sites for comparison with long-term monitoring in other sites. Deep drilling, downhole logging, and characterization of chemistry and porosity of rock recovered from drilling reveals two weathering fronts across the sites. A shallower front is marked by the extent of pervasive fracturing and oxidation of pyrite and organic carbon. A deeper weathering front is marked by the appearance of open fractures and discoloration. At the AVR site, multiple ridge tops of different hillslope length were investigated to establish the relationship between the weathering front and topography. Drilling reveals that the fraction of the hillslope relief that is weathered scales with hillslope length, consistent with the theoretical framework that guided the project relating groundwater drainage with landscape evolution. With the establishment of new boreholes, we have documented significant water storage in weathered bedrock using low-field borehole nuclear magnetic resonance (NMR) across all four sites. In the upper 4 m of bedrock, the average water contents at ACRR, AVR, SRR, and SFA are approximately 25%, 16%, 25%, 23% respectively. At the ACRR site, where long-term monitoring via neutron probe surveys and time-domain transmission sensors are available, we have demonstrated that NMR reliably records moisture content and importantly, moisture content dynamics in weathered bedrock. Across all sites, our NMR monitoring has also revealed that a considerable fraction of the water storage occurs in fractures or pores that are significantly larger than the fine-grained bedrock matrix of the parent rock. To identify the proportion of water storage occurring in fractures and large pores, we analyzed the NMR signal by evaluating both the sum of echoes and an inversion of the NMR signal for a distribution of T2 relaxation times. Both methods reveal that at least 20% of the water storage occurs in fractures and larger pores across all sites. To constrain the interpretation of the in-situ field-scale NMR measurements, we compare the NMR response in the field to independent pore-structure information obtained via helium pycnometry. Results support the inference that dynamic seasonal water storage is dominantly restricted to the fracture network, which can reach 7% of the total volume. This dynamic range in water content is comparable to that of some soils, underscoring the need to understand how this region of the Earth System is structured, and functions as a hydrologic reservoir.

58 GEOSCIENCES↗
DOE OSTI2020

Feedback regulation of crystal growth by buffering monomer concentration

Abstract Crystallization is a ubiquitous means of self-assembly that can organize matter over length scales orders of magnitude larger than those of the monomer units. Yet crystallization is notoriously difficult to control because it is exquisitely sensitive to monomer concentration, which changes as monomers are depleted during growth. Living cells control crystallization using chemical reaction networks that offset depletion by synthesizing or activating monomers to regulate monomer concentration, stabilizing growth conditions even as depletion rates change, and thus reliably yielding desired products. Using DNA nanotubes as a model system, here we show that coupling a generic reversible bimolecular monomer buffering reaction to a crystallization process leads to reliable growth of large, uniformly sized crystals even when crystal growth rates change over time. Buffering could be applied broadly as a simple means to regulate and sustain batch crystallization and could facilitate the self-assembly of complex, hierarchical synthetic structures.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Real-time signal detection for Cyclotron Radiation Emission Spectroscopy measurements using antenna arrays

Cyclotron Radiation Emission Spectroscopy (CRES) is a technique for precision measurement of the energies of charged particles, which is being developed by the Project 8 Collaboration to measure the neutrino mass using tritium beta-decay spectroscopy. Project 8 seeks to use the CRES technique to measure the neutrino mass with a sensitivity of 40 meV, requiring a large supply of tritium atoms stored in a multi-cubic meter detector volume. Antenna arrays are one potential technology compatible with an experiment of this scale, but the capability of an antenna-based CRES experiment to measure the neutrino mass depends on the efficiency of the signal detection algorithms. Here, in this paper, we develop efficiency models for three signal detection algorithms and compare them using simulations from a prototype antenna-based CRES experiment as a case-study. The algorithms include a power threshold, a matched filter template bank, and a neural network based machine learning approach, which are analyzed in terms of their average detection efficiency and relative computational cost. It is found that significant improvements in detection efficiency and, therefore, neutrino mass sensitivity are achievable, with only a moderate increase in computation cost, by utilizing either the matched filter or machine learning approach in place of a power threshold, which is the baseline signal detection algorithm used in previous CRES experiments by Project 8.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Signal Processing Based Method for Real-Time Anomaly Detection in High-Performance Computing

Performance anomalies can manifest as irregular execution times or abnormal execution events for many reasons, including network congestion and resource contention. Detecting such anomalies in real-time by analyzing the details of performance traces at scale is impractical due to the sheer volume of data High-Performance Computing (HPC) applications produce. In this paper, we propose formulating HPC performance anomaly detection as a signal-processing problem where anomalies can be treated as noise. We evaluate our proposed method in comparison with two other commonly used anomaly detection techniques of varying complexity based on their detection accuracy and scalability. Since real-time in-situ anomaly detection at a large scale requires lightweight methods that can handle a large volume of streaming data, we find that our proposed method provides the best trade-off. We then implement the proposed method in Chimbuko, the first online, distributed, and scalable workflow-level performance trace analysis framework. We compare our proposed signal-based anomaly detection algorithm with two other methods using a function of their accuracy, F1 score, and detection overhead. Our experiments demonstrate that our proposed approach achieves a 99% improvement for the benchmark datasets and a 93% improvement with Chimbuko traces.

99 GENERAL AND MISCELLANEOUS↗