Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Pipeline”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Bayesian experimental design and parameter estimation for ultrafast spin dynamics

Abstract Advanced experimental measurements are crucial for driving theoretical developments and unveiling novel phenomena in condensed matter and materials physics, which often suffer from the scarcity of large-scale facility resources, such as x-ray or neutron scattering centers. To address these limitations, we introduce a methodology that leverages the Bayesian optimal experimental design paradigm to efficiently uncover key quantum spin fluctuation parameters from x-ray photon fluctuation spectroscopy (XPFS) data. Our method is compatible with existing theoretical simulation pipelines and can also be used in combination with fast machine learning surrogate models in the event that real-time simulations are unfeasible. Our numerical benchmarks demonstrate the superior performance in predicting model parameters and in delivering more informative measurements within limited experimental time. Our method can be adapted to many different types of experiments beyond XPFS and spin fluctuation studies, facilitating more efficient data collection and accelerating scientific discoveries.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Evaluation of High Level Waste Sludge Processing Behavior

The U.S. Department of Energy’s (DOE) Hanford Site has 177 underground storage tanks that contain wastes from past nuclear fuel reprocessing and waste-management operations. Over 20% of this waste is in the form of an insoluble sludge that will require slurry modification before its transfer to the Waste Treatment and Immobilization Plant (WTP). Specific WTP acceptance criteria for waste feed delivery describe the physical and chemical characteristics of the waste that must be met before the waste is transferred to the WTP. One challenging requirement relates to the undissolved solids (UDS) composition in a waste feed because the waste contains solid particles that settle, and their concentration and relative proportion can change during the transfer of the waste in individual batches. A key uncertainty is the ability to transfer and mix wastes with large variations in UDS concentrations and resulting settling rates. To address this uncertainty, a number of small scale mixing and settling tests have been conducted to determine the mobilization performance of variable chemistry simulants. Comparison of the size and density of the particulate for each simulant to that of southeast area Hanford sludge was made using metrics for particle mobilization, suspension, settling, and pipeline transfer where dependance on particle size and density may be different, including: 1. Settling velocity, 2. Critical shear stress for erosion, 3. Just-suspended impeller speed, and 4. Pipeline critical transport velocity. Existing high-level waste sludge data has shown the effect that increasing Al concentration has on resulting settled solids. This differential settling of particles in the sludge has the possibility of resulting in solids segregation during feed preparation and uneven particle distribution during pipeline transportation or mixer jet pump operations. Understanding the predictive capabilities of HLW solids settling and transport as well as potential remedies for addressing disparate sludge behaviors can help provide technical guidance during HLW flowsheet planning.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2021

The effect of protein mutations on drug binding suggests ensuing personalised drug selection

The advent of personalised medicine promises a deeper understanding of mechanisms and therefore therapies. However, the connection between genomic sequences and clinical treatments is often unclear. We studied 50 breast cancer patients belonging to a population-cohort in the state of Qatar. From Sanger sequencing, we identified several new deleterious mutations in the estrogen receptor 1 gene (ESR1). The effect of these mutations on drug treatment in the protein target encoded by ESR1, namely the estrogen receptor, was achieved via rapid and accurate protein–ligand binding affinity interaction studies which were performed for the selected drugs and the natural ligand estrogen. Four nonsynonymous mutations in the ligand-binding domain were subjected to molecular dynamics simulation using absolute and relative binding free energy methods, leading to the ranking of the efficacy of six selected drugs for patients with the mutations. Our study shows that a personalised clinical decision system can be created by integrating an individual patient’s genomic data at the molecular level within a computational pipeline which ranks the efficacy of binding of particular drugs to variant proteins.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

CMB power spectra and cosmological parameters from Planck PR4 with CamSpec

ABSTRACT We present angular power spectra and cosmological parameter constraints derived from the Planck PR4 (NPIPE) maps of the cosmic microwave background. NPIPE, released by the Planck Collaboration in 2020, is a new processing pipeline for producing calibrated frequency maps from Planck data. We have created new versions of the CamSpec likelihood using these maps and applied them to constrain the lambda cold dark matter (ΛCDM) model and single-parameter extensions. We find excellent consistency between NPIPE and the Planck 2018 maps at the parameter level, showing that the Planck cosmology is robust to substantial changes in the mapmaking. The lower noise of NPIPE leads to ~10 per cent tighter constraints, and we see both smaller error bars and a shift towards the ΛCDM values for beyond-ΛCDM parameters including ΩK and AL.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Towards Diverse and Representative Global Pretraining Datasets for Remote Sensing Foundation Models

The design of a pretraining dataset is emerging as a critical component for the generality of foundation models. In the remote sensing realm, large volumes of imagery and benchmark datasets exist that can be leveraged to pretrain foundation models, however using this imagery in absence of a well-crafted sampling strategy is inefficient and has the potential to create biased and less generalizable models. Here, we provide a discussion and vision for the curation and assessment of pretraining datasets for remote sensing geospatial foundation models. We highlight the importance of geographic, temporal, and image acquisition diversity and review possible strategies to enable such diversity at global scale. In addition to these characteristics, support for various spatial-temporal pretext tasks within the dataset is also critical. Ultimately, our primary objective is to place emphasis on and draw attention to the data curation stage of the foundation model development pipeline. By doing so, we think it is possible to reduce biases of geospatial foundation models, as well as enable broader generalization to downstream remote sensing tasks and applications.

Arndt, Jacob↗
DOE OSTI2024

Pipeline Hydrogen Decarbonization and Repurposing Analyzers (P-HyDRAs)

The Pipeline Hydrogen Decarbonization and Repurposing Analyzers (P-HyDRAs) are a set of prototype computational tools for simulating and optimizing midstream natural gas pipeline system operations subject to location and time-dependent hydrogen blending. The models can accurately resolve dynamic gas flows through large-scale pipeline networks using non-ideal gas equations of state. The codes can be used as decision support for planning and design decisions involving intra-day energy flow schedules as well as spatiotemporal economic values of natural gas, hydrogen, and net energy delivered to consumers while ensuring that pipeline hydraulic limitations, gas compressor station constraints, operational factors, and pre-existing shipping contracts are satisfied. The inputs to the codes are a model of the pipeline system as well as time-series data that specify boundary conditions on the network. For optimization, the code module requires price and quantity offers for natural gas and hydrogen and price and quantity bids for energy, which are used as time-dependent constraints in an optimal control problem. The outputs are time-series data that provide a predictive simulation of gas flows, mass fractions, and pressures, or with additional degrees of freedom give an approximately optimal solution for gas injections/withdrawals, compressor settings, and sensitivities to the objective function that provide locational values of energy.

Zlotnik, Anatoly↗
DOE OSTI2025

WizEM

The software provides a flexible pipeline for artificial intelligence (AI) assisted analysis of data collected from an electron microscope. This tool distinctly provides near real-time image quantification results during data collection, incorporating feedback from a user to tune performance.

Akers, Sarah [Pacific Northwest National Laborator↗
DOE OSTI2021

An optimized FM-index library for nucleotide and amino acid search

Abstract Background Pattern matching is a key step in a variety of biological sequence analysis pipelines. The FM-index is a compressed data structure for pattern matching, with search run time that is independent of the length of the database text. Implementation of the FM-index is reasonably complicated, so that increased adoption will be aided by the availability of a fast and flexible FM-index library. Results We present AvxWindowedFMindex (AWFM-index), a lightweight, open-source, thread-parallel FM-index library written in C that is optimized for indexing nucleotide and amino acid sequences. AWFM-index introduces a new approach to storing FM-index data in a strided bit-vector format that enables extremely efficient computation of the FM-index occurrence function via AVX2 bitwise instructions, and combines this with optional on-disk storage of the index’s suffix array and a cache-efficient lookup table for partial k-mer searches. The AWFM-index performs exact match count and locate queries faster than SeqAn3’s FM-index implementation across a range of comparable memory footprints. When optimized for speed, AWFM-index is $$\sim $$ ∼ 2–4x faster than SeqAn3 for nucleotide search, and $$\sim $$ ∼ 2–6x faster for amino acid search; it is also $$\sim $$ ∼ 4x faster with similar memory footprint when storing the suffix array in on-disk SSD storage. Conclusions AWFM-index is easy to incorporate into bioinformatics software, offers run-time performance parameterization, and provides clients with FM-index functionality at both a high-level (count or locate all instances of a query string) and low-level (step-wise control of the FM-index backward-search process). The open-source library is available for download at https://github.com/TravisWheelerLab/AvxWindowFmIndex.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Single-nuclei transcriptome analysis of the shoot apex vascular system differentiation in Populus

ABSTRACT Differentiation of stem cells in the plant apex gives rise to aerial tissues and organs. Presently, we lack a lineage map of the shoot apex cells in woody perennials – a crucial gap considering their role in determining primary and secondary growth. Here, we used single-nuclei RNA-sequencing to determine cell type-specific transcriptomes of the Populus vegetative shoot apex. We identified highly heterogeneous cell populations clustered into seven broad groups represented by 18 transcriptionally distinct cell clusters. Next, we established the developmental trajectories of the epidermis, leaf mesophyll and vascular tissue. Motivated by the high similarities between Populus and Arabidopsis cell population in the vegetative apex, we applied a pipeline for interspecific single-cell gene expression data integration. We contrasted the developmental trajectories of primary phloem and xylem formation in both species, establishing the first comparison of vascular development between a model annual herbaceous and a woody perennial plant species. Our results offer a valuable resource for investigating the principles underlying cell division and differentiation conserved between herbaceous and perennial species while also allowing us to examine species-specific differences at single-cell resolution.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Data for Impact of Non-Irrigation on 1G and 2G Bioethanol Potential of Oilcane Feedstock: A Field to Fuel Pipeline Study

This study evaluates the bioethanol potential in response to irrigation (IR) and non-irrigation (NIR) of oilcane (OC) during a seasonal drought prior harvest. The juice was extracted through mechanical pressing of stems and fermented by Ethanol Red® yeast to produce first-generation bioethanol. Hydrothermal pretreatment followed by enzymatic hydrolysis of bagasse was performed to produce monomeric sugars from structural carbohydrates. The hydrolysates were fermented with engineered yeast for second-generation bioethanol production. The irrigated oilcane juice (276.3 ± 8.9 g/L) constitutes higher sugar concentrations than non-irrigated oilcane juice (236.5 ± 2.2 g/L). The enzymatic hydrolysis of IR-OC and NIR-OC pretreated bagasse yielded similar concentrations of 247.5 ± 2.22 and 249.7 ± 4.98 g/L fermentable sugars. Industry-relevant bioethanol titers of ≥99 g/L and ≥75 g/L were achieved from juice and hydrolysates, respectively. Therefore, the non-irrigation regime did not impact the 1G and 2G bioethanol titers. However, the overall bioethanol yield can be lower due to the reduction of stem yield (8 %) per hectare.

Biomass Analytics↗
DOE OSTI2025

Verified, Archived Library of Inputs and Data (VALID) Online Repository [Slides]

VALID continues to serve as a library of high-quality models used for evaluating SCALE and nuclear data. Large number of cases are in the pipeline. Online repository is available for use. Future plans include simplifying the process for adding cases, including additional models outside the ICSBEP Handbook (HTC models, DNCSH models, International Handbook of Evaluated Reactor Physics Benchmark Experiments [IRPhE], SINBAD, etc.), and increasing the availability of models and results for external users.

KENO↗
DOE OSTI2024

Portable Parallel Algorithms and Frameworks for Exascale Graph Analytics

Graphs (or networks) are a tool used to model the interactions among various entities. Efficiently processing large graphs has recently attracted significant attention due to the applications of graphs in various domains, such as biology, chemistry, and cyber-security. Analyzing the structure and properties of these graphs is an important component of many scientific computing pipelines. With the explosion in the volume of data, graphs have become very large and can contain hundreds of billions of vertices and trillions of edges. Therefore, it is crucial to develop high-performance methods to enable graph analysis to be done quickly and energy-efficiently. Furthermore, these solutions should be highly parallel in order to take advantage of modern parallel machines. However, designing efficient solutions is not enough. With the wide variety of computing environments available, each with different programmability and performance characteristics, it is necessary to develop solutions that are portable in terms of both performance (i.e., provide theoretical guarantees) and programmability (i.e., provide high level abstractions).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

FALCON: Framework for Anomaly Detection in Industrial Control Systems

Industrial Control Systems (ICS) are used to control physical processes in critical infrastructure. These systems are used in a wide variety of operations such as water treatment, power generation and distribution, and manufacturing. While the safety and security of these systems are of serious concern, recent reports have shown an increase in targeted attacks aimed at manipulating physical processes to cause catastrophic consequences. This trend emphasizes the need for algorithms and tools that provide resilient and smart attack detection mechanisms to protect ICS. In this paper, we propose an anomaly detection framework for ICS based on a deep neural network. The proposed methodology uses dilated convolution and long short-term memory (LSTM) layers to learn temporal as well as long term dependencies within sensor and actuator data in an ICS. The sensor/actuator data are passed through a unique feature engineering pipeline where wavelet transformation is applied to the sensor signals to extract features that are fed into the model. Additionally, this paper explores four variations of supervised deep learning models, as well as an unsupervised support vector machine (SVM) model for this problem. The proposed framework is validated on Secure Water Treatment testbed results. This framework detects more attacks in a shorter period of time than previously published methods.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2020

TESS Hunt for Young and Maturing Exoplanets (THYME). II. A 17 Myr Old Transiting Hot Jupiter in the Sco-Cen Association

We present the discovery of a transiting hot Jupiter orbiting HIP 67522 (T {sub eff} ∼ 5650 K; M {sub *} ∼ 1.2M {sub ⊙}) in the 10–20 Myr old Sco-Cen OB association. We identified the transits in the TESS data using our custom notch filter planet search pipeline and characterize the system with additional photometry from Spitzer; spectroscopy from SOAR/Goodman, SALT/HRS, LCOGT/NRES, and SMARTS/CHIRON; and speckle imaging from SOAR/HRCam. We model the photometry as a periodic Gaussian process with transits to account for stellar variability and find an orbital period of 6.9596{sub −0.000015}{sup +0.000016} days and radius of 10.02{sub −0.53}{sup +0.54} R {sub ⊕}. We also identify a single transit of an additional candidate planet with radius 8.01{sub −0.71}{sup +0.75} R {sub ⊕} that has an orbital period of ≳23 days. The validated planet HIP 67522b is currently the youngest transiting hot Jupiter discovered and is an ideal candidate for transmission spectroscopy and radial velocity follow-up studies, while also demonstrating that some young giant planets either form in situ at small orbital radii or else migrate promptly from formation sites farther out in the disk.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

autoGEMM: Pushing the Limits of Irregular Matrix Multiplication on Arm Architectures

This paper presents an open-source library that pushes the limits of performance portability for irregular General Matrix Multiplication (GEMM) on the widely-used Arm architectures. Our library, autoGEMM, is designed to support a wide range of Arm processors: from edge devices to HPC-grade CPUs. autoGEMM generates optimized kernels for various hardware configurations by auto-combining fragments of autogenerated micro-kernels that employ hand-written optimizations to maximize computational efficiency. We optimize the kernel pipeline by tuning the register reuse and the data load/store overlapping. In addition, we use a dynamic tiling scheme to generate balanced tile shapes. Finally, we position autoGEMM on top of the TVM framework where our dynamic tiling scheme prunes the search space for TVM to identify the optimal combination of parameters for code optimization. Evaluations on five different classes of Arm chips demonstrate the advantages of autoGEMM. For small matrices, autoGEMM achieves 98% of peak and up to 2.0x speedup over state-of-the-art libraries such as LIBXSMM and LibShalom. For irregular matrices (i.e. tall skinny and long rectangles), autoGEMM is 1.3-2.0x faster than widely-used libraries such as OpenBLAS and Eigen. autoGEMM is publicly available at: https://github.com/wudu98/autoGEMM.

Wu, Du↗
DOE OSTI2021

Hardening DOE R&D Software Tools for Web-based Visualization SBIR Phase I Final Report

Ubiquitous web-based visualization is essential to delivering large-scale data visualization to various stakeholders, from the scientist to the board member. These stakeholders will not tolerate a stalled application or a pop-up window asking them to wait for the processing to complete. They require a responsive and interactive visualization environment with high-quality imagery suitable for detailed analysis and boardroom presentations. At Kitware, Inc., we have accomplished web visualization to this point, leveraging state-of-the-art tools like HTML5, CSS3, SVG, Canvas, and WebGL. Solutions that leverage a combination of these technologies are necessary to handle workloads that vary significantly in data size efficiently. However, it is not always practical to move large data to the web client for visualization. Kitware's ParaView as a Service combines client-side visualization using both distributed processing and remote rendering on big data impractical to move. Existing distributed processing and remote rendering solution's interactivity is below the expectations of web-based applications. Our project examined proposed solutions to the areas outlined above in ParaView as a Service. We have investigated concurrent pipelines, streaming images, progressive rendering, and optimization of algorithms and data movement to address these concerns. For the Phase I project, we completed the proposed work plan. As a result, the project produced three prototypes of essential importance for web visualization and the ParaView as a Service community. We created a simple desktop application for an interactive streamline placement prototype, a web-based interactive streamline placement prototype, and a web-based progressive rendering utilizing raytracing prototype. These prototypes relied on the hardening of emerging software toolkits funded by the Department of Energy (DOE) Advanced Scientific Computing Research (ASCR) program (such as ParaView, VTK-m, and Mochi). We blended these components into web-based visualization prototypes that meet the industry's expectations for interactivity and responsiveness. The Phase I project had four essential focus areas: 1. Develop prototype ParaView as a Service backend server using asynchronous, non-blocking design principles. 2. Develop a prototype web application that uses the ParaView as a Service backend server for remote data visualization. 3. Implement image streaming with encoding/compression and progressive rendering capabilities in the proposed platform. 4. Evaluate the prototype developed and summarize observations, including the challenges and pitfalls of our approach. After our successful completion of Phase I, we are strongly positioned to propose a successful Phase II project.

Geveci, Berk↗
DOE OSTI2023

From BEYONDPLANCK to COSMOGLOBE: Preliminary WMAP Q -band analysis

We present the first application of the COSMOGLOBE analysis framework by analyzing nine-year WMAP time-ordered observations that uses similar machinery to that of BEYONDPLANCK for the Planck Low Frequency Instrument (LFI). We analyzed only the Q-band (41 GHz) data and report on the low-level analysis process based on uncalibrated time-ordered data to calibrated maps. Most of the existing BEYONDPLANCK pipeline may be reused for WMAP analysis with minimal changes to the existing codebase. The main modification is the implementation of the same preconditioned biconjugate gradient mapmaker used by the WMAP team. Producing a single WMAP Q1-band sample requires 22 CPU-hrs, which is slightly more than the cost of a Planck 44 GHz sample of 17 CPU-hrs; this demonstrates that a full end-to-end Bayesian processing of the WMAP data is computationally feasible. In general, our recovered maps are very similar to the maps released by the WMAP team, although with two notable differences. In terms of temperature, we find a ~2 μK quadrupole difference that most likely is caused by different gain modeling, while in polarization we find a distinct 2.5 μK signal that has been previously referred to as poorly measured modes by the WMAP team. In the COSMOGLOBE processing, this pattern arises from temperature-to-polarization leakage from the coupling between the CMB Solar dipole, transmission imbalance, and sidelobes. No traces of this pattern are found in either the frequency map or TOD residual map, suggesting that the current processing has succeeded in modeling these poorly measured modes within the assumed parametric model by using Planck information to break the sky-synchronous degeneracies inherent in the WMAP scanning strategy.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Combined tracer analysis for DESI 2024 BAO

This paper demonstrates how the Dark Energy Spectroscopic Instrument (DESI) Data Release 1 (DR1) and future baryon acoustic oscillations (BAO) analyses can optimally combine overlapping tracers (galaxies of distinct types) in the same redshift range. We make a unified catalog of Luminous Red Galaxies (LRGs) and Emission Line Galaxies (ELGs) in the redshift range 0.8 < z < 1.1 and investigate the impact on the BAO constraints. DESI DR1 contains ∼ 30% of the final DESI LRG sample and less than 25% of the final ELG sample, and the combination of LRGs and ELGs increases the number density and reduces the shot noise. We developed a pipeline to merge the overlapping tracers using galaxy bias as an approximately optimal weight and tested the pipeline on a suite of Abacus simulations, calibrated on the final version of the DESI Early Data Release. When applying our pipeline to the DESI DR1 catalog, we find an improvement in the BAO constraints of 11% for α iso and ∼ 7.0% for α AP consistent with our findings in mock catalogs. Our analysis was integrated into the DESI DR1 BAO analysis to produce theLRG+ELG result in the 0.8 < z < 1.1 redshift bin, which provided the most precise BAO measurement from DESI DR1 with a 0.86% constraint on the BAO distance scale and a 9.1σ detection of the isotropic BAO feature.

baryon acoustic oscillations↗