Search NASA⌕ Search

SEARCH · Search NASA

Results for “Science Pipeline”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Ramdb: The NASA Raman Spectral Database (version 1.00).

Given that, in most instances, minimal sample preparation is required and due to its contactless instrument design, Raman spectroscopy is one of the most versatile vibrational spectroscopic techniques for the chemical analysis of environmental and biological specimens. The diversity of applications of Raman spectroscopy ranges anywhere from art [1] to planetary science missions [2]. The advancement in the use of Raman spectroscopy in Solar System missions, notably in post-mission sample return analysis, requires a spectral library holding the broad range of specimens that could be found in Solar System sources. For this purpose, we have initiated the development of a Raman spectral database (Ramdb) at NASA Ames Research Center. Currently, the database includes experimental and theoretical Raman spectra of PAHs [3, 4], as well as laboratory Raman spectra of amino acids, carbon allotropes, minerals, and analogs relevance to Earth Sciences [5], Exobiology [6], Planetary [7], and Astrochemistry [8] to name just a few examples. Ramdb can be found on the web at www.astrochemistry.org/ramdb, where raw and processed Raman spectra can be downloaded in CSV format. The laboratory Raman spectra are measured using a laser Raman spectrometer (JASCO NRS-5500-532QRI). The Raman instrument is equipped with three excitation lasers, with wavelengths of 405, 532, and 785 nm. A clean silicon substrate is used as the internal standard for wavenumber calibration. Powdered samples were prepared (microscopic >10 um, grounded microscopic < 10 um) on glass slides. Some raw data exhibited a background signal arising as a combination of laser-induced fluorescence from the sample. To correct this background, we developed a Python pipeline that uses open-source Python libraries. Ramdb provides both raw and processed (using Python pipeline) data, which includes tabulated Raman shift transitions and other measurement details. The theoretical Raman band positions of PAHs (pyrene monomers and tetramer clusters) were computed using density functional theory (DFT) with the help of the Gaussian 16 suite of programs [9]. In the near future, Ramdb will serve as a repository of Raman spectral data from Laboratory Astrophysics and Planetary Science experiments involving the irradiation of organic compounds under simulated space and planetary conditions. In addition, online and offline tools will be developed for utilising the database for comparison to the user’s sample.

N Punnakayathil↗
DOE OSTI2024

Intro to HPC Bootcamp: Engaging New Communities Through Energy Justice Projects

The U.S. Department of Energy (DOE) is a long-standing leader in research and development of high-performance computing (HPC) in the pursuit of science. However, we face daunting challenges in fostering a robust and diverse HPC workforce. Basic HPC is not typically taught at early stages of students' academic careers, and the capacity and knowledge of HPC at many institutions are limited. Even so, such topics are prerequisites for advanced training programs, internships, graduate school, and ultimately for careers in HPC. To help address this challenge, as part of the DOE Exascale Computing Project's Broadening Participation Initiative, we recently launched the Introduction to HPC Training and Workforce Pipeline Program to provide accessible introductory material on HPC, scalable AI, and analytics. We describe the Intro to HPC Bootcamp, an immersive program designed to engage students from underrepresented groups as they learn foundational HPC skills. Here, the program takes a novel approach to HPC training by turning the traditional curriculum upside down. Instead of focusing on technology and its applications, the bootcamp focuses on energy justice to motivate the training of HPC skills through project-based pedagogy and real-life science stories. Additionally, the bootcamp prepares students for internships and future careers at DOE labs. The first bootcamp, hosted by the advanced computing facilities at Argonne, Lawrence Berkeley, and Oak Ridge National Labs and organized by Sustainable Horizons Institute, took place in August 2023.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2022

Optimal high-throughput virtual screening pipeline for efficient selection of redox-active organic materials

As global interest in renewable energy continues to increase, there has been a pressing need for developing novel energy storage devices based on organic electrode materials that can overcome the shortcomings of the current lithium-ion batteries. One critical challenge for this quest is to find materials whose redox potential (RP) meets specific design targets. In this study, we propose a computational framework for addressing this challenge through the effective design and optimal operation of a high-throughput virtual screening (HTVS) pipeline that enables rapid screening of organic materials that satisfy the desired criteria. Starting from a high-fidelity model for estimating the RP of a given material, we show how a set of surrogate models with different accuracy and complexity may be designed to construct a highly accurate and efficient HTVS pipeline. We demonstrate that the proposed HTVS pipeline construction and operation strategies substantially enhance the overall screening throughput.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Helium recovery system at IB3a

The increasing need for optimal and sustainable use of cryogenic resources to support Fermilab’s scientific mission has highlighted the necessity of improving the Laboratory’s helium management practices. An assessment of cryogenic test facilities identified the Technical Division’s Industrial Building 3a (IB3A) as a key site requiring upgrades to integrate a helium recovery system. The IB3A facility is essential for characterizing and testing superconductors, cables, and coils for various R&D projects, including the US High-Luminosity LHC Accelerator Upgrade Project (AUP), Mu2e, and other external collaborations. Currently, the facility relies on 500 L helium Dewars and vents the vaporized helium directly into the atmosphere, leading to significant helium loss. Given the non-renewable nature of helium, recovering and reusing this resource is critical for the sustainability of Fermilab’s operations. To address this challenge, a project has been initiated to connect IB3A to an existing helium purification station and refrigeration system located in another building via a dedicated pipeline pass over the roof of several buildings. This solution will enable the efficient capture of vented helium, its transfer to the purification station, and subsequent liquefaction for reuse in future operations. The project includes a detailed design phase, specifying the pipeline route, flow control mechanisms, and integration with the existing cryogenic infrastructure, followed by phased implementation and commissioning. By implementing this pipeline connection and upgrading IB3A, Fermilab aims to significantly reduce helium waste, lower operational costs, and align with its commitment to sustainability. This initiative provides a model for resource-efficient cryogenic operations and reinforces the Laboratory’s capacity to support its science mission for the long term.

Porwisiak, D. [Fermilab]↗
NASA NTRS2018

NASA Pilot-Engaged Expert Response Using IBM Watson Technology: Prototype Evaluation of Knowledge Retrieval System

NASA Langley Research Center and IBM have been investigating the use of IBM Watson technology in aerospace research and development. One application of Watson technology is the Pilot-Engaged Expert Response (PEER) use case. The PEER system is envisioned as an in-cockpit advisor that will act as a source of situationally-relevant information for pilots and other flight crew members to assist in decision making about real-time events and situations that arise in the course of aircraft operations. PEER will make available vast stores of knowledge and information quickly and directly, putting important informational resources where they are needed most. IBM has worked with NASA to develop an architecture and articulate a roadmap for the development of the PEER system. That vision is built around Watson Discovery Advisor (WDA) software solution, derived from IBM's Jeopardy!-winning automatic question answering system. PEER makes use of WDA's sophisticated question-answering capabilities as its core, adding important User Interface components and other customizations for the cockpit environment, including communication with flight systems and other external data sources. The development plan for PEER includes four development stages, with the current project constituting the first phase. In this project, a prototype instance of PEER was successfully adapted to the aviation domain, enabling users to ask questions about aviation topics and receive useful and accurate answers to these questions. Major tasks accomplished include the development of procedures for domain adaptation through automatic lexicon extraction from domain glossaries; generation of question-answer training data which was used to train the system; and assessment of the effectiveness of domain adaptation, which showed a dramatic improvement in the ability of the PEER system to answer domain-relevant questions. In addition, the vision for the PEER system was pushed forward by the articulation of a plan for the automatic enhancement of question-answering with contextual information. This initial phase focused on two main goals: 1) the targeted domain adaptation of the underlying WDA system to the aviation domain; and, 2) the design of the software systems needed to leverage flight-contextual data. Domain adaptation of the WDA system proceeds via three main activities: Domain data ingestion, lexical customization and model training. A textual corpus consisting of 1,147 individual documents with more than 7.5 million words of text was ingested into the system and this served as the basis of all further development. A domain lexicon of over 3,500 aviation-domain terms was semi-automatically generated from domain documents and used to train the system. In addition, a set of over 500 question-answer (QA) pairs relevant to the PEER use case was developed; these were used to train and assess the system. These important first steps established the basis for the PEER system. In addition, steps were taken towards the integration of the PEER system into the cockpit environment with the development of a functional design for the Contextual Data Augmentation (CDA) subsystem. This subsystem brings to bear contextual data to improve system responses. It has three main submodules: the Contextual Data Collection module, the Contextual Data Selection module, and the Contextual QA Augmentation module. These modules form a processing pipeline that addresses the problems associated with automatically integrating information from external resources into the knowledge-retrieval mechanism.

Machine learning↗
DOE OSTI2026

REDI – Readiness Engine for Data Integration

The Readiness Engine for Data Integration (REDI) is an open-source framework for automating, standardizing, and assessing the process of preparing scientific data for AI training. REDI implements a five-stage pipeline (ingest, preprocess, transform, structure, output) with per-stage provenance instrumentation via Flowcept, domain-aware transformation logic (PII anonymization, regridding, graph encoding, and more), and built-in readiness assessment and validation modes. REDI has been evaluated across climate, proteomics, materials science, and nuclear fusion datasets, demonstrating near-ideal parallel scaling to 100 nodes on OLCF's Frontier system. REDI is deployable as an agent-callable skill in coding environments such as Claude Code and OpenAI Codex, and is complemented by SetGo for FAIR compliance and catalog publication.

Brewer, Wesley [Oak Ridge National Laboratory (ORN↗
DOE OSTI2023

In-depth analysis on parallel processing patterns for high-performance Dataframes

The Data Science domain has expanded monumentally in both research and industry communities during the past decade, predominantly owing to the Big Data revolution. Artificial Intelligence (AI) and Machine Learning (ML) are bringing more complexities to data engineering applications, which are now integrated into data processing pipelines to process terabytes of data. Typically, a significant amount of time is spent on data preprocessing in these pipelines, and hence improving its efficiency directly impacts the overall pipeline performance. The community has recently embraced the concept of Dataframes as the de-facto data structure for data representation and manipulation. However, the most widely used serial Dataframes today (R, pandas) experience performance limitations while working on even moderately large data sets. We believe that there is plenty of room for improvement by taking a look at this problem from a high-performance computing point of view. In a prior publication, we presented a set of parallel processing patterns for distributed dataframe operators and the reference runtime implementation, Cylon. In this paper, we are expanding on the initial concept by introducing a cost model for evaluating the said patterns. Furthermore, we evaluate the performance of Cylon on the ORNL Summit supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

First detection of the BAO signal from early DESI data

We present the first detection of the baryon acoustic oscillations (BAOs) signal obtained using unblinded data collected during the initial 2 months of operations of the Stage-IV ground-based Dark Energy Spectroscopic Instrument (DESI). From a selected sample of 261 291 luminous red galaxies spanning the redshift interval 0.4 < z < 1.1 and covering 1651 square degrees with a 57.9 per cent completeness level, we report a ∼5σ level BAO detection and the measurement of the BAO location at a precision of 1.7 per cent. Using a bright galaxy sample of 109 523 galaxies in the redshift range 0.1 < z < 0.5, over 3677 square degrees with a 50.0 per cent completeness, we also detect the BAO feature at ∼3σ significance with a 2.6 per cent precision. These first BAO measurements represent an important milestone, acting as a quality control on the optimal performance of the complex robotically actuated, fibre-fed DESI spectrograph, as well as an early validation of the DESI spectroscopic pipeline and data management system. Based on these first promising results, we forecast that DESI is on target to achieve a high-significance BAO detection at sub-per cent precision with the completed 5-yr survey data, meeting the top-level science requirements on BAO measurements. This exquisite level of precision will set new standards in cosmology and confirm DESI as the most competitive BAO experiment for the remainder of this decade.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Using Large Language Models to help customers monitor global threat data

Large Language Models have proven adept at answering general knowledge questions. To make these generative AI tools useful to our mission customers for monitoring global threats, the data sciences team at Sandia is utilizing retrieval augmented generation (RAG) techniques to customize these models with local data. The local data we use consists of data such as research articles and patent abstracts that we've collected over the last several years using automated pipelines.

Herzer, John Andrew [Sandia National Laboratories ↗
DOE OSTI2020

Viral Annotation Pipeline in KBase

This data analysis workflow demonstrates how to process a metagenome to identify viruses using VirSorter. Then, provides taxonomic classification using vConTACT2.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Quantitative cross-species translators of cardiac myocyte electrophysiology: Model training, experimental validation, and applications

Animal experimentation is key in the evaluation of cardiac efficacy and safety of novel therapeutic compounds. However, interspecies differences in the mechanisms regulating excitation-contraction coupling can limit the translation of experimental findings from animal models to human physiology and undermine the assessment of drugs’ efficacy and safety. Here, we built a suite of translators for quantitatively mapping electrophysiological responses in ventricular myocytes across species. We trained these statistical operators using a broad dataset obtained by simulating populations of our biophysically detailed computational models of action potential and Ca 2+ transient in mouse, rabbit, and human. We then tested our translators against experimental data describing the response to stimuli, such as ion channel block, change in beating rate, and β-adrenergic challenge. We demonstrate that this approach is well suited to predicting the effects of perturbations across different species or experimental conditions and suggest its integration into mechanistic studies and drug development pipelines.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Machine learning pipeline to predict defect behavior in metallic alloy systems

The interaction between defect and solute atoms is critical to the thermodynamic and kinetic behavior of metallic alloys under exposure to high-energy radiation, causing irradiation damage in materials. Radiation can generate non-equilibrium concentrations of point defects such as vacancies and interstitials. The excess point defects not only accelerate diffusional processes such as precipitation that cause radiation embrittlement, but also change the pathway of phase transformations, including nucleation processes. Understanding these defect behaviors is complicated by the challenge and complexity of addressing each possible local and discrete distribution of environments and chemical interactions around targeted defects-solute or solute-solute complexes. To resolve the challenge, machine learning regression techniques have emerged as powerful tools that can train and construct an energy model to accurately describe the chemical interactions of solutes and defects. In Fiscal Year 2022, the work focused on the workflow development and demonstration using machine learning regression, density functional theory, cluster expansion, and Monte Carlo simulation to predict the effects of ternary solute elements (e.g., aluminum and molybdenum) and point defects on the Cr-rich $\alpha^{\prime}$ precipitation in multicomponent FeCr model alloys. The computational outcomes include the prediction of the ternary phase diagram, vacancy formation energy for different compositions, and the effect of vacancies on the nucleation of Cr-rich clusters. The simulations predict a pronounced change of Cr solubility in bcc Fe by the addition of Al and the rejection of Al atoms from $\alpha^{\prime}$ precipitates. Additionally, the simulations show the formation of Cr-vacancy clusters as the initial nuclei for stable nucleation and growth of $\alpha^{\prime}$ particles. The results demonstrate important outcomes and applications of using machine learning pipeline to study model or commercial alloys with multicomponent solute species and point defects.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2021

The Last Journey. I. An Extreme-scale Simulation on the Mira Supercomputer

The Last Journey is a large-volume, gravity-only, cosmological N-body simulation evolving more than 1.24 trillion particles in a periodic box with a side length of 5.025 Gpc. It was implemented using the HACC simulation and analysis framework on the BG/Q system Mira. The cosmological parameters are chosen to be consistent with the results from the Planck satellite. A range of analysis tools have been run in situ to enable a diverse set of science projects and, at the same time, keep the resulting data amount manageable. Analysis outputs have been generated starting at redshift z ~ 10 to allow for construction of synthetic galaxy catalogs using a semianalytic modeling approach in postprocessing. As part of our in situ analysis pipeline, we employ a new method for tracking halo substructures, introducing the concept of subhalo cores. The production of multiwavelength synthetic sky maps is facilitated by generating particle light cones in situ, also beginning at z ~ 10. We provide an overview of the simulation setup and generated data products; a first set of analysis results is presented. A subset of the data is publicly available.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2020

Data Accountability and Uncertainty Analysis for the Mars Science Laboratory

This paper presents machine learning-based approaches to automate and optimize the detection of volume loss for the downlink process of telemetry data from the Mars Curiosity Rover. The Curiosity observes volume loss and data corruption, requiring re-transmits from the rover and Ground Data System Analysts (GDSA) to monitor the data flow. To resolve this issue, we created a data pipeline to accumulate data from various data sources in the downlink process and detect where the data is missed. In this paper, we benchmarked different methodologies based on the accuracy and excitability of them to identify whether a downlink data that is received to the ground system is complete or incomplete. Our results show that machine learning methods can improve the performance of the GDSA by 55% while the user can diagnose why data is missed and provide an explanation for the data accountability problem.

Chowdhury, Ameera↗
DOE OSTI2020

Hypergraph Analytics of Domain Name System Relationships

We report on the use of novel mathematical methods in hypergraph analytice over a large quantity of DNS data. Hypergraphs generalize graphs, as used in network science, to better model complex multiway relations in cyber data. Specifically, casting DNS data from Georgia Tech's ActiveDNS repository as hypergraphs allows us to fully represent the interactions between {\em collections} of domains and IP addresses. To facilitate large-scale analytics, we fielded an analytical pipeline of two capabilities. HyperNetX (HNX) is a Python package for the exploration and visualization of hypergraphs, acting as a frontend. For the backend, the Chapel HyperGraph Library (CHGL) is a library for high performance hypergraph analytics written in the exascale programming language Chapel. CHGL was used to process gigascale DNS data, performing compute-intensive calculations for data reduction and segmentation. Identified portions are then sent to HNX for both exploratory analysis and knowledge discovery targeting known tactics, techniques, and procedures.

Joslyn, Cliff A.↗
NASA NTRS2001

ROSAT Science Data Center

This report provides a summary of the Smithsonian Astrophysical Observatory (SAO) ROSAT SCIENCE DATA CENTER (RSDC) activities for the recent years of our contract. Details have already been reported in the monthly reports. The SAO was responsible for the High Resolution Imager (HRI) detector on ROSAT. We also provided and supported the HRI standard analysis software used in the pipeline processing (SASS). Working with our colleagues at the Max Planck in Garching Germany (MPE), we fixed bugs and provided enhancements. The last major effort in this area was the port from VMS/VAX to VMS/ALPHA architecture. In 1998, a timing bug was found in the HRI standard processing system which degraded the positional accuracy because events accessed incorrect aspect solutions. The bug was fixed and we developed off-line correction routines and provided them to the community. The Post Reduction Off-line Software (PROS) package was developed by SAO and runs in the IRAF environment. Although in recent years PROS was not a contractual responsibility of the RSDC, we continued to maintain the system and provided new capabilities such as the ability to deal with simulated AXAF data in preparation for the NASA call for proposals for Chandra. Our most recent activities in this area included the debugging necessary for newer versions of IRAF which broke some of our software. At SAO we have an operating version of PROS and hope to release a patch even though almost all functionality that was lost was subsequently recovered via an IRAF patch (i.e. most of our problems were caused by an IRAF bug).

Murray, Stephen↗