Search NASA⌕ Search

SEARCH · Search NASA

Results for “data pipeline”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Intelligent experiments through real-time AI: Fast Data Processing and Autonomous Detector Control for sPHENIX and future EIC detectors

This R&D project, initiated by the DOE Nuclear Physics AI-Machine Learning initiative in 2022, leverages AI to address data processing challenges in high-energy nuclear experiments (RHIC, LHC, and future EIC). Our focus is on developing a demonstrator for real-time processing of high-rate data streams from sPHENIX experiment tracking detectors. The limitations of a 15 kHz maximum trigger rate imposed by the calorimeters can be negated by intelligent use of streaming technology in the tracking system. The approach efficiently identifies low momentum rare heavy flavor events in high-rate p+p collisions (3MHz), using Graph Neural Network (GNN) and High Level Synthesis for Machine Learning (hls4ml). Success at sPHENIX promises immediate benefits, minimizing resources and accelerating the heavy-flavor measurements. The approach is transferable to other fields. For the EIC, we develop a DIS-electron tagger using Artificial Intelligence - Machine Learning (AI-ML) algorithms for real-time identification, showcasing the transformative potential of AI and FPGA technologies in high-energy nuclear and particle experiments real-time data processing pipelines.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

A Gigaparsec-scale Hydrodynamic Volume Reconstructed with Deep Learning

The next generation of spectroscopic surveys will map the large-scale structure of the Universe at high redshifts (2 ≤ z ≤ 5) using millions of quasar spectra, enabling major advances in constraining both the standard cosmological model and its extensions. Robust cosmological analyses of these data sets require numerical simulations that both cover gigaparsec volumes and resolve features on ∼10 kpc scales and smaller. However, running such large-volume, high-resolution hydrodynamic simulations is computationally prohibitive. We present a generative deep learning model that enhances a low-resolution, gigaparsec-scale (960 h −1 Mpc) hydrodynamic simulation using a smaller (80 h −1 Mpc) high-resolution input hydrodynamic simulation as training data. The resulting enhanced simulation reproduces the line-of-sight power spectrum to within ∼10% and the three-dimensional power spectrum at the ∼20% level at intermediate to small scales (k ≲ 2 h Mpc −1 ). Our method shows strong promise for producing realistic simulations for cosmological analyses with current surveys such as the Dark Energy Spectroscopic Instrument and upcoming next-generation experiments, but further improvements are needed to accurately recover the large-scale modes. We publicly release the enhanced hydrodynamic simulation, along with a halo catalog from a companion N-body dark matter simulation to support the calibration of data analysis pipelines for these large-scale surveys.

Convolutional neural networks↗
DOE OSTI2025

SITCOMTN-149: An Interim Report on the LSSTComCam On-Sky Campaign

From 24 October to 11 December 2024, the NSF-DOE Vera C. Rubin Observatory conducted an on-sky campaign using the engineering LSST Commissioning Camera (LSSTComCam) to test the end-to-end functionality of hardware and software, as well as operational procedures. This interim report provides a preliminary technical overview of our understanding of the integrated system performance based tests and analyses conducted during the LSSTComCam on-sky campaign. The objectives are to synthesize what we have learned about the system in a timely way to inform on-going commissioning efforts, and to inform the Rubin science community on the progress of the LSSTComCam on-sky campaign. The report is organized into sections to describe major activities during the campaign, as well as multiple aspects of the demonstrated system and science performance. All of the results presented here are to be understood as work in progress using engineering data and the initial versions of the data processing pipelines; the report is a living document that will be updated as analyses are refined.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1978

Modular on-board adaptive imaging

Feature extraction involves the transformation of a raw video image to a more compact representation of the scene in which relevant information about objects of interest is retained. The task of the low-level processor is to extract object outlines and pass the data to the high-level process in a format that facilitates pattern recognition tasks. Due to the immense computational load caused by processing a 256x256 image, even a fast minicomputer requires a few seconds to complete this low-level processing. It is, therefore, necessary to consider hardware implementation of these low-level functions to achieve real-time processing speeds. The considered project had the objective to implement a system in which the continuous feature extraction process is not affected by the dynamic changes in the scene, varying lighting conditions, or object motion relative to the cameras. Due to the high bandwidth (3.5 MHz) and serial nature of the TV data, a pipeline processing scheme was adopted as the overall architecture of this system. Modularity in the system is achieved by designing circuits that are generic within the overall system.

Eskenazi, R.↗
NASA NTRS1983

Frequency-multiplexed and pipelined iterative optical systolic array processors

Optical matrix processors using acoustooptic transducers are described, with emphasis on new systolic array architectures using frequency multiplexing in addition to space and time multiplexing. A Kalman filtering application is considered in a case study from which the operations required on such a system can be defined. This also serves as a new and powerful application for iterative optical processors. The importance of pipelining the data flow and the ordering of the operations performed in a specific application of such a system are also noted. Several examples of how to effectively achieve this are included. A new technique for handling bipolar data on such architectures is also described.

Casasent, D.↗
NASA NTRS1984

Optical Kalman filtering for missile guidance

Optical systolic array processors constitute a powerful and general-purpose set of optical architectures with high computational rates. In this paper, Kalman filtering, a novel application for these architectures, is investigated. All required operations are detailed; their realization by optical and special-purpose analog electronics are specified; and the processing time of the system is quantified. The specific Kalman filter application chosen is for an air-to-air missile guidance controller. The architecture realized in this paper meets the design goal of a fully adaptive Kalman filter which processes a measurement every 1 msec. The vital issue of flow and pipelining of data and operations in a systolic array processor is addressed. The approach is sufficiently general and can be realized on an optical or digital systolic array processor.

Casasent, D.↗
NASA NTRS1989

General-Purpose Electronic System Tests Aircraft

Versatile digital equipment supports research, development, and maintenance. Extended aircraft interrogation and display system is general-purpose assembly of digital electronic equipment on ground for testing of digital electronic systems on advanced aircraft. Many advanced features, including multiple 16-bit microprocessors, pipeline data-flow architecture, advanced operating system, and resident software-development tools. Basic collection of software includes program for handling many types of data and for displays in various formats. User easily extends basic software library. Hardware and software interfaces to subsystems provided by user designed for flexibility in configuration to meet user's requirements.

Glover, Richard D.↗
NASA NTRS1997

Structure and Rotation of the Solar Interior: Initial Results from the MDI Medium-L Program

The medium-l program of the Michelson Doppler Imager instrument on board SOHO provides continuous observations of oscillation modes of angular degree, l, from 0 to approximately 300. The data for the program are partly processed on board because only about 3% of MDI observations can be transmitted continuously to the ground. The on-board data processing, the main component of which is Gaussian-weighted binning, has been optimized to reduce the negative influence of spatial aliasing of the high-degree oscillation modes. The data processing is completed in a data analysis pipeline at the SOI Stanford Support Center to determine the mean multiplet frequencies and splitting coefficients. The initial results show that the noise in the medium-l oscillation power spectrum is substantially lower than in ground-based measurements. This enables us to detect lower amplitude modes and, thus, to extend the range of measured mode frequencies. This is important for inferring the Sun's internal structure and rotation. The MDI observations also reveal the asymmetry of oscillation spectral lines. The line asymmetries agree with the theory of mode excitation by acoustic sources localized in the upper convective boundary layer. The sound-speed profile inferred from the mean frequencies gives evidence for a sharp variation at the edge of the energy-generating core. The results also confirm the previous finding by the GONG (Gough et al., 1996) that, in a thin layer just beneath the convection zone, helium appears to be less abundant than predicted by theory. Inverting the multiplet frequency splittings from MDI, we detect significant rotational shear in this thin layer. This layer is likely to be the place where the solar dynamo operates. In order to understand how the Sun works, it is extremely important to observe the evolution of this transition layer throughout the 11-year activity cycle.

Kosovichev, A. G.↗
NASA NTRS2008

Targeting and Localization for Mars Rover Operations

A design and a partially developed application framework were presented for improving localization and targeting for surface spacecraft. The program has value for the Mars Science Laboratory mission, and has been delivered to support the Mars Exploration Rovers as part of the latest version of the Maestro science planning tool. It also has applications for future missions involving either surface-based or low-altitude atmospheric robotic vehicles. The targeting and localization solutions solve the problem of how to integrate localization estimate updates into operational planning tools, operational data product generalizations, and flight software by adding expanded flexibility to flight software, the operations data product pipeline, and operations planning tools based on coordinate frame updates during a planning cycle.

Powell, Mark W.↗
NASA NTRS2012

A Peculiar Class of Debris Disks from Herschel/DUNES: A Steep Fall Off in the Far Infrared

Context. The existence of debris disks around old main sequence stars is usually explained by continuous replenishment of small dust grains through collisions from a reservoir of larger objects. Aims. We present photometric data of debris disks around HIP 103389 (HD199260), HIP 100350 (HN Peg, HD206860), and HIP 114948 (HD 219482), obtained in the context of our Herschel Open TIme Key Program DUNES (DUst around NEarby Stars). Methods. We used Herschel/PACS to detect the thermal emission of the three debris disks with a 30 sigma sensitivity of a few mJy at l00 micron and 160 micron. In addition, we obtained Herschel/PACS photometric data at 70 micron for HIP 103389. These observations are complemented by a large variety of optical to far-infrared photometric data. Two different approaches are applied to reduce the Herschel data to investigate the impact of data reduction on the photometry. We fit analytical models to the available spectral energy distribution (SED) data using the fitting method of simulated therma1 annealing as well as a classical grid search method. Results. The SEDs of the three disks potentially exhibit an unusually steep decrease at wavelengths >= 70 micron. We investigate the significance of the peculiar shape of these SEDs and the impact on models of the disks provided it is real. Using grain compositions that have been applied successfully for modeling of many other debris disks, our modeling reveals that such a steep decrease of the SEDs in the long wavelength regime is inconsistent with a power-law exponent of the grain size distribution -3.5 expected from a standard equilibrium collisional cascade. In contrast, a steep grain size distribution or, alternatively an upper grain size in the range of few tens of micrometers are implied. This suggests that a very distinct range of grain sizes would dominate the thermal. emission of such disks. However, we demonstrate that the understanding of the data of faint sources obtained with Herschel is still incomplete and that the significance of our results depends on the version of the data reduction pipeline used. Conclusions. A new mechanism to produce the dust in the presented debris disks, deviations from the conditions required for a standard equilibrium collisional cascade (grain size exponent of -3.5), and/or significantly different dust properties would be necessary to explain the potentially steep SED shape of the three debris disks presented.

Ertel, S.↗
NASA NTRS2014

Astrophysical Observations with the HEROES Balloon-borne Payload

The High Energy Replicated Optics to Explore the Sun (HEROES) payload flew on a balloon from Ft. Sumner, NM, September 21-22, 2013. HEROES is sensitive from about 20-75 keV and comprises 8 optics modules, each consisting of 13-14 nickel replicated optics shells and 8 Xenon-filled positionsensitive proportional counter detectors. HEROES is unique in that it is the first hard X-ray telescope that will observe the Sun and astrophysical targets in the same balloon flight. Our astrophysics targets include the Crab nebula and pulsar and the black hole binary GRS 1915+105. In this presentation, I will describe the HEROES mission, the data analysis pipeline and calibrations, and preliminary astrophysics results.

Wilson, C.↗
NASA NTRS2014

The HEROES Balloon-Borne Hard X-Ray Telescope

The High Energy Replicated Optics to Explore the Sun (HEROES) payload flew on a balloon from Ft. Sumner, NM, September 21-22, 2013. HEROES is sensitive from about 20-75 keV and comprises 8 optics modules (HPD approximately 33" as flown), each consisting of 13-14 nickel replicated optics shells and 8 matching Xenon-filled position-sensitive proportional counter detectors (dE/E=0.05 @ 60 keV). Our targets included the Sun, the Crab Nebula and pulsar and the black hole binary GRS 1915+105. HEROES was pointed using a day/night star camera system for astrophysical observations and a newly developed Solar Aspect System for solar observations (with a shutter protecting the star camera.) We have successfully detected the Crab Nebula. Analyses for GRS 1915+105 and the Sun are ongoing. In this presentation, I will describe the HEROES mission, the data analysis pipeline and calibrations, preliminary results, and plans for follow-on missions.

Wilson-Hodge, C.↗
NASA NTRS2017

Fireballs in the Sky: An Augmented Reality Citizen Science Program

Fireballs in the Sky is an innovative Australian citizen science program that connects the public with the research of the Desert Fireball Network (DFN). This research aims to understand the early workings of the solar system, and Fireballs in the Sky invites people around the world to learn about this science, contributing fireball sightings via a user-friendly augmented reality mobile app. Tens of thousands of people have downloaded the app world-wide and participated in the science of meteoritics. The Fireballs in the Sky app allows users to get involved with the Desert Fireball Network research, supplementing DFN observations and providing enhanced coverage by reporting their own meteor sightings to DFN scientists. Fireballs in the Sky reports are used to track the trajectories of meteors - from their orbit in space to where they might have landed on Earth. Led by Phil Bland at Curtin University in Australia, the Desert Fireball Network (DFN) uses automated observatories across Australia to triangulate trajectories of meteorites entering the atmosphere, determine pre-entry orbits, and pinpoint their fall positions. Each observatory is an autonomous intelligent imaging system, taking 1000 by 36 megapixel all-sky images throughout the night, using neural network algorithms to recognize events. They are capable of operating for 12 months in a harsh environment, and store all imagery collected. We developed a completely automated software pipeline for data reduction, and built a supercomputer database for storage, allowing us to process our entire archive. The DFN currently stands at 50 stations distributed across the Australian continent, covering an area of 2.5 million square kilometers. Working with DFN's partners at NASA's Solar System Exploration Research Virtual Institute, the team is expanding the network beyond Australia to locations around the world. Fireballs in the Sky allows a growing public base to learn about and participate in this exciting research.

Day, Brian↗
NASA NTRSDate not supplied

AAM NC Tech Talk: Intro to Amazon Web Services

The purpose of this Tech Talk is to engage with the Advance Air Mobility National Campaign community on the use of a cloud provider to promote public confidence and accelerate the realization of emerging aviation markets for passenger and cargo transportation in urban, suburban, rural, and regional environments.

AWS↗
NASA NTRSDate not supplied

Optimizing Single Nuclei Sequencing of Brain Samples From Space Flown Mice Across Age and Strain

The NASA GeneLab Sample Processing Laboratory offers high-throughput sequencing services to NASA-funded space biology researchers. Space biology studies have specific challenges such as low sample numbers, introducing susceptibility to batch effects from sample handling. These issues are compounded by complex protocols such as single-nuclei isolation and sequencing, which has recently become an attractive methodology for assessing the cellular diversity within spaceflight samples. High quality single-nuclei sequencing requires reproducible protocols to dissociate tissue and generate clean suspension of intact single nuclei. Producing single-nuclei suspension from brain tissue is particularly challenging due to cell type heterogeneity and the myelin sheath that carries over into the nuclei suspension as debris. Current procedures tend to be time consuming and sometimes include steps that can alter gene expression and create cell-type bias. Commercially available nuclei isolation kits, such as the 10X Genomics nuclei isolation kit, offers a streamlined way to process samples for nuclei isolation, thereby minimizing batch effects and enabling reproducibility. In this study, we report on the performance of the 10X Genomics nuclei isolation kit and Chromium Next GEM Single Cell Multiome ATAC + Gene Expression kit to generate sequencing libraries from space-flown mouse brain samples. Single nuclei sequencing was performed on frozen mouse brain tissue from two spaceflight missions, Rodent Research-10 (RR-10) and RR Reference Mission-2 (RRRM-2). RR-10 mice were female B6129SF2/J, euthanized at 18-19 weeks whereas RRRM-2 mice were female C57BL/6NTac, euthanized at 20 or 37 weeks. Sequencing data was processed using standard GeneLab data processing pipelines. We report evaluation of the performance of the 10X Genomics nuclei isolation kit for spaceflight samples from mouse brain, and evaluation of reproducibility across different mouse strains and age groups. We also report preliminary scientific results including cell type inference, cell clustering, and differentially expressed genes and pathways between spaceflight and ground control samples.

RR-10↗
NASA NTRSDate not supplied

SatCORPS Global Cloud Composite (GCC): the Design and Delivery of A High Quality, High Resolution, Global Cloud Product Available in Near-Real Time

The NASA Satellite ClOud and Radiation Property retrieval System (SatCORPS) supports the development of an analysis ready and cloud-optimized data transformation pipeline and geospatial service enablement of a global cloud composite (GCC) product derived from global geostationary satellite imagery. This geospatial service will be available at high temporal and spatial resolution via the SatCORPS web mapping application for visualization and analysis as well as direct ingestion to common geospatial software and custom programming. The resulting global cloud composite products from the processing pipeline can then be geospatially-service enabled as ArcGIS Image Services and Open Geospatial Consortium (OGC) Web Mapping/Coverage Services for visualization and analysis via a web mapping application and common geospatial software. Near real time global observations are created through the composition of five geostationary satellites that provides modelling and forecasting communities with the capability to provide high quality and timely information to start the projection process. The Global Cloud Composite product combines information from geostationary satellites, GOES-16, GOES-17, Himawari-8, Meteosat-11 and Meteosat-9 to create a single global composite netcdf file and images using the different products within the netcdf file. The SatCORPS team, though our Global Cloud Composite (GCC) product and web-based visualization tools including Geographical Information System (GIS) services provide near real time global cloud product information to both automated processes and traditional web users that is timely and high quality derived from geostationary satellites. The Global Cloud Composite product takes advantage of the scalable processing resources provided by the AWS batch service to provide new composites every thirty minutes. Because information from each of the low earth orbiting satellites is available on schedules tuned to the specific satellite, the processing algorithm temporally composites the final dataset as each satellite’s information becomes available. The SatCORPS team has leveraged our experience using Amazon Web Services (AWS) to build a low latency high availability tool that allows end users both human and automated to acquire high quality and high-resolution Geostationary Earth Orbiting (GEO) information at zero cost to the end user. This presentation will describe how we architected and implemented the service as well as lessons learned based on our experiences both developing and operating the system. The lessons learned include how we integrated multiple services including Amazon Batch, Amazon S3 and Amazon Lambda service to create a low cost but high-performance processing system that is capable of identifying and processing the most appropriate satellite overpass information into global cloud composites. We will also describe our web-based tools including our Geographic Information System that can be used for visualization and analysis. The products from the processing can be geospatially-service enabled as ArcGIS Image Services and Open Geospatial Consortium (OGC) Web Mapping/Coverage Services for visualization and analysis via a web mapping application and common geospatial software. The SatCORPS Global Composite Cloud product provides sophisticated global composited cloud research products with very low latency that we see that as filling a rapidly growing need in the research and modelling community with no up-front nor ongoing costs associated with downloading or using the information.

AWS AMCE SMCE GCC SATCORPS GLOBAL CLOUD COMPOSITE ↗
DOE OSTI2025

Evaluating Direct and Indirect Influence on EV Charging Stations Across the US

The adoption of new technology for electric vehicles (EV) and mobility applications can bring underappreciated vulnerabilities to the power grid. One area of potential fraud and adversarial influence is through the business ecosystem of startups that own and deploy EV technology. Yet, there are no models or analyses that map the network of organizations and people that have direct and indirect influence over technologies currently deployed in the grid. To fill this gap, we develop a multilayer network model to measure direct and indirect influence on EV charging stations. First, we create and adversarial socio-technical network (ASTN) model via a data fusion pipeline for different US regions of interest (ROI). Then, we develop an integrated ASTN for Chicago, Los Angeles, New York, and Philadelphia. We rank EV charging companies direct influence within each geographic region as well as indirect influence via social network analysis. While some companies have strong direct and indirect influence (i.e., ChargePoint) others show a mismatch between their influence over charging stations and their position within the social network. For example, Tesla has strong direct influence on stations and weak indirect influence over competitors. In contrast, 7Charge has weak direct influence over stations, but strong indirect influence over competitors.

29 - ENERGY PLANNING, POLICY AND ECONOMY↗