Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Pipeline”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

The Dark Energy Camera Plane Survey 2 (DECaPS2): More Sky, Less Bias, and Better Uncertainties

Abstract Deep optical and near-infrared imaging of the entire Galactic plane is essential for understanding our Galaxy’s stars, gas, and dust. The second data release of the Dark Energy Camera (DECam) Plane Survey extends the five-band optical and near-infrared survey of the southern Galactic plane to cover 6.5% of the sky, ∣ b ∣ ≤ 10°, and 6° > ℓ > −124°, complementary to coverage by Pan-STARRS1. Typical single-exposure effective depths, including crowding effects and other complications, are 23.5, 22.6, 22.1, 21.6, and 20.8 mag in g , r , i , z , and Y bands, respectively, with around 1″ seeing. The survey comprises 3.32 billion objects built from 34 billion detections in 21,400 exposures, totaling 260 hr open shutter time on the DECam at Cerro Tololo. The data reduction pipeline features several improvements, including the addition of synthetic source injection tests to validate photometric solutions across the entire survey footprint. A convenient functional form for the detection bias in the faint limit was derived and leveraged to characterize the photometric pipeline performance. A new postprocessing technique was applied to every detection to debias and improve uncertainty estimates of the flux in the presence of structured backgrounds, specifically targeting nebulosity. The images and source catalogs are publicly available at http://decaps.skymaps.info/ .

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Millimeter Light Curves of Sagittarius A* Observed during the 2017 Event Horizon Telescope Campaign

The Event Horizon Telescope (EHT) observed the compact radio source, Sagittarius A* (Sgr A*), in the Galactic Center on 2017 April 5–11 in the 1.3 mm wavelength band. At the same time, interferometric array data from the Atacama Large Millimeter/submillimeter Array and the Submillimeter Array were collected, providing Sgr A* light curves simultaneous with the EHT observations. These data sets, complementing the EHT very long baseline interferometry, are characterized by a cadence and signal-to-noise ratio previously unattainable for Sgr A* at millimeter wavelengths, and they allow for the investigation of source variability on timescales as short as a minute. While most of the light curves correspond to a low variability state of Sgr A*, the April 11 observations follow an X-ray flare and exhibit strongly enhanced variability. All of the light curves are consistent with a red-noise process, with a power spectral density (PSD) slope measured to be between −2 and −3 on timescales between 1 minute and several hours. Our results indicate a steepening of the PSD slope for timescales shorter than 0.3 hr. The spectral energy distribution is flat at 220 GHz, and there are no time lags between the 213 and 229 GHz frequency bands, suggesting low optical depth for the event horizon scale source. We characterize Sgr A*’s variability, highlighting the different behavior observed just after the X-ray flare, and use Gaussian process modeling to extract a decorrelation timescale and a PSD slope. We also investigate the systematic calibration uncertainties by analyzing data from independent data reduction pipelines.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

SITCOMTN-149: An Interim Report on the LSSTComCam On-Sky Campaign

From 24 October to 11 December 2024, the NSF-DOE Vera C. Rubin Observatory conducted an on-sky campaign using the engineering LSST Commissioning Camera (LSSTComCam) to test the end-to-end functionality of hardware and software, as well as operational procedures. This interim report provides a preliminary technical overview of our understanding of the integrated system performance based tests and analyses conducted during the LSSTComCam on-sky campaign. The objectives are to synthesize what we have learned about the system in a timely way to inform on-going commissioning efforts, and to inform the Rubin science community on the progress of the LSSTComCam on-sky campaign. The report is organized into sections to describe major activities during the campaign, as well as multiple aspects of the demonstrated system and science performance. All of the results presented here are to be understood as work in progress using engineering data and the initial versions of the data processing pipelines; the report is a living document that will be updated as analyses are refined.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Wildfire Smoke PM2.5 per Zipcode

This dataset contains daily aggregated measurements of daily PM2.5 from ambient wildfire smoke in the contiguous United States, spanning from 2006 to 2016. The data is sourced from a study by Childs et al. (2022), titled "Daily Local-Level Estimates of Ambient Wildfire Smoke PM2.5 for the Contiguous US" published in Environmental Science & Technology. To compute the standardized weight across different zip codes, we computed the weight calculation on a 10km grids. To compute the weight, we obtain area-weights that add up to 1 in each polygon of zipcodes. This enabled us to calculate the smoke values per zipcode for the aforementioned period. Those interested in replicating our data processing pipeline can access it at https://github.com/NSAPH-Data-Processing/smoke_aggregation.

PM2.5↗
NASA NTRS1978

Modular on-board adaptive imaging

Feature extraction involves the transformation of a raw video image to a more compact representation of the scene in which relevant information about objects of interest is retained. The task of the low-level processor is to extract object outlines and pass the data to the high-level process in a format that facilitates pattern recognition tasks. Due to the immense computational load caused by processing a 256x256 image, even a fast minicomputer requires a few seconds to complete this low-level processing. It is, therefore, necessary to consider hardware implementation of these low-level functions to achieve real-time processing speeds. The considered project had the objective to implement a system in which the continuous feature extraction process is not affected by the dynamic changes in the scene, varying lighting conditions, or object motion relative to the cameras. Due to the high bandwidth (3.5 MHz) and serial nature of the TV data, a pipeline processing scheme was adopted as the overall architecture of this system. Modularity in the system is achieved by designing circuits that are generic within the overall system.

Eskenazi, R.↗
NASA NTRS1983

Frequency-multiplexed and pipelined iterative optical systolic array processors

Optical matrix processors using acoustooptic transducers are described, with emphasis on new systolic array architectures using frequency multiplexing in addition to space and time multiplexing. A Kalman filtering application is considered in a case study from which the operations required on such a system can be defined. This also serves as a new and powerful application for iterative optical processors. The importance of pipelining the data flow and the ordering of the operations performed in a specific application of such a system are also noted. Several examples of how to effectively achieve this are included. A new technique for handling bipolar data on such architectures is also described.

Casasent, D.↗
NASA NTRS1984

Optical Kalman filtering for missile guidance

Optical systolic array processors constitute a powerful and general-purpose set of optical architectures with high computational rates. In this paper, Kalman filtering, a novel application for these architectures, is investigated. All required operations are detailed; their realization by optical and special-purpose analog electronics are specified; and the processing time of the system is quantified. The specific Kalman filter application chosen is for an air-to-air missile guidance controller. The architecture realized in this paper meets the design goal of a fully adaptive Kalman filter which processes a measurement every 1 msec. The vital issue of flow and pipelining of data and operations in a systolic array processor is addressed. The approach is sufficiently general and can be realized on an optical or digital systolic array processor.

Casasent, D.↗
NASA NTRS1989

General-Purpose Electronic System Tests Aircraft

Versatile digital equipment supports research, development, and maintenance. Extended aircraft interrogation and display system is general-purpose assembly of digital electronic equipment on ground for testing of digital electronic systems on advanced aircraft. Many advanced features, including multiple 16-bit microprocessors, pipeline data-flow architecture, advanced operating system, and resident software-development tools. Basic collection of software includes program for handling many types of data and for displays in various formats. User easily extends basic software library. Hardware and software interfaces to subsystems provided by user designed for flexibility in configuration to meet user's requirements.

Glover, Richard D.↗
NASA NTRS1997

Structure and Rotation of the Solar Interior: Initial Results from the MDI Medium-L Program

The medium-l program of the Michelson Doppler Imager instrument on board SOHO provides continuous observations of oscillation modes of angular degree, l, from 0 to approximately 300. The data for the program are partly processed on board because only about 3% of MDI observations can be transmitted continuously to the ground. The on-board data processing, the main component of which is Gaussian-weighted binning, has been optimized to reduce the negative influence of spatial aliasing of the high-degree oscillation modes. The data processing is completed in a data analysis pipeline at the SOI Stanford Support Center to determine the mean multiplet frequencies and splitting coefficients. The initial results show that the noise in the medium-l oscillation power spectrum is substantially lower than in ground-based measurements. This enables us to detect lower amplitude modes and, thus, to extend the range of measured mode frequencies. This is important for inferring the Sun's internal structure and rotation. The MDI observations also reveal the asymmetry of oscillation spectral lines. The line asymmetries agree with the theory of mode excitation by acoustic sources localized in the upper convective boundary layer. The sound-speed profile inferred from the mean frequencies gives evidence for a sharp variation at the edge of the energy-generating core. The results also confirm the previous finding by the GONG (Gough et al., 1996) that, in a thin layer just beneath the convection zone, helium appears to be less abundant than predicted by theory. Inverting the multiplet frequency splittings from MDI, we detect significant rotational shear in this thin layer. This layer is likely to be the place where the solar dynamo operates. In order to understand how the Sun works, it is extremely important to observe the evolution of this transition layer throughout the 11-year activity cycle.

Kosovichev, A. G.↗
NASA NTRS2008

Targeting and Localization for Mars Rover Operations

A design and a partially developed application framework were presented for improving localization and targeting for surface spacecraft. The program has value for the Mars Science Laboratory mission, and has been delivered to support the Mars Exploration Rovers as part of the latest version of the Maestro science planning tool. It also has applications for future missions involving either surface-based or low-altitude atmospheric robotic vehicles. The targeting and localization solutions solve the problem of how to integrate localization estimate updates into operational planning tools, operational data product generalizations, and flight software by adding expanded flexibility to flight software, the operations data product pipeline, and operations planning tools based on coordinate frame updates during a planning cycle.

Powell, Mark W.↗
NASA NTRS2012

A Peculiar Class of Debris Disks from Herschel/DUNES: A Steep Fall Off in the Far Infrared

Context. The existence of debris disks around old main sequence stars is usually explained by continuous replenishment of small dust grains through collisions from a reservoir of larger objects. Aims. We present photometric data of debris disks around HIP 103389 (HD199260), HIP 100350 (HN Peg, HD206860), and HIP 114948 (HD 219482), obtained in the context of our Herschel Open TIme Key Program DUNES (DUst around NEarby Stars). Methods. We used Herschel/PACS to detect the thermal emission of the three debris disks with a 30 sigma sensitivity of a few mJy at l00 micron and 160 micron. In addition, we obtained Herschel/PACS photometric data at 70 micron for HIP 103389. These observations are complemented by a large variety of optical to far-infrared photometric data. Two different approaches are applied to reduce the Herschel data to investigate the impact of data reduction on the photometry. We fit analytical models to the available spectral energy distribution (SED) data using the fitting method of simulated therma1 annealing as well as a classical grid search method. Results. The SEDs of the three disks potentially exhibit an unusually steep decrease at wavelengths >= 70 micron. We investigate the significance of the peculiar shape of these SEDs and the impact on models of the disks provided it is real. Using grain compositions that have been applied successfully for modeling of many other debris disks, our modeling reveals that such a steep decrease of the SEDs in the long wavelength regime is inconsistent with a power-law exponent of the grain size distribution -3.5 expected from a standard equilibrium collisional cascade. In contrast, a steep grain size distribution or, alternatively an upper grain size in the range of few tens of micrometers are implied. This suggests that a very distinct range of grain sizes would dominate the thermal. emission of such disks. However, we demonstrate that the understanding of the data of faint sources obtained with Herschel is still incomplete and that the significance of our results depends on the version of the data reduction pipeline used. Conclusions. A new mechanism to produce the dust in the presented debris disks, deviations from the conditions required for a standard equilibrium collisional cascade (grain size exponent of -3.5), and/or significantly different dust properties would be necessary to explain the potentially steep SED shape of the three debris disks presented.

Ertel, S.↗
NASA NTRS2014

Astrophysical Observations with the HEROES Balloon-borne Payload

The High Energy Replicated Optics to Explore the Sun (HEROES) payload flew on a balloon from Ft. Sumner, NM, September 21-22, 2013. HEROES is sensitive from about 20-75 keV and comprises 8 optics modules, each consisting of 13-14 nickel replicated optics shells and 8 Xenon-filled positionsensitive proportional counter detectors. HEROES is unique in that it is the first hard X-ray telescope that will observe the Sun and astrophysical targets in the same balloon flight. Our astrophysics targets include the Crab nebula and pulsar and the black hole binary GRS 1915+105. In this presentation, I will describe the HEROES mission, the data analysis pipeline and calibrations, and preliminary astrophysics results.

Wilson, C.↗
NASA NTRS2014

The HEROES Balloon-Borne Hard X-Ray Telescope

The High Energy Replicated Optics to Explore the Sun (HEROES) payload flew on a balloon from Ft. Sumner, NM, September 21-22, 2013. HEROES is sensitive from about 20-75 keV and comprises 8 optics modules (HPD approximately 33" as flown), each consisting of 13-14 nickel replicated optics shells and 8 matching Xenon-filled position-sensitive proportional counter detectors (dE/E=0.05 @ 60 keV). Our targets included the Sun, the Crab Nebula and pulsar and the black hole binary GRS 1915+105. HEROES was pointed using a day/night star camera system for astrophysical observations and a newly developed Solar Aspect System for solar observations (with a shutter protecting the star camera.) We have successfully detected the Crab Nebula. Analyses for GRS 1915+105 and the Sun are ongoing. In this presentation, I will describe the HEROES mission, the data analysis pipeline and calibrations, preliminary results, and plans for follow-on missions.

Wilson-Hodge, C.↗
NASA NTRS2017

Fireballs in the Sky: An Augmented Reality Citizen Science Program

Fireballs in the Sky is an innovative Australian citizen science program that connects the public with the research of the Desert Fireball Network (DFN). This research aims to understand the early workings of the solar system, and Fireballs in the Sky invites people around the world to learn about this science, contributing fireball sightings via a user-friendly augmented reality mobile app. Tens of thousands of people have downloaded the app world-wide and participated in the science of meteoritics. The Fireballs in the Sky app allows users to get involved with the Desert Fireball Network research, supplementing DFN observations and providing enhanced coverage by reporting their own meteor sightings to DFN scientists. Fireballs in the Sky reports are used to track the trajectories of meteors - from their orbit in space to where they might have landed on Earth. Led by Phil Bland at Curtin University in Australia, the Desert Fireball Network (DFN) uses automated observatories across Australia to triangulate trajectories of meteorites entering the atmosphere, determine pre-entry orbits, and pinpoint their fall positions. Each observatory is an autonomous intelligent imaging system, taking 1000 by 36 megapixel all-sky images throughout the night, using neural network algorithms to recognize events. They are capable of operating for 12 months in a harsh environment, and store all imagery collected. We developed a completely automated software pipeline for data reduction, and built a supercomputer database for storage, allowing us to process our entire archive. The DFN currently stands at 50 stations distributed across the Australian continent, covering an area of 2.5 million square kilometers. Working with DFN's partners at NASA's Solar System Exploration Research Virtual Institute, the team is expanding the network beyond Australia to locations around the world. Fireballs in the Sky allows a growing public base to learn about and participate in this exciting research.

Day, Brian↗
NASA NTRSDate not supplied

AAM NC Tech Talk: Intro to Amazon Web Services

The purpose of this Tech Talk is to engage with the Advance Air Mobility National Campaign community on the use of a cloud provider to promote public confidence and accelerate the realization of emerging aviation markets for passenger and cargo transportation in urban, suburban, rural, and regional environments.

AWS↗
NASA NTRSDate not supplied

Optimizing Single Nuclei Sequencing of Brain Samples From Space Flown Mice Across Age and Strain

The NASA GeneLab Sample Processing Laboratory offers high-throughput sequencing services to NASA-funded space biology researchers. Space biology studies have specific challenges such as low sample numbers, introducing susceptibility to batch effects from sample handling. These issues are compounded by complex protocols such as single-nuclei isolation and sequencing, which has recently become an attractive methodology for assessing the cellular diversity within spaceflight samples. High quality single-nuclei sequencing requires reproducible protocols to dissociate tissue and generate clean suspension of intact single nuclei. Producing single-nuclei suspension from brain tissue is particularly challenging due to cell type heterogeneity and the myelin sheath that carries over into the nuclei suspension as debris. Current procedures tend to be time consuming and sometimes include steps that can alter gene expression and create cell-type bias. Commercially available nuclei isolation kits, such as the 10X Genomics nuclei isolation kit, offers a streamlined way to process samples for nuclei isolation, thereby minimizing batch effects and enabling reproducibility. In this study, we report on the performance of the 10X Genomics nuclei isolation kit and Chromium Next GEM Single Cell Multiome ATAC + Gene Expression kit to generate sequencing libraries from space-flown mouse brain samples. Single nuclei sequencing was performed on frozen mouse brain tissue from two spaceflight missions, Rodent Research-10 (RR-10) and RR Reference Mission-2 (RRRM-2). RR-10 mice were female B6129SF2/J, euthanized at 18-19 weeks whereas RRRM-2 mice were female C57BL/6NTac, euthanized at 20 or 37 weeks. Sequencing data was processed using standard GeneLab data processing pipelines. We report evaluation of the performance of the 10X Genomics nuclei isolation kit for spaceflight samples from mouse brain, and evaluation of reproducibility across different mouse strains and age groups. We also report preliminary scientific results including cell type inference, cell clustering, and differentially expressed genes and pathways between spaceflight and ground control samples.

RR-10↗
NASA NTRSDate not supplied

SatCORPS Global Cloud Composite (GCC): the Design and Delivery of A High Quality, High Resolution, Global Cloud Product Available in Near-Real Time

The NASA Satellite ClOud and Radiation Property retrieval System (SatCORPS) supports the development of an analysis ready and cloud-optimized data transformation pipeline and geospatial service enablement of a global cloud composite (GCC) product derived from global geostationary satellite imagery. This geospatial service will be available at high temporal and spatial resolution via the SatCORPS web mapping application for visualization and analysis as well as direct ingestion to common geospatial software and custom programming. The resulting global cloud composite products from the processing pipeline can then be geospatially-service enabled as ArcGIS Image Services and Open Geospatial Consortium (OGC) Web Mapping/Coverage Services for visualization and analysis via a web mapping application and common geospatial software. Near real time global observations are created through the composition of five geostationary satellites that provides modelling and forecasting communities with the capability to provide high quality and timely information to start the projection process. The Global Cloud Composite product combines information from geostationary satellites, GOES-16, GOES-17, Himawari-8, Meteosat-11 and Meteosat-9 to create a single global composite netcdf file and images using the different products within the netcdf file. The SatCORPS team, though our Global Cloud Composite (GCC) product and web-based visualization tools including Geographical Information System (GIS) services provide near real time global cloud product information to both automated processes and traditional web users that is timely and high quality derived from geostationary satellites. The Global Cloud Composite product takes advantage of the scalable processing resources provided by the AWS batch service to provide new composites every thirty minutes. Because information from each of the low earth orbiting satellites is available on schedules tuned to the specific satellite, the processing algorithm temporally composites the final dataset as each satellite’s information becomes available. The SatCORPS team has leveraged our experience using Amazon Web Services (AWS) to build a low latency high availability tool that allows end users both human and automated to acquire high quality and high-resolution Geostationary Earth Orbiting (GEO) information at zero cost to the end user. This presentation will describe how we architected and implemented the service as well as lessons learned based on our experiences both developing and operating the system. The lessons learned include how we integrated multiple services including Amazon Batch, Amazon S3 and Amazon Lambda service to create a low cost but high-performance processing system that is capable of identifying and processing the most appropriate satellite overpass information into global cloud composites. We will also describe our web-based tools including our Geographic Information System that can be used for visualization and analysis. The products from the processing can be geospatially-service enabled as ArcGIS Image Services and Open Geospatial Consortium (OGC) Web Mapping/Coverage Services for visualization and analysis via a web mapping application and common geospatial software. The SatCORPS Global Composite Cloud product provides sophisticated global composited cloud research products with very low latency that we see that as filling a rapidly growing need in the research and modelling community with no up-front nor ongoing costs associated with downloading or using the information.

AWS AMCE SMCE GCC SATCORPS GLOBAL CLOUD COMPOSITE ↗