Search NASA⌕ Search

SEARCH · Search NASA

Results for “data pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

Geomagnetically Induced Currents Around the World During the 17 March 2015 Storm

Geomagnetically induced currents (GICs) represent a significant space weather issue for power grid and pipeline infrastructure, particularly during severe geomagnetic storms. In this study, magnetometer data collected from around the world are analyzed to investigate the GICs caused by the 2015 St. Patricks Day storm. While significant GIC activity in the high-latitude regions due to storm time substorm activity is shown for this event, enhanced GIC activity was also measured at two equatorial stations in the American and Southeast Asian sectors. This equatorial GIC activity is closely examined, and it is shown that it is present both during the arrival of the interplanetary shock at the storm sudden commencement (SSC) in Southeast Asia and during the main phase of the storm approximately 10 h later in South America. The SSC caused magnetic field variations at the equator in Southeast Asia that were twice the magnitude of those observed only a few degrees to the north, strongly indicating that the equatorial electrojet (EEJ) played a significant role. The large equatorial magnetic field variations measured in South America are also examined, and the coincident solar wind data are used to investigate the causes of the sudden changes in the EEJ approximately 10 h into the storm. From this analysis it is concluded that sudden magnetopause current increases due to increases in the solarwind dynamic pressure, and the sudden changes in the resultant magnetospheric and ionospheric current systems, are the primary drivers of equatorial GICs.

Carter, B. A.↗
NASA NTRS2018

Land Surface Reflectances from Geostationary Sensors

GEONEX is a processing pipeline that produces a suite of satellite land surface products using data streams from the latest geostationary (GEO) sensors including the GOES016/ABI and the Himawari-8/AHI. The suite, created collaboratively by scientists from NASA and NOAA, includes top-of-atmosphere (TOA) reflectances, land surface reflectances (LSRs), vegetation indices, LAI/fPAR, and other downstream products. As a key component of the GEONEX product processing, we have adapted the Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm to produce LSRs from the TOA data. Because the algorithm depends on building "stacks" of images, we first run internal geo-registration checks to ensure geo-spatial accuracy and consistency of the input (L1B) data before transferring them from the geostationary projection into a tile system in geographic grids. Scan-time is inferred from metadata and applied to calculate the sun-sensor angles for each grid cell. The MAIAC algorithm is run to detect clouds/shadows, estimate aerosol optical thickness (AOT), perform atmospheric corrections, and generate LSRs. We have processed 18-months (from 2016/04 onward) of AHI data over East Asia and Oceania at a 10-minute time step and 10-months (from 2018/01 onward) of ABI data over North and South Americas at a 15-minute time step. As a verification measure, we compare the GEONEX (AHI/ABI) surface reflectances with the standard MODIS products (MOD09GA) and the MODIS MAIAC products over pixels that have similar sun-view geometries. The results indicate general linear relationships between GEONEX and corresponding MODIS LSRs. In particular, the RMSEs between GEONEX and MOD09 data are comparable to those between MOD09 and MODIS MAIAC products, suggesting that the uncertainties of GEONEX LSRs fall into an acceptable range. However, direct comparisons of LSRs over pixels with different sun-view angles are not as straightforward and require more modeling efforts to correct the directional effects. Evaluation of such angular influences on the downstream products (e.g., vegetation indices) is also under investigation.

Geostationary satellite; Remote Sensing; Atmospher↗
NASA NTRS2022

Characterization of Response Times based on Voice Communication and Traffic Surveillance Data

A barrier to the integration of remotely piloted aircraft operations in the U.S. National Airspace System is the latency of voice communications between the air traffic controller and the remote pilot, and the latency of communication between the aircraft and the remote pilot. The latency can be substantial especially when satellite-based beyond-radio-line-of-sight communication and relay through the aircraft are employed. This study uses voice recordings of controller-pilot communications and aircraft track data to establish a baseline of pilot readback latencies and maneuver detection delays in the current piloted operations. A machine learning pipeline was developed to parse the contents of the air traffic control clearances including the callsigns using natural language processing. After manually validating the results obtained using the pipeline, the average pilot readback latency was found to be about 0.6 seconds. The average latency between the end of maneuver (inferred from track data), initiated by the pilot in response to the clearance, and the end of clearance was found to be about 176 seconds for altitude change commands, 69 seconds for heading change commands, and 182 seconds for speed change commands. The average latency between the beginning of maneuver and the end of clearance was found to be about 17 seconds for altitude change commands, 17seconds for heading change commands, and 25 seconds for speed change commands.

controller-pilot communication, communication late↗
NASA NTRS2022

Characterization of Response Times Based on Voice Communication and Traffic Surveillance Data

A barrier to the integration of remotely piloted aircraft operations in the U.S. National Airspace System is the latency of voice communications between the air traffic controller and the remote pilot, and the latency of communication between the aircraft and the remote pilot. The latency can be substantial especially when satellite-based beyond-radio-line-of-sight communication and relay through the aircraft are employed. This study uses voice recordings of controller-pilot communications and aircraft track data to establish a baseline of pilot readback latencies and maneuver detection delays in the current piloted operations. A machine learning pipeline was developed to parse the contents of the air traffic control clearances including the callsigns using natural language processing. After manually validating the results obtained using the pipeline, the average pilot readback latency was found to be about 0.6 seconds. The average latency between the end of maneuver (inferred from track data), initiated by the pilot in response to the clearance, and the end of clearance was found to be about 176 seconds for altitude change commands, 69 seconds for heading change commands, and 182 seconds for speed change commands. The average latency between the beginning of maneuver and the end of clearance was found to be about 17 seconds for altitude change commands, 17seconds for heading change commands, and 25 seconds for speed change commands.

controller-pilot communication↗
NASA NTRS2023

BatAnalysis - A Comprehensive Python Pipeline for Swift BAT Survey Analysis

The Swift Burst Alert Telescope (BAT) is a coded aperture gamma-ray instrument with a large field of view that primarily operates in survey mode when it is not triggering on transient events. The survey data consists of eighty-channel detector plane histograms that accumulate photon counts over time periods of at least 5 minutes. These histograms are processed on the ground and are used to produce the survey dataset between 14 and 195 keV. Survey data comprises >90% of all BAT data by volume and allows for the tracking of long term light curves and spectral properties of cataloged and uncataloged hard X-ray sources. Until now, the survey dataset has not been used to its full potential due to the complexity associated with its analysis and the lack of easily usable pipelines. Here, we introduce the BatAnalysis python package , a wrapper for HEASoftpy, which provides a modern, open-source pipeline to process and analyze BAT survey data. BatAnalysis allows members of the community to use BAT survey data in more advanced analyses of astrophysical sources including pulsars, pulsar wind nebula, active galactic nuclei, and other known/unknown transient events that may be detected in the hard X-ray band. We outline the steps taken by the python code and exemplify its usefulness and accuracy by analyzing survey data from the Crab Pulsar, NGC 2992, and a previously uncataloged MAXI Transient. The BatAnalysis package allows for ∼ 18 years of BAT survey to be used in a systematic way to study a large variety of astrophysical sources.

Tyler Parsotan↗
NASA NTRS2010

Robust Mosaicking of Stereo Digital Elevation Models from the Ames Stereo Pipeline

Robust estimation method is proposed to combine multiple observations and create consistent, accurate, dense Digital Elevation Models (DEMs) from lunar orbital imagery. The NASA Ames Intelligent Robotics Group (IRG) aims to produce higher-quality terrain reconstructions of the Moon from Apollo Metric Camera (AMC) data than is currently possible. In particular, IRG makes use of a stereo vision process, the Ames Stereo Pipeline (ASP), to automatically generate DEMs from consecutive AMC image pairs. However, the DEMs currently produced by the ASP often contain errors and inconsistencies due to image noise, shadows, etc. The proposed method addresses this problem by making use of multiple observations and by considering their goodness of fit to improve both the accuracy and robustness of the estimate. The stepwise regression method is applied to estimate the relaxed weight of each observation.

Kim, Tae Min↗
NASA NTRSDate not supplied

Artificial Intelligence (AI) Methods for Automating the Impact Tool Evidence Library

INTRODUCTION: The development of the Evidence Library for use with the IMPACT (Informing Mission Planning via Analysis of Complex Tradespaces) probability risk assessment tool involved a multilayered, time intensive process of data collection and analysis by subject matter experts from the Exploration Medical Capability (ExMC) Element Clinical and Science Team to produce clinical findings forms (CliFFs) for 120 medical conditions. Artificial Intelligence Large Language Models (LLMs) can be leveraged to facilitate this process, thus reducing labor and time. TOPIC: CliFFs contain information about medical conditions as they pertain to spaceflight. This includes condition definitions, incidence data, crew task impairment estimates caused by conditions, treatment protocols and references to literature used for gathering condition evidence. Guided by the Evidence Library Methods document and the CliFF development instructions, a team has leveraged Microsoft Azure AI services and open-source documentation to construct an AI-assisted automated pipeline for CliFF development. This process is designed to search, retrieve, and evaluate the applicable data, and ultimately generate a completed CliFF. The LLM evaluates the relevance of each of the source materials to spaceflight, either as direct evidence or as an analog. The model extracts keywords and generates brief summaries to enhance search and retrieval in later stages of CliFF development. For instance, it can calculate epidemiological statistical data, such as incidence rates and the likelihood of best or worst-case scenarios. APPLICATION: Large Language Models (LLMs) can efficiently summarize large amounts of text. Leveraging this technology will automate data retrieval and evidence gathering for medical databases, like the IMPACT tool, by aiding in the labor-intensive process of analyzing large bodies of literature and organizing it into a formatted document like a CliFF. This added efficiency will enable expeditious expansion of the Evidence Library with additional medical conditions and update previous CLiFFs as new technology becomes available.

Ali Al↗
NASA NTRS2022

Classifying Unidentified X-Ray Sources in the Chandra Source Catalog Using A Multiwavelength Machine-Learning Approach

The rapid increase in serendipitous X-ray source detections requires the development of novel approaches to efficiently explore the nature of X-ray sources. If even a fraction of these sources could be reliably classified, it would enable population studies for various astrophysical source types on a much larger scale than currently possible. Classification of large numbers of sources from multiple classes characterized by multiple properties (features) must be done automatically and supervised machine learning (ML) seems to provide the only feasible approach. We perform classification of Chandra Source Catalog version 2.0 (CSCv2) sources to explore the potential of the ML approach and identify various biases, limitations, and bottlenecks that present themselves in these kinds of studies. We establish the framework and present a flexible and expandable Python pipeline, which can be used and improved by others. We also release the training data set of 2941 X-ray sources with confidently established classes. In addition to providing probabilistic classifications of 66,369 CSCv2 sources (21% of the entire CSCv2 catalog), we perform several narrower-focused case studies (high-mass X-ray binary candidates and X-ray sources within the extent of the H.E.S.S. TeV sources) to demonstrate some possible applications of our ML approach. We also discuss future possible modifications of the presented pipeline, which are expected to lead to substantial improvements in classification confidences.

Hui Yang↗
NASA NTRS1986

Convolver for Pipelined-Image Processor

3 x 3 convolver produces weighted sum of nine contiguous picture elements in square. Data processed through convolver at video scanning rate of current raster line. Two previous lines stored in external buffers (N-3)element delays. Specific choice of convolution weights determines whether convolver performs smoothing, spatial-frequency filtering, edge detection, or other forms of image processing.

Wilcox, B.↗
NASA NTRS2004

Central Stars of Planetary Nebulae in the LMC

In FUSE cycle 2's program B001 we studied Central Stars of Planetary Nebulae (CSPN) in the Large Magellanic Could. All FUSE observations have been successfully completed and have been reduced, analyzed and published. The analysis and the results are summarized below. The FUSE data were reduced using the latest available version of the FUSE calibration pipeline (CALFUSE v2.2.2). The flux of these LMC post-AGB objects is at the threshold of FUSE's sensitivity, and thus special care in the background subtraction was needed during the reduction. Because of their faintness, the targets required many orbit-long exposures, each of which typically had low (target) count-rates. Each calibrated extracted sequence was checked for unacceptable count-rate variations (a sign of detector drift), misplaced extraction windows, and other anomalies. All the good calibrated exposures were combined using FUSE pipeline routines. The default FUSE pipeline attempts to model the background measured off-target and subtracts it from the target spectrum. We found that, for these faint objects, the background appeared to be over-estimated by this method, particularly at shorter wavelengths (i.e., < 1000 A). We therefore tried two other reductions. In the first method, subtraction of the measured background is turned off and and the background is taken to be the model scattered-light scaled by the exposure time. In the second one, the first few steps of the pipeline were run on the individual exposures (correcting for effects unique to each exposure such as Doppler shift, grating motions, etc). Then the photon lists from the individual exposures were combined, and the remaining steps of the pipeline run on the combined file. Thus, more total counts for both the target and background allowed for a better extraction.

Bianchi, Luciana↗
NASA NTRS2017

Searching the Gamma-Ray Sky for Counterparts to Gravitational Wave Sources Fermi Gamma-Ray Burst Monitor and Large Area Telescope Observations of LVT151012 and GW151226

We present the Fermi Gamma-ray Burst Monitor (GBM) and Large Area Telescope (LAT) observations of the LIGO binary black hole merger event GW151226 and candidate LVT151012. At the time of the LIGO triggers on LVT151012 and GW151226, GBM was observing 68% and 83% of the localization regions, and LAT was observing 47% and 32%, respectively. No candidate electromagnetic counterparts were detected by either the GBM or LAT. We present a detailed analysis of the GBM and LAT data over a range of timescales from seconds to years, using automated pipelines and new techniques for characterizing the flux upper bounds across large areas of the sky. Due to the partial GBM and LAT coverage of the large LIGO localization regions at the trigger times for both events, differences in source distances and masses, as well as the uncertain degree to which emission from these sources could be beamed, these non-detections cannot be used to constrain the variety of theoretical models recently applied to explain the candidate GBM counterpart to GW150914.

gamma rays: general – gravitational waves – me↗
NASA NTRS2020

Gamma-Ray Urgent Archiver for Novel Opportunities (GUANO): Swift/BAT Event Data Dumps on Demand to Enable Sensitive Subthreshold GRB Searches

We introduce a new capability of the Neil Gehrels Swift Observatory to provide event-level data from the Burst Alert Telescope (BAT) on demand in response to transients detected by other instruments. We show that the availability of these data can effectively increase the rate of detections and arcminute localizations of gamma-ray bursts (GRB) like GRB 170817 by >400%. We describe an autonomous spacecraft-commanding pipeline purpose built to enable this science; to our knowledge, this is the first fully autonomous extremely low-latency commanding of a space telescope for scientific purposes. This pipeline has been successfully run in its complete form since 2020 January, and has resulted in the recovery of BAT event data for >800 externally triggered events to date (gravitational waves, GWs; neutrinos; GRBs triggered by other facilities; fast radio bursts; and very high-energy detections), now running with a success rate of ∼90%. We exemplify the utility of this new capability by using the resultant data to (1) set the most sensitive upper limits on prompt 1 s duration short GRB-like emission within ±15 s around the unmodeled GW burst candidate S200114f, and (2) provide an arcminute localization for short GRB 200325A and other bursts. We also show that using data from GUANO to localize GRBs discovered by other instruments, we can increase the net rate of arcminute-localized GRBs by 10%–20% per year. Along with the scientific yield of more sensitive searches for subthreshold GRBs, the new capabilities designed for this project will serve as the foundation for further automation and rapid target of opportunity capabilities for the Swift mission, and have implications for the design of future rapid-response space telescopes.

Aaron Tohuvavohu↗
NASA NTRSDate not supplied

2022 Spring Internship Exit Presentation

As efforts of the National Aeronautics and Space Administration (NASA) and the Federal Aviation Administration (FAA) continue to digitize the air traffic management (ATM) domain, there is countless times of need for downstream natural language processing (NLP) tasks such as named entity recognition, text summarization, classification, and more. Although there are a plethora of open-sourced pre-trained transformer models in the NLP field such as BERT, RoBERTa, XLNet, and GPT-3, these models are trained on general corpora and perform poorly on domain-specific terminology and phraseology seen in ATM documents such as Notice to Airmen (NOTAMs) and Letters of Agreement (LoA). Our proposed research objective will be to first gather a large corpus of air traffic management related documents, orders, notices, books, technical papers, conference papers, articles, and other miscellaneous sources of text data from the FAA, NASA, and accredited conference and publication societies. After gathering this data, many steps will have to be taken to collate and preprocess the data into a format understandable by our test transformer models. Thirdly, we will set up training pipelines to train the RoBERTa model on its unsupervised training task masked language modelling (MLM) using resources provided by the NASA Advanced Supercomputing (NAS) facilities. Finally, these fine-tuned transformer models will be evaluated on their performance on down-stream NLP tasks as mentioned above, to show whether they will be effective when working with ATM related data or not. Once complete, this model could be made open-sourced on the HuggingFace website, where the rest of the ATM community can access and utilize this tool.

NLP↗
NASA NTRS1973

A Multidisciplinary Survey for the Management of Alaskan Resources Utilizing ERTS Imagery

The application of ERTS-1 imagery for a multidisciplinary study of Alaskan resources is discussed. ERTS-1 data form a politically neutral base from which resource surveys can be made without arousing suspicions from groups which have mutually confliciting goals. The specific use of ERTS-1 imagery for monitoring migratory animals, pipeline locations, road construction, snow cover, and vegetation growth is reported. Maps, illustrations, and photographs are provided to support the data.

Miller, J. M.↗
NASA NTRS2005

System Developed for Real-Time Blade-Flutter Monitoring in the Wind Tunnel

A real-time system has been developed to monitor flutter vibrations in turbomachinery. The system is designed for continuous processing of blade tip timing data at a rate of 10 MB/sec. A USB 2.0 interface provides uninterrupted real-time processing of the data, and the blade-tip arrival times are measured with a 50-MHz oscillator and a 24-bit pipelined architecture counter. The input stage includes a glitch catcher, which reduces the probability of detecting a ghost blade to negligible levels. A graphical user interface provides online interrogation of any blade tip from any light probe sensor. Alternatively, data from all blades and all sensors can be superimposed into a single composite scatter plot displaying the vibration amplitude of each blade.

Kurkov, Anatole P.↗
NASA NTRS1979

On-board image correction

This paper describes a processor architecture for performing on-board geometric and radiometric correction of Landsat imagery. The design uses a general purpose processor to calculate the distortion values at selected points in the image and a special purpose processor to resample (calculate distortion at each image point and interpolate the intensity) the sensor output data. A distinct special purpose processor is used for each spectral band. Because of the sensor's high output data rate, 80 M bit per second, the special purpose processors use a pipeline architecture. Sizing has been done of both the general and special purpose hardware.

Martin, D. R.↗
NASA NTRS1980

Onboard image correction

A processor architecture for performing onboard geometric and radiometric correction of LANDSAT imagery is described. The design uses a general purpose processor to calculate the distortion values at selected points in the image and a special purpose processor to resample (calculate distortion at each image point and interpolate the intensity) the sensor output data. A distinct special purpose processor is used for each spectral band. Because of the sensor's high output data rate, 80 M bit per second, the special purpose processors use a pipeline architecture. Sizing has been done on both the general and special purpose hardware.

Martin, D. R.↗
NASA NTRS1990

Intensity dependent spread processor and workstation

The Intensity Dependent Spread (IDS) is an adaptive algorithm which is modified according to the local intensity in the scene. (This results in a nonlinear process which cannot take advantage of rather nice linear transform methods.) The computation is similar to a neural net whereby intensity information is moving from each input pixel to a set of surrounding output pixels in a manner described by Cornsweet and Yellott. A prototype of a very large scale integration IDS processor is being developed and implemented in a workstation environment. The workstation consists of a SUN 3/260 and a DATACUBE pipeline processor. The IDS prototype is a board set which operates in the DATA CUBE processor. The SUN 3/260 performs control, background processing, IDS simulation and image display functions.

Westrom, George↗