Results for “pipeline data processing” — Page 10

NASA NTRS2011

Orthographic Stereo Correlator on the Terrain Model for Apollo Metric Images

A stereo correlation method on the object domain is proposed to generate the accurate and dense Digital Elevation Models (DEMs) from lunar orbital imagery. The NASA Ames Intelligent Robotics Group (IRG) aims to produce high-quality terrain reconstructions of the Moon from Apollo Metric Camera (AMC) data. In particular, IRG makes use of a stereo vision process, the Ames Stereo Pipeline (ASP), to automatically generate DEMs from consecutive AMC image pairs. Given camera parameters of an image pair from bundle adjustment in ASP, a correlation window is defined on the terrain with the predefined surface normal of a post rather than image domain. The squared error of back-projected images on the local terrain is minimized with respect to the post elevation. This single dimensional optimization is solved efficiently and improves the accuracy of the elevation estimate.

Terrain Model↗

NASA NTRS2017

Fireballs in the Sky: An Augmented Reality Citizen Science Program

Fireballs in the Sky is an innovative Australian citizen science program that connects the public with the research of the Desert Fireball Network (DFN). This research aims to understand the early workings of the solar system, and Fireballs in the Sky invites people around the world to learn about this science, contributing fireball sightings via a user-friendly augmented reality mobile app. Tens of thousands of people have downloaded the app world-wide and participated in the science of meteoritics. The Fireballs in the Sky app allows users to get involved with the Desert Fireball Network research, supplementing DFN observations and providing enhanced coverage by reporting their own meteor sightings to DFN scientists. Fireballs in the Sky reports are used to track the trajectories of meteors - from their orbit in space to where they might have landed on Earth. Led by Phil Bland at Curtin University in Australia, the Desert Fireball Network (DFN) uses automated observatories across Australia to triangulate trajectories of meteorites entering the atmosphere, determine pre-entry orbits, and pinpoint their fall positions. Each observatory is an autonomous intelligent imaging system, taking 1000 by 36 megapixel all-sky images throughout the night, using neural network algorithms to recognize events. They are capable of operating for 12 months in a harsh environment, and store all imagery collected. We developed a completely automated software pipeline for data reduction, and built a supercomputer database for storage, allowing us to process our entire archive. The DFN currently stands at 50 stations distributed across the Australian continent, covering an area of 2.5 million square kilometers. Working with DFN's partners at NASA's Solar System Exploration Research Virtual Institute, the team is expanding the network beyond Australia to locations around the world. Fireballs in the Sky allows a growing public base to learn about and participate in this exciting research.

Day, Brian↗

NASA NTRS2019

Creating Data-Driven Vector Visualizations of Satellite Orbit Tracks Using NASA GIBS and Worldview

NASA Earth Observing System (EOS) currently operates dozens of remote sensing satellites, many of which can be viewed directly in NASA’s open-source Worldview application. Much of this satellite imagery can be viewed in near-real time as it is processed and served by NASA’s Global Imagery Browse Service (GIBS). To better educate users on the time and location of imagery, GIBS serves orbit track specific layers for each satellite. Worldview has historically served these layers as raster images but recent updates have enabled the application to now serve these layers using vector tiles. With the release of Worldview v3.0, orbit track layers can be displayed using mapbox vector tiles (MVT). This visualization format allows users to not only view and change the color of orbit track layers, as they could do previously with rasters, but also inspect individual vector points and filter layers by specific parameters such as time. The data contained within a MVT is further enhanced in Worldview with the combination of a JSON description file served from GIBS used to describe the MVT data. This presentation will provide an overview of the process of consuming orbit track vector tiles and data files from GIBS using a pipeline to configure, build and ultimately display the orbit tracks in Worldview. Furthermore, the presentation aims to describe how others can leverage our open-source code to display and enhance vector layers in their own applications.

Rice, Zachary↗

NASA NTRSDate not supplied

The NASA Open Science Data Repository: Biomedical Fair Data, Analysis Tools, User Communities, Publications, and Discoveries for Deep Space Missions

Increased biomedical risks and challenges associated with deep space missions require new knowledge discovery, new health countermeasures, and development of novel ecosystems, life support, crop production, and biomedical support capabilities. To meet NASA’s Moon to Mars strategic program goals for Human and Biological Sciences, findable, accessible, interoperable, reusable (FAIR), and maximally open-access data is going to be required to enable humanity to thrive in deep space. Indeed, this cornerstone perspective on FAIR and maximally open access data was also recommended in the recent 2023-2032 Decadal Survey from the National Academies of Sciences, Engineering, and Medicine. The NASA Open Science Data Repository (OSDR) is a maximally open access and FAIR database, and meets various scientific, technical, and operational spaceflight needs. It offers public users and submitters the ability to upload, download, search, share, analyze, and visualize data across ‘omics, physiological, phenotypic, behavioral, bioimaging, video, and environmental monitoring telemetry datasets. OSDR includes NASA GeneLab, NASA Ames Life Sciences Data Archive, and the NASA Biological Institutional Scientific Collection. OSDR has >455 studies with datasets from model organisms and non-NASA human astronauts. There are ~12 datasets from the Inspiration 4 (I4) mission, spanning metagenomics, comprehensive metabolic panels, clonal hematopoiesis, spatial transcriptomics, proteomics, and cytokine panels. In the interest of data privacy, two I4 datasets have raw FASTQ and FASTA files relating to the epitranscriptome, and a new request feature is live in OSDR (with a backend review process established) which was developed based on industry norms. OSDR also recently began a collaboration with the European Space Agency (ESA) to scientifically curate and make available >200 terabytes of human and model organism space-relevant data. The OSDR submission portal is designed to ingest and curate ~25 ‘omics assay data types, and ~50 physiological-phenotypic-imaging assay data types, spanning ultrasonography, micro-computed tomography, histology, morphometric photography, rebound tonometry, gait analysis, optical coherence tomography, novel object recognition, flow cytometry, and immunohistochemistry. A suite of analysis tools are available for OSDR users including: 1) an Environmental Data Application to compare radiation, CO2, relative humidity, temperature, and other telemetry across missions and subjects, 2) the RadLab database, a collaboration between NASA, ESA, the German and Italian Space Agencies, and the Bulgarian Academy of Sciences, which compiles radiation measurements relevant to human spaceflight and provides tools for accessing and manipulating the data, and 3) a Multi-study visualization tool which enables users to look across and combine GeneLab’s omics datasets across different experiments and missions. There are ~600 volunteer OSDR Analysis Working Group (AWG) members who: 1) provide feedback on scientific standards for reuse (subject and assay metadata; processing pipelines; dataset formats and uniformed structures for machine-readability), and 2) collaborate to mine-reuse OSDR data conducting scientific analysis. OSDR has enabled 60 publications as of September 2023, many directly from AWG collaborations most notably the Cell Press package in 2020. Lastly, there are at least 15 articles which mine OSDR data part of a package of ~50 articles across Nature Portfolio with research stemming from I4, the Japan Aerospace Exploration Agency, NASA Space Biology, and the NASA Human Research Program.

space biology↗

NASA NTRSDate not supplied

NASA Open Science Data Repository: Biomedical FAIR Data, Analysis Tools, User Communities, and Discoveries for Deep Space Missions

Increased biomedical risks and challenges associated with deep space missions require new knowledge discovery, new health countermeasures, and development of novel ecosystems, life support, crop production, and biomedical support capabilities. To meet NASA’s Moon to Mars strategic program goals for Human and Biological Sciences, findable, accessible, interoperable, reusable (FAIR), and maximally open-access data is going to be required to enable humanity to thrive in deep space. Indeed, this cornerstone perspective on FAIR and maximally open access data was also recommended in the recent 2023-2032 Decadal Survey from the National Academies of Sciences, Engineering, and Medicine. The NASA Open Science Data Repository (OSDR) is a maximally open access and FAIR database, and meets various scientific, technical, and operational spaceflight needs. It offers public users and submitters the ability to upload, download, search, share, analyze, and visualize data across ‘omics, physiological, phenotypic, behavioral, bioimaging, video, and environmental monitoring telemetry datasets. OSDR includes NASA GeneLab, NASA Ames Life Sciences Data Archive, and the NASA Biological Institutional Scientific Collection. OSDR has >455 studies with datasets from model organisms and non-NASA human astronauts. There are ~12 datasets from the Inspiration 4 (I4) mission, spanning metagenomics, comprehensive metabolic panels, clonal hematopoiesis, spatial transcriptomics, proteomics, and cytokine panels. In the interest of data privacy, two I4 datasets have raw FASTQ and FASTA files relating to the epitranscriptome, and a new request feature is live in OSDR (with a backend review process established) which was developed based on industry norms. OSDR also recently began a collaboration with the European Space Agency (ESA) to scientifically curate and make available >200 terabytes of human and model organism space-relevant data. The OSDR submission portal is designed to ingest and curate ~25 ‘omics assay data types, and ~50 physiological-phenotypic-imaging assay data types, spanning ultrasonography, micro-computed tomography, histology, morphometric photography, rebound tonometry, gait analysis, optical coherence tomography, novel object recognition, flow cytometry, and immunohistochemistry. A suite of analysis tools are available for OSDR users including: 1) an Environmental Data Application to compare radiation, CO2, relative humidity, temperature, and other telemetry across missions and subjects, 2) the RadLab database, a collaboration between NASA, ESA, the German and Italian Space Agencies, and the Bulgarian Academy of Sciences, which compiles radiation measurements relevant to human spaceflight and provides tools for accessing and manipulating the data, and 3) a Multi-study visualization tool which enables users to look across and combine GeneLab’s omics datasets across different experiments and missions. There are ~600 volunteer OSDR Analysis Working Group (AWG) members who: 1) provide feedback on scientific standards for reuse (subject and assay metadata; processing pipelines; dataset formats and uniformed structures for machine-readability), and 2) collaborate to mine-reuse OSDR data conducting scientific analysis. OSDR has enabled 60 publications as of September 2023, many directly from AWG collaborations most notably the Cell Press package in 2020. Lastly, there are at least 15 articles which mine OSDR data part of a package of ~50 articles across Nature Portfolio with research stemming from I4, the Japan Aerospace Exploration Agency, NASA Space Biology, and the NASA Human Research Program.

open access↗

NASA NTRSDate not supplied

AN INTRODUCTION TO THE GEONEX LEVEL-1G PRODUCTS: TOP-OF-ATMOSPHERE REFLECTANCE AND BRIGHTNESS TEMPERATURE

This paper introduces the GeoNEX (Geostationary-NASA Earth eXchange) Level-1G products of top-of-atmosphere (TOA) reflectance and brightness temperature. The products use data streams from the latest geostationary (GEO) sensors including the GOES-16/17 ABI and the Himawari-8/9 AHI. The GeoNEX processing pipeline starts by converting digital numbers to physical quantities with the latest radiometric calibration information. It integrates algorithms to automatically detect and remove residual geolocation errors, to estimate the pixel-wise data-acquisition time, and to accurately calculate the solar illumination angles for each pixel in the domain at every time step. The outputs are reprojected to a globally tiled common grid in geographic coordinates designed to facilitate inter-comparisons and/or synergies between the GeoNEX products and existing Earth observation datasets from polar-orbiting satellites. Therefore, the GeoNEX L1G products provide accurate and consistent TOA reflectance and brightness temperature datasets for scientific analyses and downstream product development.

Geostationary satellite, GOES-16, Himawari-8, NASA↗

NASA NTRS1986

Convolver for Pipelined-Image Processor

3 x 3 convolver produces weighted sum of nine contiguous picture elements in square. Data processed through convolver at video scanning rate of current raster line. Two previous lines stored in external buffers (N-3)element delays. Specific choice of convolution weights determines whether convolver performs smoothing, spatial-frequency filtering, edge detection, or other forms of image processing.

Wilcox, B.↗

NASA NTRS1981

Real-time image enhancement

Pipelined system with "vision" algorithm is implemented on LSI chip that processes input digital image data to produce image-edge map. System contains 3 input adder, difference and absolute value cells, and adder and comparator. Data store for 1 to 2 ms, and are easily transmitted or isolated; design has reduced package count and number of interconnections for increased reliability. Applications include locating objects on moving belt, deep-sea and coal mining, and control of robotic rovers.

Wong, V. S.↗

NASA NTRS1993

A comparison of multiprocessor scheduling methods for iterative data flow architectures

A comparative study is made between the Algorithm to Architecture Mapping Model (ATAMM) and three other related multiprocessing models from the published literature. The primary focus of all four models is the non-preemptive scheduling of large-grain iterative data flow graphs as required in real-time systems, control applications, signal processing, and pipelined computations. Important characteristics of the models such as injection control, dynamic assignment, multiple node instantiations, static optimum unfolding, range-chart guided scheduling, and mathematical optimization are identified. The models from the literature are compared with the ATAMM for performance, scheduling methods, memory requirements, and complexity of scheduling and design procedures.

Storch, Matthew↗

NASA NTRS2021

Automated Data Accountability for Missions in Mars Rover Data

As the Mars Curiosity Rover transmits data to the JPL Ground Data System (GDS), it frequently observes data loss and corruption, requiring re-transmits from the rover and Ground Data System Analysts (GDSA) to monitor the downlink process. As new missions are launched, the GDSA team redistributes analysts to these new missions, causing shortages in previous missions. The GDSA team can significantly benefit from the automation and optimization of the downlink process of telemetry data. In fact, there is a need for a better understanding of why the data is corrupted, so that the GDSA team can best determine the root cause of the issues in the GDS. This paper presents machine learning and deep learning based approaches to automate and optimize the detection of data loss. We first created a pipeline to automatically accumulate data from the telemetry databases (MAROS, Telemetry Data Storage, and GDS Elastic Search Database) in the downlink process. With our newly created datasets, we perform feature selection to supplement the GDSA understanding of the downlink process and provide supplemental analysis on the importance of different features. We implement various machine learning and deep learning based models, including support vector machines, ensemble methods, and deep neural networks and evaluate their accuracies in identifying whether a downlink process is complete or incomplete. We utilize fast hyperparameter optimization methods that allow our models to quickly be re-trained, allowing them to quickly be tuned and optimized on daily incoming data in real time. This hyperparameter optimization also allows our methods to be quickly integrated into other JPL missions. Our results show that our best-performing machine learning and deep learning based models outperform the existing GDSA detection software by 6 accuracy points and can aid analysts by providing insights into the data accountability problem. Since these various machine learning and deep learning approaches vary significantly in interpretability, we provide a discussion on the tradeoffs between their performance and trustworthiness in helping detect issues in data transmission.

Divsalar, Dariush↗

NASA NTRS1992

Implementation and performance of the Magellan digital correlator subsystem

The Magellan synthetic aperture radar (SAR) produces Venus surface images from data collected by the SAR carried on board the Magellan spacecraft. The core of the primary Magellan SAR processor is the digital correlator subsystem (DCS). The pipeline DSC architecture enables the Magellan primary SAR processor (PSP) to achieve real-time data processing capability. The implementation and performance of the DSC are described. Hardware (H/W) constraints that influenced the processing algorithm design are highlighted.

Chen, M.↗

NASA NTRS1997

Joint US-Japan Observations with the Infrared Space Observatory (ISO): Deep Surveys and Observations of High-Z Objects

Several important milestones were passed during the past year of our ISO observing program: (1) Our first ISO data were successfully obtained. ISOCAM data were taken for our primary deep field target in the 'Lockman Hole'. Thirteen hours of integration (taken over 4 contiguous orbits) were obtained in the LW2 filter of a 3 ft x 3 ft region centered on the position of minimum HI column density in the Lockman Hole. The data were obtained in microscanning mode. This is the deepest integration attempted to date (by almost a factor of 4 in time) with ISOCAM. (2) The deep survey data obtained for the Lockman Hole were received by the Japanese P.I. (Yoshi Taniguchi) in early December, 1996 (following release of the improved pipeline formatted data from Vilspa), and a copy was forwarded to Hawaii shortly thereafter. These data were processed independently by the Japan and Hawaii groups during the latter part of December 1996, and early January, 1997. The Hawaii group made use of the U.S. ISO data center at IPAC/Caltech in Pasadena to carry out their data reduction, while the Japanese group used a copy of the ISOCAM data analysis package made available to them through an agreement with the head of the ISOCAM team, Catherine Cesarsky. (3) Results of our LW2 Deep Survey in the Lockman Hole were first reported at the ISO Workshop "Taking ISO to the Limits: Exploring the Faintest Sources in the Infrared" held at the ISO Science Operations Center in Villafranca, Spain (VILSPA) on 3-4 February, 1997. Yoshi Taniguchi gave an invited presentation summarizing the results of the U.S.-Japan team, and Dave Sanders gave an invited talk summarizing the results of the Workshop at the conclusion of the two day meeting. The text of the talks by Taniguchi and Sanders are included in the printed Workshop Proceedings, and are published in full on the Web. By several independent accounts, the U.S.-Japan Deep Survey results were one of the highlights of the Workshop; these data showed conclusively that the ISOCAM S/N continues to decrease as the square root of time for periods as long as 13 hours.

Sanders, David B.↗

NASA NTRS1992

Alaska SAR processor implementation of E-ERS-1

The synthetic aperture radar (SAR) data processing algorithm used by the Alaska SAR Facility (ASF) for the European Space Agency's first Remote-Sensing Satellite (E-ERS-1) SAR data are examined. Preprocessing highlights two features: signal measurement, which includes signal-to-noise ratio, replica measurement, and noise measurement; and Doppler measurement, which includes clutter lock and autofocus. The custom pipeline architecture performs the main processing with controls at the input interface, range correlator, corner-turn memory, azimuth correlator, and multi-look memory. The control software employs a flexible control scheme. The Committee on Earth Observation Satellites (CEOS) format encapsulates the ASF products. System performance for SAR image processing of E-ERS-1 data is reviewed.

Cuddy, David↗

NASA NTRS2022

Characterization of Response Times based on Voice Communication and Traffic Surveillance Data

A barrier to the integration of remotely piloted aircraft operations in the U.S. National Airspace System is the latency of voice communications between the air traffic controller and the remote pilot, and the latency of communication between the aircraft and the remote pilot. The latency can be substantial especially when satellite-based beyond-radio-line-of-sight communication and relay through the aircraft are employed. This study uses voice recordings of controller-pilot communications and aircraft track data to establish a baseline of pilot readback latencies and maneuver detection delays in the current piloted operations. A machine learning pipeline was developed to parse the contents of the air traffic control clearances including the callsigns using natural language processing. After manually validating the results obtained using the pipeline, the average pilot readback latency was found to be about 0.6 seconds. The average latency between the end of maneuver (inferred from track data), initiated by the pilot in response to the clearance, and the end of clearance was found to be about 176 seconds for altitude change commands, 69 seconds for heading change commands, and 182 seconds for speed change commands. The average latency between the beginning of maneuver and the end of clearance was found to be about 17 seconds for altitude change commands, 17seconds for heading change commands, and 25 seconds for speed change commands.

controller-pilot communication, communication late↗

NASA NTRS2022

Characterization of Response Times Based on Voice Communication and Traffic Surveillance Data

A barrier to the integration of remotely piloted aircraft operations in the U.S. National Airspace System is the latency of voice communications between the air traffic controller and the remote pilot, and the latency of communication between the aircraft and the remote pilot. The latency can be substantial especially when satellite-based beyond-radio-line-of-sight communication and relay through the aircraft are employed. This study uses voice recordings of controller-pilot communications and aircraft track data to establish a baseline of pilot readback latencies and maneuver detection delays in the current piloted operations. A machine learning pipeline was developed to parse the contents of the air traffic control clearances including the callsigns using natural language processing. After manually validating the results obtained using the pipeline, the average pilot readback latency was found to be about 0.6 seconds. The average latency between the end of maneuver (inferred from track data), initiated by the pilot in response to the clearance, and the end of clearance was found to be about 176 seconds for altitude change commands, 69 seconds for heading change commands, and 182 seconds for speed change commands. The average latency between the beginning of maneuver and the end of clearance was found to be about 17 seconds for altitude change commands, 17seconds for heading change commands, and 25 seconds for speed change commands.

controller-pilot communication↗

NASA NTRS1974

Digital registration of ERTS-1 imagery

Two requirements for the registration of Earth Resources Technology Satellite (ERTS) data are discussed. These requirements are registration of ERTS data acquired on separate passes and registration of ERTS data to a ground reference. Performances of a semi-automatic warp algorithm and an automatic pipeline processing algorithm demonstrate that either procedure is useful, depending upon the user's requirements. In two cases where the time lapse between passes of the satellite were 90 days and 18 days the automatic pipeline processor reduced the mean radial registration error to 0.28 and 0.58 pixel, respectively. It is concluded that this technique is promissing for high-volume production processing.

Bonrud, L. O.↗

NASA NTRS1992

Real-time processor for the Danish airborne SAR

A real-time processor for the Danish high-resolution SAR is presented in terms of its functional performance, algorithm, architecture, and implementation. The real-time processor is mainly intended to assist the operator in using the SAR system, but since the processor has been designed to produce high-quality images, it is expected to make off-line processing superfluous in many cases. The range-Doppler algorithm is adopted and supplemented with an extensive motion compensation, considering the special conditions related to the real-time strip mapping of large scenes. The processor is a pipeline of about 20 elements interconnected by a dedicated data path and a control bus. Only three different types of elements are involved: a programmable signal processing element, a multipurpose memory element, and a multipurpose interface element. The prototypes of these three elements have been tested with satisfactory results.

Dall, J.↗

NASA NTRS1990

Intensity dependent spread processor and workstation

The Intensity Dependent Spread (IDS) is an adaptive algorithm which is modified according to the local intensity in the scene. (This results in a nonlinear process which cannot take advantage of rather nice linear transform methods.) The computation is similar to a neural net whereby intensity information is moving from each input pixel to a set of surrounding output pixels in a manner described by Cornsweet and Yellott. A prototype of a very large scale integration IDS processor is being developed and implemented in a workstation environment. The workstation consists of a SUN 3/260 and a DATACUBE pipeline processor. The IDS prototype is a board set which operates in the DATA CUBE processor. The SUN 3/260 performs control, background processing, IDS simulation and image display functions.

Westrom, George↗