SEARCH · Search NASA
Results for “Pipeline Integrity”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Data Reduction Pipeline for the CHARIS Integral-Field Spectrograph I: Detector Readout Calibration and Data Cube Extraction
We present the data reduction pipeline for CHARIS, a high-contrast integral-field spectrograph for the Subaru Telescope. The pipeline constructs a ramp from the raw reads using the measured nonlinear pixel response and reconstructs the data cube using one of three extraction algorithms: aperture photometry, optimal extraction, or chi-squared fitting. We measure and apply both a detector flatfield and a lenslet flatfield and reconstruct the wavelength- and position-dependent lenslet point-spread function (PSF) from images taken with a tunable laser. We use these measured PSFs to implement a chi-squared-based extraction of the data cube, with typical residuals of approximately 5 percent due to imperfect models of the under-sampled lenslet PSFs. The full two-dimensional residual of the chi-squared extraction allows us to model and remove correlated read noise, dramatically improving CHARIS's performance. The chi-squared extraction produces a data cube that has been deconvolved with the line-spread function and never performs any interpolations of either the data or the individual lenslet spectra. The extracted data cube also includes uncertainties for each spatial and spectral measurement. CHARIS's software is parallelized, written in Python and Cython, and freely available on github with a separate documentation page. Astrometric and spectrophotometric calibrations of the data cubes and PSF subtraction will be treated in a forthcoming paper.
Improving Sim-to-Real Transfer in Vision-Based Robot Navigation Via Instance-Level GAN-Based Data Augmentation
Achieving robust vision-based robotic tasks requires large amounts of data, which are often difficult to obtain in real-world scenarios. Simulators and synthetic data offer a cost-effective alternative, but the visual gap between simulation and reality hinders the performance of models when deployed in real-world environments. In this paper, we present a data augmentation pipeline that integrates a foundation model (Segment Anything Model) with an unsupervised image-to-image translation model (CycleGAN) for instance-level domain transfer from simulation to reality. This pipeline enables the generation of realistic labeled data from synthetic images for training supervised machine learning models in vision-based navigation tasks. We evaluate our approach on real-world data for ego-vehicle pose estimation, a critical autonomous navigation task involving the prediction of cross-track position and heading angle relative to road center line markings. The results of our tests show that our GAN-based data augmentation pipeline significantly outperforms models trained solely on simulation data or on data processed with standard image augmentation methods for sim-to-real transfer, enhancing model robustness and generalizability in real-world scenarios. Our method provides a scalable and flexible data augmentation tool for leveraging large synthetic datasets to enhance vision-based robotic navigation tasks.
GeoNEX: A Geostationary Earth Observatory
The latest generation of geostationary satellites (Himawari 8/9, GOES-16/17, FY-4, GK-2A) carries sensors that closely mimic the spatial and spectral characteristics of widely used polar-orbiting, global monitoring sensors such as MODIS and VIIRS. When combined, data from various currently operating/planned geostationary platforms provide a geo-ring of hyper-temporal (5-10 minutes), multispectral observations at spatial resolutions as high as 500 m. These high frequency observations offer exciting new possibilities for monitoring our planet, including better retrievals of geophysical variables by overcoming cloud cover, enabling studies of diurnally varying phenomena in the atmosphere, land, and the oceans, and support operational decision-making in agriculture, hydrology and disaster management. The NASA Earth Exchange (NEX) team, in collaboration with scientists from JAXA, KARI, NOAA and other international institutions, created the GeoNEX (www.nasa.gov/geonex) pipeline to integrate data from all available geostationary platforms and produce and distribute spatially, temporally, and radiometrically consistent data for the earth science community. We envision various institutions adapting the Geo component (e.g., GeoNOAA, GeoKARI, GeoChiba, GeoJAXA, GeoCMA) and customizing the pipeline and downstream products to serve the local/regional research and applied science communities. To facilitate collaborative work among the partners, we have established the OpenNEX platform on the public cloud. OpenNEX provides researchers, developers, educators, and ordinary users with easy access to an integrated Earth science computational and data platform, enabling citizen scientists and application developers to realize the full value of GeoNEX data assets and software tools.
GeoNEX: A geostationary earth observatory at NASA Earth eXchange: Earth monitoring from operational geostationary satellite systems
The latest generation of geostationary satellites (Himawari 8/9, GOES-16/17, FY-4, GK-2A) carries sensors that closely mimic the spatial and spectral characteristics of widely used polar-orbiting, global monitoring sensors such as MODIS and VIIRS. When combined, data from various currently operating/planned geostationary platforms provide a geo-ring of hyper-temporal (5-10 minutes), multispectral observations at spatial resolutions as high as 500 m. These high frequency observations offer exciting new possibilities for monitoring our planet, including better retrievals of geophysical variables by overcoming cloud cover, enabling studies of diurnally varying phenomena in the atmosphere, land, and the oceans, and support operational decision-making in agriculture, hydrology and disaster management. The NASA Earth Exchange (NEX) team, in collaboration with scientists from JAXA, KARI, NOAA and other international institutions, created the GeoNEX (www.nasa.gov/geonex) pipeline to integrate data from all available geostationary platforms and produce and distribute spatially, temporally, and radiometrically consistent data for the earth science community. We envision various institutions adapting the Geo component (e.g., GeoNOAA, GeoKARI, GeoChiba, GeoJAXA, GeoCMA) and customizing the pipeline and downstream products to serve the local/regional research and applied science communities.
PISCES High Contrast Integral Field Spectrograph Simulations and Data Reduction Pipeline
The PISCES (Prototype Imaging Spectrograph for Coronagraphic Exoplanet Studies) is a lenslet array based integral field spectrograph (IFS) designed to advance the technology readiness of the WFIRST (Wide Field Infrared Survey Telescope)-AFTA (Astrophysics Focused Telescope Assets) high contrast Coronagraph Instrument. We present the end to end optical simulator and plans for the data reduction pipeline (DRP). The optical simulator was created with a combination of the IDL (Interactive Data Language)-based PROPER (optical propagation) library and Zemax (a MatLab script), while the data reduction pipeline is a modified version of the Gemini Planet Imager's (GPI) IDL pipeline. The simulations of the propagation of light through the instrument are based on Fourier transform algorithms. The DRP enables transformation of the PISCES IFS data to calibrated spectral data cubes.
Integrated-Circuit Active Digital Filter
Pipeline architecture with parallel multipliers and adders speeds calculation of weighted sums. Picture-element values and partial sums flow through delay-adder modules. After each cycle or time unit of calculation, each value in filter moves one position right. Digital integrated-circuit chips with pipeline architecture rapidly move 35 X 35 two-dimensional convolutions. Need for such circuits in image enhancement, data filtering, correlation, pattern extraction, and synthetic-aperture-radar image processing: all require repeated calculations of weighted sums of values from images or two-dimensional arrays of data.
NIAC Phase I Final Report Lunar South Pole Oxygen Pipeline (LSPOP)
The Lunar South Pole Oxygen Pipeline (LSPOP) Phase I NIAC, is to analyze the feasibility of constructing a pipeline at the Moon’s South Pole for transporting gaseous oxygen from point of generation to point of use. A lunar pipeline has never been pursued and will revolutionize lunar surface operations for the Artemis program and reduce cost and risk. Our starting concept is for a 5 km pipeline to transport oxygen gas from an oxygen production source, for example from our molten regolith electrolysis (MRE, currently at TRL 5, see references[1],[2],[3],[4],[5],[6] for technical discussions of the MRE process) extraction site, or any other source, to an oxygen storage/liquification plant near a lunar base. The pipeline is designed to: 1) be constructed robotically from regolith-derived metals with minimal material transferred from Earth, 2) be repairable robotically, 3) have an oxygen flow rate of ~2 kg/hour, which is commensurate with the NASA initial projected need of 10,000 kg/year, 4) operate with minimal power over the lifetime of the pipeline, with a goal of high operational reliability and the ability to survive in the lunar environment for > 10 years. We compare this concept to a pipeline concept with pipe produced on Earth and then transported to and assembled on the Moon. Both approaches will use earth-based compressors, valves, and fittings which are integrated into the pipeline on the Moon.
Development and Demonstration of a Digital NDE Pipeline for Streamlined Analysis of Ultrasonic Data
Currently when nondestructive evaluation (NDE) is performed on composite structures, the results, although recorded digitally are often manually interpreted and indicated (drawn) on the part being inspected by hand. Following this, a determination must be made on how to disposition that part. This decision could be based on engineering guidelines and best practices, rule-of-thumb, expert opinion or finite-element analysis of the part with some approximate representation of the damage. In order to streamline this process for ultrasonic inspection an effort was undertaken as part of NASA Advanced Composites Project to develop the Digital NDE Pipeline. The Digital NDE Pipeline is an integrated tool suite and associated framework that streamlines the inspection and defect disposition process through model-assisted inspection optimization, automated defect analysis of NDE data, and mapping of NDE data into finite element analysis software. This paper will provide an overview of the Digital NDE Pipeline and provide details of the individual tools developed along with the results of applying these tools to a demonstration test case.
Anomaly Detection for the Roman Space Telescope Wide Field Instrument’s Science Data Processing Pipeline
The Roman Space Telescope (RST) Wide Field Instrument (WFI) will be utilizing a preliminary Science Data Processing (SDP) pipeline during its Integration and Test, and to some extent during Operations, to track basic statistics and identify known features such as cosmic rays, snowballs as well as possible anomalies in raw detector data. In our detectors, these anomalies appear as jumps in the ramp of a readout and are classified as cosmic rays if they appear as a streak or snowballs if they’re more circular. The WFI employs an array of 18 H4RG-10 detectors that collect image samples. Each set of raw frames within a non-destructive exposure is packaged by the SDP pipeline into image cubes for each detector. Each cube is a time series of 4096 × 4096 accumulating pixel frames. The preliminary analysis pipeline is used to locate anomalies in these time-series accumulation frames and identify the type of anomaly, either natural phenomena or detector characteristic. To compare different methods, we’ve implemented both heuristic-based and data-driven methods to identify anomalies. For the heuristic-based approach, we identify snowballs and cosmic rays by the size and shape of outlier pixel clusters between consecutive frames. For data driven methods, we evaluated a Convolutional Neural Network (CNN) model, and more traditional methods like Principal Component Analysis (PCA). CNN is a supervised learning/classification method. Thus, we used a labeled dataset of anomalies to perform segmentation of the image and identify anomalies. We used previously identified cosmic rays and snowballs to measure the accuracy and efficiency of the mentioned approaches. In evaluating these methods, we aim to pick the best fit for the SDP pipeline’s anomaly detection in terms of both performance and runtime.
On the VLSI design of a pipeline Reed-Solomon decoder using systolic arrays
A new very large scale integration (VLSI) design of a pipeline Reed-Solomon decoder is presented. The transform decoding technique used in a previous article is replaced by a time domain algorithm through a detailed comparison of their VLSI implementations. A new architecture that implements the time domain algorithm permits efficient pipeline processing with reduced circuitry. Erasure correction capability is also incorporated with little additional complexity. By using a multiplexing technique, a new implementation of Euclid's algorithm maintains the throughput rate with less circuitry. Such improvements result in both enhanced capability and significant reduction in silicon area.
On the VLSI design of a pipeline Reed-Solomon decoder using systolic arrays
A new very large scale integration (VLSI) design of a pipeline Reed-Solomon decoder is presented. The transform decoding technique used in a previous article is replaced by a time domain algorithm through a detailed comparison of their VLSI implementations. A new architecture that implements the time domain algorithm permits efficient pipeline processing with reduced circuitry. Erasure correction capability is also incorporated with little additional complexity. By using multiplexing technique, a new implementation of Euclid's algorithm maintains the throughput rate with less circuitry. Such improvements result in both enhanced capability and significant reduction in silicon area.
VLSI Architecture Of A Binary Up/Down Counter
Identical stages contain relatively-few logic gates. New algorithm simplifies design of binary up/down counter. Design suitable for very-large-scale integrated circuits. Contains simple "pipeline" array of identical cells. Programmable logic unit converts increment and decrement input signals to "U" and "D" signals required by algorithm of counter.
A low-cost transportable ground station for capture and processing of direct broadcast EOS satellite data
The Earth Observing System (EOS), part of a cohesive national effort to study global change, will deploy a constellation of remote sensing spacecraft over a 15 year period. Science data from the EOS spacecraft will be processed and made available to a large community of earth scientists via NASA institutional facilities. A number of these spacecraft are also providing an additional interface to broadcast data directly to users. Direct broadcast of real-time science data from overhead spacecraft has valuable applications including validation of field measurements, planning science campaigns, and science and engineering education. The success and usefulness of EOS direct broadcast depends largely on the end-user cost of receiving the data. To extend this capability to the largest possible user base, the cost of receiving ground stations must be as low as possible. To achieve this goal, NASA Goddard Space Flight Center is developing a prototype low-cost transportable ground station for EOS direct broadcast data based on Very Large Scale Integration (VLSI) components and pipelined, multiprocessing architectures. The targeted reproduction cost of this system is less than $200K. This paper describes a prototype ground station and its constituent components.
Using Quality Attributes to Bridge Systems Engineering Gaps : A Juno Ground Data Systems Case Study
The Juno Mission to Jupiter is the second mission selected by the NASA New Frontiers Program. Juno launched August 2011 and will reach Jupiter July 2016. Juno's payload system is composed of nine instruments plus a gravity science experiment. One of the primary functions of the Juno Ground Data System (GDS) is the assembly and distribution of the CFDP (CCSDS File Delivery Protocol) product telemetry, also referred to as raw science data, for eight out of the nine instruments. The GDS accomplishes this with the Instrument Data Pipeline (IDP). During payload integration, the first attempt to exercise the IDP in a flight like manner revealed that although the functional requirements were well understood, the system was unable to meet latency requirements with the as-is heritage design. A systems engineering gap emerged between Juno instrument data delivery requirements and the assumptions behind the heritage flight-ground interactions. This paper describes the use of quality attributes to measure and overcome this gap by introducing a new systems engineering activity, and a new monitoring service architecture that successfully delivered the performance metrics needed to validate Juno IDP.
NASA POWER: Providing Analysis-Ready, Cloud-Optimized Data for AI /ML Training and Applications in Earth Science
As global demand for sustainable development grows, the integration of Earth Observation (EO) data into decision making frameworks has become a primary objective for the scientific community. The NASA Prediction of Worldwide Energy Resources (POWER) project serves as a bridge between NASA EO data and the specialized needs of the renewable energy, sustainable infrastructure and agroclimatology communities. In this poster presentation we will present an overview of POWER data products and services along with its use in diverse research to decision-making workflows. By providing over 40 years of high-resolution historical, hourly and daily solar and meteorological data, POWER transforms satellite observations and global model reanalysis into actionable, Analysis-Ready Dataset (ARD). Currently, the project delivers over 250 industry-friendly parameters to the users from different NASA datasets like CERES SYN1Deg, MERRA-2, and IMERG alongside downscaled CMIP6 climate model data, fulfilling over 16 million requests from 50,000 unique users monthly. To ensure data quality and traceability, these parameters are rigorously validated against the ground-based observations from the Baseline Surface Radiation Network (BSRN) and the Global Surface Summary of the Day (GSOD) – these results will be discussed in the presentation. A newly introduced web-based PaRameter Uncertainty ViEwer (PRUVE) tool will be presented that provides an online validation platform to the users that benchmarks satellite-based and assimilation data products against these surface measurements. To reduce technical barriers to data adoption, POWER data is accessible through RESTful APIs, ESRI ArcGIS Image Services, a web-based Data Access Viewer tool, allowing users to visualize, validate and apply the dataset. For efficient data delivery POWER data is cloud-optimized into Zarr datastore accessible through NASA managed Amazon S3 ensures high-performance allowing users to integrate EO directly into operational pipelines. These customized services will be presented. Use cases from application will be presented from the energy sector - such as for design of generation systems, performance monitoring of solar power plants, in infrastructure sector- optimizing building energy efficiency and thermal comfort, in agriculture – such as driving crop simulation and yield forecasting models to enable climate resilient farming. Furthermore, the shift toward machine learning (ML) in EO research that has positioned POWER as a key provider for training datasets which will be discussed. Use-cases will be presented to showcase how NASA data is enabling the development of predictive tools for climate variability and resource management. The poster will present POWER’s future plans including technology development to enhance data traceability and reproducibility and improving I/O performance to support the rapid integration of new EO products, ensuring that POWER remains a robust scalable backend for the evolving landscape of AI-driven Earth Science. Additionally, POWER is developing an AI Agent and an MCP-Server to enable industry AI-Agentic workflows.
Rapid Diagnostics of Onboard Sequences
Keeping track of sequences onboard a spacecraft is challenging. When reviewing Event Verification Records (EVRs) of sequence executions on the Mars Exploration Rover (MER), operators often found themselves wondering which version of a named sequence the EVR corresponded to. The lack of this information drastically impacts the operators diagnostic capabilities as well as their situational awareness with respect to the commands the spacecraft has executed, since the EVRs do not provide argument values or explanatory comments. Having this information immediately available can be instrumental in diagnosing critical events and can significantly enhance the overall safety of the spacecraft. This software provides auditing capability that can eliminate that uncertainty while diagnosing critical conditions. Furthermore, the Restful interface provides a simple way for sequencing tools to automatically retrieve binary compiled sequence SCMFs (Space Command Message Files) on demand. It also enables developers to change the underlying database, while maintaining the same interface to the existing applications. The logging capabilities are also beneficial to operators when they are trying to recall how they solved a similar problem many days ago: this software enables automatic recovery of SCMF and RML (Robot Markup Language) sequence files directly from the command EVRs, eliminating the need for people to find and validate the corresponding sequences. To address the lack of auditing capability for sequences onboard a spacecraft during earlier missions, extensive logging support was added on the Mars Science Laboratory (MSL) sequencing server. This server is responsible for generating all MSL binary SCMFs from RML input sequences. The sequencing server logs every SCMF it generates into a MySQL database, as well as the high-level RML file and dictionary name inputs used to create the SCMF. The SCMF is then indexed by a hash value that is automatically included in all command EVRs by the onboard flight software. Second, both the binary SCMF result and the RML input file can be retrieved simply by specifying the hash to a Restful web interface. This interface enables command line tools as well as large sophisticated programs to download the SCMF and RMLs on-demand from the database, enabling a vast array of tools to be built on top of it. One such command line tool can retrieve and display RML files, or annotate a list of EVRs by interleaving them with the original sequence commands. This software has been integrated with the MSL sequencing pipeline where it will serve sequences useful in diagnostics, debugging, and situational awareness throughout the mission.