Search NASA⌕ Search

SEARCH · Search NASA

Results for “pipeline monitoring”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

GeoNEX: A Geostationary Earth Observatory

The latest generation of geostationary satellites (Himawari 8/9, GOES-16/17, FY-4, GK-2A) carries sensors that closely mimic the spatial and spectral characteristics of widely used polar-orbiting, global monitoring sensors such as MODIS and VIIRS. When combined, data from various currently operating/planned geostationary platforms provide a geo-ring of hyper-temporal (5-10 minutes), multispectral observations at spatial resolutions as high as 500 m. These high frequency observations offer exciting new possibilities for monitoring our planet, including better retrievals of geophysical variables by overcoming cloud cover, enabling studies of diurnally varying phenomena in the atmosphere, land, and the oceans, and support operational decision-making in agriculture, hydrology and disaster management. The NASA Earth Exchange (NEX) team, in collaboration with scientists from JAXA, KARI, NOAA and other international institutions, created the GeoNEX (www.nasa.gov/geonex) pipeline to integrate data from all available geostationary platforms and produce and distribute spatially, temporally, and radiometrically consistent data for the earth science community. We envision various institutions adapting the Geo component (e.g., GeoNOAA, GeoKARI, GeoChiba, GeoJAXA, GeoCMA) and customizing the pipeline and downstream products to serve the local/regional research and applied science communities. To facilitate collaborative work among the partners, we have established the OpenNEX platform on the public cloud. OpenNEX provides researchers, developers, educators, and ordinary users with easy access to an integrated Earth science computational and data platform, enabling citizen scientists and application developers to realize the full value of GeoNEX data assets and software tools.

GeoNEX↗
NASA NTRS2000

Air, Ocean and Climate Monitoring Enhancing Undergraduate Training in the Physical, Environmental and Computer Sciences

Faculty in the Department of Physical, Environmental and Computer Sciences strongly believe in the concept that undergraduate research and research-related activities must be integrated into the fabric of our undergraduate Science and Technology curricula. High level skills, such as problem solving, reasoning, collaboration and the ability to engage in research, are learned for advanced study in graduate school or for competing for well paying positions in the scientific community. One goal of our academic programs is to have a pipeline of research activities from high school to four year college, to graduate school, based on the GISS Institute on Climate and Planets model.

Hope, W. W.↗
NASA NTRS2015

High-Energy Electromagnetic Offline Follow-Up of Ligo-Virgo Gravitational-Wave Binary Coalescence Candidate Events

We present two different search methods for electromagnetic counterparts to gravitational-wave (GW) events from ground-based detectors using archival NASA high-energy data from the Fermi Gamma-ray Burst Monitor (GBM) and RXTE All-sky Monitor (ASM) instruments. To demonstrate the methods, we use a limited number of representative GW background noise events produced by a search for binary neutron star coalescence over the last two months of the LIGO-Virgo S6/VSR3 joint science run. Time and sky location provided by the GW data trigger a targeted search in the high-energy photon data. We use two custom pipelines: one to search for prompt gamma-ray counterparts in GBM, and the other to search for a variety of X-ray afterglow model signals in ASM. We measure the efficiency of the joint pipelines to weak gamma-ray burst counterparts, and a family of model X-ray afterglows. By requiring a detectable signal in either electromagnetic instrument coincident with a GW event, we are able to reject a large majority of GW candidates. This reduces the signal-to-noise ratio of the loudest surviving GW background event by around 15-20 percent.

gamma-ray burst: general↗
NASA NTRS2013

Methods and Results of a Search for Gravitational Waves Associated with Gamma-Ray Bursts Using the GEO 600, LIGO, and Virgo Detectors

In this paper we report on a search for short-duration gravitational wave bursts in the frequency range 64 Hz-1792 Hz associated with gamma-ray bursts (GRBs), using data from GEO600 and one of the LIGO or Virgo detectors. We introduce the method of a linear search grid to analyze GRB events with large sky localization uncertainties such as the localizations provided by the Fermi Gamma-ray Burst Monitor (GBM). Coherent searches for gravitational waves (GWs) can be computationally intensive when the GRB sky position is not well-localized, due to the corrections required for the difference in arrival time between detectors. Using a linear search grid we are able to reduce the computational cost of the analysis by a factor of O(10) for GBM events. Furthermore, we demonstrate that our analysis pipeline can improve upon the sky localization of GRBs detected by the GBM, if a high-frequency GW signal is observed in coincidence. We use the linear search grid method in a search for GWs associated with 129 GRBs observed satellite-based gamma-ray experiments between 2006 and 2011. The GRBs in our sample had not been previously analyzed for GW counterparts. A fraction of our GRB events are analyzed using data from GEO600 while the detector was using squeezed-light states to improve its sensitivity; this is the first search for GWs using data from a squeezed-light interferometric observatory. We find no evidence for GW signals, either with any individual GRB in this sample or with the population as a whole. For each GRB we place lower bounds on the distance to the progenitor, assuming a fixed GW emission energy of 10(exp -2)Stellar Mass sq c, with a median exclusion distance of 0.8 Mpc for emission at 500 Hz and 0.3 Mpc at 1 kHz. The reduced computational cost associated with a linear search grid will enable rapid searches for GWs associated with Fermi GBM events in the Advanced detector era.

methods↗
NASA NTRS2009

Multi-Mission Automated Task Invocation Subsystem

Multi-Mission Automated Task Invocation Subsystem (MATIS) is software that establishes a distributed data-processing framework for automated generation of instrument data products from a spacecraft mission. Each mission may set up a set of MATIS servers for processing its data products. MATIS embodies lessons learned in experience with prior instrument- data-product-generation software. MATIS is an event-driven workflow manager that interprets project-specific, user-defined rules for managing processes. It executes programs in response to specific events under specific conditions according to the rules. Because requirements of different missions are too diverse to be satisfied by one program, MATIS accommodates plug-in programs. MATIS is flexible in that users can control such processing parameters as how many pipelines to run and on which computing machines to run them. MATIS has a fail-safe capability. At each step, MATIS captures and retains pertinent information needed to complete the step and start the next step. In the event of a restart, this information is retrieved so that processing can be resumed appropriately. At this writing, it is planned to develop a graphical user interface (GUI) for monitoring and controlling a product generation engine in MATIS. The GUI would enable users to schedule multiple processes and manage the data products produced in the processes. Although MATIS was initially designed for instrument data product generation,

Cheng, Cecilia S.↗
NASA NTRS1974

Scientific and economic potential of the SEASAT Program

The SEASAT satellite system is planned as a user-oriented system for timely monitoring of global ocean dynamics and mapping the global ocean geoid. The satellite instrumentation and modular concept are discussed. Operational data capabilities will include oceanographic data services, direct satellite read-out to users, and conversational retrieval and analysis of stored data. A case-study technique, generalized through physical and econometric modeling, indicates potential economic benefit from SEASAT to users in the following areas: ship routing, iceberg reconnaissance, arctic operations, Alaska pipeline ship link, and off-shore oil production.

Mccandless, S. W.↗
NASA NTRS2015

Reconfigurable Very Long Instruction Word (VLIW) Processor

Future NASA missions will depend on radiation-hardened, power-efficient processing systems-on-a-chip (SOCs) that consist of a range of processor cores custom tailored for space applications. Aries Design Automation, LLC, has developed a processing SOC that is optimized for software-defined radio (SDR) uses. The innovation implements the Institute of Electrical and Electronics Engineers (IEEE) RazorII voltage management technique, a microarchitectural mechanism that allows processor cores to self-monitor, self-analyze, and selfheal after timing errors, regardless of their cause (e.g., radiation; chip aging; variations in the voltage, frequency, temperature, or manufacturing process). This highly automated SOC can also execute legacy PowerPC 750 binary code instruction set architecture (ISA), which is used in the flight-control computers of many previous NASA space missions. In developing this innovation, Aries Design Automation has made significant contributions to the fields of formal verification of complex pipelined microprocessors and Boolean satisfiability (SAT) and has developed highly efficient electronic design automation tools that hold promise for future developments.

Velev, Miroslav N.↗
NASA NTRS2010

Reconfigurable Hardware for Compressing Hyperspectral Image Data

High-speed, low-power, reconfigurable electronic hardware has been developed to implement ICER-3D, an algorithm for compressing hyperspectral-image data. The algorithm and parts thereof have been the topics of several NASA Tech Briefs articles, including Context Modeler for Wavelet Compression of Hyperspectral Images (NPO-43239) and ICER-3D Hyperspectral Image Compression Software (NPO-43238), which appear elsewhere in this issue of NASA Tech Briefs. As described in more detail in those articles, the algorithm includes three main subalgorithms: one for computing wavelet transforms, one for context modeling, and one for entropy encoding. For the purpose of designing the hardware, these subalgorithms are treated as modules to be implemented efficiently in field-programmable gate arrays (FPGAs). The design takes advantage of industry- standard, commercially available FPGAs. The implementation targets the Xilinx Virtex II pro architecture, which has embedded PowerPC processor cores with flexible on-chip bus architecture. It incorporates an efficient parallel and pipelined architecture to compress the three-dimensional image data. The design provides for internal buffering to minimize intensive input/output operations while making efficient use of offchip memory. The design is scalable in that the subalgorithms are implemented as independent hardware modules that can be combined in parallel to increase throughput. The on-chip processor manages the overall operation of the compression system, including execution of the top-level control functions as well as scheduling, initiating, and monitoring processes. The design prototype has been demonstrated to be capable of compressing hyperspectral data at a rate of 4.5 megasamples per second at a conservative clock frequency of 50 MHz, with a potential for substantially greater throughput at a higher clock frequency. The power consumption of the prototype is less than 6.5 W. The reconfigurability (by means of reprogramming) of the FPGAs makes it possible to effectively alter the design to some extent to satisfy different requirements without adding hardware. The implementation could be easily propagated to future FPGA generations and/or to custom application-specific integrated circuits.

Aranki, Nazeeh↗
NASA NTRSDate not supplied

A Cloud-Based Operational Surface Water Extent Mapping Service from Sentinel-1 SAR

With its weather independence and day-and-night capabilities, SAR has long been known as a useful data set for flood monitoring. The recently launched Sentinel-1 (S1) C-band sensors, with their regularly acquired, free-and-open data, have finally elevated SAR to a relevant data source in operational hazard response. Leveraging these capabilities of S1, this poster introduces the HYDRO30 product, a 30-m resolution surface water extent product derived from dual-pol S1 SAR data. To enable automatic and near real-time product generation, HYDRO30 is embedded in the HydroSAR service, a cloud-based production pipeline developed by the University of Alaska Fairbanks in collaboration with the NASA Alaska Satellite Facility DAAC and the NASA Marshall and Goddard Space Flight Centers.

Franz J Meyer↗
NASA NTRS2017

Searching the Gamma-Ray Sky for Counterparts to Gravitational Wave Sources Fermi Gamma-Ray Burst Monitor and Large Area Telescope Observations of LVT151012 and GW151226

We present the Fermi Gamma-ray Burst Monitor (GBM) and Large Area Telescope (LAT) observations of the LIGO binary black hole merger event GW151226 and candidate LVT151012. At the time of the LIGO triggers on LVT151012 and GW151226, GBM was observing 68% and 83% of the localization regions, and LAT was observing 47% and 32%, respectively. No candidate electromagnetic counterparts were detected by either the GBM or LAT. We present a detailed analysis of the GBM and LAT data over a range of timescales from seconds to years, using automated pipelines and new techniques for characterizing the flux upper bounds across large areas of the sky. Due to the partial GBM and LAT coverage of the large LIGO localization regions at the trigger times for both events, differences in source distances and masses, as well as the uncertain degree to which emission from these sources could be beamed, these non-detections cannot be used to constrain the variety of theoretical models recently applied to explain the candidate GBM counterpart to GW150914.

gamma rays: general – gravitational waves – me↗
NASA NTRSDate not supplied

HydroSAR: A Cloud-based SAR Data Analysis Service to Monitor Hydrological Disasters and their Impact on Population and Agriculture

Weather-related hazards are ubiquitous around the world including: 1) hurricane storm surges, 2) rapid snowmelt and heavy rainfall, 3) severe weather leading to flash floods, and 4) seasonal freeze and thaw of rivers that may lead to ice jams. Each of these hazards affects human settlements and has the potential to impact agricultural productivity. In each setting, end-users in disaster management need access to data processing tools helpful in mapping past and current disasters. Analysis of past events supports risk mitigation by understanding what has already occurred and how to alleviate those impacts in the future. Having capabilities to generate the same products in a response setting means that lessons learned from risk analysis will carry forward to event response. Synthetic aperture radar (SAR) data are particularly useful for these activities due to their all-weather 24/7 monitoring capabilities. In this effort we present HydroSAR, a cloud-based SAR data analysis service for the mapping of meteorological and hydrological disasters as well as their impact on population and agriculture. As part of this project we have developed a series of SAR-based value added products for the monitoring of surface hydrology (image time series, change detection, flood extent, flood depth) and the assessment of impacts on population (flood depth) and agriculture (active agriculture, inundated agriculture, flood duration). We also developed a cloud-based platform for generating these products over affected areas and are working with end-users to integrate derived product into decision-making workflows The paper will briefly introduce the SAR-based products that were developed for this effort. We describe the cloud-based production pipeline that was built to automatically generate these products in near-real time over extended regions. The integration of SAR-based information into hazard preparation and response activities is described for a number of recent disasters including the 2019 forest fires in Alaska, 2019 flooding in the U.S. Midwest, the 2020 U.S. severe weather easter outbreak, 2020 tropical storm Christobal, 2020 cyclone Amphan, 2020, Alaska Spring breakup flooding, and the 2020 flood season in Eastern India, Bangladesh, and Nepal.

Franz Josef Meyer↗
NASA NTRS2012

Using Quality Attributes to Bridge Systems Engineering Gaps : A Juno Ground Data Systems Case Study

The Juno Mission to Jupiter is the second mission selected by the NASA New Frontiers Program. Juno launched August 2011 and will reach Jupiter July 2016. Juno's payload system is composed of nine instruments plus a gravity science experiment. One of the primary functions of the Juno Ground Data System (GDS) is the assembly and distribution of the CFDP (CCSDS File Delivery Protocol) product telemetry, also referred to as raw science data, for eight out of the nine instruments. The GDS accomplishes this with the Instrument Data Pipeline (IDP). During payload integration, the first attempt to exercise the IDP in a flight like manner revealed that although the functional requirements were well understood, the system was unable to meet latency requirements with the as-is heritage design. A systems engineering gap emerged between Juno instrument data delivery requirements and the assumptions behind the heritage flight-ground interactions. This paper describes the use of quality attributes to measure and overcome this gap by introducing a new systems engineering activity, and a new monitoring service architecture that successfully delivered the performance metrics needed to validate Juno IDP.

Ground Data Systems (GDS)↗
NASA NTRSDate not supplied

Developing a Cloud-Based ArcGIS Image Service for TROPOMI Level 2 Data: Preprocessing, Transformation, and Publication

TROPOMI, the Tropospheric Monitoring Instrument aboard the Sentinel-5 Precursor satellite, provides high spatiotemporal resolution atmospheric measurements. It is essential for monitoring air quality, greenhouse gases, and other trace gases. Integrating TROPOMI Level 2 data into an ArcGIS Image Service marks a significant advancement in the accessibility and utility of satellite-derived environmental information for GIS applications. This paper details the methodology for building an ArcGIS Image Service tailored to handle TROPOMI Level 2 data, with a focus on preprocessing, transformation, and cloud-based publication. The workflow utilizes Python and ArcPy for data reformatting, reprojection, and updating, ensuring efficient processing and cloud-based notifications. By establishing a robust pipeline, the ArcGIS Image Service provides real-time access to TROPOMI Level 2 data, enabling users to visualize, analyze, and interpret atmospheric phenomena effectively.

Level 2 Data, ArcGIS, Image Service, ArcPy, OGC↗
NASA NTRS2021

Automated Data Accountability for Missions in Mars Rover Data

As the Mars Curiosity Rover transmits data to the JPL Ground Data System (GDS), it frequently observes data loss and corruption, requiring re-transmits from the rover and Ground Data System Analysts (GDSA) to monitor the downlink process. As new missions are launched, the GDSA team redistributes analysts to these new missions, causing shortages in previous missions. The GDSA team can significantly benefit from the automation and optimization of the downlink process of telemetry data. In fact, there is a need for a better understanding of why the data is corrupted, so that the GDSA team can best determine the root cause of the issues in the GDS. This paper presents machine learning and deep learning based approaches to automate and optimize the detection of data loss. We first created a pipeline to automatically accumulate data from the telemetry databases (MAROS, Telemetry Data Storage, and GDS Elastic Search Database) in the downlink process. With our newly created datasets, we perform feature selection to supplement the GDSA understanding of the downlink process and provide supplemental analysis on the importance of different features. We implement various machine learning and deep learning based models, including support vector machines, ensemble methods, and deep neural networks and evaluate their accuracies in identifying whether a downlink process is complete or incomplete. We utilize fast hyperparameter optimization methods that allow our models to quickly be re-trained, allowing them to quickly be tuned and optimized on daily incoming data in real time. This hyperparameter optimization also allows our methods to be quickly integrated into other JPL missions. Our results show that our best-performing machine learning and deep learning based models outperform the existing GDSA detection software by 6 accuracy points and can aid analysts by providing insights into the data accountability problem. Since these various machine learning and deep learning approaches vary significantly in interpretability, we provide a discussion on the tradeoffs between their performance and trustworthiness in helping detect issues in data transmission.

Divsalar, Dariush↗
NASA NTRS2002

Science Goal Driven Observing and Spacecraft Autonomy

Spacecraft autonomy will be an integral part of mission operations in the coming decade. While recent missions have made great strides in the ability to autonomously monitor and react to changing health and physical status of spacecraft, little progress has been made in responding quickly to science driven events. For observations of inherently variable targets and targets of opportunity, the ability to recognize early if an observation will meet the science goals of a program, and react accordingly, can have a major positive impact on the overall scientific returns of an observatory and on its operational costs. If the onboard software can reprioritize the schedule to focus on alternate targets, discard uninteresting observations prior to downloading, or download a subset of observations at a reduced resolution, the spacecraft's overall efficiency will be dramatically increased. The science goal monitoring (SGM) system is a proof- of-concept effort to address the above challenge. The SGM will have an interface to help capture higher-level science goals from the scientists and translate them into a flexible observing strategy that SGM can execute and monitor. We are developing an interactive distributed system that will use on-board processing and storage combined with event-driven interfaces with ground-based processing and operations, to enable fast re-prioritization of observing schedules, and to minimize time spent on non-optimized observations. This paper will focus on our strategy for developing SGM and the technical challenges that we have encountered. We will discuss the SGM architecture as it applies to the proposed MIDEX-class mission Kronos. However, the architecture and interfaces will also be designed for easy adaptability to other observing platforms, including ground-based systems and to work with different scheduling and pipeline processing systems.

Jones, Jeremy↗
NASA NTRS1986

Gas dispersion measurements using a mobile Raman lidar system

The exploitation of natural gas resources to supply energy demands has resulted in the need to engineer pipelines and plants capable of handling extremely high pressures and throughputs. Consequently, more attention has been directed to evaluating the consequences of releases of material whether accidental or deliberate in nature. An important aspect of assessing the consequences of a release is an understanding of how gas disperses in the atmosphere over a wide range of release and atmospheric conditions. The most cost effective way of providing such information is through the development and use of reliable theoretical prediction methods. The need for some form of remote sensing device was identified. The various possibilities studied led to the conclusion that LIDAR (Light Detection And Ranging) offered the most suitable method. The system designed and built is described, and its recent use in monitoring operational ventings from a high pressure transmission system is discussed.

Houston, J. D.↗
NASA NTRS2002

An End-to-End Architecture for Science Goal Driven Observing

New observatories will have greater on-board storage capacity and on-board processing capabilities. The new bottleneck will be download capacity. The cost of downlink time and limitations of bandwidth will end the era where all exposure data is downloaded and all data processing is performed on the ground. In addition, observing campaigns involving inherently variable targets will need scheduling flexibility to focus observing time and data download on exposures that are scientifically interesting. The ability to quickly recognize and react to such events by re-prioritizing the observing schedule will be an essential characteristic for maximizing scientific returns. It will also be a step towards increasing spacecraft autonomy, a major goal of NASA's strategic plan. The science goal monitoring (SGM) system is a proof-of-concept effort to address these challenges. We are developing an interactive distributed system that will use on-board processing and storage combined with event-driven interfaces with ground-based processing and operations, to enable fast re-prioritization of observing schedules, and to minimize time spent on non-optimized observations. SGM is initially aimed towards time-tagged observing modes used frequently in spectroscopic studies of varying targets. In particular, the SGM is collaborating with the proposed MIDEX-class mission Kronos team. The variable targets that Kronos seeks to study make an adaptive system such as SGM particularly valuable for achieving mission goals. However, the architecture and interfaces will also be designed for easy adaptability to other observing platforms, including ground-based systems and to work with different scheduling and pipeline processing systems. This talk will focus on our strategy for developing SGM and the technical challenges that we have encountered. We will discuss the SGM architecture as it applies to the Kronos mission and explain how it is scalable to other missions.

Jones, Jeremy↗
NASA NTRS1999

SIRTF Science Operations System Design

SIRTF Science Operations System Design William B. Green Manager, SIRTF Science Center California Institute of Technology M/S 310-6 1200 E. California Blvd., Pasadena CA 91125 (626) 395 8572 Fax (626) 568 0673 bgreen@ipac.caltech.edu. The Space Infrared Telescope Facility (SIRTF) will be launched in December 2001, and perform an extended series of science observations at wavelengths ranging from 20 to 160 microns for five years or more. The California Institute of Technology has been selected as the home for the SIRTF Science Center (SSC). The SSC will be responsible for evaluating and selecting observation proposals, providing technical support to the science community, performing mission planning and science observation scheduling activities, instrument calibration during operations and instrument health monitoring, production of archival quality data products, and management of science research grants. The science payload consists of three instruments delivered by instrument Principal Investigators located at University of Arizona, Cornell, and Harvard Smithsonian Astrophysical Observatory. The SSC is responsible for design, development, and operation of the Science Operations System (SOS) which will support the functions assigned to the SSC by NASA. The SIRTF spacecraft, mission profile, and science instrument design have undergone almost ten years of refinement. SIRTF development and operations activities are highly cost constrained. The cost constraints have impacted the design of the SOS in several ways. The Science Operations System has been designed to incorporate a set of efficient, easy to use tools which will make it possible for scientists to propose observation sequences in a rapid and automated manner. The use of highly automated tools for requesting observations will simplify the long range observatory scheduling process, and the short term scheduling of science observations. Pipeline data processing will be highly automated and data-driven, utilizing a variety of tools developed at JPL, the instrument development teams, and Space Telescope Science Institute to automate processing. An incremental ground data system development approach has been adopted, featuring periodic deliveries that are validated with the flight hardware throughout the various phases of system level development and testing. This approach minimizes development time and decreases operations risk. This paper will describe the top level architecture of the SOS and the basic design concepts. A summary of the incremental development approach will be presented. Examples of the unique science user tools now under final development prior to the first proposal call scheduled for mid-2000 will be shown.

Green, William↗