Search NASA⌕ Search

SEARCH · Search NASA

Results for “pipeline data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Advancing Additive Manufacturing Through Artificial Intelligence–Powered, High-Throughput, Nondestructive Characterization and Process Optimization

This Cooperative Research and Development Agreement (CRADA) between Oak Ridge National Laboratory (ORNL) and ZEISS Industrial Metrology has demonstrated the transformative potential of artificial intelligence (AI)-enabled x-ray computed tomography (XCT) to accelerate the qualification and certification of additively manufactured (AM) parts. At the core of this effort is Simurgh, an AI-powered XCT reconstruction framework jointly advanced by ORNL and ZEISS that integrates computer-aided design (CAD) models, physics-based simulations, and deep learning to overcome the long-standing challenges of metal artifact correction, long scan durations, and limited flaw detectability in dense and geometrically complex components. Simurgh enables high-throughput, high-quality 3D reconstruction from sparse and fast scans, which reduces XCT acquisition times by more than an order of magnitude and simultaneously improves defect detection limits by up to fourfold compared with industry-standard approaches. This capability reduces scan costs by more than 50%, lowers labor overhead, and makes XCT characterization economically viable for routine industrial use. By enabling reliable flaw detection in minutes rather than hours, Simurgh facilitates real-time feedback loops for process parameter optimization, which was highlighted in a recent npj Computational Materials (a Nature journal) issue. In the published study, more than 100 alloy coupons were characterized within a single day. This work represents a tenfold acceleration in the development of novel AM alloys and processes compared with conventional workflows. The ZEISS collaboration has also demonstrated the scalability of Simurgh to diverse application domains, including aerospace, nuclear, automotive, and biomedical components; in these applications, ensuring structural integrity is paramount. By drastically reducing barriers to XCT adoption, this partnership has laid the foundation for digital twins and data-driven certification pipelines and directly addressed bottlenecks in qualifying new materials and designs. Together, ORNL and ZEISS have shown that Simurgh advances the state of the art in nondestructive evaluation and aligns with the broader mission of enabling Industry 4.0 manufacturing ecosystems, in which intelligent, cost-effective, rapid quality assurance is integral to accelerating innovation and ensuring safety in critical applications.

36 MATERIALS SCIENCE↗
DOE OSTI2026

lumicap v0.1

Automated HDR luminance imaging system designed for daylighting research and building science. It controls a fisheye-lens camera to capture time-lapse bracket sequences, merges them into calibrated HDR images, and runs a full post-processing pipeline — all unattended. Features: - Scheduled LDR bracket capture via gphoto2 - HDR merging with vignetting, ND filter, and fisheye projection corrections - Illuminance and luminance meter integration (Konica Minolta T-10A, LS-100/150) - Daylight glare probability (DGP) and solar position computation - Automated false-color rendering, JPEG thumbnails, and daily time-lapse video - CSV data logging per timestep Uses: - Long-term monitoring of daylight conditions in buildings - Glare analysis for occupant comfort research - Solar irradiance and sky luminance studies Advantages: - End-to-end automation — capture, calibration, analysis, and archiving run without manual intervention - Built on the proven Radiance toolchain, ensuring photometrically accurate HDR output - Hardware-agnostic meter support via serial auto-detection - Lightweight — no GUI overhead, deployable on a headless Raspberry Pi or similar embedded system

Wang, Taoning [Lawrence Berkeley National Laborato↗
NASA NTRS2011

Computer-Aided Parallelizer and Optimizer

The Computer-Aided Parallelizer and Optimizer (CAPO) automates the insertion of compiler directives (see figure) to facilitate parallel processing on Shared Memory Parallel (SMP) machines. While CAPO currently is integrated seamlessly into CAPTools (developed at the University of Greenwich, now marketed as ParaWise), CAPO was independently developed at Ames Research Center as one of the components for the Legacy Code Modernization (LCM) project. The current version takes serial FORTRAN programs, performs interprocedural data dependence analysis, and generates OpenMP directives. Due to the widely supported OpenMP standard, the generated OpenMP codes have the potential to run on a wide range of SMP machines. CAPO relies on accurate interprocedural data dependence information currently provided by CAPTools. Compiler directives are generated through identification of parallel loops in the outermost level, construction of parallel regions around parallel loops and optimization of parallel regions, and insertion of directives with automatic identification of private, reduction, induction, and shared variables. Attempts also have been made to identify potential pipeline parallelism (implemented with point-to-point synchronization). Although directives are generated automatically, user interaction with the tool is still important for producing good parallel codes. A comprehensive graphical user interface is included for users to interact with the parallelization process.

Jin, Haoqiang↗
DOE OSTI2024

L-PBF High-Throughput Data Pipeline Approach for Multi-modal Integration

Abstract Metal-based additive manufacturing requires active monitoring solutions for assessing part quality. Multiple sensors and data streams, however, generate large heterogeneous data sets that are impractical for manual assessment and characterization. In this work, an automated pipeline is developed that enables feature extraction from high-speed camera video and multi-modal data analysis. The framework removes the need for manual assessment through the utilization of deep learning techniques and training models in a weakly supervised paradigm. We demonstrate this pipeline’s capability over 700,000 high-speed camera frames. The pipeline successfully extracts melt pool and spatter geometries and links them to corresponding pyrometry, radiography, and processparameter information. 715 individual prints are examined to reveal melt pool areas that exceeds 0.07 mm 2 and pyrometry signal over a threshold (375 pyrometry units) were more likely to have defects. These automated processes enable massive throughput of characterization techniques.

36 MATERIALS SCIENCE↗
NASA NTRS1982

Principal Components as a Data Reduction and Noise Reduction Technique

The potential of principal components as a pipeline data reduction technique for thematic mapper data was assessed and principal components analysis and its transformation as a noise reduction technique was examined. Two primary factors were considered: (1) how might data reduction and noise reduction using the principal components transformation affect the extraction of accurate spectral classifications; and (2) what are the real savings in terms of computer processing and storage costs of using reduced data over the full 7-band TM complement. An area in central Pennsylvania was chosen for a study area. The image data for the project were collected using the Earth Resources Laboratory's thematic mapper simulator (TMS) instrument.

Imhoff, M. L.↗
DOE OSTI2026

Single‐Cell Nanodroplet Processing Proteomics Pipeline for Analysis of Human‐Derived Microglia

Single-cell omics tools provide unique insights into heterogeneous cell populations and their responses to stimuli. For example, single-cell RNA sequencing has identified several transcriptionally distinct populations of microglia, which are resident immune cells of the central nervous system (CNS) that are responsive to CNS injury, infection, and neurodegeneration. To date, single-cell studies of microglia have focused on RNA-sequencing or cytometry by time of flight (CyTOF), which provide indirect readouts of protein abundance or quantification of a limited number of targets. Herein, we present a workflow based on FACS-assisted isolation, cryopreservation, and nanodroplet-based processing for single-cell mass spectrometry proteomics analysis of the postmortem human brain cortex-derived microglia. From a single microglial cell, 1039 proteins could be identified on average. As a proof-of-principle, we applied single-cell proteomics for exploring the heterogeneity of brain microglia at the cellular level. This pilot proteomics data partially recapitulates the prior microglia subtypes. Specifically, we determined that mitochondrial proteins, in particular members of NADH dehydrogenase (Complex I), cytochrome b-c1 (Complex III), cytochrome c oxidase (Complex IV), F1-ATPase (Complex V), and Na+/K+-ATPase complex, drive variation across microglia. This pipeline offers the potential for identifying functionally and analytically relevant protein targets for microglia in Alzheimer's disease and other neurological disorders.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Leveraging Existing Assets for Long Duration Energy Storage

Increased renewables penetration to electrical grid is necessary to reduce overall emissions from the electrical power generation sector. Nonetheless, its integration creates challenges to grid operators who must match the power being generated by intermittent renewables and other traditional energy sources with the demand from consumers, while ensuring the reliability and power quality for the entire system. Energy storage has been proposed as an alternative to natural gas peaking plants and a form to deliver excess renewable energy generation at times of peak demand. For energy storage to provide benefits to end customers (energy consumers), it must be reliable, efficient, and cost effective. The Illinois Sustainable Technology Center (ISTC), one of the surveys that integrate the Prairie Research Institute (PRI), aims to develop a Center for Energy Storage at Existing Assets (CESEA) at UIUC with the participation of Waste Pressure Corp and Ecotek Engineering USA LLC. CESEA will focus on LDES systems that can integrate to existing infrastructure in a manner that reduces the initial capital expenditure and demonstrates the ability to repurpose fossil assets that would otherwise become stranded, to serve the energy transition. CESEA aims to leverage UIUC’s unique facilities to validate LDES systems performance at a relevant operating environment. UIUC’s facilities include a 85-MW combined heat and power (CHP) power plant, two (2) solar PV plants totaling over 18 MWdc of installed capacity, an electrical grid along with a substation at transmission and distribution voltages, a 22-mile gas pipeline network operating at two pressure levels, along with steam and chilled water distribution networks. The new LDES systems will connect to the existing UIUC grid through a new test electrical station, which will have the capacity to accommodate additional connections to test new devices and technologies as part of future CESEA R&D activities. The test electrical station will contain meters, instrumentation, and controls to accurately capture data and allow optimization of control algorithms. CESEA will initially focus on technologies that: i) utilize existing equipment or facilities to perform at least one of the process steps in LDES (charging, storage, or discharging), ii) leverage mature or commercially available components or controls, iii) show potential for cost-leadership in 10+ hour storage at a commercial scale. Initial technologies that were identified to meet these criteria include Compressed Gas Energy Storage (CGES), and TES. CGES stores electricity by raising the pressure of a compressible gas inside a control volume and converting the stored energy to electricity via expansion-generation. CGES is a generalization of CAES that covers any working gas (not just air). A successful CGES demo will help to circumvent many challenges faced by CAES (long development times due to site prospecting, high cost of compression and storage, heat recovery management, etc.) by: 1) utilizing existing infrastructure (compressors, pipelines, underground storage or pressure vessels) used in the transportation and storage of industrial gases for LDES charging and storage; 2) deploying over sites already-developed for industrial applications with minor additional work; 3) leveraging the price structure of commercial industrial gas to cover the costs of electricity used during charging. A previous DOE-sponsored conceptual study (DE-FE-0032018) estimated the levelized cost of energy of a 1.1 MW / 17 MWh CGES system at $0.08/kWh, with a commercial 10x scale system cost estimated at <$0.04/kWh (Giardinella, 2022). The pilot-sized system was estimated to avoid up to 2693 tons of CO2/year.

25 ENERGY STORAGE↗
NASA NTRSDate not supplied

AstroAmpSeq: Microbial Bioinformatics Education with NASA GeneLab’s Amplicon Pipeline

The prevalence and importance of large sequencing datasets in microbiology has led to a movement to share microbial ecology experimental data through open-access databases. This is particularly true of experiments that are difficult to replicate, such as those conducted in the spaceflight environment and shared via NASA GeneLab. It is now possible and indeed valuable for students to access and re-analyze these shared datasets for educational and research purposes. To provide students with experience utilizing microbial bioinformatics tools, GeneLab for Colleges and Universities (GL4U) has designed AstroAmpSeq, a week-long, virtually implemented project-based learning (PBL) minicourse to instruct undergraduate students on 16S amplicon sequencing. AstroAmpSeq was created to be accessible to students without prior bioinformatics or microbial ecology experience. During the minicourse students work in teams to process, analyze, and visualize a subsample of GeneLab dataset GLDS-280 using GeneLab’s standard amplicon processing pipeline, which is based in R. Students develop a hypothesis related to the dataset then generate and analyze figures to evaluate their hypothesis. Formative assessment of student learning is determined via pre- and post-evaluations, peer feedback, and self-reflection. Project and presentation rubrics serve as a summative assessment of student learning. GL4U AstroAmpSeq not only meets American Society for Microbiology Curriculum Guidelines, but also incites student interest in research by an inquiry-based approach and can be made part of a larger semester-long curriculum. GL4U AstroAmpSeq raises awareness of space microbiology and bioinformatics as a field and career path among undergraduates. Further, by using a GeneLab dataset and nesting microbiology techniques into the real-world application of space biology, AstroAmpSeq enforces deeper and longer-lasting student learning.

microbiology↗
NASA NTRS2017

Status of the TESS Science Processing Operations Center

The Transiting Exoplanet Survey Satellite (TESS) science pipeline is being developed by the Science Processing Operations Center (SPOC) at NASA Ames Research Center based on the highly successful Kepler Mission science pipeline. Like the Kepler pipeline, the TESS science pipeline will provide calibrated pixels, simple and systematic error-corrected aperture photometry, and centroid locations for all 200,000+ target stars, observed over the 2-year mission, along with associated uncertainties. The pixel and light curve products are modeled on the Kepler archive products and will be archived to the Mikulski Archive for Space Telescopes (MAST). In addition to the nominal science data, the 30-minute Full Frame Images (FFIs) simultaneously collected by TESS will also be calibrated by the SPOC and archived at MAST. The TESS pipeline will search through all light curves for evidence of transits that occur when a planet crosses the disk of its host star. The Data Validation pipeline will generate a suite of diagnostic metrics for each transit-like signature discovered, and extract planetary parameters by fitting a limb-darkened transit model to each potential planetary signature. The results of the transit search will be modeled on the Kepler transit search products (tabulated numerical results, time series products, and pdf reports) all of which will be archived to MAST.

high performance computing↗
NASA NTRS2019

Quantitative Comparison of Proprietary and Open-Source Georeferencing Tools for Use with Astronaut Photography

The Crew Earth Observations (CEO) Facility within the Earth Science and Remote Sensing Unit at NASA’s Johnson Space Center supports the acquisition, analysis, and curation of astronaut photography of Earth’s surface and atmosphere. Astronauts on the International Space Station (ISS) respond to requests from CEO to acquire imagery of scientific and education targets, to include high profile targets in response to activations from the International Charter for Space & Major Disasters (also known as the International Disaster Charter, or IDC) and NASA’s Disasters Program. CEO facilitates the acquisition of astronaut photography in response to IDC events and delivers georeferenced data products to the United States Geological Survey (USGS) for distribution to the disaster community. Using GeoRef, an internal web-based tool developed in collaboration with NASA’s Ames Research Center, CEO generates data packages of georeferenced imagery, uncertainty images for assessing control and tie point accuracy, and metadata documenting raw and processed data. Operational experience with the Georef software identified vulnerabilities to internal code and server errors that can significantly increase time of data production. As such, CEO developed a backup procedure in case the GeoRef software experiences front-end or back-end errors. A system using OSGEO’s open-source QGIS software combined with a semi-automated pipeline using the object-oriented Python language and the Geospatial Abstract Library for generating metadata is quantitatively compared to GeoRef’s data package for quality and productivity. Root Mean Square Error (RMSE) provides a standard measurement of data quality as it relates to ground error. Assessing RMSE measurements generated from georeferenced astronaut photographs acquired with different obliquity and focal length offers a comprehensive accuracy assessment of the software’s transformation algorithms. This assessment will indicate the software's ability to produce data products with the least ground-error or highest data quality regarding ground accuracy. In addition, a comparison of the software’s efficiency in generating a data package that includes georeferenced images, metadata, and uncertainty images for measuring tie/ground point error was performed. Initial results, based on the comparison of three nadir-facing astronaut photographs acquired with a 95mm focal length, reveal the QGIS-based system's average RMSE is 2.36 (pixels) suggesting its georectification system produces data products that meet and perhaps improve upon Georef solution's average RMSE of 32.99 (pixels). However, the QGIS system was unable to reproduce two unique Georef data products, uncertainty images for measuring tie and control point errors and a translated unwrapped image. In addition, the Georef software is designed to accept handheld camera pose information from a hardware component (Geosens) scheduled for deployment on the ISS in late 2018; this information is intended to provide increased accuracy and auto-registration capability for astronaut photographs. Future work is expected to determine the QGIS-based georectification system’s potential as an open-source alternative (and operational backup) to Georef for georeferencing the full range of resolutions and viewing angles unique to handheld digital camera imagery in support of ISS disaster response activities.

Jagge, Amy M.↗
DOE OSTI2025

Pipeline for Integrated Projects in Energy Systems (PIPES): A Tool for Integrated System Planning [Slides]

The Pipeline for Integrated Projects in Energy Systems (PIPES) is a comprehensive project, data, and workflow management tool designed for integrated modeling teams. PIPES facilitates the management of data requirements, tasks, and progress tracking, serving as a higher-level integration layer that works across various data and modeling software. This tool integrates models, data, and tools to perform large-scale, integrated analysis work at scale. PIPES is designed to streamline integrated modeling projects, enhance collaboration, and ensure the quality and efficiency of data management and workflow processes. This presentation introduces PIPES a multi-model tool for integrated system planning; it describes the underlying architecture, deep dives into common user workflows, and outlines the upcoming development roadmap beyond its current alpha state.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2024

Semi-automatic image annotation using 3D LiDAR projections and depth camera data

Efficient image annotation is necessary to utilize deep learning object recognition neural networks in nuclear safeguards, such as for the detection and localization of target objects like nuclear material containers (NMCs). This capability can help automate the inventory accounting of different types of NMCs within nuclear storage facilities. The conventional manual annotation process is labor-intensive and time-consuming, hindering the rapid deployment of deep learning models for NMC identifications. This paper introduces a novel semi-automatic method for annotating 2D images of nuclear material containers (NMCs) by combining 3D light detection and ranging (LiDAR) data with color and depth camera images collected from a handheld scan system. The annotation pipeline involves an operator manually marking new target objects on a LiDAR-generated map, and projecting these 3D locations to images, thereby automatically creating annotations from the projections. The semi-automatic approach significantly reduces manual efforts and the expertise in image annotation that is required to perform the task, allowing deep learning models to be trained on-site within a few hours. The paper compares the performance of models trained on datasets annotated through various methods, including semi-automatic, manual, and commercial annotation services. The evaluation demonstrates that the semi-automatic annotation method achieves comparable or superior results, with a mean average precision (mAP) above 0.9, showcasing its efficiency in training object recognition models. Additionally, the paper explores the application of the proposed method to instance segmentation, achieving promising results in detecting multiple types of NMCs in various formations.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
NASA NTRS2015

The Use of Field Programmable Gate Arrays (FPGA) in Small Satellite Communication Systems

This paper will describe the use of digital Field Programmable Gate Arrays (FPGA) to contribute to advancing the state-of-the-art in software defined radio (SDR) transponder design for the emerging SmallSat and CubeSat industry and to provide advances for NASA as described in the TAO5 Communication and Navigation Roadmap (Ref 4). The use of software defined radios (SDR) has been around for a long time. A typical implementation of the SDR is to use a processor and write software to implement all the functions of filtering, carrier recovery, error correction, framing etc. Even with modern high speed and low power digital signal processors, high speed memories, and efficient coding, the compute intensive nature of digital filters, error correcting and other algorithms is too much for modern processors to get efficient use of the available bandwidth to the ground. By using FPGAs, these compute intensive tasks can be done in parallel, pipelined fashion and more efficiently use every clock cycle to significantly increase throughput while maintaining low power. These methods will implement digital radios with significant data rates in the X and Ka bands. Using these state-of-the-art technologies, unprecedented uplink and downlink capabilities can be achieved in a 1/2 U sized telemetry system. Additionally, modern FPGAs have embedded processing systems, such as ARM cores, integrated inside the FPGA allowing mundane tasks such as parameter commanding to occur easily and flexibly. Potential partners include other NASA centers, industry and the DOD. These assets are associated with small satellite demonstration flights, LEO and deep space applications. MSFC currently has an SDR transponder test-bed using Hardware-in-the-Loop techniques to evaluate and improve SDR technologies.

Varnavas, Kosta↗
NASA NTRS2016

The TESS Science Processing Operations Center

The Transiting Exoplanet Survey Satellite (TESS) will conduct a search for Earth's closest cousins starting in early 2018 and is expected to discover approximately 1,000 small planets with R(sub p) less than 4 (solar radius) and measure the masses of at least 50 of these small worlds. The Science Processing Operations Center (SPOC) is being developed at NASA Ames Research Center based on the Kepler science pipeline and will generate calibrated pixels and light curves on the NASA Advanced Supercomputing Division's Pleiades supercomputer. The SPOC will also search for periodic transit events and generate validation products for the transit-like features in the light curves. All TESS SPOC data products will be archived to the Mikulski Archive for Space Telescopes (MAST).

Jenkins, Jon M.↗
DOE OSTI2024

Large reductions in Permian Basin methane intensity shown in multi-year comparison of aerially-visible methane emissions

Spanning the US states of Texas and New Mexico, the Permian Basin has been a hotspot of methane emissions from oil and natural gas activity 1–4, although studies disagree over the magnitude of these emissions. The most comprehensive measurement campaigns published were conducted in 2019 1–5 .There have been large changes in the energy industry since then, including in the prices of oil and gas, both state and federal regulatory environments, investor and activist pressure over methane emissions, and the adoption of new technologies and policies by energy operators. Understanding how any or all of these might influence methane emissions is important for policy makers, oil and gas operators, and other stakeholders. We characterize the time evolution of Permian Basin methane emissions using a series of comprehensive aerial surveys conducted every year from 2020-2023 and compare them to the 2019 results cited above. To maintain comparability, all the data sets are from surveys using Insight M point source methane sensing technology. The scope of these surveys expanded over time: from 33-46% of wells, oil production, and gas production in 2020 to 60-65% in 2021, to 84% of wells and over 90% of both oil and gas production in 2023. These surveys by Insight M also include hundreds of gas processing plants and compressor stations as well as 1000s of km of gathering and transmission pipelines. Considering only the aerially detected portion of emissions (typically the majority of the total in such surveys 4), we find reductions of more than 70% in methane emissions intensity compared to the 2019 New Mexico-only Insight M survey, with variation depending on the year 3,4. Notably, although sources below 100 kg/hr contributed less than 10% of aerially measured emissions the 2019 New Mexico survey 3, these smaller sources constitute a larger proportion of total aerially measured emissions (although not the majority) in 2020-2023. Production facilities and gathering pipelines are responsible for the larges shares of total emissions, followed by compressor stations and gas processing plants. Permian methane emissions were also measured in a comprehensive 2019 Permian-wide survey by the Carbon Mapper team 2. That analysis led to a lower total emissions estimate at the time 4. These new Insight M-based emission rates are still roughly 30-70% lower than the aerially measured portion of the 2019 Carbon Mapper-based estimates 4. Further work is needed to harmonize these surveys in space and time to create the most intercomparable numbers possible 5. Additional analysis is needed to compare our findings to the more spatially constrained 2020, 2021, and 2023 Carbon Mapper surveys in the Permian 4,6. The evidence is strong from these two survey teams that emissions intensity has declined significantly since 2019. Reasons for this trend are currently unclear but point to possible success of emissions control programs. Future work investigating frequency, source, and operator-specific intensities could provide insights into the causes of this promising trend.

methane, oil and gas, data science, remote sensing↗
DOE OSTI2024

DiffLense: a conditional diffusion model for super-resolution of gravitational lensing data

Abstract Gravitational lensing data is frequently collected at low resolution due to instrumental limitations and observing conditions. Machine learning-based super-resolution techniques offer a method to enhance the resolution of these images, enabling more precise measurements of lensing effects and a better understanding of the matter distribution in the lensing system. This enhancement can significantly improve our knowledge of the distribution of mass within the lensing galaxy and its environment, as well as the properties of the background source being lensed. Traditional super-resolution techniques typically learn a mapping function from lower-resolution to higher-resolution samples. However, these methods are often constrained by their dependence on optimizing a fixed distance function, which can result in the loss of intricate details crucial for astrophysical analysis. In this work, we introduce DiffLense , a novel super-resolution pipeline based on a conditional diffusion model specifically designed to enhance the resolution of gravitational lensing images obtained from the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP). Our approach adopts a generative model, leveraging the detailed structural information present in Hubble space telescope (HST) counterparts. The diffusion model, trained to generate HST data, is conditioned on HSC data pre-processed with denoising techniques and thresholding to significantly reduce noise and background interference. This process leads to a more distinct and less overlapping conditional distribution during the model’s training phase. We demonstrate that DiffLense outperforms existing state-of-the-art single-image super-resolution techniques, particularly in retaining the fine details necessary for astrophysical analyses.

Computer Science↗
NASA NTRS2011

Software Displays Data on Active Regions of the Sun

The Solar Active Region Display System is a computer program that generates, in near real time, a graphical display of parameters indicative of the spatial and temporal variations of activity on the Sun. These parameters include histories and distributions of solar flares, active region growth, coronal mass ejections, size, and magnetic configuration. By presenting solar-activity data in graphical form, this program accelerates, facilitates, and partly automates what had previously been a time-consuming mental process of interpretation of solar-activity data presented in tabular and textual formats. Intended for original use in predicting space weather in order to minimize the exposure of astronauts to ionizing radiation, the program might also be useful on Earth for predicting solar-wind-induced ionospheric effects, electric currents, and potentials that could affect radio-communication systems, navigation systems, pipelines, and long electric-power lines. Raw data for the display are obtained automatically from the Space Environment Center (SEC) of the National Oceanic and Atmospheric Administration (NOAA). Other data must be obtained from the NOAA SEC by verbal communication and entered manually. The Solar Active Region Display System automatically accounts for the latitude dependence of the rate of rotation of the Sun, by use of a mathematical model that is corrected with NOAA SEC active-region position data once every 24 hours. The display includes the date, time, and an image of the Sun in H light overlaid with latitude and longitude coordinate lines, dots that mark locations of active regions identified by NOAA, identifying numbers assigned by NOAA to such regions, and solar-region visual summary (SRVS) indicators associated with some of the active regions. Each SRVS indicator is a small pie chart containing five equal sectors, each of which is color-coded to provide a semiquantitative indication of the degree of hazard posed by one aspect of the activity at the indicated location. The five aspects in question are the history of solar flares, the history of coronal mass ejections, the growth or decay of activity, the overall size, and the magnetic configuration. Mouse-clicking on an active-region-marking dot, SRVS indicator, or NOAA region number causes the program to generate a solar-region summary table (SRT) for the active region in question. The SRT contains additional quantitative and qualitative data, beyond those contained in the SRVS: These data include the solar coordinates of the region, the area of the region and its change in area during the past 24 hours, the change in the number of sunspots in the region during the past 24 hours, the magnetic configuration, and the types, dates, and times of the most recent flare and coronal mass ejection.

Golightly, Mike↗
NASA NTRSDate not supplied

SatCORPS Global Cloud Composite (GCC): the Design and Delivery of A High Quality, High Resolution, Global Cloud Product Available in Near-Real Time

The NASA Satellite ClOud and Radiation Property retrieval System (SatCORPS) supports the development of an analysis ready and cloud-optimized data transformation pipeline and geospatial service enablement of a global cloud composite (GCC) product derived from global geostationary satellite imagery. This geospatial service will be available at high temporal and spatial resolution via the SatCORPS web mapping application for visualization and analysis as well as direct ingestion to common geospatial software and custom programming. The resulting global cloud composite products from the processing pipeline can then be geospatially-service enabled as ArcGIS Image Services and Open Geospatial Consortium (OGC) Web Mapping/Coverage Services for visualization and analysis via a web mapping application and common geospatial software. Near real time global observations are created through the composition of five geostationary satellites that provides modelling and forecasting communities with the capability to provide high quality and timely information to start the projection process. The Global Cloud Composite product combines information from geostationary satellites, GOES-16, GOES-17, Himawari-8, Meteosat-11 and Meteosat-9 to create a single global composite netcdf file and images using the different products within the netcdf file. The SatCORPS team, though our Global Cloud Composite (GCC) product and web-based visualization tools including Geographical Information System (GIS) services provide near real time global cloud product information to both automated processes and traditional web users that is timely and high quality derived from geostationary satellites. The Global Cloud Composite product takes advantage of the scalable processing resources provided by the AWS batch service to provide new composites every thirty minutes. Because information from each of the low earth orbiting satellites is available on schedules tuned to the specific satellite, the processing algorithm temporally composites the final dataset as each satellite’s information becomes available. The SatCORPS team has leveraged our experience using Amazon Web Services (AWS) to build a low latency high availability tool that allows end users both human and automated to acquire high quality and high-resolution Geostationary Earth Orbiting (GEO) information at zero cost to the end user. This presentation will describe how we architected and implemented the service as well as lessons learned based on our experiences both developing and operating the system. The lessons learned include how we integrated multiple services including Amazon Batch, Amazon S3 and Amazon Lambda service to create a low cost but high-performance processing system that is capable of identifying and processing the most appropriate satellite overpass information into global cloud composites. We will also describe our web-based tools including our Geographic Information System that can be used for visualization and analysis. The products from the processing can be geospatially-service enabled as ArcGIS Image Services and Open Geospatial Consortium (OGC) Web Mapping/Coverage Services for visualization and analysis via a web mapping application and common geospatial software. The SatCORPS Global Composite Cloud product provides sophisticated global composited cloud research products with very low latency that we see that as filling a rapidly growing need in the research and modelling community with no up-front nor ongoing costs associated with downloading or using the information.

AWS AMCE SMCE GCC SATCORPS GLOBAL CLOUD COMPOSITE ↗