SEARCH · Search NASA

Results for “cloud computing; data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

NASA NTRS2022

Global Energy and Water Exchanges Program Surface Radiation Budget Release 4 Integrated Product: Algorithm Theoretical Basis Document

This document represents the Algorithm Theoretical Basis Document (ATBD) describing the NASA Langley Research Center's Surface Radiation budget (SRB) developed in collaboration with the World Research Climate Programme's Global Energy and Water Exchanges Program. The top of atmosphere and surface fluxes span from July 1, 1983 through June 30, 2017. The shortwave or solar wavelengths span the entire record. The thermal infrared or longwave fluxes span globally from January 1, 1988 to December 30, 2009. Before 1988, only the land-only fluxes are available. After 2009 only the ocean fluxes are available. This document describes all the inputs and radiative transfer algorithms used to generate the TOA and surface radiative fluxes. The fluxes include upward and downward, all-sky, clear-sky and pristine sky conditions. Surface net fluxes and cloud radiative effect are computed and presented as part of the analysis. Comparisons to other data products are made to assess the top-of-atmosphere and surface fluxes. Additionally, surface measurements are utilized to validation the surface fluxes. The surface validation analysis shows the the radiative fluxes are improved compared to the previous version. Data set cautions are also presented.

surface radiation budget↗

NASA NTRS1975

Analysis of cloud polarization measurements

The polarization of sunlight reflected by various cloud types was measured in the near-infrared with a polarimeter flown aboard the NASA Convair 990. The data reduction has been completed and detailed comparisons have been made between the observations and theoretical computations. The analysis presented shows that the polarization in the near-infrared can be used on a routine basis to reliably yield the cloud particle phase (water or ice) and the average particle size in the top part of water clouds (usually within + or - 25%). In some cases the polarization also yields the cloud optical thickness (or a lower limit) and a measure of the width of the particle size distribution in the top of the water clouds.

Hansen, J. E.↗

NASA NTRS2017

Cumulus: NASA's Cloud Based Distributed Active Archive Center Prototype

NASAs Earth Science Data System (ESDS) Program serves as a central cog in order to facilitate the implementation of NASA's Earth Science strategic plan. Since 1994, the ESDS Program has committed to the full and open sharing of Earth science data obtained from NASA instruments to all users. One of the key responsibilities of the ESDS Program is to continuously evolve the entire data and information system to maximize returns on the collected NASA data. An independent review was conducted in 2015 to holistically review the EOSDIS in order to identify gaps. The review recommendations were to investigate two areas: one, whether commercial cloud providers offer potential for storage, processing, and operational efficiencies, and two, the potential development of new data access and analysis paradigms. In response, ESDS has initiated several prototypes investigating the advantages and risks of leveraging cloud computing. This paper describes one such prototyping activity named Cumulus. Cumulus is being designed and developed as a "native" cloud-based data ingest, archive and management system that can be used for all future NASA Earth science data streams. Cumulus will foster design of new analysisvisualization tools that can leverage collocated data from all of the distributed DAACs as well as elastic cloud computing resources.

Earth Science Informatics↗

NASA NTRSDate not supplied

The Future of NASA Earth Science in the Commercial Cloud: Challenges and Opportunities

NASA produces a large volume and variety of data products that are used every day to support research, decision making, and education. The widespread use of NASA’s Earth Science data is enabled by NASA’s Earth Science Data System (ESDS) program, which oversees the archiving and distribution of these data and invests in the development of new data systems and tools. However, NASA’s current approach to Earth Science data distribution — based on distributed institutional archives with individual on-premises high-performance computing capabilities — faces some significant challenges, including massive increases in data volume from upcoming missions, a greater need for transdisciplinary science that synthesizes many different kinds of observations, and a push to make science more open, inclusive, and accessible. To address these challenges, NASA is aggressively migrating its Earth Science data and related tools and services into the commercial cloud. Migration of data into the commercial cloud can significantly improve NASA’s existing data system capabilities by (1) providing more flexible options for storage and compute (including rapid, as-needed access to state-of-the-art capabilities); (2) by centralizing and standardizing data access, which gives all of NASA’s institutional data centers access to all of each other’s datasets; and (3) by facilitating “analysis-in-place”, whereby users can bring their own computational workflows and tools to the data rather than having to maintain their own copies of NASA datasets. However, migration to the commercial cloud also poses some significant challenges, including (1) managing costs under a “pay-as-you-go” model; (2) incompatibility with existing tools and data formats with object-based storage and network access; (3) vendor lock-in; (4) challenges with data access for workflows that mix on-premise and cloud computing; and (5) standardization for highly diverse data as is present in NASA’s data archive. I conclude with two examples of recent NASA activities showcasing capabilities enabled by the commercial cloud: An interactive analysis and development platform for analyzing airborne imaging spectroscopy data, and a new collection of tools and services for data discovery, analysis, publication, and data-driven storytelling (Visualization, Exploration, and Data Analysis, VEDA).

Alexey N Shiklomanov↗

NASA NTRS2004

The Goddard Profiling Algorithm (GPROF): Description and Current Applications

Atmospheric scientists use different methods for interpreting satellite data. In the early days of satellite meteorology, the analysis of cloud pictures from satellites was primarily subjective. As computer technology improved, satellite pictures could be processed digitally, and mathematical algorithms were developed and applied to the digital images in different wavelength bands to extract information about the atmosphere in an objective way. The kind of mathematical algorithm one applies to satellite data may depend on the complexity of the physical processes that lead to the observed image, and how much information is contained in the satellite images both spatially and at different wavelengths. Imagery from satellite-borne passive microwave radiometers has limited horizontal resolution, and the observed microwave radiances are the result of complex physical processes that are not easily modeled. For this reason, a type of algorithm called a Bayesian estimation method is utilized to interpret passive microwave imagery in an objective, yet computationally efficient manner.

Olson, William S.↗

NASA NTRS1991

SEPAC data analysis in support of the environmental interaction program

Data analyses of the Space Experiments with Particle Accelerators (SEPAC) data and computer modeling were conducted to investigate spacecraft environmental effects associated with injection of electron beams, plasma clouds, and neutral gas clouds from the Shuttle orbiter. The data analysis indicates that Extremely Low Frequency oscillations from 150 to 200 Hz were seen in the Langmuir probe current when the beam was fired in a continuous mode. The strongest oscillations occurred when the ambient pressure was augmented by neutral gas releases from the SEPAC plasma accelerator magnetoplasma-dynamic (MPD) arcjet. To understand the dependence of spacecraft charging potential on beam density and other plasma parameters, a two-dimensional electrostatic particle code was used to simulate the injection of electron beams from an infinite conductor into a plasma. The simulations show that the conductor charging potential depends critically on the reflection coefficient of the conductor surface, which is defined as the percentage of incident particles reflected by the conductor. The ionization effects on spacecraft charging were examined by including interactions of electrons with neutral gas. The simulations show that the conductor charging potential decreases with increasing neutral background density due to the production of secondary electrons near the conductor surface. The simulations also indicate that the beam radius is generally proportional to the beam electron gyroradius when the conductor is charged to a large potential. It appears that the charge buildup at the beam stagnation point causes the beam radial expansion. A survey of the simulation results suggests that the ratio of the beam radius to the beam electron gyroradius increases with the square root of beam density and decreases inversely with beam injection velocity. These results are useful for explaining the spacecraft charging phenomena observed during SEPAC experiments from Spacelab 1.

Lin, Chin S.↗

NASA NTRS2016

Climate Analytics as a Service

Exascale computing, big data, and cloud computing are driving the evolution of large-scale information systems toward a model of data-proximal analysis. In response, we are developing a concept of climate analytics as a service (CAaaS) that represents a convergence of data analytics and archive management. With this approach, high-performance compute-storage implemented as an analytic system is part of a dynamic archive comprising both static and computationally realized objects. It is a system whose capabilities are framed as behaviors over a static data collection, but where queries cause results to be created, not found and retrieved. Those results can be the product of a complex analysis, but, importantly, they also can be tailored responses to the simplest of requests. NASA's MERRA Analytic Service and associated Climate Data Services API provide a real-world example of climate analytics delivered as a service in this way. Our experiences reveal several advantages to this approach, not the least of which is orders-of-magnitude time reduction in the data assembly task common to many scientific workflows.

big data↗

NASA NTRS1990

Cloud spatial structure during the FIRE MS IFO

The fractal properties of clouds observed during the FIRE Marine Stratocumulus Intensive Field Observations (MS IFO) and their effects on the large scale radiative properties of the atmosphere are examined. This involves three states: (1) analysis of LANDSAT Thematic Mapper (TM) cloud data to determine the scaling properties associated with various cloud types; (2) simulation of fractal clouds with realistic scaling properties; and (3) computation of mean radiative properties of fractal clouds as a function of their scaling properties. Thirty-three LANDSAT scenes were acquired as part of the FIRE Marine Stratocumulus IFO in July 1987. They exhibit a wide variety of stratocumulus structures. Analysis has so far focused upon the July 7 scene, in which the NASA ER-2, the BMO C130 and the NCAR Electra repeatedly gathered data across a stratocumulus-fair weather cumulus transition.

Cahalan, Robert F.↗

NASA NTRSDate not supplied

Development of Computational Environmental Microbiome Workflows for the Laboratory and the International Space Station

Identification of microorganisms in the spaceflight environment is critical for crew health risk assessment on the International Space Station (ISS). Since 2017, nanopore sequencing technology has been used to support thein situ identification of microbial species during spaceflight. Beginning in 2018, a culture-independent, swab-to-sequencer method was implemented onboard the ISS to provide a more thorough insight of the ISS microbiome. Eliminating microbial culture enables identification of difficult-to-culture organisms, reduces risks associated with potentially pathogenic cultures, and could significantly reduce the time from sample-to-answer. However, this molecular-based approach generates large metagenomic datasets that require substantial computational resources for analysis. To process nanopore-generated sequencing data, the JSC Microbiology Laboratory established a bioinformatics workflow on Amazon EC2 under the security guidance of the NASA Science Managed Cloud Environment (SMCE).This resource allows for the development, testing, and accessing of computational tools for processing large and complex datasets. The work described here will address the downlinking of data from the ISS, the automated pipeline developed to identify targeted bacterial and fungal organisms, and the time from sampling onboard to microbial identification. The pipelines have been enhanced to address high and low biomass samples using optimization based on sample source (air, water, or surface) and type of collection (filter, colony, or swab).The resulting microbiome data can be assessed beyond microbial identifications to gain understanding toward population changes over time, potential selective environmental pressures, and evaluating correlations with a wide range of additional data sets. Metagenome analysis pipelines in development could allow for simultaneous identification of microbial species, gene function, and gene pathways present in the environment. Beyond the ground processing, the developed analysis pipeline is currently deployed onboard the ISS to allow for near real-time assessments of the ISS microbiome. This study serves as a critical foundation for exploration missions, where rapid microbiome analyses will be required.

G. Marie Sharp↗

NASA NTRS2019

Tuning the Performance of a Computational Persistent Homology Package

In recent years, persistent homology has become an attractive method for data analysis. It captures topological features, such as connected components, holes, and voids from point cloud data and summarizes the way in which these features appear and disappear in a filtration sequence. In this project, we focus on improving the performanceof Eirene, a computational package for persistent homology. Eirene is a 5000-line open-source software library implemented in the dynamic programming language Julia. We use the Julia profiling tools to identify performance bottlenecks and develop novel methods to manage them, including the parallelization of some time-consuming functions on multicore/manycore hardware. Empirical results show that performance can be greatly improved.

Persistent Homology↗

NASA NTRS1998

Prediction of Particle Number Density and Particle Properties in the Flow Field Observed by the Nephelometer Experiment on the Galileo Probe

This report summarizes the work performed to assist in the analysis of data returned from the Galileo Probe's Nephelometer instrument. A computation of the flow field around the Galileo Probe during its descent through the Jovian atmosphere was simulated. The behavior of cloud particles that passed around the Galileo probe was then computed and the number density in the vicinity of the Nephelometer instrument was predicted. The results of our analysis support the finding that the number density of cloud particles was not the same in each of the four sampling volumes of the Nephelometer instrument. The number densities calculated in this study are currently being used to assist in the reanalysis of the data returned from the Galileo Probe.

Naughton, Jonathan W.↗

NASA NTRS1998

Prediction of Particle Number Density and Particle Properties in the Flow Field Observed by the Nephelometer Experiment on the Galileo Probe

Naughton, Jonathan W.↗

NASA NTRS2017

Performance Enhancement of a Computational Persistent Homology Package

In recent years, persistent homology has become an attractive method for data analysis. It captures topological features, such as connected components, holes, voids, etc., from a point cloud by finding out when these features appear and disappear in the filtration sequence. In this project, we focus on improving the performance of Eirene, a fancy computational persistent homology package. Eirene is a 5000-line opensource software implemented by using the dynamic programming language Julia. We use the Julia profiling tools to identify the performance bottlenecks and develop different methods to manage the bottlenecks, including the parallelization of some time-consuming functions on the multicore/manycore hardware. The empirical results show that the performance can be greatly improved.

Profiling↗

NASA NTRS2019

Cloud Computing Methods for Near Rectilinear Halo Orbit Trajectory Design

Complicated mission design problems require innovative computational solutions. As spacecraft depart from a proposed Gateway in a Near Rectilinear Halo Orbit (NRHO), recontact analysis is required to avoid risk of collision and ensure safe operations. Escape dynamics from NRHOs are governed by multiple gravitational bodies, yielding a trajectory design space that is exhaustively large. This paper summarizes the recontact analysis for departure from the NRHO and describes how the Deep Space Trajectory Explorer (DSTE) trajectory design software incorporates high performance cloud computing to compute and visualize the orbit design space. Recent focus on exploration missions to cislunar space has kindled accelerated interest in multibody orbit solutions. Trajectory analysis in the presence of multiple gravity fields is complex, and innovative computational tools are needed to simplify complicated design spaces, to generate large quantities of data quickly, and to visualize the output for user accessibility. The Gateway mission is a prime example. The Gateway1 is proposed as a human outpost in deep space. The current baseline orbit for the Gateway is a Near Rectilinear Halo Orbit (NRHO) near the Moon.2 The NRHO exists in a regime that experiences the gravitational effects of the Earth and the Moon simultaneously, complicating orbit analysis. The mission design process benefits greatly from updated computational tools for multibody missions like the Gateway. As an example, consider the problem of assessing the risk of collision in an NRHO. As a staging location to missions to the lunar surface and beyond the Earth-Moon system, the Gateway will experience spacecraft and other objects regularly arriving and departing. Departing objects potentially include spent logistics modules, visiting crew vehicles, debris objects, wastewater particles, and cubesats. Each departure is governed by the dynamics of the Gateway orbit and the surrounding dynamical environment. Over time, any unmaintained object in such an orbit eventually departs due to the small instabilities associated with the NRHOs. A separation maneuver speeds the departure from the NRHO, but the effects of the maneuver on the spacecraft behavior depend on the location, magnitude, and direction of the burn. Escape dynamics from the NRHO with regard to these maneuver options open up an enormous potential trajectory design space where subtle changes in input can produce dramatically large changes in the results. Any departing object must avoid recontacting the Gateway as it leaves the lunar vicinity, and a recontact analysis thus involves a significant number of computations and extensive output data. To explore the dynamics of this extensive design space, the Deep Space Trajectory Explorer3 (DSTE) trajectory design software incorporates new High Performance Computing (HPC) services and novel interactive visualizations. This paper details the HPC and cloud infrastructure techniques that are implemented in the DSTE, applying the new capabilities to analysis of recontact risk with the Gateway in NRHO. NEAR RECTILINEAR HALO ORBITS The Gateway is planned to fly in a lunar NRHO as its baseline orbit. The NRHO families of orbits are subsets of the larger halo families, which originate from planar orbits near the L1 and L2 libration points; the Earth-Moon L2 halo family appears in Figure 1. Each halo orbit is perfectly periodic in the Circular Restricted 3-Body Problem (CR3BP) and becomes a quasi-periodic orbit in a higher fidelity ephemeris force model. The NRHOs are defined as those members of the halo family with bounded stability properties;2 they pass near the Moon at perilune and are nearly polar. Families exist with apolunes located both above the lunar north pole and above the lunar south pole; the Gateway is planned to reside in a southern L2 NRHO in a 9:2 resonance with the lunar synodic period. The 9:2 NRHO is characterized by a period of about 6.5 days, a perilune radius of about 3,500 km, and an apolune radius of about 71,000 km; it is strongly affected by the gravity of both the Earth and the Moon simultaneously. This NRHO offers extended communications with assets on the south pole of the Moon,4 as well as low-cost orbit maintenance and attitude control,5 favorable eclipse avoidance properties,6 and inexpensive transfers from Earth and to other destinations.5,7 The NRHO portion of the southern L2 halo family is highlighted in black in Figure 1, and the 9:2 NRHO appears in blue.

Phillips, Sean M.↗

NASA NTRSDate not supplied

Biomass Harmonization and SAR Analysis with the Multi-mission Algorithm and Analysis Platform (MAAP)

The Multi‐mission Algorithm and Analysis Platform (MAAP) is a collaborative effort between NASA and the European Space Agency (ESA) to support above ground biomass (AGB) research in an open science framework. MAAP brings together relevant data, algorithms, and computing capabilities in a common cloud environment to address the challenges of sharing and processing data from field, airborne and satellite measurements. MAAP was publicly released in October 2021, providing computing capabilities co-located with the data, a collaborative coding and analysis environment, and a set of interoperable tools and algorithms developed to support the estimation and visualization of data. MAAP has allowed scientists from both North America and Europe to collaborate on the generation and analysis/visualization of data derived from multiple, discipline-adjacent missions in an open, collaborative environment that has reached beyond traditional scientific investigation. MAAP has been used to support multiple scientific activities. To date, existing LiDAR data from multiple platforms has been calibrated with field measurements and combined for more comprehensive and accurate estimates of above ground biomass AGB; these LiDAR platforms include airborne (e.g. LVIS), the International Space Station (NASA’s Global Ecosystem Dynamics Investigation (GEDI), and satellites (e.g. ICESat-2). The current challenge is to effectively and seamlessly combine the aforementioned LiDAR-based data with new data sources such as P-band RADAR from ESA’s upcoming BIOMASS mission, existing ESA Sentinel-1 C-band SAR, and the 30 PB/yr of high cadence global coverage L-band SAR data from the upcoming NASA-ISRO SAR (NISAR) mission. Recent analysis using MAAP merged ICESat-2 and optical data (Harmonized Landsat Sentinel) produced the most comprehensively precise estimate of boreal-wide AGB to date. Another effort using MAAP is the production and open distribution of global comparisons of AGB map estimates, including from ICESat-2 and GEDI, to bolster stakeholder uptake for policy applications. These map estimates will feed into the Intergovernmental Panel on Climate Change (IPCC) database, likely aiding the next Global Carbon Stocktake of the UNFCCC. Furthermore, the biomass retrieval intercomparison exercise BRIX-2 could benefit from the MAAP providing standardized test cases (based on airborne campaign and spaceborne data) allowing the community to develop and apply retrieval algorithms based on these test cases, while forthcoming SAR data training curricula could also use the MAAP as a teaching and learning platform. The MAAP is meeting the challenges inherent in international, open science collaboration and large scale computing with a platform that is entirely open source and cloud native, using open standards for data access, manipulation, protocols, and formats. The MAAP data system consists of a dedicated data store whose data is indexed in an online catalog conforming to established metadata, application programmatic interfaces (APIs), and service interface standards, using an implementation of the open sourced NASA Common Metadata Repository. Federation of user identities allows users from either NASA or ESA to access and consume services from the other using a unified metadata catalog for the data utilized across the ESA and NASA MAAP platforms. Similarly, we are exploring how to increase interoperability to achieve a common approach to packaging, orchestrating and executing algorithms, with interoperable access to data for subsetting, fast browse, and cloud-optimized access, all using interoperable standards such as those from the Open Geospatial Consortium (OGC). Designed for interoperability, ESA and NASA utilize a common architecture for the software platform. It provides a cloud-based algorithm development environment (ADE) that enables scientists to develop algorithms collaboratively with access to the MAAP data catalog as well as other data archives. MAAP provides an Eclipse Che-based ADE supporting both Python and R languages, popular in this biomass community. Algorithms developed and containerized within the ADE can be deployed to run to thousands of computational nodes in the MAAP’s data processing system (DPS), dramatically speeding up processing and giving scientists a rapid, iterative turnaround of results. NASA’s implementation of the DPS is based on the Hybrid Science Data System (HySDS) framework, used by NASA flight projects to produce Earth science standard products.

cloud computing↗

NASA NTRS2017

Educational and Scientific Applications of Climate Model Diagnostic Analyzer

Climate Model Diagnostic Analyzer (CMDA) is a web-based information system designed for the climate modeling and model analysis community to analyze climate data from models and observations. CMDA provides tools to diagnostically analyze climate data for model validation and improvement, and to systematically manage analysis provenance for sharing results with other investigators. CMDA utilizes cloud computing resources, multi-threading computing, machine-learning algorithms, web service technologies, and provenance-supporting technologies to address technical challenges that the Earth science modeling and model analysis community faces in evaluating and diagnosing climate models. As CMDA technology and infrastructure have matured, we have developed the educational and scientific applications of CMDA. Educationally, CMDA supported the summer school of the JPL Center for Climate Sciences in 2014, 2015, and 2016. In the summer school, the students work on group research projects where CMDA provide datasets, analysis tools, and provenance support utility tools. Each student is assigned to a virtual machine with CMDA installed in Amazon Web Services. Scientifically, we have developed several science use cases of CMDA covering various topics, datasets, and analysis types. Each of the science use cases is described in terms of a scientific goal, datasets used, the analysis tools used, scientific results discovered, an analysis result such as output plots and data files, and a link to the corresponding analysis service call with all the input arguments filled.

Bao, Qihao↗

NASA NTRS1981

Severe storms and local weather research

Developments in the use of space related techniques to understand storms and local weather are summarized. The observation of lightning, storm development, cloud development, mesoscale phenomena, and ageostrophic circulation are discussed. Data acquisition, analysis, and the development of improved sensor and computer systems capability are described. Signal processing and analysis and application of Doppler lidar data are discussed. Progress in numerous experiments is summarized.

Source record↗

NASA NTRS2013

NASA Tech Briefs, June 2013

Topics include: Cloud Absorption Radiometer Autonomous Navigation System - CANS, Software Method for Computed Tomography Cylinder Data Unwrapping, Re-slicing, and Analysis, Discrete Data Qualification System and Method Comprising Noise Series Fault Detection, Simple Laser Communications Terminal for Downlink from Earth Orbit at Rates Exceeding 10 Gb/s, Application Program Interface for the Orion Aerodynamics Database, Hyperspectral Imager-Tracker, Web Application Software for Ground Operations Planning Database (GOPDb) Management, Software Defined Radio with Parallelized Software Architecture, Compact Radar Transceiver with Included Calibration, Software Defined Radio with Parallelized Software Architecture, Phase Change Material Thermal Power Generator, The Thermal Hogan - A Means of Surviving the Lunar Night, Micromachined Active Magnetic Regenerator for Low-Temperature Magnetic Coolers, Nano-Ceramic Coated Plastics, Preparation of a Bimetal Using Mechanical Alloying for Environmental or Industrial Use, Phase Change Material for Temperature Control of Imager or Sounder on GOES Type Satellites in GEO, Dual-Compartment Inflatable Suitlock, Modular Connector Keying Concept, Genesis Ultrapure Water Megasonic Wafer Spin Cleaner, Piezoelectrically Initiated Pyrotechnic Igniter, Folding Elastic Thermal Surface - FETS, Multi-Pass Quadrupole Mass Analyzer, Lunar Sulfur Capture System, Environmental Qualification of a Single-Crystal Silicon Mirror for Spaceflight Use, Planar Superconducting Millimeter-Wave/Terahertz Channelizing Filter, Qualification of UHF Antenna for Extreme Martian Thermal Environments, Ensemble Eclipse: A Process for Prefab Development Environment for the Ensemble Project, ISS Live!, Space Operations Learning Center (SOLC) iPhone/iPad Application, Software to Compare NPP HDF5 Data Files, Planetary Data Systems (PDS) Imaging Node Atlas II, Automatic Calibration of an Airborne Imaging System to an Inertial Navigation Unit, Translating MAPGEN to ASPEN for MER, Support Routines for In Situ Image Processing, and Semi-Supervised Eigenbasis Novelty Detection.

Source record↗