Search NASA⌕ Search

SEARCH · Search NASA

Results for “Cloud Architectures”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2017

Archive Management of NASA Earth Observation Data to Support Cloud Analysis

NASA collects, processes and distributes petabytes of Earth Observation (EO) data from satellites, aircraft, in situ instruments and model output, with an order of magnitude increase expected by 2024. Cloud-based web object storage (WOS) of these data can simplify the execution of such an increase. More importantly, it can also facilitate user analysis of those volumes by making the data available to the massively parallel computing power in the cloud. However, storing EO data in cloud WOS has a ripple effect throughout the NASA archive system with unexpected challenges and opportunities. One challenge is modifying data servicing software (such as Web Coverage Service servers) to access and subset data that are no longer on a directly accessible file system, but rather in cloud WOS. Opportunities include refactoring of the archive software to a cloud-native architecture; virtualizing data products by computing on demand; and reorganizing data to be more analysis-friendly. Reviewed by Mark McInerney ESDIS Deputy Project Manager.

Lynnes, Christopher↗
NASA NTRSDate not supplied

NASA's Implementation of Cloud Services for Human Space Flight

Cloud is a tried-and-true technology used throughout United States government agencies, including the National Aeronautics and Space Administration (NASA). With reliable results and infrequent downtimes, cloud allows for secure remote access, customizability, and streamlined monitoring options, creating an environment for better data integrity and availability. As NASA increasingly migrates functions to the cloud, the Space Communications and Navigation Program (SCaN) program has been investigating how this capability can be leveraged to provide communication services to its users and customers. Currently, missions such as NASA-ISRO Synthetic Aperture Radar (NISAR), Plankton, Aerosol, Cloud, ocean Ecosystem (PACE), and Roman Space Telescope (RST) are planned to incorporate cloud into their data delivery architecture. However, SCaN is looking to expand further. This conversion to using cloud services allows for greater availability of mission data for both robotic and human space flight (HSF)missions. The SCaN program and the Near Space Network (NSN) are working to consolidate resources and create a cloud environment suitable for the entirety of the SCaN program network architecture. SCaN is in the process of finalizing its cloud architecture and soon will be implementing cloud services. The new services used will adhere to federal regulations including Federal Risk and Authorization Management Program (FedRAMP), which is built upon National Institute of Standards and Technology (NIST)documentation. While keeping in mind these security requirements, an auxiliary objective of the cloud integration is to ensure the most cost-efficient solution; providing a scalable, robust and resilient system. Using cloud services, NASA will gain access to better centralized monitoring and management features, along with customizable services on a pay-per-use plan. With the ever-growing NASA mission data volume needs, maintaining ample storage space is another major constraint. Processing and storing such large amounts of data, on the order of terabytes a day, requires dynamic processing capability which is inherently a strength of cloud computing. By routing this data from ground stations through the cloud, there will be greater ease of access for both SCaN and the user community. Artificial intelligence and other built-in cloud functions can also enhance efficiency, improving data processing time. Thereby also allowing for better data availability. As we look to the future of cloud services, NASA will continue to leverage capabilities that will benefit NASA’s ability to provide cost-effective communication services. This paper further outlines the evolution of cloud use by SCaN in the context of Human Space Flight.

cloud storage↗
NASA NTRS2018

Generalizing a Data Analysis Pipeline in the Cloud to Handle Diverse Use Cases in NASA's EOSDIS

NASA's Earth Observing System Data and Information System (EOSDIS) is tasked with archiving and distributing Earth Observation data across a range of disciplines, including atmospheric science, oceanography, land processes, natural hazards, solar radiance and even socioeconomic aspects relating to the environment. Driven by rapidly rising data volumes, EOSDIS is migrating to a cloud computing based archive over the next few years. Although this simplifies data management somewhat, the main aim is to provide the data in an environment where end users can bring their analysis to the data rather than attempting to download and manage ever-increasing volumes. To that end, a cloud-based analysis platform is being constructed to enable data transformations, analyses and visualization without egressing the data from the cloud. In this endeavor, we expect a wide variety of users, algorithms and use cases. Consequently, the architecture of this cloud analytics platform is expressly designed to be based on open services, thus fostering an ecosystem that enables the efficient combination of common components with data-specific or analysis-specific components. Reviewed and approved by Andrew Mitchell, ESDIS project manager.

Cloud computing↗
NASA NTRS2017

Cloud Surprises in Moving NASA EOSDIS Applications into Amazon Web Services

NASA ESDIS has been moving a variety of data ingest, distribution, and science data processing applications into a cloud environment over the last 2 years. As expected, there have been a number of challenges in migrating primarily on-premises applications into a cloud-based environment, related to architecture and taking advantage of cloud-based services. What was not expected is a number of issues that were beyond purely technical application re-architectures. We ran into surprising network policy limitations, billing challenges in a government-based cost model, and difficulty in obtaining certificates in an NASA security-compliant manner. On the other hand, this approach has allowed us to move a number of applications from local hosting to the cloud in a matter of hours (yes, hours!!), and our CMR application now services 95% of granule searches and an astonishing 99% of all collection searches in under a second. And most surprising of all, well, you'll just have to wait and see the realization that caught our entire team off guard!

Cloud↗
NASA NTRS2022

The Mars 2020 Ground Data System Architecture

The Mars 2020 Mission’s primary objective is to collect 20 geographically unique samples during its prime mission of one and a quarter Martian years, or just over 2 Earth years. Mission planners determined the project needed to develop a system that would enable the operations team to analyze engineering and science data, make science decisions, select viable rover targets at a millimeter resolution and validate an uplink bundle for a car sized rover with more complex science instruments than any previous Mars surface mission. All this had to be done within a five hour time frame. Doing this with a small team would be a challenge, but this had to be accomplished by a large team of engineers and scientists located across North America and Europe. Achieving this level of operational efficiency was unheard of in the prime mission. In addition, the mission had another set of requirements that had nothing to do with surface operations; the Mars 2020 Ground Data System (GDS) was also expected to comply with a new set of security requirements to keep up with the ever changing cybersecurity landscape. The Mars 2020 Ground Data System (GDS) is a re-architected version of the Mars Science Laboratory GDS. The primary goal was to integrate the lessons learned from previous Mars surface missions, accommodate a set of new requirements and capabilities required to ensure mission success, and comply with a new set of cybersecurity controls. The new architecture includes several unique qualities including a data lake, language-agnostic system-wide event-based operations, containerization, automated deployment, network segmentation, infrastructure-as-code, API-driven interfaces, and the first Mars surface GDS to operate primarily in the cloud. The new architecture enabled greater access to the system’s data, tighter integration with the operations team, and a higher level of traceability. The availability of the data also enabled a new set of capabilities previously not possible on surface missions. These new capabilities include an autonomous data to information, pipeline for downlink analysis, horizontal scaling of science data processing capabilities, autonomous round trip data tracking of science and engineering data, integration of flight system state into the tactical planning cycle, high fidelity targeting utilizing kinematic data, and hierarchical image and 3d meshes data representations. This paper will introduce the requirements for the Mars 2020 Mission, the heritage architecture, and the rationale for the changes to achieve the new architecture. The paper will continue to describe the fundamental changes made to the GDS architecture, how these changes enabled a more tightly integrated GDS, and the new capabilities that were enabled by the new architecture. The paper will conclude with the lessons learned from the process of rearchitecting a heritage GDS system and from the first 200 days of operations supporting over 800 users from around the world.

Lopez-Roig, Reynaldo↗
NASA NTRS2012

Cloud Computing for Mission Design and Operations

The space mission design and operations community already recognizes the value of cloud computing and virtualization. However, natural and valid concerns, like security, privacy, up-time, and vendor lock-in, have prevented a more widespread and expedited adoption into official workflows. In the interest of alleviating these concerns, we propose a series of guidelines for internally deploying a resource-oriented hub of data and algorithms. These guidelines provide a roadmap for implementing an architecture inspired in the cloud computing model: associative, elastic, semantical, interconnected, and adaptive. The architecture can be summarized as exposing data and algorithms as resource-oriented Web services, coordinated via messaging, and running on virtual machines; it is simple, and based on widely adopted standards, protocols, and tools. The architecture may help reduce common sources of complexity intrinsic to data-driven, collaborative interactions and, most importantly, it may provide the means for teams and agencies to evaluate the cloud computing model in their specific context, with minimal infrastructure changes, and before committing to a specific cloud services provider.

cloud computing↗
NASA NTRS2014

A Geosynchronous Orbit Optical Communications Relay Architecture

NASA is planning to fly a Next Generation Tracking and Data Relay Satellite (TDRS) next decade. While the requirements and architecture for that satellite are unknown at this time, NASA is investing in communications technologies that could be deployed on the satellite to provide new communications services. One of those new technologies is optical communications. The Laser Communications Relay Demonstration (LCRD) project, scheduled for launch in December 2017 as a hosted payload on a commercial communications satellite, is a critical pathfinder towards NASA providing optical communications services on the Next Generation TDRS. While it is obvious that a small to medium sized optical communications terminal could be flown on a GEO satellite to provide support to Near Earth missions, it is also possible to deploy a large terminal on the satellite to support Deep Space missions. Onboard data processing and Delay Tolerant Networking (DTN) are two additional technologies that could be used to optimize optical communications link services and enable additional mission and network operations. This paper provides a possible architecture for the optical communications augmentation of a Next Generation TDRS and touches on the critical technology work currently being done at NASA. It will also describe the impact of clouds on such an architecture and possible mitigation techniques.

optics↗
NASA NTRS2023

Toward Physics-informed Neural Networks for 3D Multi-layer Cloud Mask Reconstruction

Three-dimensional (3D) cloud retrievals are critical for understanding their impact on climate and other applications such as aviation safety, weather prediction, and remote sensing. However, obtaining high-resolution and accurate vertical representation of clouds remains unsolved due to the limitations imposed by satellite instrumentation, viewing conditions, and the complexity of cloud dynamics. Cloud masks are essential for comprehending various cloud vertical properties, but deriving accurate 3D cloud masks from 2D satellite imagery data is a challenging task. To tackle these challenges, we introduce a physics-informed loss function for training deep learning models that can extend 2D cloud images into 3D cloud masks. The proposed loss, called CloudMask Loss, is composed of two domain knowledge-informed loss terms: one for evaluating cloud position and thickness, and the other for measuring the number of layers. By combining these loss terms, we improve the trainability of the deep learning models for more accurate and meaningful results. We apply the proposed loss function to different neural networks and demonstrate significant improvements in multi-layer cloud mask reconstruction. Utilizing the same neural network architecture, our proposed loss outperforms standard binary crossentropy loss in terms of multi-layer cloud classification accuracy, number of layers accuracy, and thickness mean absolute error (MAE). The proposed loss function can be readily integrated into various neural network architectures, resulting in substantial performance gains in 3D cloud mask generation.

multi-layer clouds↗
NASA NTRS1988

A support architecture for reliable distributed computing systems

The Clouds project is well underway to its goal of building a unified distributed operating system supporting the object model. The operating system design uses the object concept of structuring software at all levels of the system. The basic operating system was developed and work is under progress to build a usable system.

Dasgupta, Partha↗
NASA NTRS2017

Task 28: Web Accessible APIs in the Cloud Trade Study

This study explored three candidate architectures for serving NASA Earth Science Hierarchical Data Format Version 5 (HDF5) data via Hyrax running on Amazon Web Services (AWS). We studied the cost and performance for each architecture using several representative Use-Cases. The objectives of the project are: Conduct a trade study to identify one or more high performance integrated solutions for storing and retrieving NASA HDF5 and Network Common Data Format Version 4 (netCDF4) data in a cloud (web object store) environment. The target environment is Amazon Web Services (AWS) Simple Storage Service (S3).Conduct needed level of software development to properly evaluate solutions in the trade study and to obtain required benchmarking metrics for input into government decision of potential follow-on prototyping. Develop a cloud cost model for the preferred data storage solution (or solutions) that accounts for different granulation and aggregation schemes as well as cost and performance trades.

cost model↗
NASA NTRS1991

Analysis of global cloud imagery from multiple satellites

IR measurements simultaneously conducted by four GEO and two polar-orbit satellites have been used to create synoptic global cloud-field images. The composite Global Cloud Imagery (GCI), with 0.5-deg horizontal resolution and 3-hr temporal resolution, constitute a nearly instantaneous snapshot of the global cloud pattern. To facilitate analysis of global cloud behavior, the GCI has been constructed with certain homogeneous properties. An interactive Image Analysis System (IAS) is developed in order to investigate the space-time behavior of global cloud activity. Due to its customized architecture and the homogeneous properties of the GCI, the IAS can perform space-time covariance analyses of the three-dimensional data with 'interactive' speed. Statistical properties of cloud variability are given.

Salby, Murry L.↗
NASA NTRS2018

PACE Technical Report Series, Volume 5: Mission Formulation Studies

This chapter summarizes the mission architecture for the Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) mission, ranging from its scientific rationale to the history of its realized conception to itspresent-day organization and management. This volume in the PACE Technical Report series focuses ontrade studies that informed the formulation of the mission in its pre-Phase A (2014-2016; pre-formulation:define a viable and affordable concept) and Phase A (2016-2017; concept and technology development).With that in mind, this chapter serves to introduce the mission by providing: a brief summary of thescience drivers for the mission; a history of the direction of the mission to NASA's Goddard Space Flight Center (GSFC); a synopsis of the mission's and instruments' management and development structures; and a brief description of the primary components and elements that form the foundation ofthe mission, encompassing the major mission segments (space, ground, and science data processing) and their roles in integration, testing, and operations.

Cetinic, Ivona↗
NASA NTRS2019

Cloud Giovanni: Reining in Costs and Improving Performance with Analytical Data Stores Using Scalable Serverless Architecture

Giovanni is the Geospatial Interactive Online Visualization ANd aNalysis Infrastructure developed at NASA GES DISC which provides a simple and intuitive way to visualize, analyze, and access vast amounts of Earth science data. It receives large number of user requests each day for a variety of analysis and visualization services, which leads to the big data challenge of serving gradually increasing large data volumes with diverse statistical algorithms. We hereby propose a multi-dimensional accumulation method which provides fast and cost-efficient cloud analysis for diverse services including both area averaging and time averaging. This method involves the weighted volume integration over multiple variable dimensions (time and space), and is implemented in AWS using Athena providing serverless and highly scalable data analysis. Compared to the standard method, this approach dramatically reduced the computational time by order of magnitude with a minimal AWS cost incurred. For example, for a benchmark of 10-year area averaging over the 1x1 degree daily variable, the computational time was reduced from minutes to seconds, and the Athena cost is only $5 for 100,000 requests.

Zhang, Hailiang↗
NASA NTRS2017

Giovanni in the Cloud: Earth Science Data Exploration in Amazon Web Services

Giovanni is an exploration tool at the NASA Goddard Earth Sciences Data Information Services Center (GES DISC), providing 22 analysis and visualization services for over 1600 Earth Science data variables. Owing to its popularity, Giovanni has experienced a consistent growth in overall demand, with periodic usage spikes attributed to trainings by education organizations, extensive data analysis in response to natural disasters, preparations for science meetings, etc. Furthermore, the new generation of spaceborne sensors and high resolution models have resulted in an exponential growth in data volume with data distributed across the traditional boundaries of data centers. Seamless exploration of data (without users having to worry about data center boundaries) has been a key recommendation of the GES DISC User Working Group. These factors have required new strategies for delivering acceptable performance. The cloud-based Giovanni, built on Amazon Web Services (AWS), evaluates (1) AWS native solutions to provide a scalable, serverless architecture; (2) open standards for data storage in the Cloud; (3) a cost model for operations; and (4) end-user performance. Our preliminary findings indicate that the use of serverless architecture has a potential to significantly reduce development and operational cost of Giovanni. The combination of using AWS managed services, storage of data in open standards, and schema-on-read data access strategy simplifies data access and analytics, in addition to making data more accessible to the end users of Giovanni through popular programming languages.

Giovanni↗
NASA NTRSDate not supplied

Reconfiguring COSSIR for the Next Generation of Cloud and Precipitation Science

NASA originally developed the Compact Scanning Sub-millimeter Imaging Radiometer (CoSSIR) in the early 2000s, where it was own in the CRYSTAL-FACE and TC4 field experiments. With renewed interest in sub-millimeter remote sensing motivated by the upcoming launch of the EUMETSAT-sponsored Ice Cloud Imager (ICI), along with its potential for inclusion as part of the NASA Decadal Survey Aerosols, CLouds, Convection, and Precipitation (ACCP) observing architecture, CoSSIR is being refurbished to prepare for routine suborbital submillimeter measurements this decade. Upgrades include hardware (procurement of shared parts so that CoSMIR and CoSSIR can fly concurrently) and a change to the channels, with a focus on differential scattering and polarization at eight frequencies ranging from 170 to 640 GHz. As part of the refurbishment, the instrument name has been changed to Configurable Scanning Sub-millimeter Instrument/Radiometer, retaining the same acronym, in order to emphasize the science objectives that are enabled by CoSSIR's unique scanning geometry and channel set. These anticipated capabilities include two-dimensional pro ling of clouds and precipitation and characterization of the geometry (shape and orientation) of large cloud and precipitating ice particles.

Radiometry↗
NASA NTRS1990

Pattern recognition of clouds and ice in polar regions

The study is based on AVHRR imagery and results from Landsat high-spatial-resolution scenes. Among the textual features investigated are the gray level difference vector (GLDV), and sum and difference histogram (SADH) approaches as well as gray level run length, spatial-coherence, and spectral-histogram measures. The traditional stepwise discriminant analysis and neural-network analysis are used for the identification of 20 Arctic surface and cloud classes. A principal-component analysis and hybrid architecture employing a modularized competitive learning layer are utilized. It is pointed out that the cloud-classification accuracy comparable to that of back-propagation could be achieved with a training time two orders of magnitude faster.

Welch, R. M.↗