NASA's Implementation of Cloud Services for Human Space Flight
Explore the source record for details and available documents.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
This presentation will focus on the new paradigm the Golden age of Earth observations (e.g. high temporal, spatial and spectral resolution Earth observations) represent for natural resources management, and ultimately societal benefit. And what to expect for the upcoming satellite Missions that NASA is planning to launch, such as NISAR and SBG.
Explore the source record for details and available documents.
The integration of cloud services and capabilities across government and commercial sectors has continued to increase over the last several years. Cloud architectures provide on-demand process, storage, and transfer of data as a service without the need to purchase expensive hardware, infrastructure, and their associated maintenance costs. Most cloud environments charge the user for only the resources they use, making cloud implementations highly scalable and easily adjustable to the user’s changing demands.
NASA has a vast archive of engineering guidelines, standards, and best practices collected over decades. This encompasses a breadth of topics from rocketry and engineering standards to risk management and space-related health issues. This wealth of information, while invaluable to NASA engineers, staff, and the public, is too extensive for any individual to fully comprehend. To address this challenge, we have developed Atlas, a tool within NASA's Mission Cloud Platform that enables users to query these diverse sources effectively. Atlas allows users to ask natural language questions and receive answers grounded in factual information from source documents. The tool provides responses with direct quotations and links to original documents, ensuring transparency and accuracy. It can address a wide range of queries, from specific technical details like safe distances for rocket launches from lightning to broader topics such as crew health requirements for long-duration space missions, corrosion protection in low Earth orbit, and NASA's agreements with various entities. In developing Atlas, we encountered and overcame several technical challenges. Large Language Models often struggle with consistently providing accurate information, especially for highly specialized topics. We implemented strategies to prevent hallucinations and ensure the reliability of responses, even for complex questions on topics ranging from NASA Mission Classes to intricate rocket science concepts. Additionally, we addressed the challenges of delivering quick responses while maintaining cost-effectiveness. Our presentation will detail the innovative approaches we employed to optimize performance and efficiency, making Atlas a powerful and practical tool for accessing NASA's extensive knowledge base.
The Modern-Era Retrospective analysis for Research and Applications, Version (MERRA-2) dataset is now available in the AWS West Region-2 cloud. The free data downloading policy remains unchanged. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) offers cloud services such as OPeNDP subsetting, and time series services. This presentation will showcase use cases that testing the procedures and find the most efficiency workflows to access and process the data for finding climate extremes. The use cases include a) accessing and processing data directly in the cloud S3 bucket without downloading it, and b) downloading subset data to process on the user’s local system.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
This is a slide deck developed to present to a potential end-user interested in our tool. This is intended to be a presentation resource in lieu of doing a live-demonstration.
The Clouds kernel design was through several design phases and is nearly complete. The object manager, the process manager, the storage manager, the communications manager, and the actions manager are examined.
The Clouds project is well underway to its goal of building a unified distributed operating system supporting the object model. The operating system design uses the object concept of structuring software at all levels of the system. The basic operating system was developed and work is under progress to build a usable system.
Historically, cloud structures, dynamics, and precipitation processes have been observed and measured with sensors from two different spatial resolutions. Laser-based sensors have volume resolutions on the order of 10(exp -4) to 10(exp -2) cubic meters for a 1-s sample. Radar systems operating at wavelengths between 1 and 10 cm have resolutions on the order of 10(exp 4) to 10(exp 7) cubic meters. The resolutions of micro-wave systems depend primarily on the system RF bandwidth and antenna bandwidth. Both resolution regimes were useful in the study of cloud structures and processes - the former for determining resolution on the individual cloud particle scale and the latter for studying the coarse characteristics of cloud dynamics and structure. There are, however, cloud processes and structures that occur on scales that lie between these two regimes: the process of entrainment, where outside air is brought within the cloud boundaries; the mixing of in-cloud particles with different histories; cloud particle coalescence; and ice formation. The use of mm-wave (30- to 300-GHz) radars offers an opportunity to observe cloud processes at these scales and determine their influence on precipitation development, cloud albedos, cloud lifetimes, chemical cycling of tract substances, aircraft icing, and other meteorological phenomena. To determine the usefulness of a 35-GHz radar for observing these precipitation and cloud processes, a research program was initiated. The objectives of this program are to develop a 35-GHz radar, to measure scattering from precipitation and clouds, and to develop a model to compute scattering from clouds using the finite-difference time-domain (FDTD) technique.
NASA’s High-End Computing Capability (HECC) Project is periodically asked if it could be more cost effective through the use of commercial cloud resources. To answer the question, HECC’s Application Performance and Productivity (APP) team undertook a performance and cost evaluation comparing three domains: two commercial cloud providers, Amazon and Penguin, and HECC’s in-house resources—the Pleiades and Electra systems. In the study, the APP team used a combination of the NAS Parallel Benchmarks (NPB) and six full applications from NASA’s workload on Pleiades and Electra to compare performance of nodes based on three different generations of Intel Xeon processors—Haswell, Broadwell, and Skylake. Because of export control limitations, the most heavily used applications on Pleiades and Electra could not be used in the cloud; therefore, only one of the applications, OpenFOAM, represents work from the Aeronautics Research Mission Directorate and the Human and Exploration Mission Directorate. The other five applications are from the Science Mission Directorate.
Equilibrium temperature distribution, thermal emission, and limb function computed for Venus cloud cover model
Over the past year few years, an international collaboration has developed a pilot project under the auspices of Committee on Earth Observation Satellite (CEOS) Disasters team. The overall team consists of civilian satellite agencies. For this pilot effort, the development team consists of NASA, Canadian Space Agency, Univ. of Maryland, Univ. of Colorado, Univ. of Oklahoma, Ukraine Space Research Institute and Joint Research Center(JRC) for European Commission. This development team collaborates with regional , national and international agencies to deliver end-to-end disaster coverage. In particular, the team in collaborating on this effort with the Namibia Department of Hydrology to begin in Namibia . However, the ultimate goal is to expand the functionality to provide early warning over the South Africa region. The initial collaboration was initiated by United Nations Office of Outer Space Affairs and CEOS Working Group for Information Systems and Services (WGISS). The initial driver was to demonstrate international interoperability using various space agency sensors and models along with regional in-situ ground sensors. In 2010, the team created a preliminary semi-manual system to demonstrate moving and combining key data streams and delivering the data to the Namibia Department of Hydrology during their flood season which typically is January through April. In this pilot, a variety of moderate resolution and high resolution satellite flood imagery was rapidly delivered and used in conjunction with flood predictive models in Namibia. This was collected in conjunction with ground measurements and was used to examine how to create a customized flood early warning system. During the first year, the team made use of SensorWeb technology to gather various sensor data which was used to monitor flood waves traveling down basins originating in Angola, but eventually flooding villages in Namibia. The team made use of standardized interfaces such as those articulated under the Open Cloud Consortium (OGC) Sensor Web Enablement (SWE) set of web services was good [1][2]. However, it was discovered that in order to make a system like this functional, there were many performance issues. Data sets were large and located in a variety of location behind firewalls and had to be accessed across open networks, so security was an issue. Furthermore, the network access acted as bottleneck to transfer map products to where they are needed. Finally, during disasters, many users and computer processes act in parallel and thus it was very easy to overload the single string of computers stitched together in a virtual system that was initially developed. To address some of these performance issues, the team partnered with the Open Cloud Consortium (OCC) who supplied a Computation Cloud located at the University of Illinois at Chicago and some manpower to administer this Cloud. The Flood SensorWeb [3] system was interfaced to the Cloud to provide a high performance user interface and product development engine. Figure 1 shows the functional diagram of the Flood SensorWeb. Figure 2 shows some of the functionality of the Computation Cloud that was integrated. A significant portion of the original system was ported to the Cloud and during the past year, technical issues were resolved which included web access to the Cloud, security over the open Internet, beginning experiments on how to handle surge capacity by using the virtual machines in the cloud in parallel, using tiling techniques to render large data sets as layers on map, interfaces to allow user to customize the data processing/product chain and other performance enhancing techniques. The conclusion reached from the effort and this presentation is that defining the interoperability standards in a small fraction of the work. For example, once open web service standards were defined, many users could not make use of the standards due to security restrictions. Furthermore, once an interoperable sysm is functional, then a surge of users can render a system unusable, especially in the disaster domain.
This presentation coincides with AWS's announcement of their new Parallel Computing Service (PCS) which allows for easy creation of HPC-style clusters in their AWS cloud computing platform. I helped them beta test this service before it was made generally available in August. AWS asked if we would be interested in discussing our experience with the PCS service, and our experience with HPC workloads in the cloud in general, so this slideshow discusses a brief history of scientific computing at NREL and shares a bit of our experiences and approach to utilizing cloud services for HPC-style workloads.
Practical methods of computer screening cloud-contaminated pixels from data of various satellite systems are proposed. Examples are given of the location of clouds and representative landscape features in HCMM spectral space of reflectance (VIS) vs emission (IR). Methods of screening out cloud affected HCMM are discussed. The character of subvisible absorbing-emitting atmospheric layers (subvisible cirrus or SCi) in HCMM data is considered and radiosonde soundings are examined in relation to the presence of SCi. The statistical characteristics of multispectral meteorological satellite data in clear and SCi affected areas are discussed. Examples in TIROS-N and NOAA-7 data from several states and Mexico are presented. The VIS-IR cluster screening method for removing clouds is applied to a 262, 144 pixel HCMM scene from south Texas and northeast Mexico. The SCi that remain after cluster screening are sited out by applying a statistically determined IR limit.