Search NASA⌕ Search

SEARCH · Search NASA

Results for “Cloud Computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
NASA NTRS2001

A 3-Year Climatology of Cloud and Radiative Properties Derived from GOES-8 Data Over the Southern Great Plains

While the various instruments maintained at the Atmospheric Radiation Measurement (ARM) Program Southern Great Plains (SGP) Central Facility (CF) provide detailed cloud and radiation measurements for a small area, satellite cloud property retrievals provide a means of examining the large-scale properties of the surrounding region over an extended period of time. Seasonal and inter-annual climatological trends can be analyzed with such a dataset. For this purpose, monthly datasets of cloud and radiative properties from December 1996 through November 1999 over the SGP region have been derived using the layered bispectral threshold method (LBTM). The properties derived include cloud optical depths (ODs), temperatures and albedos, and are produced on two grids of lower (0.5 deg) and higher resolution (0.3 deg) centered on the ARM SGP CF. The extensive time period and high-resolution of the inner grid of this dataset allows for comparison with the suite of instruments located at the ARM CF. In particular, Whole-Sky Imager (WSI) and the Active Remote Sensing of Clouds (ARSCL) cloud products can be compared to the cloud amounts and heights of the LBTM 0.3 deg grid box encompassing the CF site. The WSI provides cloud fraction and the ARSCL computes cloud fraction, base, and top heights using the algorithms by Clothiaux et al. (2001) with a combination of Belfort Laser Ceilometer (BLC), Millimeter Wave Cloud Radar (MMCR), and Micropulse Lidar (MPL) data. This paper summarizes the results of the LBTM analysis for 3 years of GOES-8 data over the SGP and examines the differences between surface and satellite-based estimates of cloud fraction.

Khaiyer, M. M.↗
NASA NTRSDate not supplied

A Case Study on the Challenges and Opportunities for the Deployment of PHM Capabilities in Existing Engineering Systems

The field of Prognostics and Health Management (PHM) of engineering systems has experienced considerable growth over the last decade. From benefits associated with faster and more powerful hardware in the form of wireless sensors, edge devices, and general computing capabilities (GPU’s and cloud computing), to development of powerful algorithms for anomaly detection and remaining useful life (RUL) estimation, the number of engineering systems featuring advanced diagnostics and prognostics capabilities continues to grow at an increasingly faster pace. However, the deployment of PHM capabilities as part of the upgrade of existing engineering systems presents multiple challenges to the PHM practitioner charged with retrofitting such systems. Issues include a lack of specific instrumentation needed to capture the signals of interest; insufficient data and sampling rates required for fault detection and diagnosis, and for detection of failure/degradation indicators; and difficulties in the identification of a system’s nominal behavior as a result of age induced degradation. Today’s PHM practitioner must be able to quickly identify and assess these types of issues to effectively evaluate and select the optimal PHM strategies required to achieve the desired results. This paper presents results from the preliminary evaluation of the High-Pressure Gas Facility (HPGF) infrastructure at NASA’s Stennis Space Center in Hancock County, Mississippi. This evaluation is part of a feasibility study conducted prior to the deployment of prognostics and diagnostics capabilities in the pumps skids of the liquid nitrogen (LN2) system of the HPGF.

Condition Based Maintenance↗
DOE OSTI2023

Scaling Optimal Allocation of Cloud Resources Using Lagrange Relaxation

The rapid growth of Cloud Computing (CC) has increased the variety of computing resources, storage, and communication services that pose significant new challenges for the efficient use of cloud resources. The cost-efficient allocation of cloud resources has become a decisive premise for the adoption of CC services. The cost-efficient selection and scheduling of these resources to meet the demands of a scientific workflow is a challenging problem that is exacerbated by the inclusion of multiple CC providers. In this paper, we present a novel strategy for the cost-efficient selection of CC resources using Lagrange relaxation. Our approach is based on preselection of resources and demand decomposition to create a series of smaller sub-problems, which allow the estimation of the best cost-structures and selection of CC service providers for a subset of the time period of the planning horizon. Decomposition of the demand is achieved through the boundary analysis of a continuous relaxation of the problem. Using the metrics defined in terms of the cost and time of completion, we demonstrate excellent performance in relation to optimal solutions. Our method reduced the computational time from hours to seconds for a representative 36-month problem and provided high-quality solutions (< 0.05% relative error). Given the importance of selecting resources and scheduling complex scientific workflows, we believe that this novel strategy will be beneficial for many researchers and users of cloud computing resources.

De La Torre, Luis↗
DOE OSTI2018

A comparison of Amazon Web Services and Microsoft Azure cloud platforms for high performance computing

Advances in commercial cloud computing necessitate continual evaluation of the cloud's performance on a variety of applications. This work looks at compute oriented instances from Amazon Web Services and Microsoft Azure cloud platforms and evaluates them with several high-performance computing benchmarks, including HPCC and HPCG. These benchmarks illustrate that the most cost competitive solution depends on the application to be run.

Kotas, Charlotte↗
DOE OSTI2018

Job Management with mpi_jm

About this book: This book constitutes the refereed post-conference proceedings of 13 workshops held at the 33rd International ISC High Performance 2018 Conference, in Frankfurt, Germany, in June 2018: HPC I/O in the Data Center, HPC-IODC 2018; Workshop on Performance and Scalability of Storage Systems, WOPSSS 2018; 13th Workshop on Virtualization in High­-Performance Cloud Computing, VHPC 2018; Third International Workshop on In Situ Visualization, WOIV 2018; 4th International Workshop on Communication Architectures for HPC, Big Data, Deep Learning and Clouds at Extreme Scale, ExaComm 2018; International Workshop on OpenPOWER for HPC, IWOPH 2018; IXPUG Workshop: Many-Core Computing on Intel Processors; Workshop on Sustainable Ultrascale Computing Systems; Approximate and Transprecision Computing on Emerging Technologies, ATCET 2018; First Workshop on the Convergence of Large-Scale Simulation and Artificial Intelligence; Third Workshop for Open Source Supercomputing, OpenSuCo 2018; First Workshop on Interactive High-Performance Computing; Workshop on Performance Portable Programming Models for Accelerators, P^3MA 2018. The 53 full papers included in this volume were carefully reviewed and selected from 80 submissions. They cover all aspects of research, development, and application of large-scale, high performance experimental and commercial systems. Topics include HPC computer architecture and hardware; programming models, system software, and applications; solutions for heterogeneity, reliability, power efficiency of systems; virtualization and containerized environments; big data and cloud computing; and artificial intelligence.

Berkowitz, Evan↗
DOE OSTI2023

PhytoOracle: Scalable, modular phenomics data processing pipelines

As phenomics data volume and dimensionality increase due to advancements in sensor technology, there is an urgent need to develop and implement scalable data processing pipelines. Current phenomics data processing pipelines lack modularity, extensibility, and processing distribution across sensor modalities and phenotyping platforms. To address these challenges, we developed PhytoOracle (PO), a suite of modular, scalable pipelines for processing large volumes of field phenomics RGB, thermal, PSII chlorophyll fluorescence 2D images, and 3D point clouds. PhytoOracle aims to ( i ) improve data processing efficiency; ( ii ) provide an extensible, reproducible computing framework; and ( iii ) enable data fusion of multi-modal phenomics data. PhytoOracle integrates open-source distributed computing frameworks for parallel processing on high-performance computing, cloud, and local computing environments. Each pipeline component is available as a standalone container, providing transferability, extensibility, and reproducibility. The PO pipeline extracts and associates individual plant traits across sensor modalities and collection time points, representing a unique multi-system approach to addressing the genotype-phenotype gap. To date, PO supports lettuce and sorghum phenotypic trait extraction, with a goal of widening the range of supported species in the future. At the maximum number of cores tested in this study (1,024 cores), PO processing times were: 235 minutes for 9,270 RGB images (140.7 GB), 235 minutes for 9,270 thermal images (5.4 GB), and 13 minutes for 39,678 PSII images (86.2 GB). These processing times represent end-to-end processing, from raw data to fully processed numerical phenotypic trait data. Repeatability values of 0.39-0.95 (bounding area), 0.81-0.95 (axis-aligned bounding volume), 0.79-0.94 (oriented bounding volume), 0.83-0.95 (plant height), and 0.81-0.95 (number of points) were observed in Field Scanalyzer data. We also show the ability of PO to process drone data with a repeatability of 0.55-0.95 (bounding area).

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Two-dimensional materials for bio-realistic neuronal computing networks

Two-dimensional (2D) van der Waals materials have found broad utility in a diverse range of applications including electronics, optoelectronics, renewable energy, and quantum information technologies. Meanwhile, exponentially growing digital data coupled with the ubiquity of artificial intelligence algorithms have generated significant interest in edge neuromorphic computing as an alternative to centralized cloud computing. The drive to incorporate neuroscience principles into computing hardware is motivated by the low power consumption, parallel processing, and reconfigurability of the human brain. The diverse library of 2D materials with atomic-level thicknesses, exceptional electrostatic tunability, and integration versatility is particularly well-suited for realizing bio-realistic synaptic and neuronal functionality. Here, we summarize past and present work in this field and outline the frontier challenges that have not yet been overcome. Here we also delineate potential solutions and suggest that the neuroscience principles of criticality and synchrony have the potential to inspire breakthrough applications of 2D materials in neuronal computing networks.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Quantum Computing in the Cloud: Analyzing job and machine characteristics

As the popularity of quantum computing continues to grow, quantum machine access over the cloud is critical to both academic and industry researchers across the globe. And as cloud quantum computing demands increase exponentially, the analysis of resource consumption and execution characteristics are key to efficient management of jobs and resources at both the vendor-end as well as the client-end. While the analysis of resource consumption and management are popular in the classical HPC domain, it is severely lacking for more nascent technology like quantum computing. This paper is a first-of-its-kind academic study, analyzing various trends in job execution and resources consumption / utilization on quantum cloud systems. We focus on IBM Furthermore, quantum systems and analyze characteristics over a two year period, encompassing over 6000 jobs which contain over 600,000 quantum circuit executions and correspond to almost 10 billion “shots” or trials over 20+ quantum machines. Specifically, we analyze trends focused on, but not limited to, execution times on quantum machines, queuing/waiting times in the cloud, circuit compilation times, machine utilization, as well as the impact of job and machine characteristics on all of these trends. Furthermore, our analysis identifies several similarities and differences with classical HPC cloud systems. Based on our insights, we make recommendations and contributions to improve the management of resources and jobs on future quantum cloud systems.

42 ENGINEERING↗
NASA NTRS1981

Automatic cloud tracking applied to GOES and Meteosat observations

An improved automatic processing method for the tracking of cloud motions as revealed by satellite imagery is presented and applications of the method to GOES observations of Hurricane Eloise and Meteosat water vapor and infrared data are presented. The method is shown to involve steps of picture smoothing, target selection and the calculation of cloud motion vectors by the matching of a group at a given time with its best likeness at a later time, or by a cross-correlation computation. Cloud motion computations can be made in as many as four separate layers simultaneously. For data of 4 and 8 km resolution in the eye of Hurricane Eloise, the automatic system is found to provide results comparable in accuracy and coverage to those obtained by NASA analysts using the Atmospheric and Oceanographic Information Processing System, with results obtained by the pattern recognition and cross correlation computations differing by only fractions of a pixel. For Meteosat water vapor data from the tropics and midlatitudes, the automatic motion computations are found to be reliable only in areas where the water vapor fields contained small-scale structure, although excellent results are obtained using Meteosat IR data in the same regions. The automatic method thus appears to be competitive in accuracy and coverage with motion determination by human analysts.

Endlich, R. M.↗
NASA NTRSDate not supplied

Cloud-Based Time Series Analysis of Extremes: Use Cases and Applications

"Extreme weather events, such as hurricanes, tornadoes, floods, droughts, heatwaves, and blizzards, can cause widespread damage, disrupting ecosystems, agricultural production, and economies. The frequency and intensity of these events have been increasing, likely due to climate change, raising concerns and the need for more accurate analysis and predictions. NASA's Goddard Earth Sciences Data and Information Services Center (GES DISC) has migrated its long-term historical datasets, including precipitation data from MERRA-2 reanalysis, GLDAS land data assimilation, and IMERG satellite observations, to the cloud. This cloud-stored data enables scientists and researchers to utilize cloud computing for advanced modeling and forecasting of extreme weather events, eliminating the need to download large datasets. In this presentation, we will provide an overview of the cloud-based data and services managed by GES DISC; demonstrate methods for accessing and analyzing time series data stored in the cloud; and compare results across various datasets to address critical questions related to extreme precipitation. We will present use cases including: 1. Determining the average total precipitation in California during January and February from 2000 to 2024, and identifying anomalous precipitation in 2021. 2. Calculating the 10, 20, 50, and 100-year return periods for maximum daily rainfall based on 25 years of historical precipitation data (2000-2024) for Maryland.

time series↗
NASA NTRS2018

Generalizing a Data Analysis Pipeline in the Cloud to Handle Diverse Use Cases in NASA's EOSDIS

NASA's Earth Observing System Data and Information System (EOSDIS) is tasked with archiving and distributing Earth Observation data across a range of disciplines, including atmospheric science, oceanography, land processes, natural hazards, solar radiance and even socioeconomic aspects relating to the environment. Driven by rapidly rising data volumes, EOSDIS is migrating to a cloud computing based archive over the next few years. Although this simplifies data management somewhat, the main aim is to provide the data in an environment where end users can bring their analysis to the data rather than attempting to download and manage ever-increasing volumes. To that end, a cloud-based analysis platform is being constructed to enable data transformations, analyses and visualization without egressing the data from the cloud. In this endeavor, we expect a wide variety of users, algorithms and use cases. Consequently, the architecture of this cloud analytics platform is expressly designed to be based on open services, thus fostering an ecosystem that enables the efficient combination of common components with data-specific or analysis-specific components. Reviewed and approved by Andrew Mitchell, ESDIS project manager.

Cloud computing↗
DOE OSTI2018

SpaRC: scalable sequence clustering using Apache Spark

MOTIVATION: Whole genome shotgun based next-generation transcriptomics and metagenomics studies often generate 100-1000 GB sequence data derived from tens of thousands of different genes or microbial species. Assembly of these data sets requires tradeoffs between scalability and accuracy. Current assembly methods optimized for scalability often sacrifice accuracy and vice versa. An ideal solution would both scale and produce optimal accuracy for individual genes or genomes. RESULTS: In this work we describe an Apache Spark-based scalable sequence clustering application, SparkReadClust (SpaRC), that partitions reads based on their molecule of origin to enable downstream assembly optimization. SpaRC produces high clustering performance on transcriptomes and metagenomes from both short and long read sequencing technologies. It reaches near-linear scalability with input data size and number of compute nodes. SpaRC can run on both cloud computing and HPC environments without modification while delivering similar performance. Our findings demonstrate that SpaRC provides a scalable solution for clustering billions of reads from next-generation sequencing experiments, and Apache Spark represents a cost-effective solution with rapid development/deployment cycles for similar large-scale sequence data analysis problems. AVAILABILITY AND IMPLEMENTATION: https://bitbucket.org/berkeleylab/jgi-sparc.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Platform for efficient large-scale storage and analysis of multi-omics data in plant and microbial systems (Final Technical Report)

Genomic variation at the sequence level fundamentally affects the phenotypic state of all organisms at all stages of development, while dynamic processes such as changes in the epigenome (e.g. DNA methylation state) and transcriptome regulate the specific phenotype expressed at any given state of development based upon that genomic variation. In plants, DNA methylation is a particularly important mechanism for both regulating transcriptomic expression and for management of genomic variations that could be deleterious to the organism due to the presence of active retrotransposons in plant genomes. While DNA methylation is heritable, it is also dynamic through a given plant’s development and life cycle, particularly during the development from seed to mature specimen suggesting variations in DNA methylation could be critical regulators of biologically and commercially important phenotypes such as time to flowering; in addition, plant DNA methylation is more complex than that of animals, with methylation of CHG and CHH trinucleotides evident in addition to the better-known CG methylation. The complexity of plant DNA methylation and its interplay with genomic sequence variation, transcriptomics and other epigenomic factors demand a storage and analysis framework that can cope with the complexity both within a single specimen and with analyses that span many individuals and even many species, such as attempts to extend models from model organisms to commercially relevant species. In addition to complexity, the rapid development and proliferation of sequencing technology has led to an explosion of data volume that conventional storage and analysis solutions will likely be unable to cope with in the long run. We proposed to study these with suitable distributed storage and computation and therefore for the application of cloud computing to biological analyses; integrate with existing data sources and compatible with virtually any interface use case, from fully automated shell scripts to notebooks and do all these at scale in this STTR grant.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

Cloud-Based Demonstration of the Eastern Interconnection Situational Awareness Monitoring System (ESAMS)

This report describes a cloud-based implementation and field demonstration of the Eastern Interconnection Situational Awareness and Monitoring System (ESAMS). ESAMS was developed to support the detection and source localization of forced oscillations using synchrophasor measurements from tie-lines connecting areas served by different reliability coordinators (RCs), so that RCs could better coordinate their response to wide-area events. A previous effort had identified deployment barriers associated with hosting shared situational awareness tools at a single RC. To address these barriers, ESAMS was migrated to Amazon Web Services and evaluated in a six-month field demonstration. ISO New England (ISO-NE) and PJM streamed data to the platform using AWS Direct Connect and a site-to-site VPN, respectively. The resulting multi-utility measurement footprint enabled regional source localization across major portions of the U.S. Eastern Interconnection and supported routine identification of oscillation events. During the final three months of the trial, 24 events above 2 MW/MVAR were detected. The largest detected oscillation approached a 25 MW peak-to-peak amplitude, and the longest persisted intermittently for more than 11 hours. The demonstration also assessed operational considerations—including data transfer volumes, end-to-end latency, and cloud computing costs—and found that network and compute requirements were modest relative to typical cloud capabilities while providing performance comparable to prior on-premises deployments. Overall, the results indicate that cloud hosting can provide a practical path to shared interconnection-wide oscillation monitoring. The cloud ESAMS demonstration establishes a foundation for broader utility participation and for building future wide-area analytics that leverage measurements across organizational boundaries.

Follum, James D.↗
NASA NTRS2014

Software Simplifies the Sharing of Numerical Models

To ease the sharing of climate models with university students, Goddard Space Flight Center awarded SBIR funding to Reston, Virginia-based Parabon Computation Inc., a company that specializes in cloud computing. The firm developed a software program capable of running climate models over the Internet, and also created an online environment for people to collaborate on developing such models.

Source record↗
NASA NTRS2018

User Metrics in NASA Earth Science Data Systems

This presentation the collection and use of user metrics in NASA's Earth Science data systems. A variety of collection methods is discussed, with particular emphasis given to the American Customer Satisfaction Index (ASCI). User sentiment on potential use of cloud computing is presented, with generally positive responses. The presentation also discusses various forms of automatically collected metrics, including an example of the relative usage of different functions within the Giovanni analysis system.

user metrics↗
NASA NTRS2019

Analysis Ready Data in Analytics Optimized Data Stores for Analysis of Big Earth Data in the Cloud

Cloud computing offers the possibility of making the analysis of Big Data approachable for a wider community due to affordable access to computing power, an ecosystem of usable tools for parallel processing, and migration of many large datasets to archives in the cloud, allowing data-proximal computing. Generally, data analysis acceleration in the cloud comes from running multiple nodes in a split-combine-apply strategy. Data systems such as the Earth Observing System Data and Information System are in a position to "pre-split" the data by storing them in a data store that is optimized for data parallel computing, i.e., an Analytics-Optimized Data Store (AODS). A variety of approaches to AODS are possible, from highly scalable databases to scalable filesystems to data formats optimized for cloud access (e.g., zarr and cloud-optimized datasets), with the optimal choice dependent on both the types of analysis and the geospatial structure of the data. A key question is how much preprocessing of the data to do, both before splitting and as the first part of the apply step. Again, the geospatial structure of the data and the analysis type influence the decision, with the added complexity of the user type. Trans-disciplinary users who are not well-versed in the nuances of quality-filtering and georeferencing of remote sensing orbit/swath/scene data tend to ask for more highly processed data, relying on the data provider to make sensible decisions on preprocessing parameters. (This accounts for the popularity of "Level 3" gridded data, despite the lower spatial resolution it provides.) In this case, data can be preprocessed before the split, resulting in higher performance in the rest of the "apply" step, which can be transformative for use cases such as interactive data exploration at scale. Discipline researchers who are experienced with remote sensing data often prefer more flexibility in customizing the preprocessing data into Analysis Ready Data, resulting in more need for on-the-fly preprocessing.

Lynnes, Christopher↗