SEARCH · Search NASA

Results for “cloud computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

Cloud-Based Time Series Analysis of Extremes: Use Cases and Applications

"Extreme weather events, such as hurricanes, tornadoes, floods, droughts, heatwaves, and blizzards, can cause widespread damage, disrupting ecosystems, agricultural production, and economies. The frequency and intensity of these events have been increasing, likely due to climate change, raising concerns and the need for more accurate analysis and predictions. NASA's Goddard Earth Sciences Data and Information Services Center (GES DISC) has migrated its long-term historical datasets, including precipitation data from MERRA-2 reanalysis, GLDAS land data assimilation, and IMERG satellite observations, to the cloud. This cloud-stored data enables scientists and researchers to utilize cloud computing for advanced modeling and forecasting of extreme weather events, eliminating the need to download large datasets. In this presentation, we will provide an overview of the cloud-based data and services managed by GES DISC; demonstrate methods for accessing and analyzing time series data stored in the cloud; and compare results across various datasets to address critical questions related to extreme precipitation. We will present use cases including: 1. Determining the average total precipitation in California during January and February from 2000 to 2024, and identifying anomalous precipitation in 2021. 2. Calculating the 10, 20, 50, and 100-year return periods for maximum daily rainfall based on 25 years of historical precipitation data (2000-2024) for Maryland.

time series↗

DOE OSTI2022

Development of an End State Vision to Implement Digital Monitoring in Nuclear Plants

Overview – Implementation of Digital Monitoring Wireless Sensor Techno-Economic Framework Online Monitoring Data Visualization Cloud Computing Services Summary and Challenges

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

NASA NTRS2018

Generalizing a Data Analysis Pipeline in the Cloud to Handle Diverse Use Cases in NASA's EOSDIS

NASA's Earth Observing System Data and Information System (EOSDIS) is tasked with archiving and distributing Earth Observation data across a range of disciplines, including atmospheric science, oceanography, land processes, natural hazards, solar radiance and even socioeconomic aspects relating to the environment. Driven by rapidly rising data volumes, EOSDIS is migrating to a cloud computing based archive over the next few years. Although this simplifies data management somewhat, the main aim is to provide the data in an environment where end users can bring their analysis to the data rather than attempting to download and manage ever-increasing volumes. To that end, a cloud-based analysis platform is being constructed to enable data transformations, analyses and visualization without egressing the data from the cloud. In this endeavor, we expect a wide variety of users, algorithms and use cases. Consequently, the architecture of this cloud analytics platform is expressly designed to be based on open services, thus fostering an ecosystem that enables the efficient combination of common components with data-specific or analysis-specific components. Reviewed and approved by Andrew Mitchell, ESDIS project manager.

Cloud computing↗

DOE OSTI2018

SpaRC: scalable sequence clustering using Apache Spark

MOTIVATION: Whole genome shotgun based next-generation transcriptomics and metagenomics studies often generate 100-1000 GB sequence data derived from tens of thousands of different genes or microbial species. Assembly of these data sets requires tradeoffs between scalability and accuracy. Current assembly methods optimized for scalability often sacrifice accuracy and vice versa. An ideal solution would both scale and produce optimal accuracy for individual genes or genomes. RESULTS: In this work we describe an Apache Spark-based scalable sequence clustering application, SparkReadClust (SpaRC), that partitions reads based on their molecule of origin to enable downstream assembly optimization. SpaRC produces high clustering performance on transcriptomes and metagenomes from both short and long read sequencing technologies. It reaches near-linear scalability with input data size and number of compute nodes. SpaRC can run on both cloud computing and HPC environments without modification while delivering similar performance. Our findings demonstrate that SpaRC provides a scalable solution for clustering billions of reads from next-generation sequencing experiments, and Apache Spark represents a cost-effective solution with rapid development/deployment cycles for similar large-scale sequence data analysis problems. AVAILABILITY AND IMPLEMENTATION: https://bitbucket.org/berkeleylab/jgi-sparc.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2020

Platform for efficient large-scale storage and analysis of multi-omics data in plant and microbial systems (Final Technical Report)

Genomic variation at the sequence level fundamentally affects the phenotypic state of all organisms at all stages of development, while dynamic processes such as changes in the epigenome (e.g. DNA methylation state) and transcriptome regulate the specific phenotype expressed at any given state of development based upon that genomic variation. In plants, DNA methylation is a particularly important mechanism for both regulating transcriptomic expression and for management of genomic variations that could be deleterious to the organism due to the presence of active retrotransposons in plant genomes. While DNA methylation is heritable, it is also dynamic through a given plant’s development and life cycle, particularly during the development from seed to mature specimen suggesting variations in DNA methylation could be critical regulators of biologically and commercially important phenotypes such as time to flowering; in addition, plant DNA methylation is more complex than that of animals, with methylation of CHG and CHH trinucleotides evident in addition to the better-known CG methylation. The complexity of plant DNA methylation and its interplay with genomic sequence variation, transcriptomics and other epigenomic factors demand a storage and analysis framework that can cope with the complexity both within a single specimen and with analyses that span many individuals and even many species, such as attempts to extend models from model organisms to commercially relevant species. In addition to complexity, the rapid development and proliferation of sequencing technology has led to an explosion of data volume that conventional storage and analysis solutions will likely be unable to cope with in the long run. We proposed to study these with suitable distributed storage and computation and therefore for the application of cloud computing to biological analyses; integrate with existing data sources and compatible with virtually any interface use case, from fully automated shell scripts to notebooks and do all these at scale in this STTR grant.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2026

Cloud-Based Demonstration of the Eastern Interconnection Situational Awareness Monitoring System (ESAMS)

This report describes a cloud-based implementation and field demonstration of the Eastern Interconnection Situational Awareness and Monitoring System (ESAMS). ESAMS was developed to support the detection and source localization of forced oscillations using synchrophasor measurements from tie-lines connecting areas served by different reliability coordinators (RCs), so that RCs could better coordinate their response to wide-area events. A previous effort had identified deployment barriers associated with hosting shared situational awareness tools at a single RC. To address these barriers, ESAMS was migrated to Amazon Web Services and evaluated in a six-month field demonstration. ISO New England (ISO-NE) and PJM streamed data to the platform using AWS Direct Connect and a site-to-site VPN, respectively. The resulting multi-utility measurement footprint enabled regional source localization across major portions of the U.S. Eastern Interconnection and supported routine identification of oscillation events. During the final three months of the trial, 24 events above 2 MW/MVAR were detected. The largest detected oscillation approached a 25 MW peak-to-peak amplitude, and the longest persisted intermittently for more than 11 hours. The demonstration also assessed operational considerations—including data transfer volumes, end-to-end latency, and cloud computing costs—and found that network and compute requirements were modest relative to typical cloud capabilities while providing performance comparable to prior on-premises deployments. Overall, the results indicate that cloud hosting can provide a practical path to shared interconnection-wide oscillation monitoring. The cloud ESAMS demonstration establishes a foundation for broader utility participation and for building future wide-area analytics that leverage measurements across organizational boundaries.

Follum, James D.↗

NASA NTRS2014

Software Simplifies the Sharing of Numerical Models

To ease the sharing of climate models with university students, Goddard Space Flight Center awarded SBIR funding to Reston, Virginia-based Parabon Computation Inc., a company that specializes in cloud computing. The firm developed a software program capable of running climate models over the Internet, and also created an online environment for people to collaborate on developing such models.

Source record↗

NASA NTRS2018

User Metrics in NASA Earth Science Data Systems

This presentation the collection and use of user metrics in NASA's Earth Science data systems. A variety of collection methods is discussed, with particular emphasis given to the American Customer Satisfaction Index (ASCI). User sentiment on potential use of cloud computing is presented, with generally positive responses. The presentation also discusses various forms of automatically collected metrics, including an example of the relative usage of different functions within the Giovanni analysis system.

user metrics↗

NASA NTRS2019

Analysis Ready Data in Analytics Optimized Data Stores for Analysis of Big Earth Data in the Cloud

Cloud computing offers the possibility of making the analysis of Big Data approachable for a wider community due to affordable access to computing power, an ecosystem of usable tools for parallel processing, and migration of many large datasets to archives in the cloud, allowing data-proximal computing. Generally, data analysis acceleration in the cloud comes from running multiple nodes in a split-combine-apply strategy. Data systems such as the Earth Observing System Data and Information System are in a position to "pre-split" the data by storing them in a data store that is optimized for data parallel computing, i.e., an Analytics-Optimized Data Store (AODS). A variety of approaches to AODS are possible, from highly scalable databases to scalable filesystems to data formats optimized for cloud access (e.g., zarr and cloud-optimized datasets), with the optimal choice dependent on both the types of analysis and the geospatial structure of the data. A key question is how much preprocessing of the data to do, both before splitting and as the first part of the apply step. Again, the geospatial structure of the data and the analysis type influence the decision, with the added complexity of the user type. Trans-disciplinary users who are not well-versed in the nuances of quality-filtering and georeferencing of remote sensing orbit/swath/scene data tend to ask for more highly processed data, relying on the data provider to make sensible decisions on preprocessing parameters. (This accounts for the popularity of "Level 3" gridded data, despite the lower spatial resolution it provides.) In this case, data can be preprocessed before the split, resulting in higher performance in the rest of the "apply" step, which can be transformative for use cases such as interactive data exploration at scale. Discipline researchers who are experienced with remote sensing data often prefer more flexibility in customizing the preprocessing data into Analysis Ready Data, resulting in more need for on-the-fly preprocessing.

Lynnes, Christopher↗

DOE OSTI2023

Measurement and applications: Exploring the challenges and opportunities of hierarchical federated learning in sensor applications

Sensor applications have become ubiquitous in modern society as the digital age continues to advance. AI-based techniques (e.g., machine learning) are effective at extracting actionable information from large amounts of data. An example would be an automated water irrigation system that uses AI-based techniques on soil quality data to decide how to best distribute water. However, these AI-based techniques are costly in terms of hardware resources, and Internet-of-Things (IoT) sensors are resource-constrained with respect to processing power, energy, and storage capacity. These limitations can compromise the security, performance, and reliability of sensor-driven applications. To address these concerns, cloud computing services can be used by sensor applications for data storage and processing. Unfortunately, cloud-based sensor applications that require real-time processing, such as medical applications (e.g., fall detection and stroke prediction), are vulnerable to issues such as network latency due to the sparse and unreliable networks between the sensor nodes and the cloud server [1]. As users approach the edge of the communications network, latency issues become more severe and frequent. A promising alternative is edge computing, which provides cloud-like capabilities at the edge of the network by pushing storage and processing capabilities from centralized nodes to edge devices that are closer to where the data are gathered, resulting in reduced network delays [2], [3].

Po-Leen Ooi, Melanie↗

DOE OSTI2024

The ATLAS Workflow Management System Evolution in the LHC Run3 and towards the High-Luminosity LHC era

The ATLAS experiment has 18+ years of experience using workload management systems to deploy and develop workflows to process and to simulate data on the distributed computing infrastructure. Simulation, processing and analysis of LHC experiment data require the coordinated work of heterogeneous computing resources. In particular, the ATLAS experiment utilizes the resources of 250 computing centers worldwide, the power of supercomputing centres, and national, academic and commercial cloud computing resources. In this contribution, we present new techniques for cost-effectively improving efficiency introduced in workflow management system software. The evolution from a mesh framework to new types of computing facilities such as cloud and HPCs is described, as well as new types of production and analysis workflows.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NASA NTRS2017

NASA Wrangler: Automated Cloud-Based Data Assembly in the RECOVER Wildfire Decision Support System

NASA Wrangler is a loosely-coupled, event driven, highly parallel data aggregation service designed to take advantageof the elastic resource capabilities of cloud computing. Wrangler automatically collects Earth observational data, climate model outputs, derived remote sensing data products, and historic biophysical data for pre-, active-, and post-wildfire decision making. It is a core service of the RECOVER decision support system, which is providing rapid-response GIS analytic capabilities to state and local government agencies. Wrangler reduces to minutes the time needed to assemble and deliver crucial wildfire-related data.

decision support↗

DOE OSTI2018

Simplifying Energy Efficiency for Homeowners - Results from a California Utility's Pilot Study of a Scalable Solution to Assess Retrofit and Solar Opportunities: Preprint

For homeowners, typical choices to reduce net energy use in their homes include: upgrading their home with a tailored efficiency retrofit or investing in roof-top solar. Determining the unique combination of these options that will provide the most cost-effective solution requires analysis for each home. This analysis needs to be presented in a simple and actionable way to homeowners. The solution should also be easily scalable to thousands of homes to enable utilities to better target efficiency program options and support solar installations. Sacramento Municipal Utility District (SMUD) piloted a new tool developed by Vistar Energy called XeroHome that automates the assessment of energy efficiency and solar opportunities in existing homes. XeroHome demonstrates an approach to using building energy modeling, energy use data analytics and homeowner engagement to identify and target energy efficiency opportunities at scale. XeroHome uses automation to build an EnergyPlus model for each home, based on information from public data sources such as property assessment records, building permit records, aerial/satellite photography, and template models derived from ResStock. Utilizing cloud computing, machine learning and optimization algorithms, energy upgrade packages are ranked by cost-effectiveness for each home and presented over an interactive web-portal. This paper discusses SMUD's pilot project using XeroHome to conduct analysis for 1,500 existing homes in Sacramento, CA. Development of XeroHome's advanced analytics capabilities are discussed in the context of applicability for utility programs.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2018

Simplifying Energy Efficiency for Homeowners - Results from a California Utility's Pilot Study of a Scalable Solution to Assess Retrofit and Solar Opportunities

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NASA NTRS2014

RECOVER: An Automated Cloud-Based Decision Support System for Post-fire Rehabilitation Planning

RECOVER is a site-specific decision support system that automatically brings together in a single analysis environment the information necessary for post-fire rehabilitation decision-making. After a major wildfire, law requires that the federal land management agencies certify a comprehensive plan for public safety, burned area stabilization, resource protection, and site recovery. These burned area emergency response (BAER) plans are a crucial part of our national response to wildfire disasters and depend heavily on data acquired from a variety of sources. Final plans are due within 21 days of control of a major wildfire and become the guiding document for managing the activities and budgets for all subsequent remediation efforts. There are few instances in the federal government where plans of such wide-ranging scope and importance are assembled on such short notice and translated into action more quickly. RECOVER has been designed in close collaboration with our agency partners and directly addresses their high-priority decision-making requirements. In response to a fire detection event, RECOVER uses the rapid resource allocation capabilities of cloud computing to automatically collect Earth observational data, derived decision products, and historic biophysical data so that when the fire is contained, BAER teams will have a complete and ready-to-use RECOVER dataset and GIS analysis environment customized for the target wildfire. Initial studies suggest that RECOVER can transform this information-intensive process by reducing from days to a matter of minutes the time required to assemble and deliver crucial wildfire-related data.

cloud computing↗

DOE OSTI2024

Cybersecurity Considerations for Emerging Energy Technologies

AI, cloud computing, post-quantum cryptography, zero-trust architectures, microgrids, and virtual power plants. What do these things have in common? They are all emerging technologies in the clean energy space that will be a critical part of grid modernization efforts. As we work towards clean energy and decarbonization targets, these technologies, developed to solve real-world problems, will help us reach goals and achieve new efficiencies as the paradigm of grid operation shifts. However, there are growing concerns about the cybersecurity risks associated with these trending topics as they are used in critical infrastructure applications. This talk will cover gaps, challenges, and opportunities for the secure implementation of grid modernization solutions and novel energy applications of state-of-the-art networking and communications. Proactive risk mitigation strategies, including the application of cyber-informed engineering, will be discussed. Practical applications of these techniques will help provide countermeasures to the impact of cyberattacks on critical infrastructure technologies in a new, digitized grid landscape.

14 SOLAR ENERGY↗

NASA NTRS1990

ER-2 lidar measurements of stratocumulus cloud top structure on July 14, 1987

On July 14, 1987 NASA's ER-2 high altitude aircraft flew a mission to measure the structure of stratocumulus clouds off the coast of California. A flight pattern was executed so that the two-dimensional variability of the clouds could be detected. The technique of analysis of the lidar data to measure cloud tops follows. First each signal is searched for its maximum in return strength. This maximum is caused by scattering of the laser light off cloud particles or from the ocean surface. Next the variance of the signal return above the level of maximum backscatter is determined. Cloud top is assigned to a level (above the level of maximum backscatter) where the backscatter exceeds the average variance. This two-step process is necessary because the level of maximum backscatter does not correspond to the cloud top. Ocean surface returns are easily separated from cloud returns in this process, described in detail by Boers, Spinhirne, and Hart (1988). Analysis of the data so far has shown that there were very few breaks in the clouds. Furthermore the layer top was very flat with local oscillations not exceeding 30 m. Such small cloud top variations are still well within the range of detectability, because the precision of this technique of cloud top detection has previously been established to be 13 to 15 m. Data are presently being analyzed to compute cloud top distributions and fractional cloudiness. The aim of this research is to relate the fractional cloudiness to the mean thermodynamic structure of the boundary layer. Researchers plan to compute spectral scales of the cloud top variability in two dimensions to determine the orientation of the clouds with respect to the mean wind. Furthermore the lidar derived cloud top distribution will be used in the computation of the thermodynamic and radiation budget of the boundary layer.

Boers, Reinout↗

DOE OSTI2020

Softwarized Federations of Science Instruments with Edge-Continuum Containers

Significant expansion of capabilities of DOE science complex of supercomputers, instruments and networks is expected as powerful experimental facilities, exascale computers and terabit networks are added. Combined with the advances in edge and cloud computing technologies, DOE science users now have the promise of unprecedented execution of complex, continuum workflows, namely, small and latency-sensitive computations at the edge and site, massive computations at remote HPC systems, and everything in between on the cloud. But, bringing this capability to the science user requires overcoming the overwhelming complexity of forming the federations of systems, and efficiently and effectively orchestrating the workflows while ensuring high utilization of the expensive facilities. Current manual configuration of the federated systems simply will not scale, since the coordination across sites may take weeks to months, often leading to under-utilized and hard-to-diagnose compositions. A powerful, composable software stack will be developed to (i) wrap the systems so that federations can be composed fast in software, and (ii) containerize computations to be orchestrated across the edge, site, cloud and HPC resources.

Rao, Nageswara S.↗