Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud computing, data cloud”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
NASA NTRS1982

Characteristics of middle and upper tropospheric clouds as deduced from rawinsonde data

The static environment of middle and upper tropospheric clouds is characterized. Computed relative humidity with respect to ice is used to diagnose the presence of cloud layer. The deduced seasonal mean cloud cover estimates based on this technique are shown to be reasonable. The cases are stratified by season and pressure thickness, and the dry static stability, vertical wind speed shear, and Richardson number are computed for three layers for each case. Mean values for each parameter are presented for each stratification and layer. The relative frequency of occurrence of various structures is presented for each stratification. The observed values of each parameter and the observed structure of each parameter are quite variable. Structures corresponding to any of a number of different conceptual models may be found. Moist adiabatic conditions are not commonly observed and the stratification based on thickness yields substantially different results for each group.

Starr, D. D. O.↗
NASA NTRSDate not supplied

Characterizing Wildfires in Western US.: A Cloud-based Case Study for Interdisciplinary Research using NASA Resources

This presentation will demonstrate a case study of interdisciplinary research done in the Amazon Web Services (AWS) cloud platform, in addition to in the local machine. We conduct data analysis next to data by leveraging various cloud-based data in NASA Earthdata Cloud, which are distributed by different missions/NASA Distributed Active Archive Centers (DAACs), and cloud computing resources at NASA. For instance, we directly access multiple datasets stored in the AWS Simple Storage Service (S3) buckets using a Python Jupyter notebook through a JupyterHub interface hosted in AWS (without having to download data), and conduct data analysis next to data in the cloud. We will also show how to share the research results following Open Source policy. This case study characterizes the change in wildfire events in the western United States during the past 20 years. In particular, we focus on the wildfires in California in 2021, one of the most severe wildfire years occurring in the most recent 20 years in California. We will analyze the possible causes of wildfires, such as drought conditions and climate variability, and examine the impacts of wildfires on air quality and atmospheric composition, and on land cover. We will examine the data distributed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), including aerosols and meteorological data from the NASA Modern-Era Retrospective analysis for Research and Applications version 2 (MERRA-2), precipitation from the Global Precipitation Measurement (GPM) and Global Precipitation Climate Project (GPCP), and aerosol index from Ozone Monitoring Instrument (OMI). We also utilize the data distributed by the Physical Oceanography (PO) DAAC, such as Sea Surface Temperature (SST) data from the Group for High Resolution Sea Surface Temperature (GHRSST), and the data distributed by Land Processes (LP) DAAC, such as Normalized Difference Vegetation Index (NDVI).

Xiaohua Pan↗
DOE OSTI2024

Secure Federated Learning Across Heterogeneous Cloud and High-Performance Computing Resources: A Case Study on Federated Fine-Tuning of LLaMA 2

Federated learning enables multiple data owners to collaboratively train robust machine learning models without transferring large or sensitive local datasets by only sharing the parameters of the locally trained models. Here, in this article, we elaborate on the design of our Advanced Privacy-Preserving Federated Learning (APPFL) framework, which streamlines end-to-end secure and reliable federated learning experiments across cloud computing facilities and high-performance computing resources by leveraging Globus Compute, a distributed function as a service platform, and Amazon Web Services. We further demonstrate the use case of APPFL in fine-tuning an LLaMA 2 7B model using several cloud resources and supercomputers.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2019

"Sensor Web Evolution - Webs of Webs for NASA Science - Focus on small Uninhabited Aerial Systems (sUAS)"

This paper will describe the evolution of information collection, derivation and delivery mechanisms in webs of NASA sensor webs, with a focus on recent advancements in small Uninhabited Aerial Systems (sUAS). I will discuss the movement to "Fog Computing", also known as Edge Computing. Fog Computing facilitates the distribution of common operations and networking between edge devices and cloud computing facilities, optimizing the production of actionable intelligence. Initially, sUASs utilized onboard data collection as standard, with minimal data downloaded directly. Information products were derived in conventional computational environments, generally desk top computers, and information products made available to the Science Community in weeks or months. With the increased availability, and increasingly lower costs, of beyond line of sight (BLOS) satellite based communication, transmission rates and data volumes increased, and processing migrated to Cloud based services. Contemporary sUASs are moving some of that information product derivation to on vehicle services, and are creating a distributed Cloud/Fog environment. I will describe the technological advances that have made this possible, including low power multi-core Central Processing Units (CPU), and, more recently, the availability of high end Graphical Processing Units (GPU) that consume only a few watts. Intelligent system software, leveraging these hardware advances, finally allows for information product generation on-board, rather than simple data collection. Additionally, intelligent flight control systems now support mutual vehicle to vehicle collaboration, allowing sUASs to create ad-hoc sensor webs on demand, as required. Also discussed will be the lessons learned by the Authors' development of data systems for NASA's large High Altitude Long Endurance (HALE) UASs like Predator and Global Hawk, and how those lessons are being applied to sUAS development. This paper will focus on application, rather a deep dive into the technology, and will highlight improving data management through these new technologies.

Sensor Web↗
DOE OSTI2021

Priority-enabled Load Balancing for Dispersed Computing

Opportunistic managed access to local in-network compute resources can improve the performance of distributed applications and reduce the dependence on shared network resources. Instead of backhauling application data to a centralized cloud data center for processing, networked services may be adaptively and continuously dispersed into shared compute resources that are closer to the source of need. While this approach has several benefits, support for mission-aware access to computation is often an afterthought, and is implemented as a brittle extension over traditional load-balancer solutions.In this work, we investigate the design of two priority-aware resource allocation strategies and two load-balancing dispatching strategies as first class citizens in an open-source dispersed computing middleware. We present a control theoretic analysis of these load-balancing primitives to identify weaknesses and strengths in our design, and recommend future directions. In parallel, we prototype two priority-aware allocation algorithms to validate our priority predictions. In initial experiments our prototype shows substantial gains in processing prioritized load. Finally, we make our source-code and experimental configurations open source.

Paulos, Aaron↗
DOE OSTI2015

Flexible HPC Environments [Poster]

New computational paradigms require new tools and environments. A new class of data-analysis tools are available for scientists. These tools do not integrate well with traditional HPC resource management systems. Differing and often conflicting requirements exist. Some fields require validated environments including high energy physics and cosmology. By adopting cloud technologies in an HPC environment we are merging the compute-intensive and data-intensive worlds in one converged system. HPC Systems will support a Larger Set of Technologies. Scientists will be able to leverage the latest tools on HPC resources at previously unavailable scales. User-defined environments will enable rapid experimentation and adoption of emerging tools. HPC systems will support a consolidated workflow from simulation to analysis and discovery. HPC systems will have increased utility to a broader set of users.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1984

Statistical analysis and use of the VAS radiance data

Special radiosonde soundings at 75 km spacings and 3 hour intervals provided an opportunity to learn more about mesoscale data and storm-environment interactions. Relatively small areas of intense convection produce major changes in surrounding fields of thermodynamic, kinematic, and energy variables. The Red River Valley tornado outbreak was studied. Satellite imagery and surface data were used to specify cloud information needed in the radiative heating/cooling calculations. A feasibility study for computing boundary layer winds from satellite-derived thermal data was completed. Winds obtained from TIROS-N retrievals compared very favorably with corresponding values from concurrent rawisonde thermal data, and both sets of thermally-derived winds showed good agreements with observed values.

Fuelberg, H. E.↗
NASA NTRS1975

Extension of four-dimensional atmospheric models

The cloud data bank, the 4-D atmospheric model, and a set of computer programs designed to simulate meteorological conditions for any location above the earth are described in turns of space vehicle design and simulation of vehicle reentry trajectories. Topics discussed include: the relationship between satellite and surface observed cloud cover using LANDSAT 1 photographs and including the effects of cloud shadows; extension of the 4-D model to the altitude of 52 km; and addition of the u and v wind components to the 4-D model of means and variances at 1 km levels from the surface to 25 km. Results of the cloud cover analysis are presented along with the stratospheric model and the tropospheric wind profiles.

Fowler, M. G.↗
DOE OSTI2025

Classification of Cloud Particle Imagery and Thermodynamics (COCPIT): A New Databasing Tool for the Characterization of Cloud Particle Images Captured During DOE Field Campaigns

The Department of Energy for decades has explored the earth system and atmosphere through research and deployment of in-situ and remote sensing platforms during field campaigns. Among these datasets exists a vast supply of cloud particle images that provide visual insight into the complex microphysics in the clouds that span our globe. The millions of images collected over decades of deployments provides a unique opportunity to further our understanding of our atmosphere down to the crystal size. This work over the past 5 years has sought to organize these images into digestible datasets that can then be used by scientists to further our understanding of microphysics. A machine learning model was developed that categorizes over 1.5 million images across 11 weather events with over 90% accuracy according to particle type. The database was then extended to include dimensional characteristics of the particle as well as co-location of environmental properties, such as temperature and water content. Then, to initialize the connection between these data and our understanding of how crystals form and grow, weather research and forecasting simulations were run to generate the growth histories of the classified crystals. This research culminates with 2 databases per event: (1) a database of all classified crystals and their dimensional and environmental properties and (2) simulated growth histories of each crystal. Finally, a user interface was created to allow researchers to explore data statistics.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1971

Corkscrews and singularities in fruitflies - Resetting behavior of the circadian eclosion rhythm.

Description of experiments undertaken to define the phase-resetting behavior of the circadian rhythm of pupal eclosion in populations of fruitflies. An attempt is made to determine how and why the resetting response depends on the duration of a standard perturbation as well as on the time at which it is given. Plotting a three-dimensional graph of the measured emergence centroids vs the stimulus variables, the data are found to spiral up around a vertical rotation axis. Using a computer, a smooth surface, called the resetting surface, which approximately fits the helicoidal cloud of data points, is obtained and is shown to be best described as a vertical corkscrew linking together tilted planes. This corkscrew feature of the resetting surface is taken to indicate that there is an isolated perturbation following which there is either no circadian rhythm of emergence in the steady state, or one of unpredictable phase. A hypothesis concerning the clock dynamics underlying the eclosion rhythm is briefly sketched which encompasses the main features of known resetting data using single discrete pulses of any perturbing agent.

Winfree, A. T.↗
DOE OSTI2018

SpaRC: scalable sequence clustering using Apache Spark

MOTIVATION: Whole genome shotgun based next-generation transcriptomics and metagenomics studies often generate 100-1000 GB sequence data derived from tens of thousands of different genes or microbial species. Assembly of these data sets requires tradeoffs between scalability and accuracy. Current assembly methods optimized for scalability often sacrifice accuracy and vice versa. An ideal solution would both scale and produce optimal accuracy for individual genes or genomes. RESULTS: In this work we describe an Apache Spark-based scalable sequence clustering application, SparkReadClust (SpaRC), that partitions reads based on their molecule of origin to enable downstream assembly optimization. SpaRC produces high clustering performance on transcriptomes and metagenomes from both short and long read sequencing technologies. It reaches near-linear scalability with input data size and number of compute nodes. SpaRC can run on both cloud computing and HPC environments without modification while delivering similar performance. Our findings demonstrate that SpaRC provides a scalable solution for clustering billions of reads from next-generation sequencing experiments, and Apache Spark represents a cost-effective solution with rapid development/deployment cycles for similar large-scale sequence data analysis problems. AVAILABILITY AND IMPLEMENTATION: https://bitbucket.org/berkeleylab/jgi-sparc.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Cloud-Based Demonstration of the Eastern Interconnection Situational Awareness Monitoring System (ESAMS)

This report describes a cloud-based implementation and field demonstration of the Eastern Interconnection Situational Awareness and Monitoring System (ESAMS). ESAMS was developed to support the detection and source localization of forced oscillations using synchrophasor measurements from tie-lines connecting areas served by different reliability coordinators (RCs), so that RCs could better coordinate their response to wide-area events. A previous effort had identified deployment barriers associated with hosting shared situational awareness tools at a single RC. To address these barriers, ESAMS was migrated to Amazon Web Services and evaluated in a six-month field demonstration. ISO New England (ISO-NE) and PJM streamed data to the platform using AWS Direct Connect and a site-to-site VPN, respectively. The resulting multi-utility measurement footprint enabled regional source localization across major portions of the U.S. Eastern Interconnection and supported routine identification of oscillation events. During the final three months of the trial, 24 events above 2 MW/MVAR were detected. The largest detected oscillation approached a 25 MW peak-to-peak amplitude, and the longest persisted intermittently for more than 11 hours. The demonstration also assessed operational considerations—including data transfer volumes, end-to-end latency, and cloud computing costs—and found that network and compute requirements were modest relative to typical cloud capabilities while providing performance comparable to prior on-premises deployments. Overall, the results indicate that cloud hosting can provide a practical path to shared interconnection-wide oscillation monitoring. The cloud ESAMS demonstration establishes a foundation for broader utility participation and for building future wide-area analytics that leverage measurements across organizational boundaries.

Follum, James D.↗
NASA NTRS1986

Turbulent dispersion of the icing cloud from spray nozzles used in icing tunnels

To correctly simulate flight in natural icing conditions, the turbulence in an icing simulator must be as low as possible. But some turbulence is required to mix the droplets from the spray nozzles and achieve an icing cloud of uniform liquid water content. The goal for any spray system is to obtain the widest possible spray cloud with the lowest possible turbulence in the test section of a icing tunnel. This investigation reports the measurement of turbulence and the three-dimensional spread of the cloud from a single spray nozzle. The task was to determine how the air turbulence and cloud width are affected by spray bars of quite different drag coefficients, by changes in the turbulence upstream of the spray, the droplet size, and the atomizing air. An ice accretion grid, located 6.3 m downstream of the single spray nozzle, was used to measure cloud spread. Both the spray bar and the grid were located in the constant velocity test section. Three spray bar shapes were tested: the short blunt spray bar used in the NASA Lewis Icing Research Tunnel, a thin 14.6 cm chord airfoil, and a 53 cm chord NACA 0012 airfoil. At the low airspeed (56 km/hr) the ice accretion pattern was axisymmetric and was not affected by the shape of the spray bar. At the high airspeed (169 km/hr) the spread was 30 percent smaller than at the low airspeed. For the widest cloud the spray bars should be located as far upstream in the low velocity plenum of the icing tunnel. Good comparison is obtained between the cloud spread data and predicitons from a two-dimensional cloud mixing computer code using the two equation turbulence (k epsilon g) model.

Marek, C. J.↗
NASA NTRS1991

Spatial and temporal variability of global surface solar irradiance

Consideration is given to a fast scheme for computing surface solar irradiance using data from the International Satellite Cloud Climatology Project (ISCCP). Daily mean solar irradiances from the fast scheme reproduce the detailed global results from full radiative transfer model calculations to within 6 and 10 W/sq m over the ocean and land, respectively. Comparison of calculated monthly mean results using 5 m of ISCCP data (July 1983-July 1984) with climatology from the 1970s at six temperature-latitude ocean weather stations shows agreement within published estimates of interannual variability of monthly means at the individual stations. A further test against a 17-day time series at a continental site, where ground and satellite data were spatially and temporally coincident, showed an accuracy of better than 9 W/sq m on a daily basis and less than 4 percent bias in the 17-day mean. Frequently used bulk formulas for solar irradiance are also evaluated in each of these tests.

Bishop, James K. B.↗
NASA NTRS2015

Technical Challenges and Opportunities of Centralizing Space Science Mission Operations (SSMO) at NASA Goddard Space Flight Center

The NASA Goddard Space Science Mission Operations project (SSMO) is performing a technical cost-benefit analysis for centralizing and consolidating operations of a diverse set of missions into a unified and integrated technical infrastructure. The presentation will focus on the notion of normalizing spacecraft operations processes, workflows, and tools. It will also show the processes of creating a standardized open architecture, creating common security models and implementations, interfaces, services, automations, notifications, alerts, logging, publish, subscribe and middleware capabilities. The presentation will also discuss how to leverage traditional capabilities, along with virtualization, cloud computing services, control groups and containers, and possibly Big Data concepts.

Science↗
NASA NTRS2016

NASA Briefing for Unidata

The NASA representative to the Unidata Strategic Committee presented a semiannual update on NASAs work with and use of Unidata technologies. The talk covered the program of cloud computing prototypes being undertaken for the Earth Observing System Data and Information System (EOSDIS). Also discussed were dataset interoperability recommendations ratified via the EOSDIS Standards Office and the HDF Product Designer tool with respect to its possible applicability to data in network Common Data Form (NetCDF) version 4.

data integration↗
NASA NTRSDate not supplied

ML/AI Applications to the Atmosphere Science Data and Simulations (Demonstration and Vision)

Artificial Intelligence has been recognized as one of the most powerful tools for scientific research. It has a wide range of applications in atmospheric science and plays a significant role in advancing our understanding of the Earth-Atmosphere system, as well as improving our ability to monitor, predict, and respond to atmospheric observations and simulations. Different AI architectures can help us handle various problems according to our research goals. In this presentation, we will introduce several use cases and models for applying computer vision to active remote sensing data from CALIOP, analyzing cloud dynamical relationships for climate model parameterizations, and building surrogate models for lidar simulations. The presentation will highlight the advantages that AI offers compared to traditional analytical methods. By assembling pieces of the multidimensional atmospheric science puzzle, we can integrate our knowledge from observations to simulations.

Shan Zeng↗
DOE OSTI2025

Improving Cyber Situational Understanding

Effective cybersecurity operations require the ability to analyze large amounts of information to assess security risks and formulate defensive strategies against adversaries. This has become more complex in recent years as the sprawl and interconnectivity of devices grows through implementation of virtualization, cloud computing, and Internet of Things (IoT). The amount of data and analysis required for effective cybersecurity command and control decisions far exceeds humans’ capacity to perform manually. We characterize the analysis problem as cyber situational understanding. The research presented to improve cyber situational understanding focuses on vulnerability analysis and threat intelligence. Regarding vulnerabilities, entities must analyze and plan work for between thousands and tens of thousands of software vulnerabilities annually. Entities heavily use network firewalls to limit vulnerability exposure. As a result, some of these vulnerabilities permit exposure to adversarial exploitation, whereas others are inaccessible and therefore present negligible risk of exploitation. Distinguishing between high and low risk software vulnerabilities requires a deep understanding of the vulnerability, network firewall protection, and characteristics of the targeted device. This problem is solved by extracting network service features from vulnerability data features using both machine-learning and natural language processing. Then, the network firewall topology is parsed to determine which vulnerabilities are reachable by adversaries. Ultimately, a state-based safety analysis ascertains which vulnerabilities are unsafe. A related vulnerability analysis problem occurs in cybersecurity operations when associating an entity’s hardware and software assets to public vulnerability databases. Assets often reveal hardware and software through installation artifacts and network service identification, and entities store these artifacts in inventory databases. However, software and hardware vendors apply a standard Common Platform Enumeration (CPE) naming convention when publicly reporting vulnerabilities. Associating these two datasets often requires many hours to days of manual inspection. The proposed solution automates the mapping approach of human analysts using fuzzy matching techniques, natural language processing, and, ultimately, machine learning to present a small set of recommendations for mapping the two datasets. The result significantly reduces human analysis time and reduces the occurrence of false positives in vulnerability notifications. Finally, cyber threat intelligence (CTI) requires associating cyber observable artifacts, such as IP addresses, URIs, and file hashes, with cyber threat tactics, techniques, and procedures. Unfortunately, most CTI data is compartmentalized across multiple organizations and cannot be shared due to the legal and reputational risk with cyber threat being associated with the entity. The approach to solving this problem inovlves using a distributed ledger with anonymous token spending and authentication. This allows a consortium of semi-trusted entities to share the workload of curating CTI for a threat sharing community’s cooperative benefit.

Huff, Philip↗