Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud platform”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

Scalable pCT Image Reconstruction Delivered as a Cloud Service

We describe a cloud-based medical image reconstruction service designed to meet a real-time and daily demand to reconstruct thousands of images from proton cancer treatment facilities worldwide. Rapid reconstruction of a three-dimensional Proton Computed Tomography (pCT) image can require the transfer of 100 GB of data and use of approximately 120 GPU-enabled compute nodes. The nature of proton therapy means that demand for such a service is sporadic and comes from potentially hundreds of clients worldwide. We thus explore the use of a commercial cloud as a scalable and cost-efficient platform for pCT reconstruction. To address the high performance requirements of this application we leverage Amazon Web Services' GPU-enabled cluster resources that are provisioned with high performance networks between nodes. To support episodic demand, we develop an on-demand multi-user provisioning service that can dynamically provision and resize clusters based on image reconstruction requirements, priorities, and wait times. We compare the performance of our pCT reconstruction service running on commercial cloud resources with that of the same application on dedicated local high performance computing resources. We show that we can achieve scalable and on-demand reconstruction of large scale pCT images for simultaneous multi-client requests, processing images in less than 10 minutes for less than $10 per image.

Cloud computing↗

Innovation in Radiological Security, Part 2 of 2 – Insights into Developing a Cloud-hosted Security Technology

A Sentry Remote Monitoring System (Sentry-RMS) is a stand-alone security system that provides detection, assessment, and communication of priority alarms as an additional means of thwarting internal and external threats to sites that maintain radiological material. The SEntry-RMS CommUnications and REsponse (Sentry-SECURE) platform is an optional feature of the Sentry-RMS that relays priority alarm information to the identified response stakeholders. Sentry-SECURE is hosted in a cloud environment that abstracts the data owner’s and data consumer’s platforms to allow for greater information sharing. This promotes situational awareness amongst authorized users and enables future innovation among modern response platforms. When securely architecting a cloud solution such as this, the use of design paradigms can be an effective tool to increase the accuracy and reliability of cyber- and information-security-related decisions made throughout the development process. This approach also supports the categorization of design considerations into three levels: industry concepts, project approaches, and data protections for digital processes. Industry concepts consist of the notional underpinnings that guide or motivate a security process, system, or design but often lack any tangible attributes. Project approaches represent decisions made during the design and development process to prioritize a solution, method, or practice above another that may provide a comparable functional output but lacks a desired security benefit. Data protections for digital processes represent the selection, integration, and implementation of specific controls for a given asset. This paper will explore specific examples of how Sentry-SECURE has been designed to account for considerations at each of these three levels, while balancing the operational intent of the platform with the security enhancements necessary to maintain data integrity, availability, and confidentiality.

assessment, RMS, security, physical, cloud, Cyber ↗

EDX Spatial: Leveraging cloud and hybrid data management resources for spatial data

In the last few years, the National Energy Technology Laboratory has started leveraging cloud-hosted services for hosting spatial data collections published on the Energy Data eXchange (EDX). Using cloud-hosted storage and compute options offers multiple benefits for visualization and tool development through utilization of spatial data resources. The transition from on-premises services to cloud-hosted services has enabled a few key features: increased accessibility of large derivative datasets, dynamic integration of external authoritative data resources directly from outside entities through representational state transfer application programming interfaces (REST API), and the ability to produce complex mapping applications, dashboards, and online maps. As a result of the shift, NETL has launched EDX Spatial, a platform that leverages on-premises resources combined with cloud compute capabilities to enable enhanced online mapping interfaces and optimize data access. In addition, a unified workflow for handling the public release of spatial data products through EDX has been developed, including standardization of data hosting practices, metadata, symbology, and application elements. This poster reviews the opportunity of leveraging cloud-hosted and hybrid data management solutions for spatial data, and discusses the benefits and lessons learned while leveraging these services through the data repository EDX.

Morkner, Paige↗

Classification of Cloud Particle Imagery and Thermodynamics (COCPIT): A New Databasing Tool for the Characterization of Cloud Particle Images Captured During DOE Field Campaigns

The Department of Energy for decades has explored the earth system and atmosphere through research and deployment of in-situ and remote sensing platforms during field campaigns. Among these datasets exists a vast supply of cloud particle images that provide visual insight into the complex microphysics in the clouds that span our globe. The millions of images collected over decades of deployments provides a unique opportunity to further our understanding of our atmosphere down to the crystal size. This work over the past 5 years has sought to organize these images into digestible datasets that can then be used by scientists to further our understanding of microphysics. A machine learning model was developed that categorizes over 1.5 million images across 11 weather events with over 90% accuracy according to particle type. The database was then extended to include dimensional characteristics of the particle as well as co-location of environmental properties, such as temperature and water content. Then, to initialize the connection between these data and our understanding of how crystals form and grow, weather research and forecasting simulations were run to generate the growth histories of the classified crystals. This research culminates with 2 databases per event: (1) a database of all classified crystals and their dimensional and environmental properties and (2) simulated growth histories of each crystal. Finally, a user interface was created to allow researchers to explore data statistics.

54 ENVIRONMENTAL SCIENCES↗

From Data to Models and Analytics: An Integrated Scientific Computing Platform to Accelerate Environmental Research SBIR Phase I (Final Technical Report)

Phase I research was conducted to investigate the technical feasibility of an integrated open source platform GeoLib to provide turnkey solutions for model-data integration for environmental systems science (ESS), using modern web technologies, advances in HPC and cloud computing and high-performance visualization. A key feature of our platform is that it minimizes and automates data movement between data storage and computing systems. Another critical feature of the design is that it enables users to perform high-fidelity post-processing by performing high-performance visualization server-side and in-situ on the compute cluster close to the data. The investigation, which was performed over a 12-month period ending April 2019, centered on developing prototype software to demonstrate the key enabling technologies required for this system. This included: (i) rapid access to data from multiple remote sources, via a unified data interface; (ii) software for scalable geospatial visualization of raster, vector, and point cloud data on the web; (iii) development of an easy-to-use API for creating simulation and modeling workflows; (iv) remote data visualization and processing on middle-tier servers; and (v) remote data processing on high-performance computing systems. A critical part of this work is that all of these capabilities be accessible to end users through standard desktop tools such as Project Jupyter, which is widely adopted for ESS simulation and modeling.

97 MATHEMATICS AND COMPUTING↗

Short-term solar irradiance forecasting via satellite/model coupling

Here, the short-term (0-3 h) prediction of solar insolation for renewable energy production is a problem well-suited to satellite-based techniques. The spatial, spectral, temporal and radiometric resolution of instrumentation hosted on the geostationary platform allows these satellites to describe the current cloud spatial distribution and optical properties. These properties relate directly to the transient properties of the downwelling solar irradiance at the surface, which come in the form of 'ramps' that pose a central challenge to energy load balancing in a spatially distributed network of solar farms. The short-term evolution of the cloud field may be approximated to first order simply as translational, but care must be taken in how the advection is handled and where the impacts are assigned. In this research, we describe how geostationary satellite observations are used with operational cloud masking and retrieval algorithms, wind field data from Numerical Weather Prediction (NWP), and radiative transfer calculations to produce short-term forecasts of solar insolation for applications in solar power generation. The scheme utilizes retrieved cloud properties to group pixels into contiguous cloud objects whose future positions are predicted using four-dimensional (space + time) model wind fields, selecting steering levels corresponding to the cloud height properties of each cloud group. The shadows associated with these clouds are adjusted for sensor viewing parallax displacement and combined with solar geometry and terrain height to determine the actual location of cloud shadows. For mid/high-level clouds at mid-latitudes and high solar zenith angles, the combined displacements from these geometric considerations are non-negligible. The cloud information is used to initialize a radiative transfer model that computes the direct and diffuse-sky solar insolation at both shadow locations and intervening clear-sky regions. Here, we describe the formulation of the algorithm and validate its performance against Surface Radiation (SURFRAD; Augustine et al., 2000, 2005) network observations. Typical errors range from 8.5% to 17.2% depending on the complexity of cloud regimes, and an operational demonstration outperformed persistence-based forecasting of Global Horizontal Irradiance (GHI) under all conditions by ~10 W/m 2 .

14 SOLAR ENERGY↗

A GENERALIZED MODEL OF NONLINEAR DIFFUSIVE SHOCK ACCELERATION COUPLED TO AN EVOLVING SUPERNOVA REMNANT

To better model the efficient production of cosmic rays (CRs) in supernova remnants (SNRs) with the associated coupling between CR production and SNR dynamics, we have generalized an existing cr-hydro-NEI code to include the following processes: (1) an explicit calculation of the upstream precursor structure including the position-dependent flow speed, density, temperature, and magnetic field strength; (2) a momentum- and space-dependent CR diffusion coefficient; (3) an explicit calculation of magnetic field amplification; (4) calculation of the maximum CR momentum using the amplified magnetic field; (5) a finite Alfven speed for the particle scattering centers; and (6) the ability to accelerate a superthermal seed population of CRs, as well as the ambient thermal plasma. While a great deal of work has been done modeling SNRs, most work has concentrated on either the continuum emission from relativistic electrons or ions or the thermal emission from the shock heated plasma. Our generalized code combines these elements and describes the interplay between CR production and SNR evolution, including the nonlinear coupling of efficient diffusive shock acceleration, based mainly on the work of P. Blasi and coworkers, and a non-equilibrium ionization (NEI) calculation of thermal X-ray line emission. We believe that our generalized model will provide a consistent modeling platform for SNRs, including those interacting with molecular clouds, and improve the interpretation of current and future observations, including the high-quality spectra expected from Astro-H. SNR RX J1713.7-3946 is modeled as an example.

79 ASTRONOMY AND ASTROPHYSICS↗

ProvLight: Efficient Workflow Provenance Capture on the Edge-to-Cloud Continuum

Modern scientific workflows require hybrid infrastructures combining numerous decentralized resources on the IoT/Edge interconnected to Cloud/HPC systems (aka the Computing Continuum) to enable their optimized execution. Understanding and optimizing the performance of such complex Edge-to-Cloud workflows is challenging. Capturing the provenance of key performance indicators, with their related data and processes, may assist in understanding and optimizing workflow executions. However, the capture overhead can be prohibitive, particularly in resource-constrained devices, such as the ones on the IoT/Edge.To address this challenge, based on a performance analysis of existing systems, we propose ProvLight, a tool to enable efficient provenance capture on the IoT/Edge. We leverage simplified data models, data compression and grouping, and lightweight transmission protocols to reduce overheads. We further integrate ProvLight into the E2Clab framework to enable workflow provenance capture across the Edge-to-Cloud Continuum. This integration makes E2Clab a promising platform for the performance optimization of applications through reproducible experiments.We validate ProvLight at a large scale with synthetic workloads on 64 real-life IoT/Edge devices in the FIT IoT LAB testbed. Evaluations show that ProvLight outperforms state-of-the-art systems like ProvLake and DfAnalyzer in resource-constrained devices. ProvLight is 26—37x faster to capture and transmit provenance data; uses 5—7x less CPU; 2x less memory; transmits 2x less data; and consumes 2—2.5x less energy. ProvLight [1] and E2Clab [2] are available as open-source tools.

Rosendo, Daniel↗

Framework for Analysis and Visualization of Simulation Data (VizAly)

As supercomputing resources increase, cosmological scientists are able to run more detailed and larger simulations generating massive amounts of data. Analyzing these simulations with an available open-source toolkit is important for collaborative Department of Energy scientific discovery across labs, universities, and other partners. Developed software as a part of this collection include: comparing data with other existing simulations, verifying and validating results with observation databases, new halo finder algorithms, and using analytical tools to get insights into the physics of the cosmological universe. The goal of this software project is to provide a set of open-source libraries, tools, and packages for large-scale cosmology that allows scientists to visualize, analyze, and compare large-scale simulation and observational data sets. Developed software will provide a variety of methods for processing, visualization, and analysis of astronomical observation and cosmological simulation data. These tools are intended for deployment on multiple scientific computing platforms, including but not limited to personal computers, cloud computing, experimental sites (telescopes) and high performance supercomputers.

Wendelberger, Joanne↗

rSHUD v2.0: advancing the Simulator for Hydrologic Unstructured Domains and unstructured hydrological modeling in the R environment

Abstract. Hydrological modeling is a crucial component in hydrology research, particularly for projecting future scenarios. However, achieving reproducibility and automation in distributed hydrological modeling research for modeling, simulation, and analysis is challenging. This paper introduces rSHUD v2.0, an innovative, open-source toolkit developed in the R environment to enhance the deployment and analysis of the Simulator for Hydrologic Unstructured Domains (SHUD). The SHUD is an integrated surface–subsurface hydrological model that employs a finite-volume method to simulate hydrological processes at various scales. The rSHUD toolkit includes pre- and post-processing tools, facilitating reproducibility and automation in hydrological modeling. The utility of rSHUD is demonstrated through case studies of the Shale Hills Critical Zone Observatory in the USA and the Waerma watershed in China. The rSHUD toolkit's ability to quickly and automatically deploy models while ensuring reproducibility has facilitated the implementation of the Global Hydrological Data Cloud (https://ghdc.ac.cn, last access: 1 September 2023), a platform for automatic data processing and model deployment. This work represents a significant advancement in hydrological modeling, with implications for future scenario projections and spatial analysis.

Shu, Lele (ORCID:0000000269034466)↗

DOE site remedial management optimization utilizing on the cloud computing systems - 15676

Environmental cleanups at complex sites necessarily address many competing demands and answer questions such as: 'What is the best way and in a sustainable manner to minimize risk to human health and the environment during active restoration efforts and/or long term legacy management, while incorporating parameter uncertainty, management considerations, and stakeholder concerns into the process?' Physics Based Management Optimization (PBMO{sup TM}) provides a means to answer such questions. 'Physics Based' indicates incorporation of numerically computed groundwater flow and transport processes into the analysis. This enables optimal remedy design based on comprehensive mass removal/destruction metrics and optimal monitoring strategies. 'Management Optimization' indicates the ability to incorporate objective functions (e.g., management constraints, sustainability considerations) into the evaluation. PBMO{sup TM} components have been successfully implemented for optimal plume delineation (points of compliance) and monitoring, for optimal system design and to optimize existing active treatment systems in the US at multiple DOE, DOD and industrial sites since the mid 1990's. PBMO{sup TM} has been recently extended and implemented on multi-core, multi-CPU grid computing systems including computational clusters, local area networks and the Cloud. This extension allows for increased flexibility and adaptation of computational resources which can now support optimal remedy design and long term management strategies for complex soil and groundwater sites. In the PBMO{sup TM} -Grid case study, we evaluate flow and transport for a typical optimization search for a realistic project against sequential optimization. PBMO{sup TM}-Grid reduces the CPU time required to solve the optimization problem by more than a factor of 14 X compared to sequential optimization. PBMO{sup TM}-Grid can be deployed for deterministic or stochastic optimization and formally computes design risk and predicted degree of remedial action success. The numerical models used can consist of a single 'mega' model (aka a monolith), or a mixture of models and other calculations or expert systems. PBMO{sup T}M-Grid integrates the computational models in a seamless manner, and models run on the Cloud can be implemented in Windows or Linux platforms. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Globus platform-as-a-service for collaborative science applications

Globus, developed as Software-as-a-Service (SaaS) for research data management, also provides APIs that constitute a flexible and powerful Platform-as-a-Service (PaaS) to which developers can outsource data management activities such as transfer and sharing, as well as identity, profile and group management. By providing these frequently important but always challenging capabilities as a service, accessible over the network, Globus PaaS streamlines web application development and makes it easy for individuals, teams, and institutions to create collaborative applications such as science gateways for science communities. We introduce the capabilities of this platform and review representative applications.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

The disCO2ver Platform: Curating Data and Tools for Geologic Carbon Sequestration and Deep Subsurface Research Systems

The U.S. DOE National Energy Technology Laboratory has invested 12+ years of development into the data repository and digital laboratory, the Energy Data eXchange (EDX, edx.netl.doe.gov). Supporting a variety of research areas across the DOE Office of Fossil Energy and Carbon Management, the platform has successfully curated and preserved thousands of data products from DOE research. The Carbon Storage Program has successfully supported data curation, upload, and publishing of data products on EDX for many years, demonstrating a success story of how resources like EDX can effectively help with long term preservation and publishing of DOE data products. EDX continues to shift towards cloud-supported infrastructure, taking a hybrid approach combining on-premises compute and storage integrated with cloud-hosted services. The integration of cloud compute and hybrid architecture enables the development of EDX-hosted platforms that tailor the data and tools hosted on them to a specific community, enables implementation of machine learning tools for data discovery and filtering, and enables the hosting of virtual (online user interface) tools. Geologic carbon sequestration (GCS) research continues to scale up in response to the current administration goals to reduce greenhouse gas emissions and transition the energy economy. Over the last year, EDX’s disCO2ver platform has been developed in response to the need for access to data products and tools to support the scaling up of GCS research. disCO2ver provides access to data resources and tools, produced by DOE and outside authoritative external resources. The platform also provides a user-access control component for the virtualization and cloud hosting of tools. Tools that need to be virtualized, to eliminate the need for users to download the tool and use local compute resources, is essential to supporting big-data analysis and machine learning that is becoming common place in carbon storage modeling, risk analysis, and data publishing practices. This talk will review the EDX’s disCO2ver platform and the current work ongoing to curate data and tools to support GCS and deep subsurface systems research.

Morkner, Paige↗

XaaS: Acceleration as a Service to Enable Productive High-Performance Cloud Computing

High-performance computing (HPC) and the cloud have evolved independently, specializing their innovations into performance or productivity. Acceleration as a Service (XaaS) is a recipe to empower both fields with a shared execution platform that provides transparent access to computing resources, regardless of the underlying cloud or HPC service provider. Bridging HPC and cloud advancements, XaaS presents a unified architecture built on performance-portable containers. Here, our converged model concentrates on low-overhead, high-performance communication and computing, targeting resource-intensive workloads from climate simulations to machine learning. XaaS lifts the restricted allocation model of Function as a Service (FaaS), allowing users to benefit from the flexibility and efficient resource utilization of serverless computing while supporting long-running and performance-sensitive workloads from HPC.

97 MATHEMATICS AND COMPUTING↗

Enabling Exascale Hardware and Software Design through Scalable System Virtualization (Final Report)

The purpose of this project has been to extend the state of the art of systems software for high-end computing (HEC) platforms, and to use systems software to better enable the evaluation of potential future HEC platforms, for example exascale platforms. Such platforms, and their systems software, have the goal of providing scientific computation at new scales, thus enabling new research in the physical sciences and engineering. Over time, the innovations in systems software for such platforms also become applicable to more widely used computing clusters, data centers, and clouds. This was a five-institution project, centered on the Palacios virtual machine monitor (VMM) systems software, a project begun at Northwestern, and originally developed in a previous collaboration between Northwestern University and the University of New Mexico. In this project, Northwestern (including via our subcontract to the University of Pittsburgh) contributed to the continued development of Palacios, along with other team members. We took the leadership role in (1) continued extension of support for emerging Intel and AMD hardware, (2) integration and performance enhancement of overlay networking, (3) connectivity with architectural simulation, (4) binary translation, and (5) support for modern Non-Uniform Memory Access (NUMA) hosts and guests. We also took a supporting role in support for specialized hardware for I/O virtualization, profiling, configurability, and integration with configuration tools. The efforts we led (1-5) were largely successful and executed as expected, with code and papers resulting from them. The project demonstrated the feasibility of a virtualization layer for HEC computing, similar to such layers for cloud or datacenter computing. For effort (3), although a prototype connecting Palacios with the GEM5 architectural simulator was demonstrated, our conclusion was that such a platform was less useful for design space exploration than anticipated due to inherent complexity of the connection between the instruction set architecture level and the microarchitectural level. For effort (4), we found that a code injection approach proved to be more fruitful. The results of our efforts are publicly available in the open source Palacios codebase and published papers, all of which are available from the project web site, v3vee.org. Palacios is currently one of the two codebases (the other being Sandia’s Kitten lightweight kernel) that underlies the node operating system for the DOE Hobbes Project, one of two projects tasked with building a systems software prototype for the national exascale computing effort.

97 MATHEMATICS AND COMPUTING↗

Cloud-based Architecture Capabilities Summary Report

In collaborating scientific research arena it is important to have an environment where analysts have access to a shared of information documents, software tools and be able to accurately maintain and track historical changes in models. A new cloud-based environment would be accessible remotely from anywhere regardless of computing platforms given that the platform has available of Internet access and proper browser capabilities. Information stored at this environment would be restricted based on user assigned credentials. This report reviews development of a Cloud-based Architecture Capabilities (CAC) as a web portal for PRA tools.

97 MATHEMATICS AND COMPUTING↗

Framework Development Supporting the Safety Portal

In a collaborating scientific research arena it is important to have an environment where analysts have access to a shared repository of information, documents, and software tools, and be able to accurately maintain and track historical changes in models. The new Safety Portal cloud-based environment will be accessible remotely from anywhere regardless of computing platforms given that the platform has available Internet access and proper browser capabilities. Information stored at this environment would be restricted based on user assigned credentials. This report discusses current development of a cloud-based web portal for PRA tools.

97 MATHEMATICS AND COMPUTING↗

Boundary Layer Exploration of Aerosols and Clouds ON Ships (BEACONS)

BEACONS aims to demonstrate autonomous deployment of a shipborne system that routinely observes atmospheric, cloud, and aerosol properties. These observations enable scientists to study aerosol-cloud interactions, advancing energy resilience and the Department of Energy's mission to improve model prediction. Key Project Outcomes Autonomous deployment of aerosol and cloud measurement systems aboard a Pasha Hawaii Marjorie C commercial ship. Six-month campaign for continuous, high-resolution aerosol and cloud data collection with minimal crew intervention. Development and testing of instruments in two phases, advancing from essential to complex systems. AI-ready processed data on the BEACONS Data Platform for near-real-time data access. Validation of aerosol-cloud interaction hypotheses for improved modeling. Establishment of best practices for instrument deployment in marine environments. Scientific collaboration to support the U.S. Department of Energy (DOE) Biological and Environmental Research (BER) mission.

54 ENVIRONMENTAL SCIENCES↗