gismo-cloud-deployment
Tools for executing time-consuming tasks with developer-defined custom code blocks in parallel on the AWS EKS platform.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Tools for executing time-consuming tasks with developer-defined custom code blocks in parallel on the AWS EKS platform.
Recent technology trends, such as the increasing focus on data management technologies and the emergence of sustainable on-demand computing and commercial cloud facilities provide initial steps and potential building blocks for creating such a compute facility. How do we fill the gap between what we have now and the capabilities we need to make the vision of an online data facility a reality? What research challenges need to be addressed – and can be addressed – in the coming 5-10 years? How should we adjust our vision if they are not solved? Is such a facility compatible with the existing and evolving exascale resources and/or how should they evolve to be compatible? How do “beyond Moore’s law” technologies enable advanced data analysis? Those are the challeges that the Future Platform workshop was organized to address. This report is organized as follows. Section 2 summarizes data challenges and patterns related to the exemplar use cases presented during the first half day of the workshop. Section 3 summarizes the requirements presented by those use cases as well as a synthesis of those requirements into a Future Platform definition achieved on the 3rd half day of the workshop (morning of the 2nd day). Section 4 includes area breakout reports from the areas on the 2nd half day of the workshop (afternoon of the 1st day). Finally Section 5 has a report on recommendation for building methodology and metrics. The report concludes in Section 6.
Climate studies, including trend detection and other time series analyses, necessarily require stable, well-characterized and long-term data records. For satellite-based geophysical retrieval datasets, such data records often involve merging the observational records of multiple similar, though not identical, instruments. The National Aeronautics and Space Administration (NASA)cloud mask(CLDMSK) and cloud-top and optical properties(CLDPROP) products are designed to bridge the observational records of the Moderate-resolution Imaging Spectroradiometer (MODIS) onboard NASA’s Aqua satellite and the Visible Infrared Imaging Radiometer Suite (VIIRS) onboard the joint NASA/National Oceanic and Atmospheric Administration(NOAA)Suomi National Polar-orbiting Partnership (SNPP)satellite and NOAA’s new generation of operational polar-orbiting weather platforms(NOAA-20+).Early implementations of the CLDPROP algorithms on Aqua MODIS and SNPP VIIRS suffered from large intersensor biases in cloud optical properties that were traced back to relative radiometric inconsistency in analogous shortwave channels on both imagers, with VIIRS generally observing brighter top-of-atmosphere spectral reflectance than MODIS (e.g., up to 5%brighter in the 0.67μm channel). Radiometric adjustment factors for the SNPP and NOAA-20 VIIRS shortwave channels used in the cloud optical property retrievals are derived from an extensive analysis of the overlapping observational records with Aqua MODIS, specifically for homogenous maritime liquid water cloud scenes for which the viewing/solar geometry of MODIS and VIIRS match. Application of these adjustment factors to the VIIRS L1B prior to ingestion into the CLDMSK and CLDPROP algorithms yields improved intersensor agreement, particularly for cloud optical properties
Current vat photopolymerization (VP) relies on UV or visible light to start the rapid crosslinking of liquid photocurable resins into 3D-printed structures. Here, we develop a self-assembly-driven photopatterning approach to photothermally generate polymeric solids by combining thermoplasmonic nanoparticles and thermoresponsive polymers, in which near-infrared (NIR) light activates thermoplasmonic heating of nanoparticles, triggering the irreversible self-assembly of thermoresponsive polymers into insoluble mesoglobules. A small amount of thermal initiator and crosslinker leads to irreversible self-assembly of polymer nanocomposites. NIR light offers deeper penetration and reduced scattering compared to UV, enabling more uniform curing of thicker or filled materials and expanded process control for composites or opaque systems. Thermoplasmonic heat generation is achieved using surface-modified gold nanorods (AuNRs) with a longitudinal localized surface plasmon resonance peak in the NIR region. Key variables such as polymer composition, molecular weight, physical interactions at the nanoparticle–polymer interface, which can be tuned by surface functionalization, AuNR concentration, and pH can be used to tailor the assembly behavior of these systems, including photothermal effect, flocculation, and cloud point temperature, and the mechanical properties of the final structures. Collectively, these results highlight a platform for photothermally-driven microlithography of polymer solids with diverse, tunable macroscopic properties, enabled by low-energy NIR light-activated self-assembly.
Weather radar measurements from airborne or satellite platforms can be an effective remote sensing tool for examining the three-dimensional structures of clouds and precipitation. This chapter describes some fundamental properties of radar measurements and their dependence on the particle size distribution (PSD) and radar frequency. The inverse problem of solving for the vertical profile of PSD from a profile of measured reflectivity is stated as an optimal estimation problem for single- and multi-frequency measurements. Phenomena that can change the measured reflectivity Z(sub m) from its intrinsic value Z(sub e), namely attenuation, non-uniform beam filling, and multiple scattering, are described and mitigation of these effects in the context of the optimal estimation framework is discussed. Finally, some techniques involving the use of passive microwave measurements to further constrain the retrieval of the PSD are presented.
A diverse set of data analytics systems for Earth Observations are sprouting up in the Earth Science community, with a wealth of processing algorithms and analysis methods. There is a similar wealth of data resources available via myriad data providers and clearinghouses, including large institutional systems like the Earth Observing System Data and Information System, Comprehensive Large Scale Array-data Stewardship System, and Federated Earth Observation Missions gateway. With Earth system science driving a need to work with more datasets together, and the community developing more analysis tools (some of them dataset-specific), how can we develop analysis workflows that incorporate far-flung datasets and leverage analysis resources from multiple organizations? Cloud computing points the way toward a solution in two different respects. Firstly, the access to and abstraction of virtually unlimited storage and computing power provides an environment that enables more straightforward means of pulling datasets and analysis resources together. Just as importantly, however, cloud computing serves as an example of an "ecosystem" of interoperating services, since the essence of cloud computing is the presentation of all resources as a service, from hardware to infrastructure to platform to software. This enables the combination of off-the-shelf, diverse services to construct entire systems that emerge out of an equally diverse community of architects and developers. This approach can be similarly applied to the data and analysis resources in the Earth Observation community. By exposing these resources via well understood services, and consuming resources in the same way, different organizations can construct bespoke analysis workflows and systems for their own purposes. The key leap the community needs to make is to develop analysis systems in components that interact with other components via services. The result would be a rich ecosystem of analytics components that can be combined to analyze datasets at scale and in conjunction with other datasets from other sources.
Airborne observations are critical to understanding Earth’s climate system and surface-level air quality by providing both spatial and vertical information that is not possible through ground measurement networks or satellite observations. Airborne platforms allow assessments of long-range transport, vertical redistribution, boundary-layer dynamics, and cloud-aerosol-interactions that are critical to model evaluation and satellite validation. Still, the high cost and effort of aircraft operations can limit measurement campaigns to short, focused time periods. Multiple deployments are often necessary for seasonal comparisons, and results can be limited if target phenomenon are infrequent. Subsequent analyses typically focus on case studies, especially if deployment conditions are climatologically anomalous. The NASA ACTIVATE (Aerosol Cloud meTeorology Interactions oVer the western ATlantic Experiment) mission was designed to overcome these limitations by operating semi-continuously over multiple months during multiple seasons. Flight plans were intentionally simple, repetitive, and systematic to provide a statistically robust dataset that could be utilized similarly to a ground-network. Here, we present an overview of three years (2020-2022) of aerosol measurements from the NASA HU-25 Falcon aircraft supporting the ACTIVATE mission. Data from 179 flights were considered, the majority of which were based at NASA Langley Research Center and flown locally over the Wester North Atlantic Ocean just east of Hampton, VA, USA. Airmass characteristics tended to transition from continental outflow near the coast to mostly marine over the open ocean. Seasonal, diurnal, spatial, and vertical trends in aerosol microphysical, optical, and chemical properties are discussed in the context of synoptic scale meteorology. Statistical benefits of the ACTIVATE sampling strategy are discussed with regard to future model evaluation.
Explore the source record for details and available documents.
HEP data-processing software must support the disparate physics needs of many experiments. For both collider and neutrino environments, HEP experiments typically use data-processing frameworks to manage the computational complexities of their large-scale data processing needs. Data-processing frameworks are being faced with new challenges this decade. The computing landscape has changed from the past three decades of homogeneous single-core x86 batch jobs running on grid sites. Frameworks must now work on a heterogeneous mixture of different platforms: multi-core machines, different CPU architectures, and computational accelerators; and different computing sites: grid, cloud, and high-performance computing. We describe these challenges in more detail and how frameworks may confront them. Given their historic success, frameworks will continue to be critical software systems that enable HEP experiments to meet their computing needs. Frameworks have weathered computing revolutions in the past; they will do so again with support from the HEP community
During the TRacking Aerosol and Convection interaction ExpeRiment (TRACER), a suite of aerosol and cloud measurements were made using the Texas A&M Rapid Onsite Atmospheric Measurement Van (ROAM-V) mobile instrument platform. Joint ROAM-V/radiosonde deployments focused on sampling polluted marine and continental air masses to understand summertime convection along and across the sea-breeze front as it propagated through Houston. The polluted marine air mass at Seawolf Park, Galveston, TX was defined by two characteristic aerosol populations. Although the background total particle concentrations averaged 2,500 cm −3 , the air mass was also influenced by frequent but irregular periods of ship emission with significantly higher aerosol concentrations at times exceeding 34,000 cm −3 . Ship emission influenced periods, typically lasting <10 min, contained smaller, less hygroscopic particles resulting in a 69% relative decline in cloud condensation nuclei (CCN) activated fraction at 1% supersaturation compared to background periods. Measurements in continental air masses northwest of Houston revealed an average aerosol concentration of 5,208 cm −3 with a κ value near 0.1 reasonably describing the CCN population, independent of particle size. In continental air masses, substantial differences in particle size and CCN activation over small distances (<42 miles between sites) suggest considerable site-to-site variability in addition to the expected day-to-day differences. This small-scale variability makes it difficult to generalize continental air mass aerosol properties. Both coastal and inland locations had effective ice nucleating particles, but inland deployments observed the warmest nucleation temperature at −15.6°C compared to −17.8°C close to the coast. These measurements can reduce uncertainties in regional convection allowing models to improve understanding of aerosol-cloud interactions.
NASA Langley Research Center is working on a continuous wave (CW) laser based remote sensing scheme for the detection of CO2 and O2 from space based platforms suitable for ACTIVE SENSING OF CO2 EMISSIONS OVER NIGHTS, DAYS, AND SEASONS (ASCENDS) mission. ASCENDS is a future space-based mission to determine the global distribution of sources and sinks of atmospheric carbon dioxide (CO2). A unique, multi-frequency, intensity modulated CW (IMCW) laser absorption spectrometer (LAS) operating at 1.57 micron for CO2 sensing has been developed. Effective aerosol and cloud discrimination techniques are being investigated in order to determine concentration values with accuracies less than 0.3%. In this paper, we discuss the demonstration of a PN code based technique for cloud and aerosol discrimination applications. The possibility of using maximum length (ML)-sequences for range and absorption measurements is investigated. A simple model for accomplishing this objective is formulated, Proof-of-concept experiments carried out using SONAR based LIDAR simulator that was built using simple audio hardware provided promising results for extension into optical wavelengths. Keywords: ASCENDS, CO2 sensing, O2 sensing, PN codes, CW lidar
A major component of California’s yearly precipitation comes from wintertime atmospheric river (AR) events which bring large amounts of moisture from the tropics up to the midlatitudes. Understanding these systems, specifically the effects of aerosol particles on precipitation associated with these storms, was a major focus of the 2015 Atmospheric Radiation Measurement (ARM) Cloud Aerosol Precipitation Experiment (ACAPEX), which was part of the wintertime CalWater 2015 campaign. The measurement campaign provided sampling platforms on four aircraft, including the ARM Aerial Facility G-1, as well as the NOAA Ronald H. Brown research vessel and at a ground station in Bodega Bay, CA. Measurements of ice nucleating particles (INPs) were made with the Colorado State University (CSU) Continuous Flow Diffusion Chamber (CFDC) aboard the G-1, and aerosol filters were collected on the G-1, at the Bodega Bay site and on the Ronald H. Brown for post-processing via immersion freezing in the CSU Ice Spectrometer. Aerosol composition was measured aboard the G-1 with the Aerosol Time-of-Flight Mass Spectrometer (ATOFMS). Here we present INP concentrations and aerosol chemical compositions during the course of the aircraft campaign. During the AR event we found that marine aerosol was the dominant aerosol type and that marine INPs were dominant at cloud activation temperatures, which is in stark contrast to the dominance of dust INPs during the AR events in the CalWater 2011 campaign.
This white paper presents a comprehensive methodology for assessing utilities’ cloud postures and frameworks. It aims to produce a roadmap and strategy for a cloud-enabled grid future, providing guidance for integrators, asset owners, and operators. Instead of offering a yes or no answer for cloud implementation, this framework offers strategic guidance on responsibly preparing for and deploying cloud solutions. This paper delves into cloud-service models pertinent to the electric sector, dissecting the shared responsibility model and elucidating what on-premise infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS) entail. A pivotal consideration within the context of the shared responsibility model is the allocation of responsibility for foundational security aspects—a decision that will be informed by a comprehensive risk assessment. The ensuing discussion will present a checklist of certifications necessary for a secure cloud transition, equipping utilities with the knowledge to navigate this digital transformation with confidence and with a strategic roadmap. Furthermore, the paper outlines gaps in understanding the U.S. government’s role in shaping technology development and responsible use. Its purpose is to aid decision-making by offering support for risk-informed solutions that benefit those managing assets and operating in the cloud environment. The primary objective is to enhance the resilience and future readiness of a decarbonized electric grid, with cloud solutions as one viable option. The paper synthesizes information on current and future grid architectures and applications, considering both conservative and progressive energy transitions, along with scalable and distributed computing considerations.
For improved understanding of chaotic processes and the diurnal cycle, an advanced GOES imager must also have the multi-spectral spectral bands used by low earth orbit (LEO) imagers, with on-orbit calibration for all bands. A synergy between GEO and LEO radiometry would enable earth system scientists to fuse the remote sensing data from all the spaceborne platforms. These additional radiometric capabilities are designed to observe important physical processes that vary rapidly and unpredicably: smoke, fires, precipitation, ozone, volcanic ash, cloud phase and height, and surface temperature. We believe the technology now exists to develop an imaging system that can meet future weather reporting and earth system science needs. To meet this need, we propose a design for a comprehensive geosynchronous atmospheric imager. This imager is envisioned to fly on a GOES-N class spacecraft, within the volume, weight and power constraints of a platform similar to GOES-N while delivering 100 times more data and radiometric quality than the GOES-N imager. The higher data rate probably requires its own ground station, which could serve as a systems prototype for NOAA's next generation of operational satellites. For operational compatibility, our proposed advanced GOES imaging system contains the GOES-R requirements as a subset, and the GOES-N imager capabilities (and the sounder's imaging channels) as a further subset.
The objective of the PICARD Program is to develop fieldable sensing platforms for the rapid chemical identification of aerosol particles in plumes. Standoff detection involves interrogating the aerosol cloud from a distance using optical methods and probing the signal returned from direct backscattering from the aerosol particles or transmitted through the plume after reflection from a retroreflector or surface of opportunity (SOO). The identification of chemical species, however, in aerosols is complicated by their complex compositions and morphologies, chemical interferants, and non-uniform particle sizes. To advance standoff detection of aerosols, modelling the infrared transmittance, reflectance and scattering spectra of aerosolized liquid and solid chemical compounds is required, and then testing that model experimentally via laboratory and field experiments. To perform accurate modeling, the infrared optical constants (n/k), i.e., the complex refractive index, of the compounds of interest are required. Thus, PNNL was tasked to provide the optical constants for a set of analytes relevant to the PICARD program. This report describes the experimental techniques used to derive the optical constants of both liquid and solid compounds using established “gold-standard” protocols, how the experimental data are processed to produce the wavenumber dependent optical constant vectors, and how the data are used in the aerosol absorption spectra modeling.
A system study of the potential of space technology to monitor climate changes and improve the understanding of the coupling beteen CO2 and cloud cover is summarized. The basis for the study is scientific data requirements pertinent to the U.S. Department of Energy's CO2 Research Program. The capabilities of space-based sensor systems are matched to meet these requirements. New sensor system concepts are identified, including a Space Shuttle-launched recalibration package to provide for continuity of measurement and recalibration between satellites, a high-orbit radiation budget satellite, or parallax sensor to measure cloud altitude, a passive method for the direct measurements of CO2 and a high-altitude powered platform to monitor select regional parameters. Space-based sensor systems that could be the development focus for the time frames of 0-5, 5-10 and 10-20 years are recommended.
Wear is unavoidable, whether it occurs in a nuclear facility or abridge on the highway its detection is crucial to preventing the failure of a system. While tools capable of detecting localized damage exist, industry lacks methods to evaluate the health of larger infrastructure. Routine inspection of nuclear facilities must be undertaken to ensure the safety of personnel and proper containment of nuclear materials. Most systems in a nuclear facility are designated as radioactive and confined spaces. To constantly monitor such systems in person would be hazardous but with a remote inspection tool it can be done in a safer and easier way. Lidar: Operates by emitting a laser pulse and looking for the reflected light. Allows user to gain 3D information of the environment. Project Objective: To develop an inspection tool for evaluating the health of large scale infrastructure using Lidar and machine learning. Autonomous Platform + Lidar + Data Fusion + Big Data. Use SLAM to build a 2D map of the environment. Point cloud created by transforming the points between reference frames. Point cloud to Surface: Inherent representation of surface geometry; Difficult to process with conventional techniques. Point cloud to Image: Easily processed by conventional machine learning algorithms. The quantity of data generated will necessitate a big data framework to manage the data for processing. Additional Sensors: Detecting the temperature (a) chemical makeup (b) and radiation emission (c) of a surface could assist in detecting problems. Human inspection can detect irregularities in both physical objects and their point-cloud representations. This is a good indication that mapping nuclear sites using Lidar will provide qualitative insight into the health of the facility. If deep learning can be used to automatically identify problem areas then the completed tool could become an invaluable asset.
NASA Langley Research Center is working on a continuous wave (CW) laser based remote sensing scheme for the detection of CO2 and O2 from space based platforms suitable for ACTIVE SENSING OF CO2 EMISSIONS OVER NIGHTS, DAYS, AND SEASONS (ASCENDS) mission. ASCENDS is a future space-based mission to determine the global distribution of sources and sinks of atmospheric carbon dioxide (CO2). A unique, multi-frequency, intensity modulated CW (IMCW) laser absorption spectrometer (LAS) operating at 1.57 micron for CO2 sensing has been developed. Effective aerosol and cloud discrimination techniques are being investigated in order to determine concentration values with accuracies less than 0.3%. In this paper, we discuss the demonstration of a pseudo noise (PN) code based technique for cloud and aerosol discrimination applications. The possibility of using maximum length (ML)-sequences for range and absorption measurements is investigated. A simple model for accomplishing this objective is formulated, Proof-of-concept experiments carried out using SONAR based LIDAR simulator that was built using simple audio hardware provided promising results for extension into optical wavelengths.