Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud computing, data cloud”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

DOE Repository Metadata Profile (DRMP): A Metadata Framework for Advancing Interoperability and AI Readiness Across Scientific Repositories

The Department of Energy (DOE) funds a diverse and distributed ecosystem of repositories that steward scientific data, publications, and software across its research programs, user facilities, and national laboratories. While significant progress has been made in standardizing dataset-level metadata, the metadata describing repositories themselves (their identity, governance, access interfaces, policies, and technical capabilities) remains inconsistent and fragmented across DOE-funded systems. This variability limits discoverability, interoperability, automated validation, and AI-driven analysis, all of which are increasingly essential for modern scientific workflows. To address this gap, the DOE Data Curation Working Group (DCWG) developed the DOE Repository Metadata Profile (DRMP). The DRMP is a practical, community-driven framework that defines how repositories can describe themselves in a consistent, machine-actionable, and scalable manner. The DRMP is not a new metadata schema. Instead, it is a mapping profile and structured element set capturing the essential characteristics of DOE repositories. It harmonizes repository-level metadata across six widely adopted community schemas: RE3Data; DCAT-US v3; Schema.org; Dublin Core; DataCite 4.6; and PREMIS 3.0. This harmonization eliminates reinvention and enables interoperability within DOE and across the broader scientific ecosystem. A core objective of the DRMP is to reduce burden on repositories by allowing them to reuse their existing metadata through a Rosetta-style crosswalk rather than redesigning local implementations. The profile introduces a three-level conformance model that supports incremental adoption: • Level 1 – Minimum Viable Record (MVR): foundational identification elements required for workflows, project registration, and basic repository presence. • Level 2 – Interoperable: structured metadata enabling alignment with national and international discovery systems. • Level 3 – AI-Ready: enhanced provenance, policy transparency, fixity, semantic context, and capabilities that support automated reasoning, model training governance, and machine-assisted curation. To support implementation, the DRMP includes JSON Schema definitions, OpenAPI patterns, and MCP templates that allow repositories to publish machine-readable metadata directly within existing platforms. These resources are modular and lightweight, enabling adoption without major architectural change. Adopting the DRMP enables repositories to: • Enhance discoverability and interoperability by aligning identifiers, classifications, and descriptive elements across widely used schema standards. • Support federated discovery and cross-registration across DOE systems, Data.gov, and international catalogs. • Enable AI agents and workflow orchestration systems to interpret repository-level metadata within the American Science Cloud (AmSC) through Model Context Protocol (MCP)-based context publication. • Demonstrate alignment with DOE’s open science, stewardship, and FAIR data priorities. This guidance represents a community-driven step forward. Through voluntary adoption and continued feedback, the DRMP advances a cohesive, machine-actionable description of DOE repositories that supports FAIR data practices, preparing the infrastructure for AI-enabled research, and strengthening the discoverability and reuse of DOE’s scientific outputs.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2013

Novel biofuel formulations for enhanced vehicle performance

This interdisciplinary research program at Michigan State University, in collaboration with Ford Motor Company, has explored the application of tailored or designed biofuels for enhanced vehicle performance and reduced emissions. The project has included a broad range of experimental research, from chemical and biological formation of advanced biofuel components to multicylinder engine testing of blended biofuels to determine engine performance parameters. In addition, the project included computation modeling of biofuel physical and combustion properties, and simulation of advanced combustion modes in model engines and in single cylinder engines. Formation of advanced biofuel components included the fermentation of five-carbon and six-carbon sugars to n-butanol and to butyric acid, two four-carbon building blocks. Chemical transformations include the esterification of the butyric acid produced to make butyrate esters, and the esterification of succinic acid with n-butanol to make dibutyl succinate (DBS) as attractive biofuel components. The conversion of standard biodiesel, made from canola or soy oil, from the methyl ester to the butyl ester (which has better fuel properties), and the ozonolysis of biodiesel and the raw oil to produce nonanoate fuel components were also examined in detail. Physical and combustion properties of these advanced biofuel components were determined during the project. Physical properties such as vapor pressure, heat of evaporation, density, and surface tension, and low temperature properties of cloud point and cold filter plugging point were examined for pure components and for blends of components with biodiesel and standard petroleum diesel. Combustion properties, particularly emission delay that is the key parameter in compression ignition engines, was measured in the MSU Rapid Compression Machine (RCM), an apparatus that was designed and constructed during the project simulating the compression stroke of an internal combustion engine under highly instrumented conditions. Simulation of and experimentation on combustion in single and multicylinder engines was carried out in detail throughout the project. The combustion behavior of biofuel blends neat and in petroleum were characterized in the MSU optical engine, in part to validate results obtained in the RCM and to provide data for comparison with simulations. Simulation of in- cylinder, low-temperature combustion included development of an extensive fuel injection model that included fuel spray breakup, evaporation, and ignition, along with prediction of cylinder temperature, pressure, and work produced. Single cylinder and multicylinder engine tests under advanced low-temperature combustion conditions conducted at Ford Motor Company validated experimental and simulation results obtained in the MSU engine and in MSU simulations. Single cylinder engine tests of an advanced biofuel containing biodiesel and dibutyl succinate, carried out under low-temperature combustion conditions, showed similar power generation and gas-phase emissions (CO, HC, NOx), but a reduction in particulates of as much as 60% relative to neat biodiesel and 95% relative to petroleum diesel at the same operating conditions. This remarkable finding suggests that biofuels may be able to play a role in eliminating the need for particulate removal systems in diesel vehicles. The multicylinder engine tests at Ford, carried out using butyl nonanoate as an advanced biofuel, also gave promising results, showing a strong decline in particulate emissions and simultaneously a modest decrease in NOx emissions relative to standard petroleum diesel at the same conditions. In summary, this project has shown that advanced biofuels and their blends are capable of maintaining performance while reducing emissions, particularly particulates (soot), in 3 compression ignition engines. The interdisciplinary nature of biofuel production and testing has identified fuel properties that are capable of producing such performance, thus providing direction for the implementation of renewable fuels for U.S. transportation. The testing and simulation studies have deepened our understanding of combustion 1) by advancing the rigor with which simulations can be carried out and 2) by illustrating that differences in biofuel and petroleum fuel properties can be used to predict differences in combustion behavior in engines. The future viability of biofuels for compression ignition (diesel) engines is now subject to economic (cost) uncertainty more so than to technical barriers, as the advanced biofuel blends developed here can improve cold-weather fuel properties, provide similar engine performance, and reduce emissions.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2016

Combined High Spectral Resolution Lidar and Millimeter Wavelength Radar Measurement of Ice Crystal Precipitation

The goal of this research has been to improve measurements of snowfall using a combination of millimeter-wavelength radar and High Spectral Resolution Lidar (HSRL) Observations. Snowflakes are large compared to the 532nm HSRL wavelength and small compared to the 3.2 and 8.6 mm wavelength radars used in this study. This places the particles in the optical scattering regime of the HSRL, where extinction cross-section is proportional to the projected area of the particles, and in the Rayleigh regime for the radar, where the backscatter cross-section is proportional to the mass-squared of the particles. Forming a ratio of the radar measured cross-section to the HSRL measured cross section eliminates any dependence on the number of scattering particles, yielding a quantity proportional to the average mass-squared of the snowflakes over the average area of the flakes. Using simultaneous radar measurements of particle fall velocities, which are dependent particle mass and cross-sectional area it is possible to derive the average mass of the snow flakes, and with the radar measured fall velocities compute the snowfall rate. Since this retrieval requires the optical extinction cross-section we began by considering errors this quantity. The HSRL is particularly good at measuring the backscatter cross-section. In previous studies of snowfall in the high Arctic were able to estimate the extinction cross-section directly as a fixed ratio to the backscatter cross-section. Measurements acquired in the STORMVEX experiment in Colorado showed that this approach was not valid in mid-latitude snowfalls and that direct measurement of the extinction cross-section is required. Attempts to measure the extinction directly uncovered shortcomings in thermal regulation and mechanical stability of the newly deployed DOE HSRL systems. These problems were largely mitigated by modifications installed in both of the DOE systems. We also investigated other sources of error in the HSRL direct measurement of extinction (see appendix II of this report). We also developed improved algorithms to extract extinction from the HSRL data. These have been installed in the standard HSRL data processing software and are now available to all users of HSRL data. Validation of snowfall measurements has proven difficult due to the unreliability of conventional snowfall measurements coupled with the complexity of considering the vast variety of snowflake geometries. It was difficult to tell how well the algorithm’s approach to accommodating differences in snowflakes was working without good measurements for comparison. As a result, we decided to apply this approach to the somewhat simpler, but scientifically important, problem of drizzle measurement. Here the particle shape is known and the conventional measurement are more reliable. These algorithms where successfully applied to drizzle data acquired during the ARM MAGIC study of marine stratus clouds between California and Hawaii (see Appendix I). This technique is likely to become a powerful tool for studying lifetime of the climatically important marine stratus clouds.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Impact of dry intrusion events on the composition and mixing state of particles during the winter Aerosol and Cloud Experiment in the Eastern North Atlantic (ACE-ENA)

Abstract. Long-range transport of continental emissions has a far-reaching influence over remote regions, resulting in substantial change in the size, morphology, and composition of the local aerosol population and cloud condensation nuclei (CCN) budget. Here, we investigate the physicochemical properties of atmospheric particles collected on board a research aircraft flown over the Azores during the winter 2018 Aerosol and Cloud Experiment in the Eastern North Atlantic (ACE-ENA) campaign. Particles were collected within the marine boundary layer (MBL) and free troposphere (FT) after long-range atmospheric transport episodes facilitated by dry intrusion (DI) events. Chemical and physical properties of individual particles were investigated using complementary capabilities of computer-controlled scanning electron microscopy and X-ray spectromicroscopy to probe particle external and internal mixing state characteristics. Furthermore, real-time measurements of aerosol size distribution, cloud condensation nuclei (CCN) concentration, and back-trajectory calculations were utilized to help bring into context the findings from offline spectromicroscopy analysis. While carbonaceous particles were found to be the dominant particle type in the region, changes in the percent contribution of organics across the particle population (i.e., external mixing) shifted from 68 % to 43 % in the MBL and from 92 % to 46 % in FT samples during DI events. This change in carbonaceous contribution is counterbalanced by the increase in inorganics from 32 % to 57 % in the MBL and 8 % to 55 % in FT. The quantification of the organic volume fraction (OVF) of individual particles derived from X-ray spectromicroscopy, which relates to the multi-component internal composition of individual particles, showed a factor of 2.06 ± 0.16 and 1.11 ± 0.04 increase in the MBL and FT, respectively, among DI samples. We show that supplying particle OVF into the κ-Köhler equation can be used as a good approximation of field-measured in situ CCN concentrations. We also report changes in the κ values in the MBL from κMBL, non-DI=0.48 to κMBL, DI=0.41, while changes in the FT result in κFT, non-DI=0.36 to κFT, DI=0.33, which is consistent with enhancements in OVF followed by the DI episodes. Our observations suggest that entrainment of particles from long-range continental sources alters the mixing state population and CCN properties of aerosol in the region. The work presented here provides field observation data that can inform atmospheric models that simulate sources and particle composition in the eastern North Atlantic.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

Major improvements on the longwave radiative interactions between surface and clouds in the polar regions in atmospheric global circulation model

Motivations: One of the most important tasks in the study of physical and socioeconomic aspects of climate change is the simulation of polar climate and how it changes in response to global warming. One import component of the polar energy budget (and thus of polar climate) is the radiative interaction between surface and clouds. As of today, the vast majority of state-of-the-art global climate models (GCMs), including all three US flagship GCMs (NCAR CESM, GFDL CM3, and GISS E2-H/E2-R), still assume constant surface emissivity overall spectral bands in their longwave radiation treatment. Similarly, a majority of state-of-the-art GCMs assume non-scattering clouds in the longwave spectrum, including both the NCAR CESM and GFDL CM3. The issues of ignoring spectral variation in surface emissivity and scattering by clouds manifest themselves in the far-infrared (IR) over the polar continents because (1) there is so little water vapor in such areas that surface far-IR emission can reach clouds; and (2) the imaginary part of the refractive index of ice has a local minimum over 350-550 cm-1, which implies possible strong scattering effects over this spectral region both within and between the snow surface and ice clouds. Our sensitivity studies using CloudSat-retrieved hydrometeor profiles over the high-elevation Antarctic continent show that, in winter, an appropriate inclusion of snow surface spectral emissivity and ice cloud scattering in radiative transfer calculations can noticeably reduce the monthly-mean surface net downward far-IR flux and net atmospheric far-IR emission over the entire region. For the far-IR alone, the magnitude of these effects is ~1Wm-2 or even larger. Projection Objectives: We propose to carry out the following studies: (1) Develop or update schemes that can be used in the CESM to compute the optical properties of the snow surface and ice clouds consistently across the longwave and the shortwave spectrum; (2) Modify the longwave radiation scheme in the CESM such that it can incorporate surface spectral emissivity and scattering of clouds; (3) Validate schemes developed for (1) and (2) against measurements or benchmark models; (4) Evaluate the effects of these new treatments on the simulated polar energy budget and climate by the CESM. Technical Approaches:1.Use a detailed snowpack radiative transfer model to simulate spectrally resolved emissivity of snow surfaces as a function of snow effective radius. The model will be validated against available high spectral-resolution measurements of snow spectral emissivity (ASTER spectra library v2.0). Currently the CESM prognoses vertically-resolved snow effective radius, and snow shortwave albedo is parameterized based on snow effective radius. Our treatment of snow spectral emissivity here will be consistent with the treatment of snow albedo in the shortwave and seamlessly connected with the snow physical module in the CESM. 2. Update the parameterization of ice cloud optical properties in the CESM, for both longwave and shortwave, based on the latest ice-cloud scattering database developed by co-I Prof. Yang. 3. Modify RRTMG_LW, the current longwave radiation scheme in the CESM, to include the surface spectral emissivity and the scattering of clouds in the longwave calculation. Validate modified scheme against a line-by-line radiative transfer model with multiple scattering capability (LBLRTM/CHARTS). 4. Implement products from Steps 1-3 into the CESM and conduct simulations to evaluate their effects on the simulated polar climate and energy budget. These runs will be compared to control runs and evaluated using routine model-observation diagnostics, including data from the ARM NSA (North Slope of Alaska) site. Moreover, a novel aspect in this step is that we will evaluate the simulated longwave band-by-band fluxes and cloud radiative effect against observational counterparts over both polar continents, using a product recently derived by the lead PI Prof. Huang from collocated AIRS and CERES satellite observations from 2002 to 2013. Benefits and outcomes: This project will significantly improve the longwave radiation scheme in the CESM and its successors and provide more realistic treatment of radiative interactions between polar surfaces and clouds. In the future, it will also be straightforward to extend this framework to include spectral emissivity of all surface types. This is responsive to identified DoE Earth System Modeling Science Drivers to improve the performance of the fully coupled climate system modeling and to help address questions on how rapid changes in cryospheric systems interact with the climate system.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Custom surface reflectance, shade mask, and equivalent water thickness maps for the Colorado Headwaters Ecological Spectroscopy Study (2025)

This dataset contains land surface reflectance estimates and additional derived products generated from NEON Imaging Spectrometer (NIS) data collected in the Upper Gunnison river basin during June and July of 2025. Data was collected over three domains: the Upper East River (CRBU), Almont Triangle (ALMO), and the Upper Taylor Basin (UPTA). These products were derived from radiance and LiDAR data collected by the NEON Airborne Observation Platform (AOP) campaign funded by the Colorado Headwaters Ecological Spectroscopy Study (CHESS) (doi:10.15485/3017965). Products include per-pixel surface reflectance (rfl) and reflectance uncertainty (rfl_unc), observational data (obs), canopy equivalent water thickness (ewt), and shade masks. Atmospheric correction was performed per flightline using the ISOFIT (Imaging Spectrometer Optimal FITting) optimal estimation framework to estimate surface reflectance and the associated per-band reflectance uncertainty. Reflectance retrievals achieved a mean absolute error of 1.5% across diverse validation surfaces (see validation report.pdf). Equivalent water thickness was calculated from surface reflectance using the Beer–Lambert absorption of liquid water. Shade masks were generated based on the geometry between the sun angle, ground surface, and sensor at the time of flight. Data products are provided per-flightline and as mosaics for each domain. Flightline data products are provided as ENVI-formatted binary files (rfl, rfl_unc, ewt) and GeoTIFFs (shade). Reflectance and uncertainty mosaics are provided as tiled NetCDFs, while all other mosaicked products are provided as cloud-optimized GeoTIFFs. These formats are supported by common geospatial software (e.g., QGIS, ArcGIS, ENVI) and programmatic libraries in Python (e.g., rasterio, xarray, spectral, netCDF4) and R (e.g., terra, ncdf4). Processing workflows were designed to be equivalent to those used to generate the 2018 CHESS campaign airborne imaging spectroscopy data products (doi:10.15485/3013527). All outputs were co-registered to a common spatial grid to support time series analyses. CHESS Project Description: The Colorado Headwaters Ecological Spectroscopy Study (CHESS) comprised a multi-week airborne remote sensing and field observation campaign in the Upper Gunnison Basin, Colorado, conducted in June and July of 2025. Airborne remote sensing was conducted by the National Ecological Observatory Network Airborne Observation Platform (NEON AOP), concurrent with a field campaign run by the Rocky Mountain Biological Laboratory (RMBL), the Lawrence Berkeley National Laboratory (LBNL) and SLAC National Accelerator Laboratory Watershed Function Science Focus Area (SFA), and NASA-JPL (Jet Propulsion Laboratory) Earth Surface Mineral Dust Source Investigation (EMIT) program. Between June 10 and July 18, 2025, the NEON AOP flight team collected high-resolution aerial imaging spectroscopy and Light Detection and Ranging (LiDAR) data over three domains: the Upper East River (CRBU), Almont Triangle (ALMO), and the Upper Taylor Basin (UPTA). In coordination with the flights, a field campaign acquired ground-truth observations, including observations of vegetation composition, foliar traits, forest demography, and subsurface properties in 18 core sampling areas within the domains. Additional surface water observations were taken at over 380 point locations. All CHESS campaign datasets can be found within the CHESS ESS-DIVE data portal: https://data.ess-dive.lbl.gov/portals/chess. Funding Acknowledgment: Data acquisition was performed under a grant from the National Aeronautics and Space Administration (80NSSC24K1005). Computational research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration (80NM0018D0004) and was funded by EMIT Extended Mission Phase E Science.

2018 NEON and 2025 CHESS Campaigns↗
DOE OSTI2023

Smart Mobility in the Cloud: Enabling Real-Time Situational Awareness and Cyber-Physical Control Through a Digital Twin for Traffic

This article presents the design, implementation, and use cases of the Chattanooga Digital Twin (CTwin) towards the vision for next-generation smart city applications for urban mobility management. CTwin is an end-to-end web-based platform that incorporates various aspects of the decision-making process for optimizing urban transportation systems in Chattanooga, Tennessee, to reduce traffic congestion, incidents, and vehicle fuel consumption. The platform serves as a cyberinfrastructure to collect and integrate multi-domain urban mobility data from various online repositories and Internet of Things (IoT) sensors, covering multiple urban aspects (e.g., traffic, natural hazards, weather, and safety) that are relevant to urban mobility management. The platform enables advanced capabilities for: (a) real-time situational awareness on traffic and infrastructure conditions on highways and urban roads, (b) cyber-physical control for optimizing traffic signal timing, and (c) interactive visual analytics on big urban mobility data and various metrics for traffic prediction and transportation performance evaluation. The platform is designed using a multi-level componentization paradigm and is implemented using modular and adaptive architecture, rendering it as a generalizable and extendable prototype for other urban management applications. We present several use cases to demonstrate CTwin's core capabilities for supporting decision-making in smart urban mobility management.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2025

Reducing Data Center Peak Cooling Demand and Energy Costs with Underground Thermal Energy Storage (UTES)

By recent estimates, data center energy demands are projected to consume between 6.7% and 12% of U.S. annual electricity generation by the year 2028, driven primarily by expanded demands from cloud services, big data analytics, and Artificial Intelligence (AI) (Shehabi et al., 2024). As much as 40% of data center total energy consumption are loads associated with the site infrastructure cooling systems, and these are often highly water consumptive (Aljbour et al., 2024). For energy system planners, this presents significant challenges to meeting and managing the anticipated loads, and especially the peak loads of projected data center deployments. Geothermal technologies offer two unique solutions to these challenges: 1) by serving loads through the deployment of new conventional and/or next-generation geothermal power technologies such as EGS and 2) through an often-overlooked opportunity to reduce data center peak cooling loads. The latter is the focus of this paper which explores Cold Underground Thermal Energy Storage ("Cold UTES") as an emerging industrial-scale geothermal cooling solution. This cooling solution is energy efficient, non-water-consumptive, and utilizes long duration energy storage (LDES) on both diurnal and seasonal time scales. Cold UTES has the potential to also function as a virtual power plant (VPP). The US Department of Energy's Geothermal Technologies Office is supporting R&D to understand the grid and system-wide value, costs, and impacts of deploying this emergent cooling solution at scale.

AI↗
DOE OSTI2025

Exploring the interaction between the MW and LMC with a large sample of blue horizontal branch stars from the DESI survey

The Large Magellanic Cloud (LMC) is a Milky Way (MW) satellite that is massive enough to gravitationally attract the MW disc and inner halo, causing significant motion of the inner MW with respect to the outer halo. In this work, we probe this interaction by constructing a sample of 9866 blue horizontal branch (BHB) stars with radial velocities from the DESI spectroscopic survey out to 120 kpc from the Galactic centre. This is the largest spectroscopic set of BHB stars in the literature to date, and it contains four times more stars with Galactocentric distances beyond 50 kpc than previous BHB catalogues. Using the DESI BHB sample combined with SDSS BHBs, we measure the bulk radial velocity of stars in the outer halo and observe that the velocity in the Southern Galactic hemisphere is different by 3.7σ from the North. Modelling the projected velocity field shows that its dipole component is directed at a point 22 deg away from the LMC along its orbit, which we interpret as the travel direction of the inner MW. The velocity field includes a monopole term that is –24 km s –1 ⁠, which we refer to as compression velocity. This velocity is significantly larger than predicted by the current models of the MW and LMC interaction. This work uses DESI data from its first 2 yr of observations, but we expect that with upcoming DESI data releases, the sample of BHB stars will increase and our ability to measure the MW–LMC interaction will improve significantly.

Galaxy: evolution↗
DOE OSTI2021

A mathematical framework for ejecta cloud dynamics with application to source models and piezoelectric mass measurements

We present a mathematical framework for describing the dynamical evolution of an ejecta cloud generated by a generic ejecta source model. We consider a piezoelectric sensor fielded in the path of an ejecta cloud, for experimental configurations in which the ejecta are created at a singly shocked planar surface and fly ballistically through vacuum to the stationary sensor. To do so, we introduce the concept of a time- and velocity-dependent ejecta “areal mass function.” We derive expressions for the analytic (“true”) accumulated ejecta areal mass at the sensor and the measured (“inferred”) value obtained via the standard method for analyzing piezoelectric voltages. In this way, we derive an exact expression and upper bound for the error imposed upon a piezoelectric ejecta mass measurement (in a perfect system) by the assumption of instantaneous creation, which is commonly required for momentum diagnostic analyses. This error term is zero for truly instantaneous source models; otherwise, the standard piezoelectric analysis is guaranteed to overestimate the true mass. When combined with a piezoelectric dataset, this framework provides a unique solution for the ejecta particle velocity distribution, subject to the assumptions inherent in the data analysis. The framework also leads to strong boundary conditions that any ejecta source model must satisfy in order to be consistent with apparently global properties of piezoelectric measurements from a wide range of experiments. We demonstrate this methodology by applying it to the Richtmyer–Meshkov instability+self-similar velocity distribution ejecta source model currently under development at Los Alamos National Laboratory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A National Infrastructure for Artificial Intelligence on the Grid (NI4AI) (Final Scientific/Technical Report)

Electric utilities have traditionally taken a very pragmatic yet myopic approach with grid sensors and the resulting collected data. Sensors are purchased and deployed to solve a specific, known problem that has risen to sufficient awareness as to justify the effort of deploying sensors and the needed capital investment. This sensor data flows into proprietary software packages with limited functionality intended only to address the initial problem. This approach aligns with the financial incentives of the utility to deploy capital into fixed hardware assets for which the corporations earn a rate of return. This mentality stands in stark contrast to the big data revolution that started nearly 25 years ago with the rise of Google. In this worldview, data is a fundamental business asset; successful organizations collect, store, explore, merge, and exploit as much data as possible to not only solve problems well understood today but also to tackle new problems that will inevitably rise tomorrow. The ARPA-E Open Innovation 2018 project entitled A National Infrastructure for Artificial Intelligence on the Grid or NI4AI for short was designed to demonstrate this alternative paradigm for using data. To do this, the project was composed of three key thrust areas. The first major component deployed a variety of high-frequency grid sensors and captured terabytes of both wide-scale and localized grid measurements, generating high-value datasets for grid research and algorithm development. The second aspect made available PingThings’ PredictiveGridTM, a horizontally scalable, cloud-based data management and AI platform built for time series data to explore and exploit the collected data. Finally, the project fostered a diverse and open research community composed of experts from numerous fields through focused educational content, code sharing, and data science competitions. Shifting away from “single use” sensors and closed data silos within electric utilities is a major benefit to the public at large. This legacy approach to data is incredibly (1) capital intensive (new sensors must be deployed for each new problem and problems tend to arise continuously) and (2) painfully slow (new problems must be identified first and then new sensors must be deployed to collect data to begin to address the issue). The transition to a carbon neutral grid requires a massive transformation of the existing grid infrastructure and will continue to challenge the legacy grid in unforeseen ways. The only way to make the energy transition cost effective is for utilities to abandon this dated data paradigm and adopt more contemporary approaches. NI4AI has shown that it is technically possible and economically feasible to ingest, explore, and exploit grid data collected from even very high frequency sensing, such as continuous point on wave sensors collecting measurements 10,000 times a second. In fact, the PredictiveGrid platform used is commercially available and deployed at several utilities in the United States. Project accomplishments were numerous and included (1) making available a state of the art time series platform to the community, (2) collecting over 520 streams of time series data from grid sensors totaling over 1 trillion grid measurements, and (3) developing and nurturing a community within the industry focused on the use of data to create value for utilities and, ultimately, end consumers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

A Public-Private-Acadmic Partnership to Advance Solar Power Forecasting

The National Center for Atmospheric Research (NCAR) is pleased to have led a partnership to advance the state-of-the-science of solar power forecasting by designing, developing, building, deploying, testing, and assessing the SunCast™ Solar Power Forecasting System. The project has included cutting edge research, testing in several geographically- and climatologically-diverse high penetration solar utilities and Independent System Operators (ISOs), and wide dissemination of the research results to raise the bar on solar power forecasting technology. The partners include three other national laboratories, six universities, and industry partners. This public-private-academic team has worked in concert to perform use-inspired research to advance solar power forecasting through cutting-edge research to advance both the necessary forecasting technologies and the metrics for evaluating them. The project has culminated in a year-long, full-scale demonstration of provide irradiance and power forecasts to utilities and ISOs to use in their operations. The project focused on providing elements of a value chain, beginning with the weather that causes a deviation from clear sky irradiance and progresses through monitoring and observations, modeling, forecasting, dissemination and communication of the forecasts, interpretation of the forecast, and through decision-making, which produces outcomes that have an economic value. The system has been evaluated using metrics developed specifically for this project, which has provided rich information on model and system performance. Research was accomplished on the very short range (0-6 hours) Nowcasting system as well as on the longer term (6-72 hour) forecasting system. The shortest range forecasts are based on observations in the field. The shortest range system, built by Brookhaven National Laboratory (BNL) and based on Total Sky Imagers (TSIs) is TSICast, which operates on the shortest time scale with a latency of only a few minutes and forecasts that currently go out to about 15 min. This project has facilitated research in improving the hardware and software so that the new high definition cameras deployed at multiple nearby locations allow discernment of the clouds at varying levels and advection according to the winds observed at those levels. Improvements over “smart persistence” are about 29% for even these very short forecasts. StatCast is based on pyranometer data measured at the site as well as concurrent meteorological observations and forecasts. StatCast is based on regime-dependent artificial intelligence forecasting techniques and has been shown to improve on “smart persistence” forecasts by 15-50%. A second category of short-range forecasting systems employ satellite imagery and use that information to discern clouds and their motion, allowing them to project the clouds, and the resulting blockage of irradiance, in time. CIRACast (the system produced by the Cooperative Institute for Atmospheric Research [CIRA] at Colorado State University) was already one of the more advanced cloud motion systems, which is the reason that team was brought to this project. During the project timeframe, the CIRA team was able to advance cloud shadowing, parallax removal, and implementation of better advecting winds at different altitudes. CIRACast shows generally a 25-40% improvement over Smart Persistence between sunrise and approximately 1600 UTC (Coordinated Universal Time) . A second satellite-based system, MADCast (Multi-sensor Advective Diffusive foreCast system), assimilates data from multiple satellite imagers and profilers to assimilate a fully three-dimensional picture of the cloud into the dynamic core of WRF. During 2015, MADCast (provided at least 70% improvement over Smart Persistence, with most of that skill being derived during partly cloudy conditions. That allows advection of the clouds via the Weather Research and Forecasting (WRF) model dynamics directly. After WRF-Solar™ showed initial success, it was also deployed in nowcasting mode with coarser runs out to 6 hours made hourly. It provided improvements on the order of 50-60% over Smart Persistence for forecasts up to 1600 UTC. The advantages of WRF-Solar-Nowcasting and MADCast were then blended to develop the new MAD-WRF model that incorporates the most important features of each of those models, both assimilating satellite cloud fields and using WRF-So far physics to develop and dissipate clouds. MAE improvements for MAD-WRF for forecasts from 3-6 hours are improved over WRF-Solar-Now by 20%. While all the Nowcasting system components by themselves provide improvement over Smart Persistence, the largest benefit is derived when they are smartly blended together by the Nowcasting Integrator to produce an integrated forecast. The development of WRF-Solar™ under this project has provided the first numerical weather prediction (NWP) model specifically designed to meet the needs of irradiance forecasting. The first augmentation improved the solar tracking algorithm to account for deviations associated with the eccentricity of the Earth’s orbit and the obliquity of the Earth. Second, WRF-Solar™ added the direct normal irradiance (DNI) and diffuse (DIF) components from the radiation parameterization to the model output. Third, efficient parameterizations were implemented to either interpolate the irradiance in between calls to the expensive radiative transfer parameterization, or to use a fast radiative transfer code that avoids computing three-dimensional heating rates but provides the surface irradiance. Fourth, a new parameterization was developed to improve the representation of absorption and scattering of radiation by aerosols (aerosol direct effect). A fifth advance is that the aerosols now interact with the cloud microphysics, altering the cloud evolution and radiative properties, an effect that has been traditionally only implemented in atmospheric computationally costly chemistry models. A sixth development accounts for the feedbacks that sub-grid scale clouds produce in shortwave irradiance as implemented in a shallow cumulus parameterization Finally, WRF-Solar™ also allows assimilation of infrared irradiances from satellites to determine the three dimensional cloud field, allowing for an improved initialization of the cloud field that increases the performance of short-range forecasts. We find that WRF-Solar™ can improve clear sky irradiance prediction by 15-80% over a standard version of WRF, depending on location and cloud conditions. In a formal comparison to the NAM baseline, WRF-Solar™ showed improvements in the Day-Ahead forecast of 22-42%. The SunCast™ system requires substantial software engineering to blend all of the new model components as well as existing publically available NWP model runs. To do this we use an expert system for the Nowcasting blender and the Dynamic Integrated foreCast (DICast®) system for the NWP models. These two systems are then blended, we use an empirical power conversion method to convert the irradiance predictions to power, then apply an analog ensemble (AnEn) approach to further tune the forecast as well as to estimate its uncertainty. The AnEn module decreased RMSE (root mean squared error) by 17% over the blended SunCast™ power forecasts and provided skill in the probabilistic forecast with a Brier Skill Score of 0.55. In addition, we have also developed a Gridded Atmospheric Forecast System (GRAFS) in parallel, leveraging cost share funds. An economic evaluation based on Production Cost Modeling in the Public Service Company of Colorado showed that the observed 50% improvement in forecast accuracy will save their customers $819,200 with the projected MW deployment for 2024. Using econometrics, NCAR has scaled this savings to a national level and shown that an annual expected savings for this 50% forecast error reduction ranges from $11M in 2015 to $43M expected in 2040 with increased solar deployment. This amounts to a $455M discounted savings over the 26 year period of analysis.

14 SOLAR ENERGY↗
DOE OSTI2013

Automated Cache Performance Analysis And Optimization

While there is no lack of performance counter tools for coarse-grained measurement of cache activity, there is a critical lack of tools for relating data layout to cache behavior to application performance. Generally, any nontrivial optimizations are either not done at all, or are done ”by hand” requiring significant time and expertise. To the best of our knowledge no tool available to users measures the latency of memory reference instructions for particular addresses and makes this information available to users in an easy-to-use and intuitive way. In this project, we worked to enable the Open|SpeedShop performance analysis tool to gather memory reference latency information for specific instructions and memory addresses, and to gather and display this information in an easy-to-use and intuitive way to aid performance analysts in identifying problematic data structures in their codes. This tool was primarily designed for use in the supercomputer domain as well as grid, cluster, cloud-based parallel e-commerce, and engineering systems and middleware. Ultimately, we envision a tool to automate optimization of application cache layout and utilization in the Open|SpeedShop performance analysis tool. To commercialize this software, we worked to develop core capabilities for gathering enhanced memory usage performance data from applications and create and apply novel methods for automatic data structure layout optimizations, tailoring the overall approach to support existing supercomputer and cluster programming models and constraints. In this Phase I project, we focused on infrastructure necessary to gather performance data and present it in an intuitive way to users. With the advent of enhanced Precise Event-Based Sampling (PEBS) counters on recent Intel processor architectures and equivalent technology on AMD processors, we are now in a position to access memory reference information for particular addresses. Prior to the introduction of PEBS counters, cache behavior could only be measured reliably in the aggregate across tens or hundreds of thousands of instructions. With the newest iteration of PEBS technology, cache events can be tied to a tuple of instruction pointer, target address (for both loads and stores), memory hierarchy, and observed latency. With this information we can now begin asking questions regarding the efficiency of not only regions of code, but how these regions interact with particular data structures and how these interactions evolve over time. In the short term, this information will be vital for performance analysts understanding and optimizing the behavior of their codes for the memory hierarchy. In the future, we can begin to ask how data layouts might be changed to improve performance and, for a particular application, what the theoretical optimal performance might be. The overall benefit to be produced by this effort was a commercial quality easy-to- use and scalable performance tool that will allow both beginner and experienced parallel programmers to automatically tune their applications for optimal cache usage. Effective use of such a tool can literally save weeks of performance tuning effort. Easy to use. With the proposed innovations, finding and fixing memory performance issues would be more automated and hide most to all of the performance engineer expertise ”under the hood” of the Open|SpeedShop performance tool. One of the biggest public benefits from the proposed innovations is that it makes performance analysis more usable to a larger group of application developers. Intuitive reporting of results. The Open|SpeedShop performance analysis tool has a rich set of intuitive, yet detailed reports for presenting performance results to application developers. Our goal was to leverage this existing technology to present the results from our memory performance addition to Open|SpeedShop. Suitable for experts as well as novices. Application performance is getting more difficult to measure as the hardware platforms they run on become more complicated. This makes life difficult for the application developer, in that they need to know more about the hardware platform, including the memory system hierarchy, in order to understand the performance of their application. Some application developers are comfortable in that sceario, while others want to do their scientific research and not have to understand all the nuances in the hardware platform they are running their application on. Our proposed innovations were aimed to support both experts and novice performance analysts. Useful in many markets. The enhancement to Open|SpeedShop would appeal to a broader market space, as it will be useful in scientific, commercial, and cloud computing environments. Our goal was to use technology developed initially at the and Lawrence Livermore National Laboratory combined with the development and commercial software experience of the Argo Navis Technologies, LLC (ANT) to form a powerful combination to delivery these objectives.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

SPARTAN and IMPROVE Comparison Experiment (SPICE) Interim Campaign Report

SPICE (the SPARTAN and IMPROVE Comparison Experiment) aims to obtain and quantify comparisons between aerosol PM 2.5 mass concentration measurements from the University of Oklahoma (OU) Surface Particulate Matter Network (SPARTAN) station and the U.S. Environmental Protection Agency (EPA) Interagency Monitoring of Protected Visual Environments (IMPROVE) station hosted by the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) User Facility at ARM’s Southern Great Plains (SGP) observatory in Oklahoma. Aerosols—both natural and anthropogenic—affect human populations in multiple ways. Much of ARM’s research focuses on how aerosols influence weather and climate through their optical and radiative effects, as well as their impacts on clouds and precipitation. However, aerosols also pose direct risks to humans and other organisms through inhalation, with the severity of health impacts depending on particle size, chemical composition, and duration of exposure. The IMPROVE network was established by the U.S. EPA to monitor air quality, including visible clarity as well as total and chemically speciated aerosol mass concentrations. The ARM SGP site hosts the IMPROVE SOGP station. Separately, the SPARTAN network operates a globally distributed set of stations similar to IMPROVE but with an emphasis on remote deployment and semi-autonomous operation for use beyond the borders of the United States (IMPROVE only operates within the U.S.). The University of Oklahoma operates a SPARTAN station. To establish confidence in the OU SPARTAN instrumentation and measurement protocol relative to the EPA-certified IMPROVE station, the OU SPARTAN station is currently deployed at SGP in close proximity to the IMPROVE SOGP station. The SPICE campaign was envisioned as a contiguous calendar-year effort for 2025 to capture seasonal variation in mass loading as well as composition. However, independent of the SPICE campaign, the SPARTAN network adopted a new filter construction part-way through the year, interrupting our contiguous data set. Thus, to obtain a contiguous data set with a uniform consistent configuration, SPICE desires an extension through 2026.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Digital Twin for Hydropower System Object Modeling: Alder Dam (FY2023)

Hydropower is the world's largest source of renewable electricity, and hydropower plants are distributed all over the world. Typical major components of a hydropower plant are the governor, excitation, generator, thrust bearing, hydraulic turbine, transformer, the main lead, metering and control, tailwater depression, and dissolved oxygen. For each component, various measures are taken. The measurements are acquired by various heterogeneous systems, including standalone sensors, programmable logic controllers (PLC), Supervisory control and data acquisition (SCADA), Internet of Things (IoT), and data acquisition and integration platforms such as OSI/PI. The measured data are often archived within the plant by a data management platform, and many institutions have cloud-based archive systems, such as Hydropower Research Institution (HRI), U.S. Army Corps of Engineers (USACE), and Columbia River Data Access in Real Time (DART). Object Modeling is a general framework for designing information systems. It focuses on objects, the actions they perform, and the messages they send to one another to cause those actions to be taken. The major differences among object modeling, network modeling, data modeling, and process modeling are that in the first we focus on the actions in response to information, objects which form the system, the actions they perform, and how they pass information to one another, while in the second we concentrate on where, when and how much information is moved, while in the third we focus on what information is moved and where it is moved, while in the last we focus on how it is moved and when it is moved. Object modeling was developed basically as a method to develop object-oriented systems and to support object-oriented programming. It describes the static structure of the system. The object Modeling Technique is easy to draw and use. That is why we choose object modeling to connect physical hydropower plants to Digital Twin. It recognizes the objects and the relationship between them. It identifies the attributes and functions of each class. Dynamic Modeling: It explains how objects respond to events. Functional Modeling indicates the processes executed in an object and how data changes when it moves to objects. It has been used in many applications like telecommunication, transportation, etc.

13 HYDRO ENERGY↗
DOE OSTI2012

ON THE ORIGIN OF THE HIGH COLUMN DENSITY TURNOVER IN THE H I COLUMN DENSITY DISTRIBUTION

We study the high column density regime of the H I column density distribution function and argue that there are two distinct features: a turnover at N{sub H{sub I}} Almost-Equal-To 10{sup 21} cm{sup -2}, which is present at both z = 0 and z Almost-Equal-To 3, and a lack of systems above N{sub H{sub I}} Almost-Equal-To 10{sup 22} cm{sup -2} at z = 0. Using observations of the column density distribution, we argue that the H I-H{sub 2} transition does not cause the turnover at N{sub H{sub I}} Almost-Equal-To 10{sup 21} cm{sup -2} but can plausibly explain the turnover at N{sub H{sub I}} {approx}> 10{sup 22} cm{sup -2}. We compute the H I column density distribution of individual galaxies in the THINGS sample and show that the turnover column density depends only weakly on metallicity. Furthermore, we show that the column density distribution of galaxies, corrected for inclination, is insensitive to the resolution of the H I map or to averaging in radial shells. Our results indicate that the similarity of H I column density distributions at z = 3 and 0 is due to the similarity of the maximum H I surface densities of high-z and low-z disks, set presumably by universal processes that shape properties of the gaseous disks of galaxies. Using fully cosmological simulations, we explore other candidate physical mechanisms that could produce a turnover in the column density distribution. We show that while turbulence within giant molecular clouds cannot affect the damped Ly{alpha} column density distribution, stellar feedback can affect it significantly if the feedback is sufficiently effective in removing gas from the central 2-3 kpc of high-redshift galaxies. Finally, we argue that it is meaningful to compare column densities averaged over {approx} kpc scales with those estimated from quasar spectra that probe sub-pc scales due to the steep power spectrum of H I column density fluctuations observed in nearby galaxies.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Precursor Analysis Report: Conti Ransomware Attack on the Health Service Executive of Ireland 2021

The Conti Ransomware Attack on the Health Service Executive (HSE) of Ireland 2021 Precursor Analysis Report leverages publicly available information about the attack and catalogs anomalous observables for each technique employed by the adversary. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. The HSE provides public healthcare corporate services and operational services throughout Ireland, with critical functions including the acute national ambulance service, acute hospital service, and community healthcare service. On 14 May 2021, Conti ransomware encrypted 80 percent of the HSE’s Information Technology (IT) infrastructure across corporate, hospital, community, and electronic health record services. Conti is a ransomware-as-a-service operation that encrypts local files, uses double extortion against victims, and is facilitated by many intrusion tools. The attack forced the HSE to shut down its entire IT infrastructure to contain the ransomware, forcing employees to revert to pen and paper recordkeeping and leading to the cancellation of many appointments and procedures. The adversary also exfiltrated 700 GB of data, compromising the confidentiality of patients’ protected health information. Had the adversary targeted the COVID-19 cloud systems or operational technology assets, such as Internet of Medical Things medical devices or smart building management systems, the impact of the attack would almost certainly have been far more severe. Researchers and analysts identified 21 unique techniques (used in a sequence of 23 steps) likely utilized during the attack with a total of 1,185 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Twenty-one of the identified techniques used during the attack on the HSE were precursors to the triggering event. Analysis identified 1,086 observables associated with these precursor techniques, 850 of which were assessed to have an increased likelihood of being perceived in the 57 days preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2021

Energy Exascale Earth System Model v2.0

First release of version 2 of the Energy Exascale Earth System Model. The atmosphere component remains EAM. Major changes since version 1 include: all column-physics parameterizations are computed on a separate grid that has approximately half the number of points of the dynamics grid, a new nonhydrostatic dynamical core (running in hydrostatic mode) with semi-Lagrangian tracer transport, CLUBB updated from v1 to v2, a new convective trigger (dCAPE/ULL) based on the dynamic Convective Available Potential Energy (CAPE) (dCAPE) and the Unrestricted Launch Level (ULL) concepts is used in ZM. minimum cloud droplet number changed, gravity wave drag energy conservation fixed and new tunings used, dust emission size distribution changed to emit more coarse dust particles The land component is still ELM. Major changes since version 1 include: using SNICAR-AD for radiation in snow to match the sea-ice model and fixing bugs in snow compaction and water state calculation. The ocean component remains MPAS-ocean. Major change since version 1 include: Redi isopycnal mixing has been updated, tested, and tuned in combination with the Gent-McWilliams parameterization, a sign error was fixed in the 3rd-order flux routines, the mesh used in low-resolution coupled cases was modified and the time steps adjusted, new regionally refined meshes were created, one focused on North America and another on the Southern Ocean, including ice shelf cavities. The sea-ice component remains MPAS-seaice. Major changes since version 1 include: A new heat- and freshwater-conserving coupling of frazil ice, turning off of SSH filtering, addition of SNICAR-AD and snow grain aging. The land-ice component remains MPAS-Albany-landIce (MALI) and is a static ice sheet. There is more out-of-the-box support for cryosphere configurations including the new regionally refined configuration around Antarctica with ice shelf cavities. The river model is MOSART. The half-degree river mesh was redone so it no longer treats Black and Caspian seas as ocean. The coupler remains cpl7/MCT. Major changes since version 1 include handling of ice shelf melt fluxes (heat / freshwater exchange with the ocean), and data icebergs. All components allow regional refinement of their meshes and two separate example of refinement, one in and around North America and one around Antarctica and the Southern Ocean, are provided.

E3SM Project, DOE↗