Search NASA⌕ Search

SEARCH · Search NASA

Results for “model credibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A NICER View of the Massive Pulsar PSR J0740+6620 Informed by Radio Timing and XMM-Newton Spectroscopy

We report on Bayesian estimation of the radius, mass, and hot surface regions of the massive millisecond pulsar PSR J0740+6620, conditional on pulse-profile modeling of Neutron Star Interior Composition Explorer X-ray Timing Instrument event data. We condition on informative pulsar mass, distance, and orbital inclination priors derived from the joint North American Nanohertz Observatory for Gravitational Waves and Canadian Hydrogen Intensity Mapping Experiment/Pulsar wideband radio timing measurements of Fonseca et al. We use XMM-Newton European Photon Imaging Camera spectroscopic event data to inform our X-ray likelihood function. The prior support of the pulsar radius is truncated at 16 km to ensure coverage of current dense matter models. We assume conservative priors on instrument calibration uncertainty. We constrain the equatorial radius and mass of PSR J0740+6620 to be ${12.39}_{-0.98}^{+1.30}$ km and ${2.072}_{-0.066}^{+0.067}$ M ⊙ respectively, each reported as the posterior credible interval bounded by the 16% and 84% quantiles, conditional on surface hot regions that are non-overlapping spherical caps of fully ionized hydrogen atmosphere with uniform effective temperature; a posteriori, the temperature is ${\mathrm{log}}_{10}(T\,[{\rm{K}}])={5.99}_{-0.06}^{+0.05}$ for each hot region. All software for the X-ray modeling framework is open-source and all data, model, and sample information is publicly available, including analysis notebooks and model modules in the Python language. Furthermore, our marginal likelihood function of mass and equatorial radius is proportional to the marginal joint posterior density of those parameters (within the prior support) and can thus be computed from the posterior samples.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Swift and extensive Omicron outbreak in China after sudden exit from ‘zero-COVID’ policy

In late 2022, China transitioned from a strict ‘zero-COVID’ policy to rapidly abandoning nearly all interventions and data reporting. This raised great concern about the presumably-rapid but unreported spread of the SARS-CoV-2 Omicron variant in a very large population of very low pre-existing immunity. By modeling a combination of case count and survey data, we show that Omicron spread extremely rapidly, at a rate of 0.42/day (95% credibility interval: [0.35, 0.51]/day), translating to an epidemic doubling time of 1.6 days ([1.6, 2.0] days) after the full exit from zero-COVID on Dec. 7, 2022. Consequently, we estimate that the vast majority of the population (97% [95%, 99%], sensitivity analysis lower limit of 90%) was infected during December, with the nation-wide epidemic peaking on Dec. 23. Overall, our results highlight the extremely high transmissibility of the variant and the importance of proper design of intervention exit strategies to avoid large infection waves.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Probing quarkyonic matter in neutron stars with the Bayesian nuclear-physics multimessenger astrophysics framework

The interiors of neutron stars contain matter at the highest densities realized in our Universe. Interestingly, theoretical studies of dense matter, in combination with the existence of two-solar-mass neutron stars, indicate that the speed of sound $c_s$ has to increase to values well above the conformal limit ($c_s^2$ = 1/3) before decreasing again at higher densities. Further, the decrease could be explained by either a strong first-order phase transition or a crossover transition from hadronic to quark matter. The latter scenario leads to a pronounced peak in the speed of sound, reaching values above the conformal limit, naturally explaining the inferred behavior. In this work, we use the nuclear-physics multimessenger astrophysics (NMMA) framework to compare predictions of the quarkyonic matter model with astrophysical observations of neutron stars, with the goal of constraining model parameters. Assuming quarkyonic matter to be realized within neutron stars, we find that there can be a significant amount of quarks inside the cores of neutron stars with masses in the two-solar-mass range, amounting to up to ≈0.13$M$ ⊙ , contributing ≈ 5.9% of the total mass. Furthermore, for the quarkyonic matter model investigated here, the radius of a 1.4$M$ ⊙ neutron star would be $13.44_{–1.54}^{+1.69}(13. 54_{–1.04}^{+1.02})$ km, at 95% credibility, without (with) the inclusion of AT2017gfo.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Regional Oil and gas Aerial Methane Synthesis model (ROAMS) v2.0

The Regional Oil and gas Aerial Methane Synthesis model is a tool to convert the results of wide-area, source-resolved aerial methane remote sensing surveys of oil and natural gas infrastructure in a given region into methane emissions inventories (estimates of the magnitude and breakdown of methane emissions from the surveyed infrastructure). The tool leverages databases of source-resolved methane emissions detected in aerial surveys, aerial survey coverage information (which areas were measured and when), data summarizing surveyed oil and natural gas infrastructure and production (derived from third-party databases), as well as state-of-the-art mechanistic emissions simulation tools to characterize emissions too small for the aerial system to see. The regional methane emissions estimates produced by this tool are much more granular in both space and asset type than common satellite- or flux tower-based regional estimates. Unlike other tools for converting site-level measurements into regional emissions estimates, our unique geostatistical approach integrates aerially measured emissions with limited need for statistical extrapolation, which can be highly sensitive to modeler assumptions. As a result, ROAMS-based estimates of regional methane emissions from oil and gas activity are widely viewed as highly credible, as evidenced by the success of Dr. Sherwin's recent paper in Nature.

Sherwin, Evan [Lawrence Berkeley National Laborato↗
DOE OSTI2025

Regional Oil and gas Aerial Methane Synthesis model (Analytica) (ROAMS Analytica) v1.5.2

The Regional Oil and gas Aerial Methane Synthesis model (Analytica) is a tool to convert the results of wide-area, source-resolved aerial methane remote sensing surveys of oil and natural gas infrastructure in a given region into methane emissions inventories (estimates of the magnitude and breakdown of methane emissions from the surveyed infrastructure). This version is written in the Analytica programming language, and this version accompanies a correction in preparation for submission to Sherwin et al. 2024 (Nature). The tool leverages databases of source-resolved methane emissions detected in aerial surveys, aerial survey coverage information (which areas were measured and when), data summarizing surveyed oil and natural gas infrastructure and production (derived from third-party databases), as well as state-of-the-art mechanistic emissions simulation tools to characterize emissions too small for the aerial system to see. The regional methane emissions estimates produced by this tool are much more granular in both space and asset type than common satellite- or flux tower-based regional estimates. Unlike other tools for converting site-level measurements into regional emissions estimates, our unique geostatistical approach integrates aerially measured emissions with limited need for statistical extrapolation, which can be highly sensitive to modeler assumptions. As a result, ROAMS-based estimates of regional methane emissions from oil and gas activity are widely viewed as highly credible, as evidenced by the success of Dr. Sherwin's recent paper in Nature.

Sherwin, Evan [Lawrence Berkeley National Laborato↗
DOE OSTI2020

Assessment of Grizzly Capabilities for Reactor Pressure Vessels and Reinforced Concrete Structures

Over the last several years, capabilities to simulate the progression and effects of degradation in critical structures in light water reactor (LWR) nuclear power plants have been under development in the Grizzly code. Age-related material degradation is important for a number of systems in LWRs, but the main focus for Grizzly development has been on reactor pressure vessels (RPVs) and concrete structures because of their central role and the difficulty of replacement of these structures if they are found to be degraded to an unacceptable degree. The capabilities for analyzing both RPVs and concrete structures in Grizzly have reached a point where the feature sets are sufficiently complete to perform credible analyses of those types of structures. Because of this, the emphasis has shifted from foundational development to assessing the accuracy of these modeling capabilities on representative problems of interest and on improving the physical basis of those models to improve their ability to predict the actual response of those structural systems. This report documents a first set of test cases that have been developed to assess the Grizzly code on real-world problems for these two types of structures. For RPVs, these test cases consist of a set of benchmark problems where Grizzly is compared to another code. For concrete structures, a set of models of experimental specimens designed to characterize the multidimensional swelling response of reinforced concrete members due to alkali-silica reaction (ASR) has been developed and compared with experimental results. Both the RPV and concrete test cases developed here are much more computationally intensive than the small regression tests that are used in the testing that Grizzly undergoes every time a proposed set of changes is made to ensure that those changes do not adversely affect previously established behavior. A system for regularly running these large problems to monitor the behavior of the code as it is developed has been instituted. While these test cases generally indicate good comparison with the benchmark results, they also indicate areas where further development is warranted.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

Proton-rich Production of Lanthanides: The vi Process

The astrophysical origin of the lanthanides is an open question in nuclear astrophysics. Besides the widely studied s, i, and r processes in moderately to strongly neutron-rich environments, an intriguing alternative site for lanthanide production could in fact be robustly proton-rich matter outflows from core-collapse supernovae under specific conditions—in particular, high-entropy winds with enhanced neutrino luminosity and fast dynamical timescales. In this environment, excess protons present after charged-particle reactions have ceased can continue to be converted to neutrons by (anti)neutrino interactions, producing a neutron-capture reaction flow up to A ∼ 200. This scenario, christened the νi process in a recent paper, has previously been discussed as a possibility. Here, we examine the prospects for the νi process through the lenses of stellar abundance patterns, bolometric light curves, and galactic chemical evolution models, with a particular focus on hypernovae as candidate sites. We identify specific lanthanide signatures for which the νi process can provide a credible supplement to the r/i processes.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS2012

The Race To Understand A Changing Planet

The Earth's climate is changing rapidly. In some respects, the rate of change is outpacing the predictions of only a few years ago. The challenge to Earth Science is to put forward credible projections of possible future climates so that the public and policy makers can make science-based decisions about energy development strategies. Models, observations and experiments all play strong roles in improving knowledge and increasing confidence in our predictions. The models have progressed from simple, coarse-resolution descriptions of atmospheric dynamics and physics only twenty years ago, to full-up Earth System models (ESMs) that include complete descriptions of the oceans and cryosphere. It has been convincingly argued that such complexity - the construction of realistic "toy" Earth's - is necessary to address the complex processes involved in climate change, including not only the physical atmosphere, oceans and cryosphere, but also the carbon cycle - both its natural and anthropogenic components - and the biosphere. Observations, particularly satellite observations, have more or less kept pace with the demands of the modelers, being able to observe progressively more and different facets of the Earth system, but the global satellite fleet is in need of an overhaul very soon. Lastly, field experiments and process studies confront the models with facts and allow us to develop more sophisticated and accurate satellite data algorithms. The challenges facing our relatively small Earth and planetary science communities are considerable and the stakes are significant. The stakeholders, now numbering 7 billion but soon to be 10 billion, will be relying on our results and capabilitie's to guide them into the future.

Sellers, Piers J.↗
NASA NTRS2017

Towards a New Food System Assessment: AgMIP Coordinated Global and Regional Assessments of Climate Change

Agricultural stakeholders need more credible information on which to base adaptation and mitigation policy decisions. In order to provide this, we must improve the rigor of agricultural modelling. Ensemble approaches can be used to address scale issues and integrated teams can overcome disciplinary silos. The AgMIP Coordinated Global and Regional Assessments of Climate Change and Food Security (CGRA) has the goal to link agricultural systems models using common protocols and scenarios to significantly improve understanding of climate effects on crops, livestock and livelihoods across multiple scales. The AgMIP CGRA assessment brings together experts in climate, crop, livestock, economics, and food security to develop Protocols to guide the process throughout the assessment. Scenarios are designed to consistently combine elements of intertwined storylines of future society including, socioeconomic development, greenhouse gas concentrations, and specific pathways of agricultural sector development. Through these approaches, AgMIP partners around the world are providing an evidence base for their stakeholders as they make decisions and investments.

Ensembles↗
NASA NTRSDate not supplied

Pretest Simulations of a Supersonic Mixing and Combustion Validation Experiment to Assess Sensitivities

The reliance on CFD simulations to develop, design, and optimize scramjet systems (or components) has become commonplace. This reliance inevitably hinges on the ability of the computational analyst to quantify the level of confidence in their computational results. Unfortunately, nearly all the measured data available for this assessment comes from antiquated experimental datasets, or from tests that focused on the extraction of scramjet system (or component) performance. The objective of a CFD validation experiment is to quantify the predictive accuracy of one or more of the CFD physics submodels, implying that other uncertainties related to replicating the facility flow environment (e.g., knowledge of boundary conditions) must be minimized to the extent possible. This inevitably places stringent requirements on the quality and quantity of measurements taken to accurately specify inflow, outflow, and surface conditions for the CFD simulations; in addition to the measurements taken for the validation of physics submodels. This places additional demands on the experimental process above and beyond those for test article performance assessment. A recent high speed code credibility workshop series sponsored by AFRL identified a gap in existing validation data for fundamental assessments of turbulent mixing and combustion CFD closure models at scramjet engine relevant conditions. To address this gap, engineers at AFRL have designed a coaxial jet flame configuration that will be tested at two facilities (Research Cell 19 at the Air Force Research Lab, and at Purdue University). The effort described here documents pretest simulations of this validation experiment with the goal of fleshing out the extent of the facility flowpath that must be included to adequately reproduce the facility test section flow environment. The findings indicate that the flow around the support structure for the fuel injection centerbody upstream of the facility nozzle generates disturbances that persist throughout the nozzle expansion process; corrupting the azimuthal symmetry that was desired in the fuel/air mixing region of the test section. Simulations without this support structure maintained a high degree of azimuthal symmetry up until the fuel injection plane. However, even in this scenario the azimuthal symmetry was not maintained once the centerbody boundary layer transitioned to a wake flow downstream of the fuel injection plane.

CFD↗
NASA NTRSDate not supplied

Pretest Simulations of a Supersonic Mixing and Combustion Validation Experiment to Assess Sensitivities

The reliance on CFD simulations to develop, design, and optimize scramjet systems (or components) has become commonplace. This reliance inevitably hinges on the ability of the computational analyst to quantify the level of confidence in their computational results. Unfortunately, nearly all the measured data available for this assessment comes from antiquated experimental datasets, or from tests that focused on the extraction of scramjet system (or component) performance. The objective of a CFD validation experiment is to quantify the predictive accuracy of one or more of the CFD physics submodels, implying that other uncertainties related to replicating the facility flow environment (e.g., knowledge of boundary conditions) must be minimized to the extent possible. This inevitably places stringent requirements on the quality and quantity of measurements taken to accurately specify inflow, outflow, and surface conditions for the CFD simulations; in addition to the measurements taken for the validation of physics submodels. This places additional demands on the experimental process above and beyond those for test article performance assessment. A recent high speed code credibility workshop series sponsored by AFRL identified a gap in existing validation data for fundamental assessments of turbulent mixing and combustion CFD closure models at scramjet engine relevant conditions. To address this gap, engineers at AFRL have designed a coaxial jet flame configuration that will be tested at two facilities (Research Cell 19 at the Air Force Research Lab, and at Purdue University). The effort described here documents pretest simulations of this validation experiment with the goal of fleshing out the extent of the facility flowpath that must be included to adequately reproduce the facility test section flow environment. The findings indicate that the flow around the support structure for the fuel injection centerbody upstream of the facility nozzle generates disturbances that persist throughout the nozzle expansion process; corrupting the azimuthal symmetry that was desired in the fuel/air mixing region of the test section. Simulations without this support structure maintained a high degree of azimuthal symmetry up until the fuel injection plane. However, even in this scenario the azimuthal symmetry was not maintained once the centerbody boundary layer transitioned to a wake flow downstream of the fuel injection plane.

CFD↗
DOE OSTI2021

Bayesian analysis of a future β decay experiment's sensitivity to neutrino mass scale and ordering

Bayesian modeling techniques enable sensitivity analyses that incorporate detailed expectations regarding future experiments. A model-based approach also allows one to evaluate inferences and predicted outcomes, by calibrating (or measuring) the consequences incurred when certain results are reported. In this work, we present procedures for calibrating predictions of an experiment's sensitivity to both continuous and discrete parameters. Using these procedures and a new Bayesian model of the β-decay spectrum, we assess a high-precision β-decay experiment's sensitivity to the neutrino mass scale and ordering for one assumed design scenario. We find that such an experiment could measure the electron-weighted neutrino mass within ~40 meV after 1 year (90% credibility). Neutrino masses > 500 meV could be measured within ≈5 meV. Using only β decay and external reactor neutrino data, we find that next-generation β-decay experiments could potentially constrain the mass ordering using a two-neutrino spectral model analysis. By calibrating mass ordering results, we identify reporting criteria that can be tuned to suppress false ordering claims. In some cases, a two-neutrino analysis can reveal that the mass ordering is inverted, an unobtainable result for the traditional one-neutrino analysis approach.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1972

Supplementary notes on sea surface temperature anomalies and model-generated meteorological histories

In seasonal computations, the Mintz-Arakawa two-level model is found to be sensitive to a minor alteration in the computational program. Effects of the program change on monthly mean sea level pressure fields are small in the first month, but large in the second and third months, although the meteorological histories generated by both the original and modified programs are equally credible. The inherited effects of a transient (one month) sea surface temperature (SST) anomaly on the computed monthly mean sea level pressure fields over a period of a season are about as large in absolute magnitude as those generated in the model by a persistent (seasonal) SST anomaly. The effects of a transient SST anomaly in the North Pacific Ocean on monthly and seasonal temperature and precipitation in the eastern United States may be large enough to produce a change of one or two class intervals in these predicted weather elements. The model-generated precipitation in the equatorial region is also found to be sensitive to the sea surface temperature field in the North Pacific.

Spar, J.↗
NASA NTRS1984

Why credible propeller noise measurements are possible in the acoustically untreated NASA Lewis 8 ft by 6 ft wind tunnel

An explanation is presented for the lack of acoustic reflections in noise studies of propfan models in the NASA-Lewis 8 x 6 ft wind tunnel, where trials were run at Mach numbers 0.5-0.85. The highly directional propeller noise, i.e., mainly in the plane of rotation, experiences a convective effect due to the high subsonic axial Mach number. Reflected sounds are carried downstream, out of range of the acoustic sensors in the tunnel. Furthermore, reflected noise is less audible, and therefore does not affect measurements near peak values. It is suggested that some data contamination may occur below Mach 0.6, and that measurements be performed on higher harmonics generated by low level reflected noise.

Dittmar, J. H.↗
NASA NTRS2003

Simulating the ARES Aircraft in the Mars Environment

NASA Langley proposed the Aerial Regional-scale Environmental Survey (ARES) of Mars science mission in response to the NASA Office of Space Science 2002 Mars Scout Opportunity. The science-driven mission proposal began with trade studies and determined that a rocket powered aircraft was the best suited platform to complete the ARES science objectives. A high fidelity six degree of freedom flight simulation was required to provide credible evidence that the aircraft design fulfilled mission objectives and to support the aircraft design process by providing performance evaluations. The aircraft was initially modeled using the aero, propulsion, and flight control system components of other aircraft models. As the proposed aircraft design evolved, the borrowed components were replaced with new models. This allowed performance evaluations to be performed as the design was maturing. Basic autopilot features were also developed for the ARES aircraft model. Altitude hold and track hold modes allowed different mission scenarios to be evaluated for both science merit and aircraft performance. Platform stability and data rate requirements were identified for each of the instruments and the aircraft performance was evaluated against those requirements. The results of the simulation evaluations indicate that the ARES design and mission profiles are sound and meet the science objectives.

Kenney, P. Sean↗
DOE OSTI2026

A Benchmarking Framework for Evaluating Large Language Model Capabilities in Nuclear Reactor Safety Applications

Large language models (LLMs) are increasingly capable of answering technical questions, synthesizing domain knowledge, and supporting engineering workflows. For nuclear science and engineering, these capabilities require careful, domain-specific evaluation before they can be credibly incorporated into safety-related activities, regulatory review, or technical decision support. This paper presents preliminary results from benchmarking framework for evaluating LLM capabilities in nuclear contexts. The framework is organized into three evaluation categories: nuclear fundamentals, general dual-use knowledge, and plant specific knowledge. These categories are intended to distinguish general nuclear engineering competence from broader technical reasoning and more context-dependent nuclear knowledge. Initial evaluations focus on nuclear fundamentals using questions representative of the knowledge expected of a nuclear professional engineer. Results indicate that contemporary frontier models perform at a high level and substantially exceed the performance of older model generations, with some models approaching saturation of the current benchmark. These findings suggest both the rapid improvement of LLM capabilities in specialized technical domains and the need for more discriminating evaluation methods. The paper presents the benchmark structure, preliminary model-comparison results, and ongoing work. This work supports development of verifiable, responsible, and safety-conscious methods for assessing AI systems in nuclear engineering applications.

21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗
NASA NTRS2021

Dynamical and Trace Gas Responses of the Quasi-Biennial Oscillation to Increased CO2

Long-term projections of the Quasi-Biennial Oscillation (QBO) remain uncertain. Using the high-top \Middle Atmosphere" (MA) version of the NASA Goddard Institute for Space Studies (GISS) ModelE (E2.2-AP), we show that increased CO2 reduces the QBO period and weakens its amplitude, consistent with results from recent multimodel studies. The amplitude response is asymmetric, with the easterly phase weakening more than the westerly phase. Results from both coupled atmosphere{ocean and climatologically fixed sea surface temperature simulations link the period response to changes in parameterized convective gravity wave fluxes, whereas the amplitude changes are more closely tied to changes in tropical upwelling. Comparisons of simulations with and without ozone-dynamical feedbacks reveal that ozone feedbacks increase the QBO amplitude and period in pre-industrial control simulations, and reduce the response of the amplitude to increased CO2. The response of stratospheric tracers to projected changes in the QBO is also investigated. Similarity between results using fully interactive (trace gas and aerosol) versus linearized ozone chemical mechanisms suggests that the QBO response to CO2 does not depend on additional chemical feedbacks beyond those associated with ozone, which implies that projected QBO changes may be credibly represented using simpli fied stratospheric ozone schemes.

GISS Model E2.2-AP↗
DOE OSTI2020

Austin Sustainable and Holistic Integration of Energy Storage and Solar PV [Austin SHINES]. Final Report, Version 2

The Austin SHINES project and solution is a software management platform, for an electric grid with a high penetration of dispersed photovoltaic (PV) solar generation sites, which maintains the traditional power quality and reliability associated with grid service. This project developed and deployed the platform as a Distributed Energy Resource Management System (DERMS), engaging multiple advanced controls, to evaluate operation and optimization of a fleet of diverse DER assets, installed at several locations among Austin Energy’s customers and distribution system. The project also produced a methodology to create a replicable DERMS template, adaptable to other regions and market structures. Last, Austin SHINES aimed to demonstrate the solution’s methodology would enable the DER grid ecosystem to serve load at a technical cost (System Levelized Cost of Electricity, or System LCOE) of less than the U.S. Department of Energy SHINES program metric of $0.14/kWh, in a defined boundary, while enabling a high penetration of distributed PV. Research was categorized in 6 reports (Final Deliverables = FD) listed below, with titles and descriptions indicating which area of understanding was investigated: FD-1: System Levelized Cost of Electricity (System LCOE) Methodology The creation and use of the System LCOE to Serve Load metric that encompasses the holistic, system-level costs and benefits of all resources, and enables them to be evaluated based on their ability to support an efficient and low-cost integrated grid ecosystem. FD-2: Software Platform Product Description The creation of new DER control methodologies deployable within a utility-grade software platform that enable DER's to maximize their benefit within a grid, that is capable of serving load enabling a high penetration of distributed PV generation. FD-3: Optimal Design Methodology Optimal design methodologies for individual DER installations that enable utilities to determine the optimal combinations and sizing for individual DER sites. FD-4: Austin SHINES Ownership and Operation Models for DER System Performance A comparison of multiple DER aggregation and ownership methodologies including direct utility control, third-party aggregator, and autonomous. FD-5: Economic Modeling & Optimization A comparison of multiple DER technology mixes and configurations within the distribution system, providing insight into an optimal blend of technologies that best enable the distribution system to serve load at the lowest cost at high penetrations of solar. FD-6: Fielded Assets Deployed DER assets within the Austin Energy SHINES circuits. Austin SHINES provided an opening for state-of-the-art technology products to be deployed, providing a rich opportunity for improving how each of the products perform as stand-alone products, and in concert with other complementary products. The Austin SHINES project comprised of two key metrics for System LCOE: SystemLCOE_SHINES<$0.14/kWh Modeled ΔSystemLCOE_SHINES/ΔSystemLCOE_Base≥20% at same solar penetration The System LCOE calculation uses the costs of the utility-owned infrastructure as it exists today, the cost of the DERs that exist in the system today, and the cost of the purchase of energy from ERCOT wholesale markets over the course of the calendar year. All costs are on an annualized basis. The capital and operating costs are derived from the rate case, which produces a yearly cost. The net cost of energy and services imported to the system is integrated over the test year, as is the load served and solar penetration. The first metric was easily achieved by every scenario considered. The goal was set when the Department of Energy’s SHINES Funding Opportunity Announcement was written in 2015 and was a more difficult target at the time. Due mostly to rapidly declining costs for DERs and the significant decrease in the Electric Reliability Council of Texas (ERCOT) energy market prices, which results in lower net cost of energy purchases, the System LCOE is well below this target for all scenarios considered. A fleet of DERs can assume different mixtures, each of which serves the load at a different LCOE. The optimal mixture of DERs serves load at the smallest System LCOE. The second metric (hereinafter %delta metric) asks that the holistic DERMS controls reduce the incremental cost above the baseline of going to a high solar penetration future by at least 20% as compared to the case of a DER deployment with no sophisticated controls (autonomous). Many comparison sets were created throughout this project. Physical technology was installed for informing utility engineering and testing several types of operational control schemes, through the DERMS. The types of operational control which were compared for valuation of the System LCOE Metric were: Holistic control = using the full suite of the DERMS platform to decide and optimize how/why the systems operate depending on weather, market, and reliability signal input. Autonomous control = a local mode at the asset site, wherein a schedule operates the asset, with visibility into performance only No control = the baseline for comparing value against the other two types of control The types of ownership control included: Direct Utility control = the utility dispatches a signal to each asset Third-Party Aggregator = a third party aggregates a fleet of assets and the utility dispatches one signal for all Autonomous = a local mode is set for operation at the asset site, wherein a schedule operates the asset, with visibility into performance only The types of control methodologies deployable within a utility-grade software platform included: Utility Peak Load Reduction = Lower transmission cost obligation Day-Ahead Energy Arbitrage = Realize economic value through price differential Real-Time Price Dispatch = Realize economic value from real-time price spikes Voltage support = Reduce losses and increase solar generation Distribution Congestion Management = Increase local grid reliability Demand Charge Reduction = Lower customer bills and realize system benefit The fielded assets deployed for the project were: Utility Scale Kingsbery Energy Storage System: 1.5 MW / 3 MWh Li-Ion battery storage Mueller Energy Storage System: 1.75 MW / 3.2 MWh Li-Ion battery storage, 7 Energy Storage Units (250 kW each) La Loma Community Solar: 2.6 MW Commercial Scale Aggregated storage installations at 3 sites, with existing solar (300+ kW): One 18 kW / 36 kWh Li-Ion battery storage Two 72 kW / 144 kWh Li-Ion battery storage Residential Scale Aggregated storage installations: -Six stationary battery storage systems (10 kWh each) at homes with existing solar -One Electric Vehicle installed as Vehicle-to-Grid (V2G) Utility-Controlled Solar via Smart Inverters at 12 homes Autonomously-Controlled Smart Inverters at 6 homes Over the course of the project, Austin SHINES undertook installing more than 3 MW of distributed battery energy storage, smart PV inverters, a DER control platform, and other enabling technologies utilizing customer and utility locations and aggregation models. All of these resources were to be integrated and optimized at the utility level. DER assets and control methodologies were designed to achieve a credible pathway to a System LCOE for energy delivered to load of $0.14//kWh or less by 2020, while maximizing distributed solar generation and maintaining acceptable standards of power quality. The project also established a template for other regions to follow, to maximize the adoption of distributed solar PV in support of an economic and efficient grid. In total, the Austin SHINES project added value to the DER subject area in each layer of integration. From utility, to commercial to residential scales, the sheer hierarchy of communication and coordination was a significant accomplishment in addition to learnings from what these communications revealed was unique to each. Economically, the most effective method demonstrated was the criticality of planning phases. Contingencies and multiple projection scenarios helped guide the project to deploy optimal design as close as feasible, in real world conditions. The project and reports will serve public benefit by outlining specific areas of DER strategy and installation where many stakeholders and needs can be addressed with improved efficiency. Overall, communities and utilities should use the results to guide the increasing options available for powering the grid with DER, renewables, and carbon considerate energy.

14 SOLAR ENERGY↗