Search NASA⌕ Search

SEARCH · Search NASA

Results for “Test protocol”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Development and experimental qualification of novel disruption prevention techniques on DIII-D

Novel disruption prevention solutions spanning a range of control regimes are being developed and tested on DIII-D to enable ITER success. First, a new real-time control algorithm has been developed and tested for regulating nearness to stability limits and maintaining safety-margins. Its first application has been for reliable prevention of vertical displacement events (VDEs) by adjusting plasma elongation (κ) and the inner-gap between the plasma and inner-wall in response to real- time open-loop VDE growth rate (γ) estimators. VDEs were robustly prevented up to average open-loop growth rates of 800 rad/s with initial tunings, with only applying shape modification when near safety limits. Second, the disruption risk during fast, emergency shutdown after large tearing and locked modes can be significantly improved by transitioning to a limited topology during shutdown. More than 50% of emergency limited shutdowns after locked modes reach a final normalized current I_N < 0.3 before terminating, scaling to the 3 MA ITER requirement. Furthermore, this is in contrast to diverted shutdowns, the majority of which disrupt at I_N > 0.8. Despite improvements, these results highlight the critical importance of early prevention. Third, a novel emergency shut down method has been developed which excites instabilities to form a warm, helical core post-thermal quench. The current quench extends to ~100ms and avoids VDEs and runaway electron generation. Novel real-time machine learning disruption prediction has been integrated with the DIII-D proximity controller, and a real- time compatible multi-mode MHD spectroscopy technique has been developed. Results presented here were enabled by a focused effort, the Disruption Free Protocol, in DIII-D’s 2019-20 campaign to complement disruption prevention experiments with a large piggy-back program. In addition to testing novel techniques, it is estimated to have helped avoid 32 potential disruptions in piggyback operations with rapid, early shutdowns after large rotating n=1 or locked modes.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Camera settings and biome influence the accuracy of citizen science approaches to camera trap image classification

Scientists are increasingly using volunteer efforts of citizen scientists to classify images captured by motion-activated trail cameras. The rising popularity of citizen science reflects its potential to engage the public in conservation science and accelerate processing of the large volume of images generated by trail cameras. While image classification accuracy by citizen scientists can vary across species, the influence of other factors on accuracy is poorly understood. Inaccuracy diminishes the value of citizen science derived data and prompts the need for specific best-practice protocols to decrease error. We compare the accuracy between three programs that use crowdsourced citizen scientists to process images online: Snapshot Serengeti, Wildwatch Kenya, and AmazonCam Tambopata. We hypothesized that habitat type and camera settings would influence accuracy. To evaluate these factors, each photograph was circulated to multiple volunteers. All volunteer classifications were aggregated to a single best answer for each photograph using a plurality algorithm. Subsequently, a subset of these images underwent expert review and were compared to the citizen scientist results. Classification errors were categorized by the nature of the error (e.g., false species or false empty), and reason for the false classification (e.g., misidentification). Our results show that Snapshot Serengeti had the highest accuracy (97.9%), followed by AmazonCam Tambopata (93.5%), then Wildwatch Kenya (83.4%). Error type was influenced by habitat, with false empty images more prevalent in open-grassy habitat (27%) compared to woodlands (10%). For medium to large animal surveys across all habitat types, our results suggest that to significantly improve accuracy in crowdsourced projects, researchers should use a trail camera set up protocol with a burst of three consecutive photographs, a short field of view, and determine camera sensitivity settings based on in situ testing. Accuracy level comparisons such as this study can improve reliability of future citizen science projects, and subsequently encourage the increased use of such data.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Axenisation of oleaginous microalgal cultures via anoxic photosensitisation

Growing interest in sustainable food and biofuel research has necessitated high quality axenic oleaginous microalgal strains. Unfortunately, most strains available in culture banks contain commensal microbes such as bacteria and the default decontamination method involves antibiotic treatment which has begun to exacerbate the emergence of antibiotic resistance. To overcome this problem, anoxic photosensitisation was investigated as an alternate approach. Four oleaginous microalgal species (Tetradesmus obliquus, Desmodesmus armatus, Chlorella vulgaris and Nannochloropsis limnetica) were incubated in varying concentrations of Rose Bengal (0 μM, 1 μM, 3 μM or 9 μM) either in normal (oxic) or anoxic conditions, for 72 h under light (8.85 ± 0.4 W/m 2 ) in a specially designed heterotrophic growth complex (HGC) medium, followed by 72 h in standard Bold's Basal Medium (BBM). Commonly used antibiotics-based protocol was used as the control method. Post treatment, cell numbers and percentage populations were counted with Flow Cytometry, and viability was tested using standard plating methods using BBM and LB. Additionally, the contaminating microbes in the cultures were profiled using 16Ss rRNA sequencing. Anoxic conditions were able to significantly decrease bacterial content, albeit with an equally detrimental effect on the microalgal population. Although the responses differed between the microalgae, anoxic incubation along with Rose Bengal at 3 μM was able to completely decontaminate N. limnetica and C. vulgaris, while D. armatus and T. obliquus could be decontaminated with an additional streak-plating step. None of the cultures could be decontaminated using antibiotics treatment. These results suggest that axenisation of microalgal cultures was largely due to anoxy, that was synergistically enhanced by Rose Bengal at a concentration of ≥3 μM.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Hardware-in-the-Loop Testing of Wide-Area Damping Controller for Field Implementation in Large-scale Power Grid

In our previous work, an adaptive measurement-driven wide-area damping controller (WADC) for suppressing inter-area oscillations has been proposed and a hardware prototype was developed and validated through hardware-in-the-loop tests. As a continuation of the work, this paper introduces a WADC software prototype to handle the realistic challenges for field implementation in the control room of the power grid. The WADC software is developed and operated as an openPDC adapter with a graphical user interface (GUI) to monitor the WADC inputs and output, the communication delays and other variables. The software prototype has been fully tested through an enhanced hardware-in-the-loop (HIL) test setup. Its performance is verified under various realistic communication uncertainties, such as random time delays and data losses, with different communication protocols. The experiment results have proven the WADC software can deliver sufficient damping to suppress the targeted oscillation mode in handling various communication uncertainties for future field deployment.

Jia, Xinlan↗
DOE OSTI2025

Real-Time Testbed for Studying Cyberattacks and Defense in DER-integrated Smart Inverter Systems

In this paper, we propose a Hardware-in-the-Loop (HIL) simulation testbed suitable for the implementation and testing of realistic cyberattacks on grid-tied smart inverter systems integrated with Distributed Energy Resources (DER) that use the Distributed Network Protocol-3 (DNP3) protocol for communications between grid components. Specifically, our testbed combines a Real-Time Digital Simulator (RTDS) NovaCor device, outfitted with GNETx2 network interface cards, a gridtied DER topology implemented via the RTDS software package RSCAD, and a custom virtual network that emulates a man in the middle attacker. The Man-in-the-Middle (MITM) attacker captures DNP3 traffic and falsifies telemetry data in DNP3 packets to trigger unwarranted commands from a DNP3 controller that exploit smart inverter grid support functions. We choose DNP3 and implement grid support functions according to the IEEE Std. 1547-2018 mandated for the interconnection and interoperability of DER power systems with associated power components. Furthermore, we develop a protocol payload agnostic attack detection framework that leverages the round-trip time (RTT) anomalies between DNP3 requests and responses and can detect the presence of attacks without having to analyze the payload’s contents, while balancing trade-offs between false alarm counts, missed detections, and time to detection. To facilitate further research, we publicly release benign and attack network traffic exchanged between various sensors, controllers, and actuators in our grid-tied inverter testbed.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Digital zero noise extrapolation for quantum error mitigation

Zero-noise extrapolation (ZNE) is an increasingly popular technique for mitigating errors in noisy quantum computations without using additional quantum resources. We review the fundamentals of ZNE and propose several improvements to noise scaling and extrapolation, the two key components in the technique. We introduce unitary folding and parameterized noise scaling. Furthermore, these are digital noise scaling frameworks, i.e. one can apply them using only gate-level access common to most quantum instruction sets. We also study different extrapolation methods, including a new adaptive protocol that uses a statistical inference framework. Benchmarks of our techniques show error reductions of 18X to 24X over non-mitigated circuits and demonstrate ZNE effectiveness at larger qubit numbers than have been tested previously. In addition to presenting new results, this work is a self-contained introduction to the practical use of ZNE by quantum programmers.

quantum computing↗
DOE OSTI2026

Augmenting LLM-Based Agents for Improved Performance in Pentesting and Commissioning Operational Technology in Critical Infrastructure

Artificial intelligence (AI), and more specifically large language models (LLMs) have the potential for use in penetration testing (“pentesting”) against devices, networks, and computer systems in information technology (IT). We explore the possibility of extending pentesting from IT systems to operational technology (OT) systems, which are more obscure than IT systems in their protocols and design. A challenge therefore exists when applying pretrained LLMs to OT systems as corpora are likely to underrepresent OT systems in comparison to other more prevalent systems. We evaluate augmentations of LLMs with various methods, especially retrieval augmented generation (RAG), to improve performance of the LLMs in the OT domain. In addition to pentesting, some of the testing of these OT devices may include commissioning to ensure that the newly installed devices work correctly. Our framework may also be applied in such cases.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Classically verifiable quantum advantage from a computational Bell test

Existing experimental demonstrations of quantum computational advantage have had the limitation that verifying the correctness of the quantum device requires exponentially costly classical computations. Here we propose and analyse an interactive protocol for demonstrating quantum computational advantage, which is efficiently classically verifiable. Our protocol relies on a class of cryptographic tools called trapdoor claw-free functions. Although this type of function has been applied to quantum advantage protocols before, our protocol employs a surprising connection to Bell’s inequality to avoid the need for a demanding cryptographic property called the adaptive hardcore bit, while maintaining essentially no increase in the quantum circuit complexity and no extra assumptions. Leveraging the relaxed cryptographic requirements of the protocol, we present two trapdoor claw-free function constructions, based on Rabin’s function and the Diffie–Hellman problem, which have not been used in this context before. We also present two independent innovations that improve the efficiency of our implementation and can be applied to other quantum cryptographic protocols. First, we give a scheme to discard so-called garbage bits, removing the need for reversibility in the quantum circuits. Second, we show a natural way of performing postselection that reduces the fidelity needed to demonstrate quantum advantage. Combining these results, we describe a blueprint for implementing our protocol on Rydberg atom-based quantum devices, using hardware-native operations that have already been demonstrated experimentally.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Product Consistency Test and Vapor Hydration Test Comparisons of a Radioactive Hanford Waste Glass with its Non-Radioactive Simulant Glass

Product Consistency Tests (PCT) and Vapor Hydration Tests (VHT) were performed on a radioactive glass made from Hanford tank AP-107 waste (i.e., R-AP-107) and its non-radioactive equivalent glass made from simulated tank waste (i.e., S-AP-107). The radioactive and non-radioactive experiments were conducted in parallel with identical equipment, protocols, and staff to remove additional experimental uncertainties. PCT and VHT results are reported for all samples and are shown in Table S.1. Average normalized mass loss of B ( NL B ) and Na ( NL Na ) calculated from the three replicate test results for each glass with their respective standard deviations (SD) are presented for PCT results while individual VHT rates ( r VHT ) with their propagated errors ( σ r prop ) from individual thickness measurements on each sample are presented for the VHT results. Some VHT samples experienced more than 50% evaporation loss during the experiments. However, no visible alteration was detected on any VHT samples regardless on the amount of evaporation loss detected as shown in Table S.1.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2022

Detailed Laboratory Evaluation of Electric Demand Load Shifting Potential of Controlled Heat Pump Water Heaters

The demand profile management of electric end uses is vital research for utilities and policymakers planning greenhouse gas emission reductions. In this study, detailed laboratory research was conducted on the load shifting potential of 4 grid-connected HPWHs and one electric resistance water heaters (ERWH). The testing applied different CTA-2045 shed and critical peak command designs under three water draw profiles. Highly-controlled laboratory experiments were conducted in Florida. One of the four HPWHs was a prototype incorporating the new CTA-2045-B protocol feature allowing ‘advanced’ load-up above tank setpoint. A three-hour morning (6 – 9 AM) and four-hour evening curtailment (4 – 8 PM) were defined as the shed or critical peak periods reflecting high-value control periods for utility coincident load for system-wide electric demand reductions. Tests were performed under baseline conditions (no load shift) and under varied load-shifting schemes, including load up and advanced load up, ahead of shed and critical peak commands. Data were collected from December 2020 – February 2022 in the laboratory and compared with field experiments in Florida and the Pacific Northwest. Grid-connected HPWHs were found to reduce peak demand by up to 0.47 kW compared to uncontrolled HPWH units, depending on time of day, control scheme, draw profile, and temperature cluster. The load up strategy demonstrated the ability of all units to utilize heat pump mode for extended periods ahead of peak events. Demand reductions for the HPWHs were much larger when compared with the ERWH— up to 1.64 kW with large hot water draws in winter.

Fenaughty, Karen↗
DOE OSTI2021

Review of Design Elements within Power Infrastructure Cyber–Physical Test Beds as Threat Analysis Environments

Cyber–physical systems (CPSs) are an integral part of modern society; thus, enhancing these systems’ reliability and resilience is paramount. Cyber–physical testbeds (CPTs) are a safe way to test and explore the interplay between the cyber and physical domains and to cost-effectively enhance the reliability and resilience of CPSs. Here a review of CPT elements, broken down into physical components (simulators, emulators, and physical hardware), soft components (communication protocols, network timing protocols), and user interfaces (visualization-dashboard design considerations) is presented. Various methods used to validate CPS performance are reviewed and evaluated for potential applications in CPT performance validation. Last, initial simulated results for a CPT design, based on the IEEE 33 bus system, are presented, along with a brief discussion on how model-based testing and fault–injection-based testing (using scaling and ramp-type attacks) may be used to help validate CPT performance.

14 SOLAR ENERGY↗
DOE OSTI2021

Dip leg plug formation evaluation and abatement testing in support of the H-Canyon accelerated basin de-inventory

As requested in the Technical Task Request (TTR), the Savannah River National Laboratory (SRNL) performed testing on the current typical bubbler dip leg design used in the H-Area Canyon (HCA). The TTR has a functional classification of General Services. Testing was initiated to ascertain if the plugging described in the TTR could be reproduced and, if so, to obtain general metrics on the resulting plugging dynamics, and evaluate possible mitigating methods, including design changes and cleaning/purging protocols. The bubblers were to be suspended in a simulant typical of H-Canyon solutions containing aluminum nitrate and silica at a temperature of 25°C using a nominal bubbler air flowrate of 7.5 SCFH. Emphasis was placed on minimizing equipment changes necessary to accomplish the mitigations, as equipment changes in the existing facility would be costly. Testing evolved into four (4) phases, concluding that plugging could indeed be reproduced (believed to be attributed to localized evaporation and bridging of crystals in the dip legs). During testing, plug formation information was obtained, including time, pressure, and temperature traces leading to plugging were defined; solution compositions leading to plugging were determined, and various physical characteristics (visual and mechanical) of the crystallized plug were obtained.

42 ENGINEERING↗
DOE OSTI2024

Marine Algae Industrialization Consortium (MAGIC): Combining biofuel and high-value bioproducts to meet the RFS

The Marine Algae Industrialization Consortium (MAGIC) was formed to address pressing challenges in the commercialization of microalgae as a source of biofuel. The “Marine Algae Industrialization Consortium (MAGIC): Combining biofuel and high-value bioproducts to meet the RFS” project formally addressed two US Department of Energy Bioenergy Technologies Office (BETO) goals: (1) Model the sustainable supply of 1 million metric tonnes ash free dry weight (AFDW) cultivated algal biomass and (2) Demonstrate valuable co-products produced along with biofuel intermediates to increase value of algal biomass by 30%. To achieve these goals, the project demonstrated and validated high-value co-products to drive down the cost of biofuel by increasing the value of algae “co-products” towards increasing the selling price of total algae biomass as one of the key drivers of economics and adoption. This was accomplished through five core, interdependent tasks including: (1) strain selection to identify and deliver strains for mass culture, (2) mass culture using a hybrid cultivation system and following key operating parameters for downstream applications to provide algae feedstock, (3) recovery and conversion to evaluate two alternative methods to separate dry algae biomass into oil and residuals for downstream testing, (4) product assessment to determine biofuel, aquafeed or poultry feed product efficacy using algae biomass fractions as well as to provide critical performance data for valuation and (5) commercialization to use technoeconomic and life cycle assessments (TEA/LCA) as iterative design and assessment tools including consideration of target markets, competitors, and distribution channels to guide product assessment, development and valuation. A total of 46 peer-review publications, many open-access, provide detail of much of the work carried out and the results of the tasks. Additional reports and presentations provide other technical and public engagement material. At a high level, using a variety of approaches, more than 1000 marine microalgae strains were evaluated to ultimately identify the seven winners that were down-selected to be grown in mass culture. Strain selection demonstrated that there were no ‘super strains’ and that each candidate had strengths and limitations for specific products, growth conditions or operational considerations. Mass culture growth of these seven strains at >5000 L / 29 m 2 scale found that four them were suitable for product assessment. More than 250 kg of biomass was produced across hundreds of pond runs along with thousands of cultivation entries on the growth and biomass characteristics as well as environmental parameters. In the process, dozens of standard operating procedures were generated as was custom software to process and analyze cultivation data. Recovery and conversion of algae biomass demonstrated that a hexane solvent based extraction protocol was most effective at recovering oil (biocrude) from algae and four strains were processed to produce oil and lipid extracted algae (residuals) for downstream testing. Membrane-based oil separation was less successful, but may still be applicable to other commercial applications in the future. Product testing demonstrated that algae biocrude is of high quality and hydrotreating generated numerous fractions of high quality composition for fuel and lubricate based applications. Aquafeed studies performed at a variety of scales showed that both whole and defatted (lipid extracted algae) microalgae were suitable as a feed ingredient, but that the specifics of the fed animal and biochemical composition of the algae are critical factors when determining formulation. Similarly, poultry studies on whole and defatted microalgae generally showed positive outcomes on animal growth and health, with some microalgae providing enhanced nutritional composition of the animal product. Economic and life cycle assessments covered a wide range of possible commercialization and sustainability scenarios. Replacement value, improved product value added, consumer values marketing added valuation and improved animal health were considered as alternatives for microalgae valuation. Using the open pond system, algae productivity was identified as the key driver of commercialization economics, but combination of co-products (e.g. animal feed) with biofuel production substantially increased the total selling price of algae. Modeled microalgae selling price exceeded $\$$1500/tonne and could generate competitive biofuel selling prices below $\$$5 gallon gas equivalents using realistic algal productivities. Short (process scale) and longer (decadal trends) sustainability assessments show that marine microalgae can enhance the sustainability of energy production and lead to other realized benefits in water, fertilizer and land use for other sectors (e.g. agriculture). This project successfully demonstrated all of the components of an end-to-end process from mass microalgae cultivation and dewatering, to recovery and conversion of algae biomass components, to final product demonstration and process valuation; the combined results provide a framework for future commercialization of algae based biofuels.

09 BIOMASS FUELS↗
DOE OSTI2024

Hamiltonian switching control of noisy bipartite qubit systems

Abstract We develop a Hamiltonian switching ansatz for bipartite control that is inspired by the quantum approximate optimization algorithm, to mitigate environmental noise on qubits. We demonstrate the control for a central spin coupled to bath spins via isotropic Heisenberg interactions, and then make physical applications to the protection of quantum gates performed on superconducting transmon qubits coupling to environmental two-level-systems (TLSs) through dipole-dipole interactions, as well as on such qubits coupled to both TLSs and a Lindblad bath. The control field is classical and acts only on the system qubits. We use reinforcement learning with policy gradient to optimize the Hamiltonian switching control protocols, using a fidelity objective for specific target quantum gates. We use this approach to demonstrate effective suppression of both coherent and dissipative noise, with numerical studies achieving target gate implementations with fidelities over 0.9999 (four nines) in the majority of our test cases and showing improvement beyond this to values of 0.999 999 999 (nine nines) upon a subsequent optimization by GRadient Ascent Pulse Engineering (GRAPE). We analyze how the control depth, total evolution time, number of environmental TLS, and choice of optimization method affect the fidelity achieved by the optimal protocols and reveal some critical behaviors of bipartite control of quantum gates.

Physics↗
DOE OSTI2022

Development of a Configurable DERMS Test System in GridAPPS-D

Distributed Energy Resources and their increased penetration within the power grid are a suitable method to provide essential reliability services. In order to do so, they must operate in aggregate. Aggregation of a wide variety of systems with varying standards and protocols is a complex task, which could benefit from rules and standards to support interoperability. To this end, the Portland State University Power Lab has developed the Energy Grid of Things project, which includes a Distributed Energy Resource Management System to provide aggregation and demonstrate the usage of an Energy Services Interface: a set of rules governing the interface between resource owners and aggregator businesses. This thesis presents a Modeling Environment that provides a simulation and test platform reflecting the complex needs of an aggregator. A grid simulator is required that is able to reflect the effects of thousands of Distributed Energy Resources operating in tandem; furthermore, these effects must be communicated to external actors such as the Grid Operator, which provides grid service requests to the aggregator and requires feedback. The Modeling Environment provides all of this functionality by leveraging the GridAPPS-D platform to develop an interoperable class-based system that provides configurable simulations with modular input and output APIs. The Modeling Environment uses abstractions to allow any feasible type of input to be reflected within the model as a Distributed Energy Resource, and serves as a proof-of-concept for the development of aggregation simulation and testing within GridAPPS-D.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

Timing and synchronization of the DUNE neutrino detector

The DUNE neutrino experiment far detector has a fiducial mass of 40 kt. The O(1 M) readout channels are distributed over the 4 x 10kt modules and need to be synchronized to O(10 ns) with a reliable, simple, affordable system. For the majority of channels a simple DC-balanced protocol is used, with clock and synchronization information encoded on the same fibre. The remaining channels use “White Rabbit” (IEEE-1588). Small scale tests show a timing jitter of <100 ps. The DUNE timing system has been successfully prototyped at the ProtoDUNE-SP detector at CERN.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Impact of Electrolyte Additives on the Lifetime of High Voltage NMC Lithium-Ion Pouch Cells

This work involves improving the lifetime of lithium-ion cells during high voltage cycling using electrolyte additives. Three generations of electrolyte additives were investigated and screened in NMC442/graphite pouch cells using a 24 h voltage-hold protocol at 40 degrees C to accelerate oxidative reactions occurring at 4.4 V. Once promising additives and combinations were identified, they were then tested in cobalt-free NMC640/graphite cells for long-term cycling to upper cutoff voltages of 4.3, 4.4, and 4.5 V at temperatures of 20, 40, and 55 degrees C. Degradation mechanisms were probed using dV/dQ analysis, micro-X-ray fluorescence spectroscopy, and electrochemical impedance spectroscopy. The primary failure mode of cells held at high voltages is due to increase in cell impedance, which is correlated to the dissolution of transition metals, specifically manganese, originating from the positive electrode. We believe this dissolution is presumably due to the formation of a high impedance rock salt surface layer on the NMC positive electrode particles. Such deleterious outcomes can be limited by selecting an appropriate electrolyte additive package. It is hoped that this paper can provide a starting point for developing NMC Li-ion cells that can operate to voltages as high as 4.4 V and still display long lifetimes.

Azam, Saad↗
DOE OSTI2023

Open Source Evaluation Framework for Solar Forecasting

The Solar Forecast Arbiter is an open-source evaluation framework for solar forecasting. The framework enables evaluations of solar irradiance, solar power, and net-load forecasts that are impartial, repeatable and auditable. The Solar Forecast Arbiter addresses stakeholder-informed use cases including evaluation of forecast skill, comparisons to reference data sets, private forecast trials, and evaluation of probabilistic forecast skill. The framework includes a data validation toolkit, reference data sources, data privacy protocols, and benchmark forecast capabilities for intra-hour and day ahead forecast horizons. Reports and metrics communicate the relative merits of the test and benchmark forecasts. The reports are created from standardized templates and include graphics for qualitatively evaluating deterministic and probabilistic forecasts and standard metrics for quantitatively evaluating forecasts. The Solar Forecast Arbiter is designed to support all solar forecasting stakeholders, including Solar Forecasting 2 Topic Area 2 and Topic Area 3 teams.

14 SOLAR ENERGY↗