Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 847 records · Page 47

Multivariate exploration of non-intrusive load monitoring via spatiotemporal pattern network

Non-intrusive load monitoring (NILM) of electrical demand for the purpose of identifying load components has thus far mostly been studied using univariate data, e.g., using only whole building electricity consumption time series to identify a certain type of end-use such as lighting load. However, using additional variables in the form of multivariate time series data may provide more information in terms of extracting distinguishable features in the context of energy disaggregation. In this work, a novel probabilistic graphical modeling approach, namely the spatiotemporal pattern network (STPN) is proposed for energy disaggregation using multivariate time-series data. The STPN framework is shown to be capable of handling diverse types of multivariate time-series to improve the energy disaggregation performance. The technique outperforms the state of the art factorial hidden Markov models (FHMM) and combinatorial optimization (CO) techniques in multiple real-life test cases. Furthermore, based on two homes' aggregate electric consumption data, a similarity metric is defined for the energy disaggregation of one home using a trained model based on the other home (i.e., out-of-sample case). The proposed similarity metric allows us to enhance scalability via learning supervised models for a few homes and deploying such models to many other similar but unmodeled homes with significantly high disaggregation accuracy.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Evaluating electrolyte additives for lithium-ion cells: A new Figure of Merit approach

Electrolyte additives are known to improve the performance of lithium-ion cells. In this work we examine the performance of Li 1.03 Ni 0.5 Mn 0.3 Co 0.3 O 2 -graphite (NMC532/Gr) cells containing combinations of lithium bis(oxalate)borate (LiBOB), vinylene carbonate (VC), trivinylcyclotriboroxane (tVCBO), prop-1-ene-1,3-sultone (PES), phenyl boronic acid ethylene glycol ester (PBE), tris(trimethylsilyl) phosphite (TMSPi), triethylphosphite (TEPi), and lithium difluoro(oxalate)borate (LiDFOB) added to our baseline (1.2M LiPF 6 in EC:EMC, 3:7 w/w) electrolyte. In order to rank performance of the various electrolytes, we developed two separate figures of merit (FOM), which are based on the energy retention and power retention of the cells. Using these two metrics in conjunction, we show that only one of the fifteen electrolyte formulations tested significantly outperforms the baseline electrolyte: this electrolyte contains the 0.25 wt% tVCBO + 1 wt% TMSPi additive mix. Little correlation was observed between the FOMs for energy retention and power retention, which indicates that the mechanisms that govern these performance parameters are likely independent of each other. In conclusion, our FOM approach has general applicability and can be used to develop electrolyte and electrode formulations that prolong the life of lithium-ion batteries.

25 ENERGY STORAGE↗

An in silico method to assess antibody fragment polyreactivity

Antibodies are essential biological research tools and important therapeutic agents, but some exhibit non-specific binding to off-target proteins and other biomolecules. Such polyreactive antibodies compromise screening pipelines, lead to incorrect and irreproducible experimental results, and are generally intractable for clinical development. Here, we design a set of experiments using a diverse naïve synthetic camelid antibody fragment (nanobody) library to enable machine learning models to accurately assess polyreactivity from protein sequence (AUC > 0.8). Moreover, our models provide quantitative scoring metrics that predict the effect of amino acid substitutions on polyreactivity. We experimentally test our models’ performance on three independent nanobody scaffolds, where over 90% of predicted substitutions successfully reduced polyreactivity. Importantly, the models allow us to diminish the polyreactivity of an angiotensin II type I receptor antagonist nanobody, without compromising its functional properties. We provide a companion web-server that offers a straightforward means of predicting polyreactivity and polyreactivity-reducing mutations for any given nanobody sequence.

60 APPLIED LIFE SCIENCES↗

Realistic galaxy image simulation via score-based generative models

ABSTRACT We show that a denoising diffusion probabilistic model (DDPM), a class of score-based generative model, can be used to produce realistic mock images that mimic observations of galaxies. Our method is tested with Dark Energy Spectroscopic Instrument (DESI) grz imaging of galaxies from the Photometry and Rotation curve OBservations from Extragalactic Surveys (PROBES) sample and galaxies selected from the Sloan Digital Sky Survey. Subjectively, the generated galaxies are highly realistic when compared with samples from the real data set. We quantify the similarity by borrowing from the deep generative learning literature, using the ‘Fréchet inception distance’ to test for subjective and morphological similarity. We also introduce the ‘synthetic galaxy distance’ metric to compare the emergent physical properties (such as total magnitude, colour, and half-light radius) of a ground truth parent and synthesized child data set. We argue that the DDPM approach produces sharper and more realistic images than other generative methods such as adversarial networks (with the downside of more costly inference), and could be used to produce large samples of synthetic observations tailored to a specific imaging survey. We demonstrate two potential uses of the DDPM: (1) accurate inpainting of occluded data, such as satellite trails, and (2) domain transfer, where new input images can be processed to mimic the properties of the DDPM training set. Here we ‘DESI-fy’ cartoon images as a proof of concept for domain transfer. Finally, we suggest potential applications for score-based approaches that could motivate further research on this topic within the astronomical community.

79 ASTRONOMY AND ASTROPHYSICS↗

Combining Machine Learning and Comparative Effectiveness Methodology to Study Primary Care Pharmacotherapy Pathways for Veterans With Depression

Our objective is to demonstrate an innovative method combining machine learning with comparative effectiveness research techniques and to investigate a hitherto unstudied question about the effectiveness of common prescribing patterns. For Operation Enduring Freedom/Operation Iraqi Freedom veterans with major depressive disorder, we generate pharmacotherapy pathways (of antidepressants) using process mining and machine learning. We select the medication episodes that were started at subtherapeutic doses by the first assigned primary care physician and observe the paths that those medication episodes follow. Using 2-stage least squares, we test the effectiveness of starting at a low dose and staying low for longer versus ramping up fast while balancing observable and unobservable characteristics of patients and providers through instrumental variables. We leverage predetermined provider practice patterns as instruments. We collected outpatient pharmacy data for selective serotonin reuptake inhibitors and selective norepinephrine reuptake inhibitors, patient and provider characteristics (as control variables), and the instruments for our cohort. All data were extracted for the period between 2006 and 2020. There is a statistically significant positive effect (0.68, 95% CI 0.11–1.25) of “ramping up fast” on engagement in care. When we examine the effect of “ramping up slow”, we see an insignificant negative impact on engagement in care (−0.82, 95% CI −1.89 to 0.25). As expected, the probability of drop-out also seems to have a negative effect on engagement in care (−0.39, 95% CI −0.94 to 0.17). We further validate these results by testing with medication possession ratios calculated periodically as an alternative engagement in care metric. Our findings contradict the “Start low, go slow” adage, indicating that ramping up the dose of an antidepressant faster has a significantly positive effect on engagement in care for our population.

60 APPLIED LIFE SCIENCES↗

On the Impact of High-Order Harmonic Generation in Electrical Distribution Systems

The modern power grid has seen a rise in the integration of non-linear loads, presenting a significant concern for operators. These loads introduce unwanted harmonics, leading to potential issues such as overheating and improper functioning of circuit breakers. In pursuing a more sustainable grid, the adoption of electric vehicles (EVs) and photovoltaic (PV) systems in residential networks has increased. Understanding and examining the effects of high-order harmonic frequencies beyond $1.5$ kHz is crucial to understanding their impact on the operation and planning of electrical distribution systems under varying nonlinear loading conditions. This study investigates a diverse set of critical power electronic loads within a household modeled using PSCAD/EMTdc, analyzing their unique harmonic spectra. This information is utilized to run the time-series harmonic analysis program in OpenDSS on a modified IEEE 34 bus test system model. The impact of high-order harmonics is quantified using metrics that evaluate total harmonic distortion (THD), transformer harmonic-driven eddy current loss component, and propagation of harmonics from the source to the substation transformer.

Peerzada, Aaqib A. [BATTELLE (PACIFIC NW LAB)]↗

SU-E-T-581: A Comparative Study of Standard IMRT and VMAT Planning Techniques for Unilateral and Bilateral Head and Neck Irradiation

Purpose: To compare dosimetric properties and monitor units (MU) of IMRT plans with several VMAT head and neck (H and N) plans. Methods: Seventeen unilateral H and N (UHN) and five bilateral H and N (BHN) patients initially treated with IMRT were replanned with VMAT. Several arc arrangements were studied for each patient: 1)for UHN, two 360° arcs, two 260° arcs, two 210° arcs, two 360° arcs with contralateral avoidance sectors, and 2)for BHN, two 360° arcs, two 360° arcs with bilateral avoidance sectors, two 360° arcs with bilateral avoidance sectors and a third arc limited to the upper neck. Optimization constraints were adjusted for each patient and plan. All plans were normalized to achieve the same highest-dose PTV coverage. Percent differences (IMRT-VMAT)/VMAT in MU, dose homogeneity (HI=maximum point dose/prescription dose), and organ-at-risk (OAR) metrics are reported and statistical significance evaluated (p<0.05; paired Student t-test). Results: Average reduction in MU with VMAT was 28% for UHN (p<0.0001) and 63% for BHN (p<0.0001). Average HI for UHN IMRT and 360° arc VMAT plans was 1.08 and for plans with arcs <360° average HI=1.10. Average HI for BHN IMRT was 1.07, for three-arc VMAT 1.08, and for two-arc VMAT 1.11. For UHN, two 210° arcs achieved lower contralateral parotid max (−2.6 Gy, p<0.02) and mean (−1.2 Gy, p=0.06) dose. For BHN two-arc plans, contralateral parotid mean dose increased (3.3 Gy, p<0.04) and larynx max dose increased (2.9 Gy, p<0.02) with no change in larynx mean dose. Conclusion: For UHN, 360degree arc VMAT consistently produced plans dosimetrically comparable to IMRT with the benefit of lower MU. VMAT with arcs <360degrees produced plans inferior to IMRT in dose homogeneity and without significantly improved OAR sparing. For BHN, three-arc plans were dosimetrically comparable to IMRT with lower MU, while two-arc plans were inferior to IMRT in HI and OAR dose. Research supported in part by a Kaye Family Award.

60 APPLIED LIFE SCIENCES↗

Gene expression of functionally-related genes coevolves across fungal species: detecting coevolution of gene expression using phylogenetic comparative methods

Researchers often measure changes in gene expression across conditions to better understand the shared functional roles and regulatory mechanisms of different genes. Analogous to this is comparing gene expression across species, which can improve our understanding of the evolutionary processes shaping the evolution of both individual genes and functional pathways. One area of interest is determining genes showing signals of coevolution, which can also indicate potential functional similarity, analogous to co-expression analysis often performed across conditions for a single species. However, as with any trait, comparing gene expression across species can be confounded by the non-independence of species due to shared ancestry, making standard hypothesis testing inappropriate. We compared RNA-Seq data across 18 fungal species using a multivariate Brownian Motion phylogenetic comparative method (PCM), which allowed us to quantify coevolution between protein pairs while directly accounting for the shared ancestry of the species. Our work indicates proteins which physically-interact show stronger signals of coevolution than randomly-generated pairs. Interactions with stronger empirical and computational evidence also showing stronger signals of coevolution. We examined the effects of number of protein interactions and gene expression levels on coevolution, finding both factors are overall poor predictors of the strength of coevolution between a protein pair. Simulations further demonstrate the potential issues of analyzing gene expression coevolution without accounting for shared ancestry in a standard hypothesis testing framework. Furthermore, our simulations indicate the use of a randomly-generated null distribution as a means of determining statistical significance for detecting coevolving genes with phylogenetically-uncorrected correlations, as has previously been done, is less accurate than PCMs, although is a significant improvement over standard hypothesis testing. These methods are further improved by using a phylogenetically-corrected correlation metric. Our work highlights potential benefits of using PCMs to detect gene expression coevolution from high-throughput omics scale data. This framework can be built upon to investigate other evolutionary hypotheses, such as changes in transcription regulatory mechanisms across species.

59 BASIC BIOLOGICAL SCIENCES↗

Wave Energy Prize - 1/20th Testing - SEWEC Attenuator

Data from the 1/20th scale testing data completed on the Wave Energy Prize for the SEWEC team, including the 1/20th scale test plan, raw test data, video, photos, and data analysis results. The top level objective of the 1/20th scale device testing is to obtain the necessary measurements required for determining Average Climate Capture Width per Characteristic Capital Expenditure (ACE) and the Hydrodynamic Performance Quality (HPQ), key metrics for determining the Wave Energy Prize (WEP) winners. * Note: During the TG4 judging meeting, the Wave Energy Prize judges reviewed the data collected during the testing of SEWEC's device at Carderock and determined that the data were inconclusive and did not allow an ACE value to be calculated for the device. Consequently, the SEWEC device was deemed ineligible to be considered for the Wave Energy Prize.

1/20th scale↗

Silicon Damage Response Function Derivation and Verification: Assessment of Impact on ASTM Standard E722

Unsuccessful attempts by members of the radiation effects community to independently derive the Norgett-Robinson-Torrens (NRT) damage energy factors for silicon in ASTM standard E722-14 led to an investigation of the software coding and data that produced those damage energy factors. The ad hoc collaboration to discover the reason for lack of agreement revealed a coding error and resulted in a report documenting the methodology to produce the response function for the standard. The recommended changes in the NRT damage energy factors for silicon are shown to have significant impact for a narrow energy region of the 1-MeV(Si) equivalent fluence response function. However, when evaluating integral metrics over all neutrons energies in various spectra important to the SNL electronics testing community, the change in the response results in a small decrease in the total 1- MeV(Si) equivalent fluence of ~0.6% compared to the E722-14 response. Response functions based on the newly recommended NRT damage energy factors have been produced and are available for users of both the NuGET and MCNP codes.

36 MATERIALS SCIENCE↗

Identify and Quantify the Mechanistic Sources of Sensor Performance Variation Between Individual Sensors SN1 and SN2

This Technical Letter Report satisfies the M3AR-14PN2301022 milestone, and is focused on identifying and quantifying the mechanistic sources of sensor performance variation between individual 22-element, linear phased-array sensor prototypes, SN1 and SN2. This effort constitutes an iterative evolution that supports the longer term goal of producing and demonstrating a pre-manufacturing prototype ultrasonic probe that possesses the fundamental performance characteristics necessary to enable the development of a high-temperature sodium-cooled fast reactor inspection system. The scope of the work for this portion of the PNNL effort conducted in FY14 includes performing a comparative evaluation and assessment of the performance characteristics of the SN1 and SN2 22 element PA-UT probes manufactured at PNNL. Key transducer performance parameters, such as sound field dimensions, resolution capabilities, frequency response, and bandwidth are used as a metric for the comparative evaluation and assessment of the SN1 and SN2 engineering test units.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Gen 3 Particle Pilot Plant (G3P3) -- High-Temperature Particle System for Concentrating Solar Power (Phases 1 and 2)

The U.S. Department of Energy Solar Energy Technologies Office initiated the Generation 3 Concentrating Solar Power (CSP) program to achieve higher operating temperatures (>700 °C) to enable next-generation CSP high-temperature power cycles such as the supercritical CO 2 (sCO2) Brayton Cycle. Three teams were selected to pursue high-temperature gas, liquid, and solid pathways for the heat-transfer media. Phases 1 and 2, which lasted from 2018 – 2020, consisted of design, modeling, and testing activities to further de-risk each of the technologies and develop a design for construction, commissioning, and operation of a pilot-scale facility in Phase 3 (2021 – 2024). This report summarizes the activities in Phases 1 and 2 for the solid-particle pathway led by Sandia National Laboratories. In Phases 1 and 2, Sandia successfully de-risked key elements of the proposed Gen 3 Particle Pilot Plant (G3P3) by improving the design, operation, and performance of key particle component technologies including the receiver, storage bins, particle-to-sCO2 heat exchanger, particle lift, and data acquisition and controls. Modeling and testing of critical components have led to optimized designs that meet desired performance metrics. Detailed drawings, piping and instrumentation diagrams, and process flow diagrams were generated for the integrated system, and structural analyses of the assembled tower structure were performed to demonstrate compliance with relevant codes and standards. Instrumentation and control systems of key subsystems were also demonstrated. Together with Bridgers & Paxton, Bohannan Huston, and Sandia Facilities, we have completed a 100% G3P3 tower design package with stamped engineering drawings suitable for construction bid in Phase 3.

14 SOLAR ENERGY↗

Architecture and Functionality of the Advanced Life Support On-Line Project Information System (OPIS)

An ongoing effort is underway at NASA Amcs Research Center (ARC) tu develop an On-line Project Information System (OPIS) for the Advanced Life Support (ALS) Program. The objective of this three-year project is to develop, test, revise and deploy OPIS to enhance the quality of decision-making metrics and attainment of Program goals through improved knowledge sharing. OPIS will centrally locate detailed project information solicited from investigators on an annual basis and make it readily accessible by the ALS Community via a web-accessible interface. The data will be stored in an object-oriented relational database (created in MySQL(Trademark) located on a secure server at NASA ARC. OPE will simultaneously serve several functions, including being an R&TD status information hub that can potentially serve as the primary annual reporting mechanism. Using OPIS, ALS managers and element leads will be able to carry out informed research and technology development investment decisions, and allow analysts to perform accurate systems evaluations. Additionally, the range and specificity of information solicited will serve to educate technology developers of programmatic needs. OPIS will collect comprehensive information from all ALS projects as well as highly detailed information specific to technology development in each ALS area (Waste, Water, Air, Biomass, Food, Thermal, and Control). Because the scope of needed information can vary dramatically between areas, element-specific technology information is being compiled with the aid of multiple specialized working groups. This paper presents the current development status in terms of the architecture and functionality of OPIS. Possible implementation approaches for OPIS are also discussed.

Hogan, John A.↗

Autonomous Multi-Sensor Coordination: The Science Goal Monitor

Many dramatic earth phenomena are dynamic and coupled. In order to fully understand them, we need to obtain timely coordinated multi-sensor observations from widely dispersed instruments. Such a dynamic observing system must include the ability to Schedule flexibly and react autonomously to sciencehser driven events; Understand higher-level goals of a sciencehser defined campaign; Coordinate various space-based and ground-based resources/sensors effectively and efficiently to achieve goals. In order to capture transient events, such a 'sensor web' system must have an automated reactive capability built into its scientific operations. To do this, we must overcome a number of challenges inherent in infusing autonomy. The Science Goal Monitor (SGM) is a prototype software tool being developed to explore the nature of automation necessary to enable dynamic observing. The tools being developed in SGM improve our ability to autonomously monitor multiple independent sensors and coordinate reactions to better observe dynamic phenomena. The SGM system enables users to specify what to look for and how to react in descriptive rather than technical terms. The system monitors streams of data to identify occurrences of the key events previously specified by the scientisther. When an event occurs, the system autonomously coordinates the execution of the users' desired reactions between different sensors. The information can be used to rapidly respond to a variety of fast temporal events. Investigators will no longer have to rely on after-the-fact data analysis to determine what happened. Our paper describes a series of prototype demonstrations that we have developed using SGM and NASA's Earth Observing-1 (EO-1) satellite and Earth Observing Systems' Aqua/Terra spacecrafts' MODIS instrument. Our demonstrations show the promise of coordinating data from different sources, analyzing the data for a relevant event, autonomously updating and rapidly obtaining a follow-on relevant image. SGM was used to investigate forest fires, floods and volcanic eruptions. We are now identifying new Earth science scenarios that will have more complex SGM reasoning. By developing and testing a prototype in an operational environment, we are also establishing and gathering metrics to gauge the success of automating science campaigns.

Koratkar, Anuradha↗

Autonomous Multi-sensor Coordination: The Science Goal Monitor

Next-generation science and exploration systems will employ new observation strategies that will use multiple sensors in a dynamic environment to provide high quality monitoring, self-consistent analyses and informed decision making. The Science Goal Monitor (SGM) is a prototype software tool being developed to explore the nature of automation necessary to enable dynamic observing of earth phenomenon. The tools being developed in SGM improve our ability to autonomously monitor multiple independent sensors and coordinate reactions to better observe the dynamic phenomena. The SGM system enables users to specify events of interest and how to react when an event is detected. The system monitors streams of data to identify occurrences of the key events previously specified by the scientist/user. When an event occurs, the system autonomously coordinates the execution of the users desired reactions between different sensors. The information can be used to rapidly respond to a variety of fast temporal events. Investigators will no longer have to rely on after-the-fact data analysis to determine what happened. Our paper describes a series of prototype demonstrations that we have developed using SGM and NASA's Earth Observing-1 (EO-1) satellite and Earth Observing Systems Aqua/Terra spacecrafts MODIS instrument. Our demonstrations show the promise of coordinating data from different sources, analyzing the data for a relevant event, autonomously updating and rapidly obtaining a follow-on relevant image. SGM is being used to investigate forest fires, floods and volcanic eruptions. We are now identifying new earth science scenarios that will have more complex SGM reasoning. By developing and testing a prototype in an operational environment, we are also establishing and gathering metrics to gauge the success of automating science campaigns.

Koratkar, Anuradha↗

Architecture and Functionality of the Advanced Life Support On-Line Project Information System

An ongoing effort is underway at NASA Ames Research Center (ARC) to develop an On-line Project Information System (OPIS) for the Advanced Life Support (ALS) Program. The objective of this three-year project is to develop, test, revise and deploy OPIS to enhance the quality of decision-making metrics and attainment of Program goals through improved knowledge sharing. OPIS will centrally locate detailed project information solicited from investigators on an annual basis and make it readily accessible by the ALS Community via a Web-accessible interface. The data will be stored in an object-oriented relational database (created in MySQL) located on a secure server at NASA ARC. OPE will simultaneously serve several functions, including being an research and technology development (R&TD) status information hub that can potentially serve as the primary annual reporting mechanism for ALS-funded projects. Using OPIS, ALS managers and element leads will be able to carry out informed R&TD investment decisions, and allow analysts to perform accurate systems evaluations. Additionally, the range and specificity of information solicited will serve to educate technology developers of programmatic needs. OPIS will collect comprehensive information from all ALS projects as well as highly detailed information specific to technology development in each ALS area (Waste, Water, Air, Biomass, Food, Thermal, Controls and Systems Analysis). Because the scope of needed information can vary dramatically between areas, element-specific technology information is being compiled with the aid of multiple specialized working groups. This paper presents the current development status in terms of the architecture and functionality of OPIS. Possible implementation approaches for OPIS are also discussed.

Hogan, John A.↗

Post-Flight Analysis of the Guidance, Navigation, and Control Performance During Orion Exploration Flight Test 1

The first test flight of the Orion Multi-Purpose Crew Vehicle presented additional challenges for guidance, navigation and control as compared to a typical re-entry from the International Space Station or other Low Earth Orbit. An elevated re-entry velocity and steeper flight path angle were chosen to achieve aero-thermal flight test objectives. New IMU's, a GPS receiver, and baro altimeters were flight qualified to provide the redundant navigation needed for human space flight. The guidance and control systems must manage the vehicle lift vector in order to deliver the vehicle to a precision, coastal, water landing, while operating within aerodynamic load, reaction control system, and propellant constraints. Extensive pre-flight six degree-of-freedom analysis was performed that showed mission success for the nominal mission as well as in the presence of sensor and effector failures. Post-flight reconstruction analysis of the test flight is presented in this paper to show whether that all performance metrics were met and establish how well the pre-flight analysis predicted the in-flight performance.

Barth, Andrew↗

Development and Validation of an Empirical Ocean Color Algorithm with Uncertainties: A Case Study with the Particulate Backscattering Coefficient

We explored how algorithm (model) and in situ measurement (observation) uncertainties can effectively be incorporated into empirical ocean color model development and assessment. In this study we focused on methods for deriving the particulate backscattering coefficient at 555 nm, b(bp)(555)/(m). We developed a simple empirical algorithm for deriving b(bp)(555) as a function of a remote sensing reflectance line height (LH) metric. Model training was performed using a high-quality bio-optical dataset that contains coincident in situ measurements of the spectral remote sensing reflectances, R(rs)(λ)/(sr), and the spectral particulate backscattering coefficients, b(bp)(λ). The LH metric used is defined as the magnitude of Rrs(555) relative to a linear baseline drawn between R(rs)(490) and R(rs)(670). Using an independent validation dataset, we compared the skill of the LH-based model with two other models. We used contemporary validation metrics, including bias and mean absolute error (MAE), that were corrected for model and observation uncertainties. The results demonstrated that measurement uncertainties do indeed impact contemporary validation metrics such as mean bias and MAE. Zeta-scores and z-tests for overlapping confidence intervals were also explored as potential methods for assessing model skill.

ocean color↗