Search NASA⌕ Search

SEARCH · Search NASA

Results for “blind comparison”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

A computational modeling framework for pre-clinical evaluation of cardiac mapping systems

There are a variety of difficulties in evaluating clinical cardiac mapping systems, most notably the inability to record the transmembrane potential throughout the entire heart during patient procedures which prevents the comparison to a relevant “gold standard”. Cardiac mapping systems are comprised of hardware and software elements including sophisticated mathematical algorithms, both of which continue to undergo rapid innovation. The purpose of this study is to develop a computational modeling framework to evaluate the performance of cardiac mapping systems. The framework enables rigorous evaluation of a mapping system’s ability to localize and characterize (i.e., focal or reentrant) arrhythmogenic sources in the heart. The main component of our tool is a library of computer simulations of various dynamic patterns throughout the entire heart in which the type and location of the arrhythmogenic sources are known. Our framework allows for performance evaluation for various electrode configurations, heart geometries, arrhythmias, and electrogram noise levels and involves blind comparison of mapping systems against a “silver standard” comprised of computer simulations in which the precise transmembrane potential patterns throughout the heart are known. A feasibility study was performed using simulations of patterns in the human left atria and three hypothetical virtual catheter electrode arrays. Activation times (AcT) and patterns (AcP) were computed for three virtual electrode arrays: two basket arrays with good and poor contact and one high-resolution grid with uniform spacing. The average root mean squared difference of AcTs of electrograms and those of the nearest endocardial action potential was less than 1 ms and therefore appears to be a poor performance metric. In an effort to standardize performance evaluation of mapping systems a novel performance metric is introduced based on the number of AcPs identified correctly and those considered spurious as well as misclassifications of arrhythmia type; spatial and temporal localization accuracy of correctly identified patterns was also quantified. This approach provides a rigorous quantitative analysis of cardiac mapping system performance. Proof of concept of this computational evaluation framework suggests that it could help safeguard that mapping systems perform as expected as well as provide estimates of system accuracy.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

An international benchmark for wind plant wakes from the American WAKE ExperimeNt (AWAKEN)

This article introduces the first benchmark study within the International Energy Agency Wind Task 57 framework, focusing on wind plant wakes. Leveraging data from the American WAKE ExperimeNt (AWAKEN), the benchmark aims to assess the accuracy of simulation tools in modeling wind plant wakes and their impact on the downstream flow under diverse inflow conditions. The AWAKEN field campaign, conducted in Oklahoma from 2022 to 2024, provides unprecedented observations of wind plant-atmosphere interactions, thus offering a large dataset to validate numerical models of different complexity. The benchmark will include three phases—code calibration, blind comparison, and iteration—allowing participants to refine their numerical models based on the feedback from the benchmark team. This article describes the benchmark case study selected from observations providing details on atmospheric conditions, wake evidence, and wind turbine operation. The benchmark’s structure and timeline, along with the expected publication of results, are discussed as well. This collaborative effort aims to enhance the accuracy of wind plant wake simulations, thus contributing to the improvement of wind energy production estimates.

17 WIND ENERGY↗
DOE OSTI2025

Feature review of photovoltaic modeling software utilizing blind performance assessment

While confidence in photovoltaic (PV) modeling software has always been essential, the rapid pace of new PV plant developments makes accuracy and credibility more critical than ever. Independent assessments, particularly through blind modeling comparisons, are therefore necessary to ensure unbiased benchmarking across PV modeling software. Previous studies have been limited by a narrow range of models compared, anonymized results, or system size. This study presents results from the first-ever onymous blind modeling comparison, evaluated using both lab- and utility-scale fixed-tilt, monofacial, south-facing systems at sub-hourly time intervals. Seven commercially used PV software tools were compared: 3E SynaptiQ, PlantPredict, PVsyst, RatedPower, SAM, SolarFarmer, and Solargis Evaluate. Predictions were submitted directly by software representatives, providing unique insights into each software’s implementation and resulting prediction behavior. Notable features, including plane-of-array (POA) transposition model, module temperature model, shading model, and performance model were analyzed and compared. Four summary tables compile these features of the software, serving as a resource to help users understand the methodological differences and select the most suitable software for their applications. The software tools show deviations from mean error in annual yield up to 2.5 % in the lab-scale system, increasing to 6.0 % for the utility-scale system. These differences arise from a combination of user decisions and the inherent behavior of the software, indicating the need for continuous and rigorous validation of modeling methods using these software tools against complex, real-world systems.

14 SOLAR ENERGY↗
DOE OSTI2023

PV Performance Modeling - Data and Resources

The Photovoltaic (PV) Performance Modeling Collaborative (PVPMC) organized a blind PV performance modeling intercomparison to allow PV modelers to blindly test their models and modeling ability against real system data. Measured weather and irradiance data were provided along with detailed descriptions of PV systems from two locations (Albuquerque, New Mexico, USA and Roskilde, Denmark). Participants were asked to simulate the plane-of-array irradiance, module temperature, and DC power output from six systems and submit their results to Sandia for processing. This dataset includes seven MS-Excel sheets with instructions, notes and all necessary data (weather, irradiance, temperature, power) used for the data analysis of the blind modeling comparison. The hourly data represent six different systems from Albuquerque, NM and Roskilde, Denmark over a period of one year. These data are useful for PV performance model validation studies.

14 SOLAR ENERGY↗
DOE OSTI2023

Blind photovoltaic modeling intercomparison: A multidimensional data analysis and lessons learned

The Photovoltaic (PV) Performance Modeling Collaborative (PVPMC) organized a blind PV performance modeling intercomparison to allow PV modelers to blindly test their models and modeling ability against real system data. Measured weather and irradiance data were provided along with detailed descriptions of PV systems from two locations (Albuquerque, New Mexico, USA, and Roskilde, Denmark). Participants were asked to simulate the plane-of-array irradiance, module temperature, and DC power output from six systems and submit their results to Sandia for processing. The results showed overall median mean bias (i.e., the average error per participant) of 0.6% in annual irradiation and –3.3% in annual energy yield. While most PV performance modeling results seem to exhibit higher precision and accuracy as compared to an earlier blind PV modeling study in 2010, human errors, modeling skills, and derates were found to still cause significant errors in the estimates.

14 SOLAR ENERGY↗
DOE OSTI2023

Tidal Turbine Benchmarking Project: Stage I - Steady Flow Blind Predictions: Preprint

This paper presents the first blind prediction stage of the Tidal Turbine Benchmarking Project being conducted and funded by the UK's EPSRC and Supergen ORE Hub. In this first stage, only steady flow conditions, at low and elevated turbulence (3.1%) levels, were considered. Prior to the blind prediction stage, a large laboratory scale experiment was conducted in which a highly instrumented 1.6m diameter tidal rotor was towed through a large towing tank in well-defined flow conditions with and without an upstream turbulence grid. Details of the test campaign and rotor design were released as part of this community blind prediction exercise. Participants were invited to use a range of engineering modelling approaches to simulate the performance and loads of the turbine. 26 submissions were received from 12 groups from across academia and industry using solution techniques ranging from blade resolved computational fluid dynamics through actuator line, boundary integral element methods, vortex methods to engineering Blade Element Momentum methods. The comparisons between experiments and blind predictions were extremely positive helping to provide validation and uncertainty estimates for the models, but also validating the experimental tests themselves. The exercise demonstrated that the experimental turbine data provides a robust data set against which researchers and design engineers can test their models and implementations to ensure robustness in their processes, helping to reduce uncertainty and provide increased confidence in engineering processes. Furthermore, the data set provides the basis by which modellers can evaluate and refine approaches.

benchmarking↗
DOE OSTI2024

Outcomes and Conclusions from the 2022 AM Bench Measurements, Challenge Problems, Modeling Submissions, and Conference

The Additive Manufacturing Benchmark Test Series (AM Bench) provides rigorous measurement data for validating additive manufacturing (AM) simulations for a broad range of AM technologies and material systems. AM Bench includes extensive in situ and ex situ measurements, simulation challenges for the AM modeling community, and a corresponding conference series. In 2022, the second round of AM Bench measurements, challenge problems, and conference were completed, focusing primarily upon laser powder bed fusion (LPBF) processing of metals, and both material extrusion processing and vat photopolymerization of polymers. In all, more than 100 people from 10 National Institute of Standards and Technology (NIST) divisions and 21 additional organizations were directly involved in the AM Bench 2022 measurements, data management, and conference organization. The international AM community submitted 138 sets of blind modeling simulations for comparison with the in situ and ex situ measurements, up from 46 submissions for the first round of AM Bench in 2018. Analysis of these submissions provides valuable insight into current AM modeling capabilities. The AM Bench data are permanently archived and freely accessible online. The AM Bench conference also hosted an embedded workshop on qualification and certification of AM materials and components.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Enhanced Detection of Primary Biological Aerosol Particles Using Machine Learning and Single-Particle Measurement

Accurately identifying primary biological aerosol particles (PBAPs) using analytical techniques poses inherent challenges due to their resemblance to other atmospheric carbonaceous particles. Here, we present a study of an enhanced method for detecting PBAPs by combining single-particle measurement with advanced supervised machine learning (SML) techniques. We analyzed ambient particles from a variety of environments and lab-generated standards, focusing on chemical composition for traditional rule-based and clustering approaches and incorporating morphological features into the SML approaches, neural networks and XGBoost, for improved accuracy. This study demonstrates that SML methods outperform traditional methods in quantifying PBAPs, achieving significant improvements in precision, recall, F1-score, and accuracy, leading to an increased number of detected PBAPs by at least 19%. The adaptability of the proposed XGBoost-based SML model is showcased in comparison to traditional methods in categorizing PBAPs for blind data sets from different geographical locations. Two field case studies were investigated, over agricultural land and Amazonia rain forest, representing relatively low and high concentrations of PBAPs, respectively, where XGBoost consistently detected up to 3.5 times more PBAPs than traditional methods. Precise detection of PBAPs in the atmosphere could significantly improve the prediction of climatic impacts by them.

42 ENGINEERING↗
DOE OSTI2024

The seventh blind test of crystal structure prediction: structure generation methods

A seventh blind test of crystal structure prediction was organized by the Cambridge Crystallographic Data Centre featuring seven target systems of varying complexity: a silicon and iodine-containing molecule, a copper coordination complex, a near-rigid molecule, a cocrystal, a polymorphic small agrochemical, a highly flexible polymorphic drug candidate, and a polymorphic morpholine salt. In this first of two parts focusing on structure generation methods, many crystal structure prediction (CSP) methods performed well for the small but flexible agrochemical compound, successfully reproducing the experimentally observed crystal structures, while few groups were successful for the systems of higher complexity. A powder X-ray diffraction (PXRD) assisted exercise demonstrated the use of CSP in successfully determining a crystal structure from a low-quality PXRD pattern. The use of CSP in the prediction of likely cocrystal stoichiometry was also explored, demonstrating multiple possible approaches. Crystallographic disorder emerged as an important theme throughout the test as both a challenge for analysis and a major achievement where two groups blindly predicted the existence of disorder for the first time. Additionally, large-scale comparisons of the sets of predicted crystal structures also showed that some methods yield sets that largely contain the same crystal structures.

Chemistry↗
DOE OSTI2024

Optimization of a Lethal, Combat-Relevant Model of Sterile Inflammation in Mice for Drug Candidate Screening

ABSTRACT Introduction Extensive trauma, commonly seen in wounded military Service Members, often leads to a severe sterile inflammation termed systemic inflammatory response syndrome (SIRS), which can progress to multiple organ dysfunction syndrome (MODS) and death. MODS is a serious threat to wounded Service Members, historically causing 10% of all deaths in trauma admissions at a forward deployed combat hospital. The importance of this problem will be exacerbated in large-scale combat operations, in which evacuation will be delayed and care of complex injuries at lower echelons of care may be prolonged. The main goal of this study was to optimize an existing mouse model of lethal SIRS/MODS as a therapeutic screening platform for the evaluation of immunomodulatory drugs. Materials and Methods Male C57BL/6 mice were euthanized, and the bones and muscles were collected and blended into a paste termed tissue–bone matrix (TBX). The TBX at 12.5%–20% relative to body weight of each recipient mouse was implanted into subcutaneous pouches created on the dorsum of anesthetized animals. Mice were observed for clinical scores for up to 48 hours postimplantation and euthanized at the preset point of moribundity. To test effects of anesthetics on TBX-induced mortality, animals received isoflurane or ketamine/xylazine (K/X). In a separate set of studies, mice received TBX followed by intraperitoneal injection with 20 mg/kg or 40 mg/kg Eritoran or a placebo carrier. All Eritoran studies were performed in a blinded fashion. Results We observed that K/X anesthesia significantly increased the lethality of the implanted TBX in comparison to inhaled anesthetics. Although all the mice anesthetized with isoflurane and implanted with 12.5% TBX survived for 24 hours, 60% of mice anesthetized with K/X were moribund by 24 hours postimplantation. To mimic more closely the timing of lethal SIRS/MODS following polytrauma in human patients, we extended observation to 48 hours. We performed TBX dose–response studies and found that as low as 15%, 17.5%, and 20% TBX caused moribundity/mortality in 50%, 80%, and 100% mice, respectively, over a 48-hour time period. With 17.5% TBX, we tested if moribundity/mortality could be rescued by anti-inflammatory drug Eritoran, a toll-like receptor 4 antagonist. Neither 20 mg/kg nor 40 mg/kg doses of Eritoran were found to be effective in this model. Conclusions We optimized a TBX mouse model of SIRS/MODS for the purpose of evaluating novel therapeutic interventions to prevent trauma-related pathophysiologies in wounded Service Members. Negative effects of K/X on lethality of TBX should be further evaluated, particularly in the light of widespread use of ketamine in treatment of pain. By mimicking muscle crush, bone fracture, and necrosis, the TBX model has pleiotropic effects on physiology and immunology that make it uniquely valuable as a screening tool for the evaluation of novel therapeutics against trauma-induced SIRS/MODS.

General & Internal Medicine↗
DOE OSTI2023

Broadband, 920-nm mirror thin film damage competition

This year’s competition proposed to survey the state-of-the-art broadband, near-IR multilayer dielectric (MLD) mirrors designed for ultra-short, pulsed laser applications. The requirements for the coatings were a minimum reflection of 99.5% at 45-degree incidence angle for S-polarization from 830 nm to 1010 nm and group delay dispersion (GDD) < ± 50 fs 2 . The participants in this effort selected the coating materials, coating design, and deposition method. Samples were damage tested at a single testing facility to enable direct comparison among the participants using a 25 ± 5 fs OPCPA laser system operating at 5 Hz. A double blind test assured sample and submitter anonymity. The damage performance results, sample rankings, details of the deposition processes, coating materials and substrate cleaning methods are shared here. We found that multilayer coatings using tantala and/or hafnia as high index materials were top performers within several coating deposition groups. Specifically, dense coatings by ion-beam sputtering (IBS), magnetron sputtering (MS), and electron-beam ion assisted deposition (e-beam IAD) exhibited highest damage initiation onset (LIDT) while e-beam coatings were low performers. In addition, damage growth onset (LDGT) was also examined and the results are reported here for all samples as this performance metric plays an important role in establishing the safe operational conditions for larger aperture, ultrashort pulsed lasers. As a result, not all coating samples in the survey met the GDD requirements stated above and associated measurements are discussed in the context of the present and past competitions focused on similar broadband, near-IR MLD coatings.

42 ENGINEERING↗
DOE OSTI2024

Compositional Variation Tests on DuraMelter 100 with LAW Sub-Envelope A1 Feed (LAWA44 Glass) in Support of the LAW Pilot Melter, VSL-02R62N0-4, Rev. 0 (Nov 2002)

The two tests described in this report utilized blended feed (glass formers plus waste simulant) prepared by Optima Chemicals according to VSL specifications generating about 1. 7 metric tons of glass. Sugar was added (at VSL) to the nominal feed at a "sugar ratio" of 0.5 for each of the two variation tests1; however, since the sugar addition is assumed to be "blind" to the variations, the actual sugar ratios were 0.57 and 0.44. The DMl00-WV melter was used in order to provide a direct comparison with the LAW tests previously conducted on the same melter. Two 100-hour melter tests were conducted: one with a 15% deficiency in simulant and one with 15% excess in simulant. Key operating parameters including cold cap coverage, feed rate, and glass pool temperature were held constant to investigate the effects of the glass compositional changes on processing characteristics (including salt formation) and the product glass. The bubbling rate was adjusted to provide the desired glass production rate with a near complete cold cap (90-100% of melt surface covered with feed). Quantitative measurements of glass production rates, melter operating conditions (temperatures, pressures, power, flows, etc.), and off-gas characteristics (NOx, SO 2, CO, particulate load and composition, and acid gases) were made for each test.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗