Search NASA⌕ Search

SEARCH · Search NASA

Results for “pilot modeling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Preliminary Pilot-scale Study and Techno-economic Analysis of Chemical Heat Pump with Conventional Nuclear Reactors

The energy economy is continually evolving, particularly in terms of primary energy sources, their conversions to useful forms such as electricity and heat, and their utilization in different sectors, in response to socio-political factors. Because nuclear source is clean and non-carbon-emitting energy source, it is crucial to consider its role in the evolution of the energy economy. A pilot-scale study was conducted for Chemical Heat Pump (ChHP) system at three different scales (25-, 100-, and 1000-kW thermal outputs) using steady state thermal model. To establish the viability of selling heat rather than electricity alone, techno-economic analysis was undertaken for advanced Small Modular Reactor (SMR), and SMR paired with varied thermal output ChHP systems was referred to as the nuclear hybrid energy system (NHES) in this study. Using the U.S. Energy Information Administration statistics, pricing for electricity and natural gas were anticipated for U.S. regions (California, Northwest, Midwest, Southwest, New England, and PJM). Based on thermal output from ChHP, advanced SMR with 100 MWth and four alternative NHES scenarios were explored, namely 50-, 10-, 5-, and 1-MWth. Net present value, payback period, discounted cash flow return, and levelized cost of energy were evaluated for all scenarios. Based on the economic analysis, selling heat to high-temperature industrial processes is more profitable compared to selling electricity only. Higher carbon taxes showed significant improvement in economic parameters for NHESs. Providing heat to high-temperature industries could be beneficial, helping to reduce the greenhouse gas emissions by slashing the fossil fuel consumption.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

Overview of Virtual Test Bed FY23 Activities

The National Reactor Innovation Center (NRIC) was created for accelerating the deployment of novel reactor concepts. This will be achieved by providing both physical and virtual spaces for building and testing various components, systems, and complete pilot plants. The Virtual Test Bed (VTB) represents the virtual arm. It is being developed in collaboration with Department of Energy’s (DOE) Nuclear Energy Advanced Modeling and Simulation (NEAMS) program.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

Pilot Development Progress to Demonstrate Electric Thermal Energy Storage (ETES) Using Low-Cost Particles

The rapid growth of future energy demand increases the need to economically store electrical energy over durations up to several days in long-duration energy storage (LDES) applications. LDES can reduce the grid stress for improving the resilience of the grid. The integration of renewable power and storage has several significant and positive impacts including expanding the renewable energy portion of electricity generation, meeting peak-load demand, and coordinating supply and demand. Several energy storage approaches including mechanical, chemical and electrochemical methods are currently deployed or under development. However, LDES requirements pose unique challenges for scalability, energy capacity, and cost. Thermal energy storage (TES) has the ability to store a large capacity of energy with improved site flexibility compared to incumbent LDES technologies (i.e., pumped hydro) and has attracted significant interest. Our development in TES technology using solid particles as the storage medium has shown feasibility and potential in serving LDES purposes. Particle TES has various advantages over commercialized molten nitrate salt TES and has emerged as a promising storage technology originating from Generation 3 concentrating solar power (CSP) development. Stable, inexpensive silica sand produced in the Midwestern U.S. was identified as a suitable storage medium and has been validated through lab heating and cycling tests. We have successfully developed particle TES for bidirectional electric energy storage to broaden the applications for TES beyond CSP technologies and into standalone electric-thermal energy storage (ETES) systems. Key component designs and performance were verified with prototype testing and model simulations. The ETES system uses high-temperature, low-cost particle thermal energy storage coupled with a unique pressurized fluidized bed heat exchanger that supports a high-efficiency, air-Brayton combined power cycle. At times of low energy demand or high renewable production, an electric particle heater heats large quantities of solid particles to high temperatures in excess of 1100 degrees C. The particles are stored in internally insulated silos. At times of high electricity demand, the hot particles are gravity-fed through the heat exchanger where stored thermal energy is transferred to the power cycle working fluid to drive the gas turbine power generation system, thereby converting the thermal energy back into electricity for the grid. The cold particles leaving the heat exchanger are returned to the storage silos by a well-insulated particle conveyor. This ETES technology using particle TES can be sited anywhere and is predicted to be economic and efficient. Moreover, this ETES technology could be co-located with a retired coal or gas plant to leverage existing power generation and grid infrastructure. Our work has focused on the development of each of the key components (e.g., the electric particle heater, particle storage silos, and the particle-to-gas pressurized fluidized bed heat exchanger) through conceptual designs, prototype testing, and computational modeling. These developments demonstrate technological feasibility and are an important step towards the next phase of development of this promising LDES technology. This presentation will show the progress in developing a pilot demonstration of the ETES system on a commercially relevant scale.

25 ENERGY STORAGE↗
DOE OSTI2022

Estimation of process steady state with autoregressive models and Bayesian inference

To improve efficiency, separations engineers will typically design process circuits containing recirculating streams, which mix one or more of the process outputs with the feed material. Doing so can improve efficiency, but will cause a delay in the system reaching steady state conditions until the recirculating load mass flows stabilize. In testing separation circuits, engineers will often test a variety of factors and complete an analysis from sample results. Knowledge of if a process is at steady state, as well as the steady state conditions of a process, is essential for a valid techno-economic analysis. However, the definition of process steady state is often poorly defined, or does not include uncertainty quantification. If the performance of a process operating under two different sets of conditions are compared, an engineer who does not test for steady state or quantify steady state conditions risks producing a faulty analysis. In this work, a Bayesian statistical method for testing if all streams are at steady state is further motivated and then derived. Then after testing for steady state, the same model is used with a prior distribution that enforces a steady state assumption to estimate steady state conditions. Further, these methods were validated in a solvent extraction pilot plant where steady state conditions for all outflows were inferred with uncertainty quantification. Analysis is completed with functions available to the reader as part of the BayesMassBal (V 1.1.0) software package written in R.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2024

Investigation of Thermal Radiation under Pressurized Oxy-combustion Conditions

Thermal radiation of the gaseous and particle phases in a pilot-scale pressurized oxy-combustor is computationally studied. In particular, the radiation characteristics of gases and particles are estimated by employing the statistical narrow-band model and the large-particle model. It is found that thermal radiation of the particle cloud dominates in the combustor under a furnace temperature of 1500 K and when there is no substantial loss of particles to the walls. Another important observation is that radiation from the gas and particles can be approximately treated as a graybody under these conditions. More specifically, the results on the spectral radiation intensity of a gas comprising 40% (vol) H 2 O and 60% CO 2 show that when the pressure is increased to 15 bar, and the radiation pathlength is 100 cm, the spectral radiation profile of the gas phase approaches that of a blackbody at the respective temperature. In addition, the emissivity of the particulate cloud has been evaluated as a function of the particle concentration and diameter by employing the large-particle model. It is shown that the emissivity grows with the particle concentration but decreases with the particle size for the same mass of the particles. Finally, this outcome of the present study is expected to be used to validate the assumption of the gray-gas model adopted in the numerical simulations of pressurized oxy-combustion.

large-particle model↗
DOE OSTI2024

Platform for Remote Deployment and Training for Enhanced Building Operation Practices (Building Re-Tuning and On-going Commissioning)

While a building’s energy usage is driven largely by its design and use, building operator behavior has a strong influence on its energy consumption. This project developed and piloted a specific, data-driven coaching methodology to help operators understand how they can adjust operations and/or affect no/low-cost repairs or upgrades to their specific building HVAC systems to reduce energy consumption. Named BuildingCoach, the operational optimization method used is based on the Building Re-tuning approach developed by the Pacific Northwest National Laboratory. A building operations analytics market has matured over the past decade, though its potential to affect energy-saving changes has not been fully realized. Training operators to understand the methods for operational optimization with the explicit approach of using building-system performance data is hypothesized to create a more effective, longer lasting result in building energy efficiency, and this strategy is the fundamental premise of this project. With the support of an Industry Advisory Board, the project succeeded in developing materials and recruiting for and delivering three pilot cohorts. Deliverables included twenty-two self-paced training modules (accessed via a Learning Management System) and a web-based platform that includes access to real-time building system data and a repository for building system documentation. The project set out to have 100 participants from 50 buildings in three pilot cohorts. In the end, there were 28 participants from 17 buildings, i.e., a significant shortfall. The first two pilot cohorts had only two buildings in each, and this was partially due to difficulties in deploying the Building Operator Coaching Solution (“the BOCS”), which is technology that extracts the data from the controls network and presents it as prescribed for coaching. In the third cohort, the project team deployed the BOCS successfully to 13 buildings, the methodology was piloted as intended, and numerous opportunities for optimization were identified. The BuildingCoach business plan charts a path to an economically sustainable effort. However, even with a licensing model captured in the final version of the business plan, the scalability is still limited to keeping less than 1,000 buildings affected by 2033. Even so, there are unexplored paths to greater scalability that are being considered. CUNY BPL is working to perpetuate and grow the use of BuildingCoach. As of this writing, about twenty buildings have either been connected or will be connected with operators coached / to be coached in the NYC municipal portfolio, twelve buildings across four campuses in NY State will use BuildingCoach, a NY upstate county wishes for six or seven buildings to participate with the support of funding from NYSERDA, and others have also expressed interest. In the decades to come, there will be an increasing percentage of large and mid-sized buildings that incorporate automated system optimization (ASO), and the building operators’ role will shift to spend more time on maintenance and monitoring. Meanwhile, programs such as BuildingCoach will play a critical role in optimizing operations. And, regardless of the emergence of ASO, operators will still need to understand how their systems operate so that they can monitor them properly. Within that context, BuildingCoach is an important step towards operators’ understanding of efficient building system operations.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Thermal radiation analysis of a pressurized oxy-coal combustor with discrete ordinate model

Concerns over climate change have led to numerous efforts in developing low-carbon energy technologies. Pressurized oxy-combustion (POC) is a promising candidate to reduce carbon emissions in power generation. Pressure strongly impacts thermal radiation, leading to a substantial difference in heat transfer between POC and conventional atmospheric pressure combustion. A good understanding of thermal radiation in POC is needed to aid new combustor development. The present work is a step in this direction, initiating a systematic analysis of thermal radiation and heat transfer in a pilot-scale POC combustor, which has been developed at Washington University in St. Louis (WUSTL). In a POC process, pulverized coal is burned under elevated pressure and O2-CO2 environment. Few researchers have investigated the thermal radiation of pressurized oxy-combustion. This work adopted the discrete ordinate model to simulate the radiation of a 1-D cylinder with predetermined conditions (temperature, pressure, gas compositions) to predict the influence of radiative properties of flue gas on the radiation in the combustor.

Li, Lei↗
DOE OSTI2021

WALLABY pre-pilot survey: two dark clouds in the vicinity of NGC 1395

ABSTRACT We present the Australian Square Kilometre Array Pathfinder (ASKAP) WALLABY pre-pilot observations of two ‘dark’ H i sources (with H i masses of a few times 108 $\rm {M}_\odot$ and no known stellar counterpart) that reside within 363 kpc of NGC 1395, the most massive early-type galaxy in the Eridanus group of galaxies. We investigate whether these ‘dark’ H i sources have resulted from past tidal interactions or whether they are an extreme class of low surface brightness galaxies. Our results suggest that both scenarios are possible, and not mutually exclusive. The two ‘dark’ H i sources are compact, reside in relative isolation, and are more than 159 kpc away from their nearest H i-rich galaxy neighbour. Regardless of origin, the H i sizes and masses of both ‘dark’ H i sources are consistent with the H i size–mass relationship that is found in nearby low-mass galaxies, supporting the possibility that these H i sources are an extreme class of low surface brightness galaxies. We identified three analogues of candidate primordial ‘dark’ H i galaxies within the TNG100 cosmological, hydrodynamic simulation. All three model analogues are dark matter dominated, have assembled most of their mass 12–13 Gyr ago, and have not experienced much evolution until cluster infall 1–2 Gyr ago. Our WALLABY pre-pilot science results suggest that the upcoming large-area H i surveys will have a significant impact on our understanding of low surface brightness galaxies and the physical processes that shape them.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

An adaptable technique for comparative image assessment: Application to crosswell electromagnetic survey design for fluid monitoring

Reservoir integrity stewardship accompanying carbon capture and sequestration considers fluid extraction and reinjection as a risk-mitigating method against overpressuring that could lead to caprock damage and ensuing [Formula: see text] leakage. Crosswell electromagnetics offers a technically viable monitoring method with the spatial volume coverage necessary for reservoir-encompassing pressure management. However, a certain logistic dilemma for deep gas sequestration into saline and thus electrically conductive aquifers is that crosswell magnetic-field measurements underperform in the imaging of more resistive plume bodies, further exacerbated when vertical arrays intersect, as opposed to surround, plumes. Furthermore, comparative synthetic-data plume imaging of such scenarios rates the information content of magnetic-field versus electric-field 3D crosswell layouts for reservoir and infrastructure conditions of a representative pilot site in a coastal area in Florida. The image quality of the resulting plume replications can be ranked numerically through a newly proposed semblance qualifier, appraising the model goodness of fit to a given reference. In contrast to common least-squares measures for goodness of fit, the semblance formulation uses classifying logistic function types, thus enabling a better distinction of predefined anomaly features.

58 GEOSCIENCES↗
DOE OSTI2023

Analysis of thermal radiation of a gas-particulate-cloud in a pressurized oxy-coal combustor

Pressurized oxy-combustion (POC) aims to reduce carbon emissions from coal-fired power plants. In the POC process, pulverized coal is burned at elevated pressure in an O2-CO2 environment. The elevated pressure and high CO2 and H2O concentrations strongly impact thermal radiation, thereby significantly distinguishing POC system from conventional atmospheric-pressure air-fired and oxy-fuel combustion. Consequently, a thorough understanding of thermal radiation in POC is required to promote the development of novel combustors. The present computational work analyzes thermal radiation of the gaseous and particle phases in the pilot-scale POC combustor in Washington University in St. Louis. Particularly, the radiation characteristics of the gaseous and particulate cloud were estimated by employing the statistical narrow band model and the large-particle model. It is found that thermal radiation of a particulate cloud dominates in the combustor under the conditions of a furnace temperature of 1500 K and no substantial particle loss. Another important result is that the gas-and-particulate cloud can be approximately treated as a graybody under the same conditions. To be more specific, the results of spectral radiation intensity of a gas comprised of 40% (vol) H2O and 60% CO2 show that when the pressure is increased to 15 bar, and the radiation pathlength to 100 cm, the spectral radiation profile of the gas phase approaches that of a blackbody at the respective temperature. In addition, the emissivity of the particulate cloud has been evaluated as a function of the particle concentration and diameter by means of the large particle model. It is shown that the emissivity grows with the particle concentration but decreases with the particle size for the same mass of a particulate cloud. The emissivity of a particulate cloud in oxy- combustion exceeds that in air-fired combustion. This work could be used to validate the assumption of the gray gas model adopted in numerical simulations.

Li, Lei↗
DOE OSTI2025

Predicting cutoff L-shells of solar protons using the GPPSn particle dataset

Solar energetic protons (SEPs) arriving at the Earth trigger severe radiation storms in the near-Earth space, directly impacting space missions operating at various altitudes. Therefore, monitoring SEP events and predicting the penetration depths of solar protons are critical for aerospace sectors. Building on previous efforts, here we demonstrate the feasibility of using proton measurements from the Global Prompt Proton Sensor network (GPPSn), enabled by Los Alamos National Laboratory developed combined X-ray dosimeters aboard GPS satellites, to characterize and predict the penetration of solar protons into the geomagnetic field. The inclined medium-Earth-orbits (MEOs) of the global GPS constellation offer a unique advantage of allowing simultaneous measurements of penetrating solar protons inside both open- and closed-field line regions. Therefore, the L-profiles of ∼10s–100 MeV solar protons and their associated cutoff L-shells can be determined from the GPPSn dataset, using predefined threshold proton flux values rather than traditional flux ratios. After examining a list of SEP event intervals across solar cycles 23, 24 and 25—including the 2024 Mother’s Day superstorm, we showcase how the latest GPPSn proton dataset (release v1.10), reprocessed and calibrated, can not only be used to monitor solar proton distributions inside the dynamic geomagnetic field for individual events, but also to derive a new empirical model linking cutoff L-shells with several key space weather parameters. This newly developed SEPCL-MEO model demonstrates high predictive performance; for example, predictions for > 30 MeV solar protons yield a correlation coefficient of 0.85 and performance efficiency of 0.67 when validated against GPPSn observations. Results from this pilot study underscores the scientific and operational value of the GPPSn dataset, and this dataset—when paired with machine-learning techniques—can play a critical role in observing and predicting the effects of future incoming SEP events, including extreme ones.

58 GEOSCIENCES↗
DOE OSTI2024

Sequential Design of Experiments for Pilot Testing of Novel Solvent System

The CCSI2 program is supporting a six-month test campaign at the National Carbon Capture Center (NCCC) for evaluation of a novel water-lean solvent. This presentation describes CCSI2’s efforts in process modeling of the solvent system for both coal and natural gas-based flue gas sources and initial uncertainty quantification (UQ) work to estimate parametric uncertainty in key sub-models of interest (e.g., thermodynamics, mass transfer, reaction kinetics). Moreover, perspective is provided on how UQ and sequential design of experiments (SDoE) tools are used to assess the impact of model uncertainty on projected process performance, use this information to optimize data collection during the campaign, and refine process models through data collection. This framework is expected to reduce the overall model uncertainty, and thus risk associated with scale-up as the process moves towards commercialization.

Morgan, Joshua↗
DOE OSTI2022

Immersive Industrialized Construction Environments for Energy Efficiency Construction Workforce

The National Renewable Energy Laboratory is actively developing and testing Immersive Industrialized Construction Environments (IICE) for construction automation and worker-machine interaction to investigate possible solutions and increase workforce productivity. At full scope and matured functionality, IICE allows us to accelerate the development of and better explore industrialized construction approaches such as prefabrication. IICE also enables wider adoption of energy-efficient products and Industry 4.0 construction automation through worker-machine interaction pilots. Industry 4.0 and industrialized construction approaches can encourage workforce specialization in energy efficiency construction, address the lack of multi-skilled workers, and increase workforce productivity with construction automation. However, recent attempts to integrate these concepts with the industry have only been moderately successful. To address this, focusing the pedagogy on using a digital twin, its digital models, and virtual reality could make the experience of continuing education on construction automation more affordable, accessible, scalable, immersive, and safer, and could greatly improve the efficiency and robustness of the building and construction industry. IICE accurately represents the realities of construction uncertainties without having to create full scale physical prototypes of machines. In this paper, we address the following research question: How can a digital twin and its models in virtual reality enhance the learning experience and productivity of energy efficiency construction workers to gain the skills in operating Industry 4.0 components such as construction automation and handling energy-efficient products in industrialized construction factories and on-site? We introduce original research on developing IICE and present preliminary findings from time and motion pilot studies.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Forecasting Battery Electrode Performance via Electrochemical Fluorescence Microscopy and Machine-Learning

Predicting lithium-ion battery performance is hindered by microscale electrode heterogeneities invisible to conventional diagnostics. Here, we combine electrochemical fluorescence microscopy (EFM), which maps electronic connectivity by visualizing an electrofluorophore reaction distribution, with a multitask ElasticNet regression to forecast discharge capacity from spatial heterogeneity. Analyzing 196 images from six pilot-scale LiNi 0.5 Mn 0.3 Co 0.2 O 2 cathodes with varying carbon loadings, we extract 62 descriptors that capture morphology and texture. A compact five-feature model predicts capacity across eight discharge rates, achieving a per-target R 2 of up to 0.63 and an overall R 2 of 0.92, with a mean absolute percentage error of less than 2%. This performance rivals impedance-based approaches while avoiding their reliance on postformation data and incomplete electronic network information. Our facile and rapid, image-driven method may enable electrode quality control upstream of costly cell assembly to offer a transformative tool for data-driven battery research and manufacturing.

battery electrodes↗
DOE OSTI2024

Framework for Optimization, Quantification of Uncertainty, and Surrogates, and CCSI2 Toolset Capabilities

This poster will present key features and capabilities of the CCSI2 software platform to relevant technical and research audience members at the 2024 FECM / NETL Carbon Management Research Project Review Meeting. Attendees will learn about available process models in the CCSI2 Toolset, as well as design of experiments and machine learning features in the Framework for Optimization, Quantification of Uncertainty, and Surrogates (FOQUS) software to support pilot plant work.

Paul, Brandon↗
DOE OSTI2025

Deep Neural Network Assisted Distributed Strain and Temperature Fiber Sensor System for Natural Gas Pipeline Monitoring

Natural gas pipeline integrity monitoring is crucial to detect potential leaks, find structural issues, and prevent environmental damage. This article presents a system of natural gas pipeline monitoring that uses a specialized double Brillouin peak sensing fiber along with the Brillouin optical time domain analysis (BOTDAs) technique. The calibrated sensing fiber coefficients for strain and temperature are 41.8 kHz/ με and 0.9 MHz/°C for peak 1; and 47.2 kHz/ με , and 1.11 MHz/°C for peak 2, respectively. Initially, lab tests were performed by installing a short section of double Brillouin peak fiber (DBPF) on a 1-in steel pipe under pressure up to 1000 per square inch (psi) at elevated temperatures. Simultaneous distributed measurements of temperature and pressure-induced hoop strain were successfully measured. Considering the long processing speed to extract Brillouin frequency shift (BFS), we employ a novel probabilistic deep neural network (PDNN) framework for rapid BFS prediction. Additionally, using the Finite Element Method, the effects of the pipeline pressure on hoop strain were modeled and compared to the experimental hoop strain under the same set of pipeline conditions. Finally, an actual 4-in outer diameter steel natural gas pipeline was used for pilot-scale tests, where hoop strain was measured at various pressure levels. Leaks were simulated to demonstrate accurate pipeline integrity monitoring. At an internal pipe pressure of 1000 psi, hoop strain of approximately 300 με was observed, and the sensitivity was calculated as 0.28 με /psi. The results of this pilot-scale study demonstrated that the system is capable of performing distributed monitoring sufficient to detect pipeline pressure and the presence of leaks to ensure the safe operation of gas pipelines in the field.

03 NATURAL GAS↗
DOE OSTI2025

Real-time reconstruction of ground motion during small magnitude earthquakes: A pilot study

This study presents a pilot investigation into a novel method for reconstructing real-time ground motion during small magnitude earthquakes (M < 4.5), removing the need for computationally expensive source characterization and simulation processes to assess ground shaking. Small magnitude earthquakes, which occur frequently and can be modeled as point sources, provide ideal conditions for evaluating real-time reconstruction methods. Utilizing sparse observation data, the method applies the Gappy Auto-Encoder (Gappy AE) algorithm for efficient field data reconstruction. This is the first study to apply the Gappy AE algorithm to earthquake ground motion reconstruction. Numerical experiments conducted with SW4 simulations demonstrate the method’s accuracy and speed across varying seismic scenarios. The reconstruction performance is further validated using real seismic data from the Berkeley area in California, USA, demonstrating the potential for practical application of real-time earthquake data reconstruction using Gappy AE. As a pilot investigation, it lays the groundwork for future applications to larger and more complex seismic events.

58 GEOSCIENCES↗
DOE OSTI2022

Development of Training Materials for Pathologists to Provide Machine Learning Validation Data of Tumor-Infiltrating Lymphocytes in Breast Cancer

The High Throughput Truthing project aims to develop a dataset for validating artificial intelligence and machine learning models (AI/ML) fit for regulatory purposes. The context of this AI/ML validation dataset is the reporting of stromal tumor-infiltrating lymphocytes (sTILs) density evaluations in hematoxylin and eosin-stained invasive breast cancer biopsy specimens. After completing the pilot study, we found notable variability in the sTILs estimates as well as inconsistencies and gaps in the provided training to pathologists. Using the pilot study data and an expert panel, we created custom training materials to improve pathologist annotation quality for the pivotal study. We categorized regions of interest (ROIs) based on their mean sTILs density and selected ROIs with the highest and lowest sTILs variability. In a series of eight one-hour sessions, the expert panel reviewed each ROI and provided verbal density estimates and comments on features that confounded the sTILs evaluation. We aggregated and shaped the comments to identify pitfalls and instructions to improve our training materials. From these selected ROIs, we created a training set and proficiency test set to improve pathologist training with the goal to improve data collection for the pivotal study. We are not exploring AI/ML performance in this paper. Instead, we are creating materials that will train crowd-sourced pathologists to be the reference standard in a pivotal study to create an AI/ML model validation dataset. The issues discussed here are also important for clinicians to understand about the evaluation of sTILs in clinical practice and can provide insight to developers of AI/ML models.

60 APPLIED LIFE SCIENCES↗