Search NASA⌕ Search

SEARCH · Search NASA

Results for “mean squared error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53

Reactor Power Level Estimation by Fusing Multi-Modal Sensor Measurements

Estimates of the power level of a nuclear reactor based on measurements from an independent monitoring sensor system can help in the compliance verification of its declared operations. We present a three-level fusion method to estimate the power level of a nuclear reactor using features derived from infrared, electromagnetic, and acoustic sensor measurements collected in proximity to the reactor. Based on a simplified analytical model of the secondary coolant system of the reactor, we identify partial regression functions of the power level in terms of the temperature difference between inlet and outlet of coolant pipes, and the activity levels of four fans and four pumps, which are estimated as features from the sensor measurements. The power level estimator employs a combination of aggregate and complementary fusion steps at three levels to incorporate the multi-modal features in a structure that reflects the secondary cooling system and its partial regression functions. Using the measurements from a test campaign at an operational reactor, we show that this estimator achieves 3.47% or lower root mean square error under 5-fold cross validation. More generally, these results illustrate a progressive reduction in estimation error as additional modalities are appropriately incorporated, and that the fuser outperforms single modality features and their sub-combinations.

Rao, Nageswara↗

A framework for calibrating and validating an air loop dynamic model in an HVAC system in Modelica

The use of Modelica for simulating the dynamic behaviors of building heating, ventilation, and air conditioning (HVAC) systems has gained popularity. Calibration of a model that represents large and complex HVAC systems in Modelica involves the determination of hundreds of parameters using real-world operation data. Considering the coupling effects among various components, such a calibration process is complex and time-consuming. In this study, we propose a systematic framework to efficiently calibrate and validate a complex HVAC system model in Modelica. The framework includes strategies to solicit real-system operation data and to decouple the model. The goal of this calibration framework is to accurately and efficiently determine a set of Modelica model parameters that provide a good match between the simulated and real system behaviors. To demonstrate the validity of the framework, it was applied to calibrate the parameters of a Modelica model of the air loop subsystem of a real AHU-VAV system. The results show that the calibrated model can generate simulated results, including VAV air flow rate, outdoor air flow, and fan power consumption, that match well with the operational data of the real system. The coefficient of the variation of the root mean square error, CV(RMSE), of the air flow rate and power consumption are 14.6 % and 11.5 %, respectively. The results prove that the framework is valid and effective, and it can be used to calibrate other complex HVAC system Modelica models in the future.

Li, Yicheng↗

Prediction of Research/Motor Octane Number and Octane Sensitivity Using Artificial Neural Networks

Octane sensitivity (OS), dfined as the research octane number (RON) minus the motor octane number (MON) of a given fuel, has gained interest among researchers due to its apparent ffect on knocking conditions in internal combustion engines. Compounds with a high OS enable higher efficiencies, especially with respect to advanced compression ignition engines. RON/MON must be experimentally tested to determine OS; however, the experimental methods utilized require a substantial amount of time, a significant monetary investment, and specialized equipment. To this end, predictive computational models trained with existing experimental data and molecular properties would allow for the preemptive screening of compounds prior to performing these experiments. The present work proposes two methods for predicting the OS of a given compound: using artficial neural networks (ANNs) trained with quantitative structure-property relationship (QSPR) descriptors to predict RON and MON individually to compute OS from RON/MON predictions (derived octane sensitivity, dOS), and using an ANN trained with QSPR descriptors to directly predict OS. ANNs trained to predict RON and MON achieved test set root-mean-square errors (RMSEs) of 10.499 and 7.551 respectively. dOS calculations were found to have a test set RMSE of 6.432 while predicting OS directly resulted in a test set RMSE of 7.019, showing it is more bene cial to obtain OS from RON/MON predictions than predicting it directly. Furthermore, relationships between individual QSPR descriptors and RON/MON/OS are discussed, highlighting correlations between specfic molecular features and these properties.

09 BIOMASS FUELS↗

On the minimum number of radiation field parameters to specify gas cooling and heating functions

Fast and accurate approximations of gas cooling and heating functions are needed for hydrodynamic galaxy simulations. We use machine learning to analyze atomic gas cooling and heating functions in the presence of a generalized incident local radiation field computed by Cloudy. We characterize the radiation field through binned radiation field intensities instead of the photoionization rates used in our previous work. We find a set of 6 energy bins whose intensities exhibit relatively low correlation. We use these bins as features to train machine learning models to predict Cloudy cooling and heating functions at fixed metallicity. We compare the relative SHapley Additive exPlanation (SHAP) value importance of the features. From the SHAP analysis, we identify a feature subset of 3 energy bins (0.5-1, 1-4, and 13-16Ry) with the largest importance and train additional models on this subset. We compare the mean squared errors and distribution of errors on both the entire training data table and a randomly selected 20% test set withheld from model training. The machine learning models trained with 3 and 6 bins, as well as 3 and 4 photoionization rates, have comparable accuracy everywhere, with errors ≳10 times smaller than for the interpolation table of Gnedin and Hollon (2012). We conclude that 3 energy bins (or 3 analogous photoionization rates: molecular hydrogen photodissociation, neutral hydrogen HI, and fully ionized carbon CVI) are sufficient to characterize the dependence of the gas cooling and heating functions on our assumed incident radiation field model.

79 ASTRONOMY AND ASTROPHYSICS↗

SLOPE daily and 250 m gross primary productivity (GPP) for the CONUS, 2000-2019, Carbon Monitoring System (CMS)

The SatelLite Only Photosynthesis Estimation (SLOPE) GPP product is a daily, 250 m GPP dataset covering the Contiguous United States (CONUS) from 2000 to present with 1 day latency. Gross primary productivity (GPP) quantifies the amount of carbon dioxide (CO2) fixed by plants through photosynthesis. Although as a key quantity of terrestrial ecosystems, there is a lack of high-spatial-and-temporal-resolution, real-time, and observation-based GPP products. This product has been developed to address this critical gap, leveraging a number of MODIS land and atmosphere products. There are three distinct features of the SLOPE GPP production algorithm. (1) SLOPE couples machine learning models with MODIS atmosphere and land products to accurately estimate PAR. (2) SLOPE couples highly efficient and pragmatic gap-filling and filtering algorithms with surface reflectance acquired by both Terra and Aqua MODIS satellites to derive a soil-adjusted NIRv (SANIRv) dataset. (3) SLOPE couples a temporal pattern recognition approach with a long-term Crop Data Layer (CDL) product to predict dynamic C4 crop fraction. PAR, SANIRv and C4 fraction are used to drive a parsimonious model with only two slope parameters to estimate GPP along with a quantitative uncertainty on a per-pixel and daily basis. The slope GPP product has a R2 = 0.84 and a root-mean-square error (RMSE) of 1.65 gC m-2 d-1, evaluated against from 50 AmeriFlux eddy covariance sites (332 site-years). Archived data from 2000 to 2019 are publicly available at the NASA's Oak Ridge National Laboratory Distributed Active Archive Center (ORNL DAAC). Data from 2020 are available from the authors upon request. All data are projected in the standard MODIS Land Integerized Sinusoidal tile map projection. Each processing tile is in size of 4800 pixels by 4800 pixels, representing approximately 1200 km by 1200 km land region. In addition to the GPP product, SLOPE PAR, SANIRV, and C4 fraction, along with their uncertainties, are also released.

Jiang, Chongya↗

Advances in statistical methods for cancer surveillance research: an age-period-cohort perspective

Background: Analysis of Lexis diagrams (population-based cancer incidence and mortality rates indexed by age group and calendar period) requires specialized statistical methods. However, existing methods have limitations that can now be overcome using new approaches. Methods: We assembled a “toolbox” of novel methods to identify trends and patterns by age group, calendar period, and birth cohort. We evaluated operating characteristics across 152 cancer incidence Lexis diagrams compiled from United States (US) Surveillance, Epidemiology and End Results Program data for 21 leading cancers in men and women in four race and ethnicity groups (the “cancer incidence panel”). Results: Nonparametric singular values adaptive kernel filtration (SIFT) decreased the estimated root mean squared error by 90% across the cancer incidence panel. A novel method for semi-parametric age-period-cohort analysis (SAGE) provided optimally smoothed estimates of age-period-cohort (APC) estimable functions and stabilized estimates of lack-of-fit (LOF). SAGE identified statistically significant birth cohort effects across the entire cancer panel; LOF had little impact. As illustrated for colon cancer, newly developed methods for comparative age-period-cohort analysis can elucidate cancer heterogeneity that would otherwise be difficult or impossible to discern using standard methods. Conclusions: Cancer surveillance researchers can now identify fine-scale temporal signals with unprecedented accuracy and elucidate cancer heterogeneity with unprecedented specificity. Birth cohort effects are ubiquitous modulators of cancer incidence in the US. The novel methods described here can advance cancer surveillance research.

60 APPLIED LIFE SCIENCES↗

EdgeAI: Machine learning via direct attached accelerator for streaming data processing at high shot rate x-ray free-electron lasers

We present a case for low batch-size inference with the potential for adaptive training of a lean encoder model. We do so in the context of a paradigmatic example of machine learning as applied in data acquisition at high data velocity scientific user facilities such as the Linac Coherent Light Source-II x-ray Free-Electron Laser. We discuss how a low-latency inference model operating at the data acquisition edge can capitalize on the naturally stochastic nature of such sources. We simulate the method of attosecond angular streaking to produce representative results whereby simulated input data reproduce high-resolution ground truth probability distributions. By minimizing the mean-squared error between the decoded output of the latent representation and the ground truth distributions, we ensure that the encoding layers and resulting latent representation maintains full fidelity for any downstream task, be it classification or regression. We present throughput results for data-parallel inference of various batch sizes, some with throughput exceeding 100 k images per second. We also show in situ training below 10 s per epoch for the full encoder–decoder model as would be relevant for streaming and adaptive real-time data production at our nation’s scientific light sources.

97 MATHEMATICS AND COMPUTING↗

Allometric Models for Predicting Aboveground Biomass and Carbon Stock of Tropical Perennial C 4 Grasses in Hawaii

Biomass is a promising renewable energy option that provides a more environmentally sustainable alternative to fossil resources by reducing the net flux of greenhouse gasses to the atmosphere. Yet, allometric models that allow the prediction of aboveground biomass (AGB), biomass carbon (C) stock non-destructively have not yet been developed for tropical perennial C 4 grasses currently under consideration as potential bioenergy feedstock in Hawaii and other subtropical and tropical locations. The objectives of this study were to develop optimal allometric relationships and site-specific models to predict AGB, biomass C stock of napiergrass, energycane, and sugarcane under cultivation practices for renewable energy and validate these site-specific models against independent data sets generated from sites with widely different environments. Several allometric models were developed for each species from data at a low elevation field on the island of Maui, Hawaii. A simple power model with stalk diameter (D) was best related to AGB and biomass C stock for napiergrass, energycane, and sugarcane, (R 2 = 0.98, 0.96, and 0.97, respectively). The models were then tested against data collected from independent fields across an environmental gradient. For all crops, the models over-predicted AGB in plants with lower stalk D, but AGB was under-predicted in plants with higher stalk D. The models using stalk D were better for biomass prediction compared to dewlap H (Height from the base cut to most recently exposed leaf dewlap) models, which showed weak validation performance. Although stalk D model performed better, however, the mean square error (MSE)-systematic was ranged from 23 to 43 % of MSE for all crops. A strong relationship between model coefficient and rainfall was existed, although these were irrigated systems; suggesting a simple site-specific coefficient modulator for rainfall to reduce systematic errors in water-limited areas. These allometric equations provide a tool for farmers in the tropics to estimate perennial C4 grass biomass and C stock during decision-making for land management and as an environmental sustainability indicator within a renewable energy system.

54 ENVIRONMENTAL SCIENCES↗

Estimating Leaf Area Index in Row Crops Using Wheel-Based and Airborne Discrete Return Light Detection and Ranging Data

Leaf area index (LAI) is an important variable for characterizing plant canopy in crop models. It is traditionally defined as the total one-sided leaf area per unit ground area and is estimated by both direct and indirect methods. This paper explores the effectiveness of using light detection and ranging (LiDAR) data to estimate LAI for sorghum and maize with different treatments at multiple times during the growing season from both a wheeled vehicle and Unmanned Aerial Vehicles. Linear and nonlinear regression models are investigated for prediction utilizing statistical and plant structure-based features extracted from the LiDAR point cloud data with ground reference obtained from an in-field plant canopy analyzer (indirect method). Results based on the value of the coefficient of determination ( R 2 ) and root mean squared error for predictive models ranged from ∼0.4 in the early season to ∼0.6 for sorghum and ∼0.5 to 0.80 for maize from 40 Days after Sowing to harvest.

59 BASIC BIOLOGICAL SCIENCES↗

Optimizing resource allocation in Miscanthus breeding via sparse testing designs for genomic prediction

Phenotyping high-biomass perennial crops is laborious and the rate of genetic gain in conventional perennial crop breeding programs is typically low. So, it is especially important to identify methods that produce efficiency gains in the breeding process. Miscanthus is a C4 perennial grass with favorable characteristics for producing biomass as a feedstock for biofuels and diverse bio-based products. Increasing biomass yield will increase profitability and environmental benefits, so it is a key target for Miscanthus breeding. In addition, the identification of well-adapted genotypes across a wide range of environmental conditions requires the establishment of multi-environment trials (METs). Sparse testing is a genomic prediction-based strategy that reduces the phenotyping costs in METs by selecting a subset of genotypes to evaluate in a subset of environments and then predicts the performance of the unobserved genotype-environment combinations. A Miscanthus sacchariflorus (MSA) population comprising 336 genotypes observed across three environments was analyzed implementing sparse testing designs. Three prediction models considering main effects (environments, genotypes, genomic) and interaction effects (genotype-by-environment; G×E interaction) were implemented for forecasting dry biomass yield (YDY), total culm (TCM), average internode length (AIL), and culm node number (CNN). Multiple calibration sets based on different compositions and sizes were considered to evaluate performance in terms of the predictive ability (PA) and the mean square error (MSE) for a fixed testing set size. The training set size ranged from 52 to 112 to predict a fixed set of 224 unobserved genotypes across all three environments. The results showed that the model accounting for G×E interaction consistently presented the highest PA and the lowest MSE: for CNN (PA: ~0.77, MSE: ~0.5) and YDY (PA: ~0.70, MSE: ~1.3) while for TCM and AIL these ranged from ~0.28 to 0.41 and ~1.3 to 4.3, respectively. Overall, varying training sets and allocation strategies did not affect PA and MSE, with 52 non-overlapping and 0 overlapping genotypes per environment as the optimal cost-effective allocation framework. This suggests that implementing sparse testing designs could significantly reduce phenotyping costs by fivefold, without compromising PA in breeding programs for perennial crops such as Miscanthus.

Miscanthus sacchariflorus (MSA)↗

Automated Cloud Based Long Short-Term Memory Neural Network Based SWE Prediction

Snow derived water is a critical component of the US water supply. Measurements of the Snow Water Equivalent (SWE) and associated predictions of peak SWE and snowmelt onset are essential inputs for water management efforts. This paper aims to develop an integrated framework for real-time data ingestion, estimation, prediction and visualization of SWE based on daily snow datasets. In particular, we develop a data-driven approach for estimating and predicting SWE dynamics using the Long Short-Term Memory neural network (LSTM) method. Our approach uses historical datasets (precipitation, air temperature, SWE, and snow thickness) collected at NRCS Snow Telemetry (SNOTEL) stations to train the LSTM network and current year data to predict SWE behavior. The performance of our prediction was compared for different prediction dates and prediction training datasets. Our results suggest that the proposed LSTM network can be an efficient tool for forecasting the SWE timeseries, as well as Peak SWE and snowmelt timing. Results showed that the window size impacts the model performance (where the Nash Sutcliffe efficiency (NSE) ranged from 0.96 to 0.85 and the Rooted Mean Square Error (RMSE) ranged from 0.038 to 0.07) with an optimum number that should be calibrated for different stations and climate conditions. In addition, by implementing the LSTM prediction capability in a cloud based site-monitoring platform, we automate model-data integration. By making the data accessible through a graphical web interface and an underlying API which exposes both training and prediction capabilities. The associated results can be made easily accessible to a broad range of stakeholders.

54 ENVIRONMENTAL SCIENCES↗

Remaining Useful Strength (RUS) Prediction of SiCf-SiCm Composite Materials Using Deep Learning and Acoustic Emission

Prognosis techniques for prediction of remaining useful life (RUL) are of crucial importance to the management of complex systems for they can lead to appropriate maintenance interventions and improvements in reliability. While various data-driven methods have been introduced to predict the remaining useful life (RUL) of machinery systems or batteries, no research has been reported on the remaining useful strength (RUS) prediction of silicon carbide fiber reinforced silicon carbide matrix (SiCf-SiCm) materials with pivotal role in its potential usage as a structural material in nuclear reactors and turbine engines. Knowledge of its degradation process is of the utmost importance to the manufacturers. For this purpose, two approaches based on the machine-learning techniques of random-forest (RF) and convolutional neural network (CNN) are proposed to predict the RUS of SiCf-SiCm using only acoustic emission (AE) signals generated during the material’s stress applying process. Experimental results show that the CNN models achieved better predictive performance than the RF models but the latter with expert-engineered features achieves better prediction for AE signals in the early stage of degradation. Additionally, our results demonstrate that both models can correctly predict the SiCf-SiCm RUS as evaluated by our robust testing method from which the best average root mean square error (RMSE) and Pearson correlation coefficient of 3.55 ksi units and 0.85 were obtained.

36 MATERIALS SCIENCE↗

Retrieval of Cloud Liquid Water Using Microwave Signals from LEO Satellites: A Feasibility Study through Simulations

A novel method, using low Earth orbit (LEO) satellite microwave communication links for cloud liquid water measurements, is introduced in this paper. The feasibility of this approach is studied through simulations of the retrieval system including a LEO satellite communicating with a group of ground receivers equipped with signal-to-noise ratio (SNR) estimators, a synthetic cloud attenuation field and a tomographic retrieval algorithm. Rectangular and Gaussian basis functions are considered to define the targeted field. Simulation results suggest that the proposed least-squares based retrieval algorithm produces satisfactory outcomes for both types of basis functions. The root-mean-square error of the retrieved field is around 0.2 dB/km, with the range of the reference field as 0 to 2.35 dB/km. It is also confirmed that the partial retrieval of the cloud field is achievable when a limited number of receivers with restricted locations are available. The retrieval outcomes exhibit properties of high resolution and low error, indicating that the proposed approach has great potential for cloud observations.

54 ENVIRONMENTAL SCIENCES↗

An Exploratory Approach Using Regression and Machine Learning in the Analysis of Mass Absorption Cross Section of Black Carbon Aerosols: Model Development and Evaluation

Mass absorption cross-section of black carbon (MAC BC ) describes the absorptive cross-section per unit mass of black carbon, and is, thus, an essential parameter to estimate the radiative forcing of black carbon. Many studies have sought to estimate MAC BC from a theoretical perspective, but these studies require the knowledge of a set of aerosol properties, which are difficult and/or labor-intensive to measure. We therefore investigate the ability of seven data analytical approaches (including different multivariate regressions, support vector machine, and neural networks) in predicting MAC BC for both ambient and biomass burning measurements. Our model utilizes multi-wavelength light absorption and scattering as well as the aerosol size distributions as input variables to predict MAC BC across different wavelengths. We assessed the applicability of the proposed approaches in estimating MAC BC using different statistical metrics (such as coefficient of determination (R 2 ), mean square error (MSE), fractional error, and fractional bias). Overall, the approaches used in this study can estimate MAC BC appropriately, but the prediction performance varies across approaches and atmospheric environments. Based on an uncertainty evaluation of our models and the empirical and theoretical approaches to predict MAC BC , we preliminarily put forth support vector machine (SVM) as a recommended data analytical technique for use. We provide an operational tool built with the approaches presented in this paper to facilitate this procedure for future users.

54 ENVIRONMENTAL SCIENCES↗

Skill of Mesoscale Models in Forecasting Springtime Macrophysical Cloud Properties at the Savannah River Site in the Southeastern US

Predicting boundary layer clouds is important for the accurate modeling of pollutant dispersion. Higher resolution mesoscale models would be expected to produce better forecasts of cloud properties that affect dispersion. Using ceilometer observations, we assess the skill of two operational mesoscale models (RAMS and WRF) to forecast cloud base altitude and cloud fraction at the Savannah River Site in the southeastern US during the springtime. Verifications were performed at small spatial and temporal scales necessary for dispersion modeling. Both models were unreliable with a 50% (RAMS) and a 46% (WRF) rate of predicting clouds observed by the ceilometer which led to low cloud fraction predictions. Results indicated that WRF better predicted daytime cloud bases from convection that occurred frequently later in the period and RAMS better predicted nighttime cloud bases. Using root mean squared error (RMSE) to score the forecast periods also highlighted this diurnal dichotomy, with WRF scores better during the day and RAMS scores better at night. Analysis of forecast errors revealed divergent model cloud base biases—WRF low and RAMS high. A hybrid solution which weighs more heavily the RAMS nighttime forecasts and WRF daytime forecasts will likely provide the best prediction of cloud properties for dispersion.

54 ENVIRONMENTAL SCIENCES↗

Implementing and Improving CBMZ-MAM3 Chemistry and Aerosol Modules in the Regional Climate Model WRF-CAM5: An Evaluation over the Western US and Eastern North Pacific

The representation of aerosols in climate-chemistry models is important for air quality and climate change research, but it can require significant computational resources. The objective of this study was to improve the representation of aerosols in climate–chemistry models, specifically in the carbon bond mechanism, version Z (CBMZ), and modal aerosol modules with three lognormal modes (MAM3) in the WRF-CAM5 model. The study aimed to enhance the model’s chemistry capabilities by incorporating biomass burning emissions, establishing a conversion mechanism between volatile organic compounds (VOCs) and secondary organic carbons (SOCs), and evaluating its performance against observational benchmarks. The results of the study demonstrated the effectiveness of the enhanced chemistry capabilities in the WRF-CAM5 model. Six simulations were conducted over the western U.S. and northeastern Pacific region, comparing the model’s performance with observational benchmarks such as reanalysis, ground-based, and satellite data. The findings revealed a significant reduction in root-mean-square errors (RMSE) for surface concentrations of black carbon (BC) and organic carbon (OC). Specifically, the model exhibited a 31% reduction in RMSE for BC concentrations and a 58% reduction in RMSE for OC concentrations. These outcomes underscored the importance of accurate aerosol representation in climate-chemistry models and emphasized the potential for improving simulation accuracy and reducing errors through the incorporation of enhanced chemistry modules in such models.

54 ENVIRONMENTAL SCIENCES↗

An Explainable Machine-Learning Model for Compensatory Reserve Measurement: Methods for Feature Selection and the Effects of Subject Variability

Tracking vital signs accurately is critical for triaging a patient and ensuring timely therapeutic intervention. The patient’s status is often clouded by compensatory mechanisms that can mask injury severity. The compensatory reserve measurement (CRM) is a triaging tool derived from an arterial waveform that has been shown to allow for earlier detection of hemorrhagic shock. However, the deep-learning artificial neural networks developed for its estimation do not explain how specific arterial waveform elements lead to predicting CRM due to the large number of parameters needed to tune these models. Alternatively, we investigate how classical machine-learning models driven by specific features extracted from the arterial waveform can be used to estimate CRM. More than 50 features were extracted from human arterial blood pressure data sets collected during simulated hypovolemic shock resulting from exposure to progressive levels of lower body negative pressure. A bagged decision tree design using the ten most significant features was selected as optimal for CRM estimation. This resulted in an average root mean squared error in all test data of 0.171, similar to the error for a deep-learning CRM algorithm at 0.159. By separating the dataset into sub-groups based on the severity of simulated hypovolemic shock withstood, large subject variability was observed, and the key features identified for these sub-groups differed. This methodology could allow for the identification of unique features and machine-learning models to differentiate individuals with good compensatory mechanisms against hypovolemia from those that might be poor compensators, leading to improved triage of trauma patients and ultimately enhancing military and emergency medicine.

60 APPLIED LIFE SCIENCES↗

Evaluating Machine Learning-Based MRI Reconstruction Using Digital Image Quality Phantoms

Quantitative and objective evaluation tools are essential for assessing the performance of machine learning (ML)-based magnetic resonance imaging (MRI) reconstruction methods. However, the commonly used fidelity metrics, such as mean squared error (MSE), structural similarity (SSIM), and peak signal-to-noise ratio (PSNR), often fail to capture fundamental and clinically relevant MR image quality aspects. To address this, we propose evaluation of ML-based MRI reconstruction using digital image quality phantoms and automated evaluation methods. Our phantoms are based upon the American College of Radiology (ACR) large physical phantom but created in k-space to simulate their MR images, and they can vary in object size, signal-to-noise ratio, resolution, and image contrast. Our evaluation pipeline incorporates evaluation metrics of geometric accuracy, intensity uniformity, percentage ghosting, sharpness, signal-to-noise ratio, resolution, and low-contrast detectability. We demonstrate the utility of our proposed pipeline by assessing an example ML-based reconstruction model across various training and testing scenarios. The performance results indicate that training data acquired with a lower undersampling factor and coils of larger anatomical coverage yield a better performing model. The comprehensive and standardized pipeline introduced in this study can help to facilitate a better understanding of the performance and guide future development and advancement of ML-based reconstruction algorithms.

47 OTHER INSTRUMENTATION↗