Search NASA⌕ Search

SEARCH · Search NASA

Results for “evaluation datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

AI Benchmark Democratization and Carpentry

Benchmarks are a cornerstone of modern machine learning, enabling reproducibility, comparison, and scientific progress. However, AI benchmarks are increasingly complex, requiring dynamic, AI-focused workflows. Rapid evolution in model architectures, scale, datasets, and deployment contexts makes evaluation a moving target. Large language models often memorize static benchmarks, causing a gap between benchmark results and real-world performance. Beyond traditional static benchmarks, continuous adaptive benchmarking frameworks are needed to align scientific assessment with deployment risks. This calls for skills and education in AI Benchmark Carpentry. From our experience with MLCommons, educational initiatives, and programs like the DOE's Trillion Parameter Consortium, key barriers include high resource demands, limited access to specialized hardware, lack of benchmark design expertise, and uncertainty in relating results to application domains. Current benchmarks often emphasize peak performance on top-tier hardware, offering limited guidance for diverse, real-world scenarios. Benchmarking must become dynamic, incorporating evolving models, updated data, and heterogeneous platforms while maintaining transparency, reproducibility, and interpretability. Democratization requires both technical innovation and systematic education across levels, building sustained expertise in benchmark design and use. Benchmarks should support application-relevant comparisons, enabling informed, context-sensitive decisions. Dynamic, inclusive benchmarking will ensure evaluation keeps pace with AI evolution and supports responsible, reproducible, and accessible AI deployment. Community efforts can provide a foundation for AI Benchmark Carpentry.

von Laszewski, Gregor [Virginia U.]↗

Immersive Scientific Visualization of Molten-Salt Reactor Waste Characteristics Using Virtual Reality

Immersive visualization is changing how we explore, communicate, and understand complex scientific systems. In nuclear energy, an area in which data are often multidimensional, time-dependent, and difficult to interpret, virtual reality (VR) represents a powerful and intuitive informational medium. This work introduces a VR-based platform that visualizes the post-shutdown behavior and waste management lifecycle of molten-salt reactors (MSRs), a next-generation reactor type with unique operational and safety characteristics. The platform, built in Unity, is streamed on the Meta Quest 3 headset. It transforms high-fidelity simulation data into an interactive, immersive experience. Users can explore time-dependent reactor characteristics such as nuclide decay, which is a key factor for evaluating reactor waste strategies. The datasets were generated using the MOOSE (Multiphysics Object-Oriented Simulation Environment) framework and then processed through ParaView scripting for smooth integration into Unity. From a visualization standpoint, the platform emphasizes spatial storytelling, temporal exploration, and user-centered interaction. Users can navigate 3D reactor geometries, slice through volumetric data, and manipulate time to observe how physical phenomena evolve. Real-scale rendering and embodied interaction make the experience feel tangible. The interface is designed to be accessible, even to those without nuclear or simulation expertise. This lowers the barrier for stakeholders, policymakers, and the general public, while still supporting expert analysis and collaborative decision-making. This work shows how immersive visualization can function as both a scientific tool and a communication interface. By integrating simulation, processing, and visualization into a cohesive workflow, we offer a scalable framework for immersive scientific storytelling. The modular design supports future extensions to other reactor types and lifecycle stages, from shutdown to long-term storage, making the platform adaptable for both research and outreach.

99 - GENERAL AND MISCELLANEOUS↗

Radiatively important parameters best estimate (RIPBE) value-added product (VAP)

Currently, to calculate radiative heating rate profiles for the Broadband Heating Rate Profile (BBHRP) product, radiatively important parameters (water vapor, ozone, surface albedo, aerosol properties, and cloud properties) from multiple VAPs and datastreams are combined into input text files that are then used to run the RRTM radiative transfer codes. These input parameters have different temporal and spatial scales and are difficult to extract from the text files to be used for other purposes such as running other radiative transfer codes, analyzing results, or error tracking. The purpose of the Radiatively Important Parameters Best Estimate (RIPBE) VAP is to improve this process by creating a clearly identified set of inputs for BBHRP (and other radiation codes) on a uniform vertical and temporal grid. This process will decouple the input/output portion of the BBHRP from the core physics (the RRTM radiative transfer model) and will add error tracking and version information to the input data set. Critical parameters (which must exist for the radiation code to be run) will be designated; for other parameters, climatological or fixed values will be used when the preferred values are missing. This should increase the number of cases for which radiative transfer calculations can be run. In all cases, flags will clearly identify the source for each parameter. RIPBE will serve multiple functions: (1) it will provide a clearly identifiable set of inputs for BBHRP, (2) it will facilitate the use of BBHRP as a retrieval and radiation code development testbed by providing a vehicle for easily extracting and swapping input parameters needed to conduct radiative transfer calculations, and (3) it will be a complement to the Climate Modeling Best Estimate (CMBE) VAP and will provide a significantly expanded set of parameters for model evaluation in a showcase data set form. At the ASR meeting, we will present examples and evaluation of the initial RIPBE dataset at SGP.

54 ENVIRONMENTAL SCIENCES↗

Performance Evaluation of Weather@home2 Simulations over West African Region

Weather and climate forecasting, using climate models, have become essential tools and life-savers in the West African region; in spite of the fact that climate models do not fully comply with attributes of forecast qualities—RASAP: reliability, association, skill, accuracy, and precision. The objective of this paper is to quantitatively evaluate, in comparison to CRU and ERA5 datasets, the RASAP compliance-level of the weather@home2 modeling system (w@h2). Findings from some statistical evaluations show that, to a moderately significant extent, w@h2 model provides useful information during the monsoon seasons; skills to capture the Little Dry Season over the Guinea zone; predictive skills for the onset season; ability to reproduce all the annual characteristics of the surface maximum air temperature over the region; as well as skill to detect heat waves that usually ravage West Africa during the boreal spring. The model displays traces of attributes that are needed for seasonal climate predictions and applications. Deficiencies in the quantitative reproducibility point to the facts that the model does provide a reliability akin to that of regional climate models. This paper further furnishes a prospective user with information on whether the model might be “useful or not” for a particular application.

West Africa↗

Next-Level Energy Management in Manufacturing: Facility-Level Energy Digital Twin Framework Based on Machine Learning and Automated Data Collection

This research introduces an energy prediction framework at the facility level supported by automated data collection and machine learning models. It investigates whether reducing the prediction time scale allows for applying more complex machine learning techniques and if those techniques improve the prediction accuracy. The primary advantages of this framework lie in its automation of the energy prediction process and its provision of real-time energy data suitable for use in energy dashboards or digital twins. A sitewide dataset was created by combining 15 min energy and daily production data of five shops—assembly, battery, body (electric), body (gas), and paint—from a globally recognized electric vehicle manufacturer. Various machine learning models were evaluated on daily, weekly, and monthly datasets, including, in increasingly complex order: naïve, simple linear regression, net regularized generalized linear regression, principal component regression, k-nearest neighbor, random forest, and Bayesian regularized neural network. Compared to the current state-of-the-art energy consumption prediction for the industrial facility level, this research investigates more complex models and smaller time intervals for higher accuracy. The findings revealed that the more complex monthly models require a minimum of a year and a half of data to operate, while weekly models demand a year of data to achieve improved accuracy. Daily models can operate with only six months of data but exhibit poor performance due to reduced prediction accuracy of production. Key challenges identified include access to reliable, high-quality energy and production data and the initial demand for human labor.

digital twin↗

A multi-dataset data-collection strategy produces better diffraction data

Theoretical analysis and experimental validation prove that a multi-dataset data-collection strategy produces better diffraction data. The readiness test is a simple and sensitive method for X-ray data-collection system evaluation and a benchmark. A multi-dataset (MDS) data-collection strategy is proposed and analyzed for macromolecular crystal diffraction data acquisition. The theoretical analysis indicated that the MDS strategy can reduce the standard deviation (background noise) of diffraction data compared with the commonly used single-dataset strategy for a fixed X-ray dose. In order to validate the hypothesis experimentally, a data-quality evaluation process, termed a readiness test of the X-ray data-collection system, was developed. The anomalous signals of sulfur atoms in zinc-free insulin crystals were used as the probe to differentiate the quality of data collected using different data-collection strategies. The data-collection results using home-laboratory-based rotating-anode X-ray and synchrotron X-ray systems indicate that the diffraction data collected with the MDS strategy contain more accurate anomalous signals from sulfur atoms than the data collected with a regular data-collection strategy. In addition, the MDS strategy offered more advantages with respect to radiation-damage-sensitive crystals and better usage of rotating-anode as well as synchrotron X-rays.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Is the assumption of normality or log-normality for continuous response data critical for benchmark dose estimation?

Continuous responses (e.g. body weight) are widely used in risk assessment for determining the benchmark dose (BMD) which is used to derive a U.S. EPA reference dose. One critical question that is not often addressed in dose–response assessments is whether to model the continuous data as normally or log-normally distributed. Additionally, if lognormality is assumed, and only summarized response data (i.e., mean ± standard deviation) are available as is usual in the peer-reviewed literature, the BMD can only be approximated. In this study, using the “hybrid” method and relative deviation approach, we first evaluate six representative continuous dose–response datasets reporting individual animal responses to investigate the impact on BMD/BMDL estimates of (1) the distribution assumption and (2) the use of summarized versus individual animal data when a log-normal distribution is assumed. We also conduct simulation studies evaluating model fits to various known distributions to investigate whether the distribution assumption has influence on BMD/BMDL estimates. Our results indicate that BMDs estimated using the hybrid method are more sensitive to the distribution assumption than counterpart BMDs estimated using the relative deviation approach. The choice of distribution assumption has limited impact on the BMD/BMDL estimates when the within dose-group variance is small, while the lognormality assumption is a better choice for relative deviation method when data are more skewed because of its appropriateness in describing the relationship between mean and standard deviation. Additionally, the results suggest that the use of summarized data versus individual response data to characterize log-normal distributions has minimal impact on BMD estimates. - Highlights: • We investigate to what extent the distribution assumption can affect BMD estimates. • Both real data analysis and simulation study are conducted. • BMDs estimated using hybrid method are more sensitive to distribution assumption. • Summarized continuous data are adequate for BMD estimation.

60 APPLIED LIFE SCIENCES↗

An Evaluation of Representation Learning Methods in Particle Physics Foundation Models

We present a systematic evaluation of representation learning objectives for particle physics within a unified framework. Our study employs a shared transformer-based particle-cloud encoder with standardized preprocessing, matched sampling, and a consistent evaluation protocol on a jet classification dataset. We compare contrastive (supervised and self-supervised), masked particle modeling, and generative reconstruction objectives under a common training regimen. In addition, we introduce targeted supervised architectural modifications that achieve state-of-the-art performance on benchmark evaluations. This controlled comparison isolates the contributions of the learning objective, highlights their respective strengths and limitations, and provides reproducible baselines. We position this work as a reference point for the future development of foundation models in particle physics, enabling more transparent and robust progress across the community.

Chen, Michael [Caltech]↗

SU-D-17A-02: Four-Dimensional CBCT Using Conventional CBCT Dataset and Iterative Subtraction Algorithm of a Lung Patient

Purpose: The Iterative Subtraction Algorithm (ISA) method generates retrospectively a pre-selected motion phase cone-beam CT image from the full motion cone-beam CT acquired at standard rotation speed. This work evaluates ISA method with real lung patient data. Methods: The goal of the ISA algorithm is to extract motion and no- motion components form the full reconstruction CBCT. The workflow consists of subtracting from the full CBCT all of the undesired motion phases and obtain a motion de-blurred single-phase CBCT image, followed by iteration of this subtraction process. ISA is realized as follows: 1) The projections are sorted to various phases, and from all phases, a full reconstruction is performed to generate an image CTM. 2) Generate forward projections of CTM at the desired phase projection angles, the subtraction of projection and the forward projection will reconstruct a CTSub1, which diminishes the desired phase component. 3) By adding back the CTSub1 to CTm, no motion CBCT, CTS1, can be computed. 4) CTS1 still contains residual motion component. 5) This residual motion component can be further reduced by iteration.The ISA 4DCBCT technique was implemented using Varian Trilogy accelerator OBI system. To evaluate the method, a lung patient CBCT dataset was used. The reconstruction algorithm is FDK. Results: The single phase CBCT reconstruction generated via ISA successfully isolates the desired motion phase from the full motion CBCT, effectively reducing motion blur. It also shows improved image quality, with reduced streak artifacts with respect to the reconstructions from unprocessed phase-sorted projections only. Conclusion: A CBCT motion de-blurring algorithm, ISA, has been developed and evaluated with lung patient data. The algorithm allows improved visualization of a single phase motion extracted from a standard CBCT dataset. This study has been supported by National Institute of Health through R01CA133539.

60 APPLIED LIFE SCIENCES↗

Assessing Current and Future Infrastructure Hazards

Project Objective Execute intelligent analytics via an advanced analytical framework, to assess the current state of offshore infrastructure, evaluate infrastructure life, and identify technologies to reduce infrastructure hazards, costs, and extend infrastructure life. Approach &amp; Results thus Far• Build comprehensive dataset• Perform data-driven analytics to evaluate infrastructure integrity<p> 1. Remaining lifespan</p><p> 2. Likelihood of future risk</p><p>• Apply data-driven advanced spatial, statistical, and Machine Learning (ML) models to quantify existing infrastructure integrity</p><p>• Release data and models through a smart, online platform hosted by Energy Data eXchange (EDX)</p>

Romeo, Lucy F.↗

Evaluation of historical CMIP6 model simulations of extreme precipitation over contiguous US regions

Simulated historical precipitation is evaluated for Coupled Model Intercomparison Project Phase 6 (CMIP6) models using precipitation indices defined by the Expert Team on Climate Change Detection and Indices. The model indices are evaluated against corresponding indices from the CPC unified gauge-based analyses of precipitation over seven geographical regions across the contiguous US (CONUS). The regions assessed match those in recent US National Climate Assessment Reports. To estimate observational uncertainty, precipitation indices for three other observational datasets (HadEx2, Livneh and PRISM) are evaluated against the CPC analyses. Both the moderate and extreme mean precipitation intensities are overestimated over the western CONUS and underestimated in the areas of the Central Great Plains (CGP) in most CMIP6 models tested. Most CMIP6 models overestimate the mean and variability of wet spell durations and underestimate the mean and variability of dry spell durations across the CONUS. Biases in interannual variability of most of the indices have similar patterns to those in corresponding mean biases. The median and interquartile model spreads in CMIP6 model biases are clearly smaller than those in CMIP5 model biases for wet spell durations. Multimodel medians of CMIP6 (CMIP6-MMM) and CMIP5 (CMIP5-MMM) have similar biases in climatology and variability but biases tend to be smaller in CMIP6-MMM. Depending on the index, extreme precipitation is slightly better in parts of the eastern half of the CONUS in CMIP6-MMM, otherwise, the biases in climatology and variability are similar to CMIP5-MMM. CMIP6-MMM performs better than individual models and even observational datasets in some cases. Differences between observational datasets for most indices are comparable to the CMIP6 interquartile model spread. The better-performing observational and model datasets are different in different parts of the CONUS.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Review and Analysis of Algorithmic Approaches Developed for Prognostics on CMAPSS Dataset

Benchmarking of prognostic algorithms has been challenging due to limited availability of common datasets suitable for prognostics. In an attempt to alleviate this problem several benchmarking datasets have been collected by NASA's prognostic center of excellence and made available to the Prognostics and Health Management (PHM) community to allow evaluation and comparison of prognostics algorithms. Among those datasets are five C-MAPSS datasets that have been extremely popular due to their unique characteristics making them suitable for prognostics. The C-MAPSS datasets pose several challenges that have been tackled by different methods in the PHM literature. In particular, management of high variability due to sensor noise, effects of operating conditions, and presence of multiple simultaneous fault modes are some factors that have great impact on the generalization capabilities of prognostics algorithms. More than 70 publications have used the C-MAPSS datasets for developing data-driven prognostic algorithms. The C-MAPSS datasets are also shown to be well-suited for development of new machine learning and pattern recognition tools for several key preprocessing steps such as feature extraction and selection, failure mode assessment, operating conditions assessment, health status estimation, uncertainty management, and prognostics performance evaluation. This paper summarizes a comprehensive literature review of publications using C-MAPSS datasets and provides guidelines and references to further usage of these datasets in a manner that allows clear and consistent comparison between different approaches.

Uncertainty↗

On piecewise interpolation techniques for estimating solar radiation missing values in Kedah

This paper discusses the use of piecewise interpolation method based on cubic Ball and Bézier curves representation to estimate the missing value of solar radiation in Kedah. An hourly solar radiation dataset is collected at Alor Setar Meteorology Station that is taken from Malaysian Meteorology Deparment. The piecewise cubic Ball and Bézier functions that interpolate the data points are defined on each hourly intervals of solar radiation measurement and is obtained by prescribing first order derivatives at the starts and ends of the intervals. We compare the performance of our proposed method with existing methods using Root Mean Squared Error (RMSE) and Coefficient of Detemination (CoD) which is based on missing values simulation datasets. The results show that our method is outperformed the other previous methods.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Evaluating Nighttime CALIOP 0.532 micron Aerosol Optical Depth and Extinction Coefficient Retrievals

NASA Cloud Aerosol Lidar with Orthogonal Polarization (CALIOP) Version 3.01 5-km nighttime 0.532 micron aerosol optical depth (AOD) datasets from 2007 are screened, averaged and evaluated at 1 deg X 1 deg resolution versus corresponding/co-incident 0.550 micron AOD derived using the US Navy Aerosol Analysis and Prediction System (NAAPS), featuring two-dimensional variational assimilation of quality-assured NASA Moderate Resolution Imaging Spectroradiometer (MODIS) and Multi-angle Imaging Spectroradiometer (MISR) AOD. In the absence of sunlight, since passive radiometric AOD retrievals rely overwhelmingly on scattered radiances, the model represents one of the few practical global estimates available from which to attempt such a validation. Daytime comparisons, though, provide useful context. Regional-mean CALIOP vertical profiles of night/day 0.532 micron extinction coefficient are compared with 0.523/0.532 micron ground-based lidar measurements to investigate representativeness and diurnal variability. In this analysis, mean nighttime CALIOP AOD are mostly lower than daytime (0.121 vs. 0.126 for all aggregated data points, and 0.099 vs. 0.102 when averaged globally per normalised 1 deg. X 1 deg. bin), though the relationship is reversed over land and coastal regions when the data are averaged per normalised bin (0.134/0.108 vs. 0140/0.112, respectively). Offsets assessed within single bins alone approach +/- 20 %. CALIOP AOD, both day and night, are higher than NAAPS over land (0.137 vs. 0.124) and equal over water (0.082 vs. 0.083) when averaged globally per normalised bin. However, for all data points inclusive, NAAPS exceeds CALIOP over land, coast and ocean, both day and night. Again, differences assessed within single bins approach 50% in extreme cases. Correlation between CALIOP and NAAPS AOD is comparable during both day and night. Higher correlation is found nearest the equator, both as a function of sample size and relative signal magnitudes inherent at these latitudes. Root mean square deviation between CALIOP and NAAPS varies between 0.1 and 0.3 globally during both day/night. Averaging of CALIOP along-track AOD data points within a single NAAPS grid bin improves correlation and RMSD, though day/night and land/ocean biases persist and are believed systematic. Vertical profiles of extinction coefficient derived in the Caribbean compare well with ground-based lidar observations, though potentially anomalous selection of a priori lidar ratios for CALIOP retrievals is likely inducing some discrepancies. Mean effective aerosol layer top heights are stable between day and night, indicating consistent layer-identification diurnally, which is noteworthy considering the potential limiting effects of ambient solar noise during day.

aerosol optical depth↗

Investigation of Spiral Bevel Gear Condition Indicator Validation via AC-29-2C Using Fielded Rotorcraft HUMS Data

This report presents the analysis of gear condition indicator data collected on a helicopter when damage occurred in spiral bevel gears. The purpose of the data analysis was to use existing in-service helicopter HUMS flight data from faulted spiral bevel gears as a Case Study, to better understand the differences between HUMS data response in a helicopter and a component test rig, the NASA Glenn Spiral Bevel Gear Fatigue Rig. The reason spiral bevel gear sets were chosen to demonstrate differences in response between both systems was the availability of the helicopter data and the availability of a test rig that was capable of testing spiral bevel gear sets to failure. The objective of the analysis presented in this paper was to re-process helicopter HUMS data with the same analysis techniques applied to the spiral bevel rig test data. The damage modes experienced in the field were mapped to the failure modes created in the test rig. A total of forty helicopters were evaluated. Twenty helicopters, or tails, experienced damage to the spiral bevel gears in the nose gearbox. Vibration based gear condition indicators data was available before and after replacement. The other twenty tails had no known anomalies in the nose gearbox within the time frame of the datasets. These twenty tails were considered the baseline dataset. The HUMS gear condition indicators evaluated included gear condition indicators (CI) Figure of Merit 4 (FM4), Root Mean Square (RMS) or Diagnostic Algorithm 1 (DA1) and +/- 3 Sideband Index (SI3). Three additional condition indicators, not currently calculated on-board, were calculated from the archived data. These three indicators were +/- 1 Sideband Index (SI1), the DA1 of the difference signal (DiffDA1) and the peak-to-peak of the difference signal (DP2P). Results found the CI DP2P, not currently available in the on-board HUMS, performed the best, responding to varying levels of damage on thirteen of the fourteen helicopters evaluated. Two additional CIs also not in the on-board system, DiffDA1and SI1, also performed well responding to twelve and ten of the fourteen helicopters evaluated respectively. Of the three CIs currently available in the MSPU, DA1, FM4 and SI3, SI3, responded to eight, DA1 responded to six and FM4 responded to four of the fourteen helicopters evaluated. FM4, the poorest performing CI, was not as responsive to damage as the other five CIs. Conversely, when compared to the other two, it was the only CI that responded to damage on two helicopters. CI response could not be correlated to specific failure modes due to limited pictures and subjective descriptions found within the TDA. Flight regime did affect CI response to some gear faults. Due to the range of operating conditions for each regime, more studies are required to determine their sensitivity to regimes.

rotorcraft↗

A Centralized AI Lakehouse Framework for Brain Tumor MRI Classification and Segmentation, University KPI Forecasting, and Water Potability Prediction

In many university and healthcare projects, models are built for very different data types such as tables, institutional time series, and medical images, but they are deployed as separate applications. In this work, that separation made testing and maintenance difficult because each module had its own pipeline and runtime requirements. This paper presents an integrated AI lakehouse-style implementation that runs three model pipelines inside one containerized backend. For medical imaging, we used MRI datasets from IEEE DataPort: a four-class classification set with 7012 images (5708 train/1304 test) and a segmentation set with 3063 image–mask pairs. The classification model (ResNet50 transfer learning) is evaluated using a proper train–validation–test protocol across multiple splits (80/10/10, 70/10/20, 60/10/30, and 10/30/60), achieving a test accuracy of 99.00% under the standard 80/10/10 split. Additionally, a patient-level evaluation is conducted using an external glioma dataset to provide a more realistic assessment without data leakage. The segmentation model (DeepLabV3-ResNet50) achieved 83.09% validation mIoU and 88.79% Dice score. For university KPI forecasting, we used annual IPEDS and NSF HERD data from 2010 to 2023 for three universities (BSU, EOU, and UAB). To examine the effect of preprocessing on forecasting performance, two case studies are conducted. In the first case, linear interpolation is applied to generate semester-level data. In the second case, the original annual data is used directly without interpolation. Random Forest regression and ARIMA models are evaluated using MAE, RMSE, MAPE, and R 2 . The results showed that interpolation improved apparent forecasting performance due to smoothing, while evaluation on the original annual data provided a more realistic assessment of model behavior. To further validate the framework on a larger dataset, an additional case study is conducted using a student dropout dataset. For water potability, we trained and compared multiple tabular classifiers on a large dataset (1,048,575 samples). A Random Forest model (100 trees, max depth 10) achieved 85.86% test accuracy and high recall for unsafe samples (0.8447). All modules are served via FastAPI and deployed together using Docker, with workflow automation routing requests to the correct endpoint. System-level benchmarking indicates that the backend maintains stable throughput and latency under concurrent requests.

97 MATHEMATICS AND COMPUTING↗

Systematic and quantitative assessment of the effect of chronic kidney disease on CYP2D6 and CYP3A4/5

Recent reviews suggest that chronic kidney disease (CKD) can affect the pharmacokinetics of nonrenally eliminated drugs, but the impact of CKD on individual elimination pathways has not been systematically evaluated. In this study we developed a comprehensive dataset of the effect of CKD on the pharmacokinetics of CYP2D6- and CYP3A4/5-metabolized drugs. Drugs for evaluation were selected based on clinical drug–drug interaction (CYP3A4/5 and CYP2D6) and pharmacogenetic (CYP2D6) studies. Information from dedicated CKD studies was available for 13 and 18 of the CYP2D6 and CYP3A4/5 model drugs, respectively. Analysis of these data suggested that CYP2D6-mediated clearance is generally decreased in parallel with the severity of CKD. There was no apparent relationship between the severity of CKD and CYP3A4/5-mediated clearance. The observed elimination-route dependency in CKD effects between CYP2D6 and CYP3A4/5 may inform the need to conduct clinical CKD studies with nonrenally eliminated drugs for optimal use of drugs in patients with CKD.

60 APPLIED LIFE SCIENCES↗

Domain Shift Analysis in Chest Radiographs Classification in a Veterans Healthcare Administration Population

This study aims to assess the impact of domain shift on chest X-ray classification accuracy and to analyze the influence of ground truth label quality and demographic factors such as age group, sex, and study year. We used a DenseNet121 model pre-trained MIMIC-CXR dataset for deep learning-based multi-label classification using ground truth labels from radiology reports extracted using the CheXpert and CheXbert Labeler. We compared the performance of the 14 chest X-ray labels on the MIMIC-CXR and Veterans Healthcare Administration chest X-ray dataset (VA-CXR). The validation of ground truth and the assessment of multi-label classification performance across various NLP extraction tools revealed that the VA-CXR dataset exhibited lower disagreement rates than the MIMIC-CXR datasets. Additionally, there were notable differences in AUC scores between models utilizing CheXpert and CheXbert. When evaluating multi-label classification performance across different datasets, minimal domain shift was observed in the unseen VA dataset, except for the label “Enlarged Cardiomediastinum.” The subgroup with the most significant variations in multi-label classification performance was study year. These findings underscore the importance of considering domain shift in chest X-ray classification tasks, paying particular attention to the temporality of the exam. Our study reveals the significant impact of domain shift and demographic factors on chest X-ray classification, emphasizing the need for improved transfer learning and robust model development. Addressing these challenges is crucial for advancing medical imaging research and improving patient care.

chest X-ray image classification↗