Search NASA⌕ Search

SEARCH · Search NASA

Results for “Selection Bias”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Core-collapse contamination in photometric samples of Type Ia Supernovae

This is an exciting time for cosmology with type Ia supernovae (SNe Ia). The recentlyconcluded Dark Energy Survey SN programme (DES-SN) has obtained the largest anddeepest high-redshift cosmological SN Ia sample, and the Vera Rubin Observatory isexpected to observe at least one order of magnitude more SNe Ia in the next decade.In both these experiments, only a limited fraction (.10 per cent) of the SNe can bespectroscopically classified. This leaves us with large ‘photometric’ SN samples, withthe potential for significant contamination by core-collapse SNe that may bias SN Iacosmological measurements. This thesis demonstrates how this contamination can bemodelled and accounted for in current and future cosmological analyses.First, I present state-of-the-art simulations of the SN universe. These are designedto accurately model the population of SNe Ia, peculiar SNe Ia and core-collapse SNe, aswell as their host galaxies. To improve the diversity and quality of the simulated corecollapseSNe, I build a new library of core-collapse SN templates using spectroscopicand photometric (optical and near-ultraviolet) data of 67 core-collapse SNe from theliterature. I account for our incomplete knowledge of core-collapse SN properties bygenerating a set of SN simulations (rather than a single one), each exploring differentmodelling choices and template libraries. I then characterise selection effects in theDES-SN survey and incorporate them in the simulations, thus obtaining a series ofDES-like simulated SN samples that can be compared to the observed DES-SN data.The agreement between the simulations and data is excellent across many observed SNproperties, including Hubble residuals. These simulations are the first to reproduce theobserved photometric SN and host galaxy properties in high-redshift surveys with no fine-tuning of the input parameters.I use my simulation framework to train and test the performance of SuperNNova,a photometric SN classifier based on recurrent neural networks. I explore differenttraining and validation strategies and show that, across all the DES-SN simulationstested, SuperNNova reduces core-collapse SN contamination to 0.8–3.5 per cent. Ithen show that biases due to contamination on the equation-of-state of dark energy,w, are < 0:008 when using our reference SuperNNova model. This compares to anexpected statistical uncertainty on w from the DES-SN sample of 0:039, thus showing that contamination is not a limiting systematic for the cosmological analysis of theDES-SN sample.The results presented in this thesis are the foundation of the DES SN Ia cosmologicalanalysis; they also provide important implications for the future of SN cosmology,as they demonstrate that contamination is not expected to significantly degrade thecosmological figure of merit of the Rubin SN Ia analysis.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Bridging microscopy with molecular dynamics and quantum simulations: an atomAI based pipeline

Recent advances in (scanning) transmission electron microscopy have enabled a routine generation of large volumes of high-veracity structural data on 2D and 3D materials, naturally offering the challenge of using these as starting inputs for atomistic simulations. In this fashion, the theory will address experimentally emerging structures, as opposed to the full range of theoretically possible atomic configurations. However, this challenge is highly nontrivial due to the extreme disparity between intrinsic timescales accessible to modern simulations and microscopy, as well as latencies of microscopy and simulations per se. Addressing this issue requires as a first step bridging the instrumental data flow and physics-based simulation environment, to enable the selection of regions of interest and exploring them using physical simulations. Here we report the development of the machine learning workflow that directly bridges the instrument data stream into Python-based molecular dynamics and density functional theory environments using pre-trained neural networks to convert imaging data to physical descriptors. Additionally, the pathways to ensure structural stability and compensate for the observational biases universally present in the data are identified in the workflow. This approach is used for a graphene system to reconstruct optimized geometry and simulate temperature-dependent dynamics including adsorption of Cr as an ad-atom and graphene healing effects. However, it is universal and can be used for other material systems.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Miniaturized Magnetoelastic Sensor System

This article describes the design, assembly, and implementation of a hand-held, magnetic-field-based sensor system that can be adapted for a variety of sensing applications. The miniaturized system is based on Chemical Identification by Magneto-Elastic Sensing (ChIMES) technology, which uses three concentric solenoid coils to wirelessly interrogate a sensor body comprised of a response material coupled to a magnetoelastic wire. The response material expands when it encounters a target, imposing mechanical stress on the wire and altering its magnetic permeability. The sensor bodies are passive, requiring no external power source, and they are small, measuring about 15 mm in length and 3.0 mm in diameter. Up to four sensor bodies can be configured as an evenly-spaced linear array. The sensor system operates by applying a low-frequency, current-stabilized, filtered triangle wave to a uniform-density excitation coil to switch the magnetic domains within the wire. Further, the responses from the sensors are picked up by a detection coil as stress-induced changes in the Faraday voltage, and the strong magnetic field induced by the excitation coil in the detection coil is nullified by a cancellation coil reverse-wound in series with the detection coil. The responses of the sensors in an array are separated in time by a linear gradient dc biasing coil. The sensors can be interrogated through metallic and nonmetallic barriers. The signals from the detection coil and the excitation coil are digitized by a pair of bipolar analog-to-digital converters (ADCs). A Raspberry Pi single-board computer (SBC) and associated software perform data acquisition and control all aspects of the sensor system hardware. The program allows the user to select the number of sensors in the array, the type of signal that is being collected, and the number of samples to take. The program also allows for signal processing of the sensor data, such as baseline correction. The program can differentiate sensor peaks from each other and calculate the magnitude of each sensor response with less than 1% error. The data are then displayed along with a graph of the signal.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

The Effects of Threshold Voltage and Number of Fins per Transistor on the TID Response of GF 12LP Technology

This abstract presents a comprehensive analysis of total ionizing dose (TID) response in GlobalFoundries’ 12LP 12nm bulk FinFET technology using 10keV X-rays. Devices with higher threshold voltages (VTs) demonstrated lower increases in off-state leakage current (IDS,OFF) post-irradiation, highlighting the mitigating role of high VT in TID response. Our data shows that transistors with fewer fins exhibit superior TID resistance, implying lower susceptibility to radiation effects. Our study also probed two bias conditions, "Gate-On" and "Pass-Gate," with the former displaying more severe TID degradation. Interestingly, p-type devices displayed negligible degradation, underscoring their inherent resilience to TID effects. Additionally, medium thick n-type devices echoed the fin-count-dependent TID response observed in other transistor types, further strengthening our findings. These results underscore the importance of strategic transistor selection and design for enhancing the TID resilience of future CMOS FinFET architectures, particularly critical in radiation-intense environments.

42 ENGINEERING↗
DOE OSTI2026

Short Circuit Detection and Voltage Sense for High Voltage Ionization Tube Power Supplies

Fermilab's PIP-II upgrade requires new rack-mounted power supplies for Beam Loss Monitor (BLM) ionization tubes compatible with the microTCA 4.1 standard, supporting high-side current measurement for short circuit detection, voltage sense telemetry, and low-side current measurement for beam loss readout. This work presents the design and preliminary schematic of such a supply. A resistive current sensing approach was chosen over inductive, optical, and Hall-effect alternatives for its independence from cable length and component availability. A 500 mΩ inline shunt steps the 2 kV common mode voltage down to roughly 70 V, producing a ~27 μV pulse during a short, which is amplified 500x, compared against a 14-bit DAC threshold, and latched to drive a fault line. Voltage sense steps the 2 kV output down at a 1V/1000V ratio, buffers it with a low bias current amplifier, and digitizes it via a 24-bit ADC over SPI. Onboard power is regulated from 12 V down to isolated 5 V and 3.3 V rails, with capacitive isolation on FPGA communication lines. Lumped parameter models were developed for the ionization tube (0.327 nF, 20 to 200 GΩ dynamic resistance) and the RG-58 coaxial cable to support the design. Component selection is complete and a preliminary schematic has been drawn in Altium Designer, forming the foundation for future prototyping and validation.

Yu, Kellen [Cornell U.]↗
DOE OSTI2023

Annotated genome sequence of a fast-growing diploid clone of red alder ( Alnus rubra Bong.)

Abstract Red alder (Alnus rubra Bong.) is an ecologically significant and important fast-growing commercial tree species native to western coastal and riparian regions of North America, having highly desirable wood, pigment, and medicinal properties. We have sequenced the genome of a rapidly growing clone. The assembly is nearly complete, containing the full complement of expected genes. This supports our objectives of identifying and studying genes and pathways involved in nitrogen-fixing symbiosis and those related to secondary metabolites that underlie red alder's many interesting defense, pigmentation, and wood quality traits. We established that this clone is most likely diploid and identified a set of SNPs that will have utility in future breeding and selection endeavors, as well as in ongoing population studies. We have added a well-characterized genome to others from the order Fagales. In particular, it improves significantly upon the only other published alder genome sequence, that of Alnus glutinosa. Our work initiated a detailed comparative analysis of members of the order Fagales and established some similarities with previous reports in this clade, suggesting a biased retention of certain gene functions in the vestiges of an ancient genome duplication when compared with more recent tandem duplications.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Systematic comparison and cross-validation of fixed-node diffusion Monte Carlo and phaseless auxiliary-field quantum Monte Carlo in solids

Quantum Monte Carlo (QMC) methods are some of the most accurate methods for simulating correlated electronic systems. Here, we investigate the compatibility, strengths, and weaknesses of two such methods, namely, diffusion Monte Carlo (DMC) and auxiliary-field quantum Monte Carlo (AFQMC). The multideterminant trial wave functions employed in both approaches are generated using the configuration interaction using a perturbative selection made iteratively (CIPSI) technique. Complete basis-set full configuration interaction energies estimated with CIPSI are used as a reference in this comparative study between DMC and AFQMC. By focusing on a set of canonical finite–size solid-state systems, we show that both QMC methods can be made to systematically converge towards the same energy once basis-set effects and systematic biases have been removed. AFQMC shows a much smaller dependence on the trial wave function than DMC while simultaneously exhibiting a much larger basis-set dependence. We outline some of the remaining challenges and opportunities for improving these approaches.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2022

Machine Learning Assisted HPC Workload Trace Generation for Leadership Scale Storage Systems

Monitoring and analyzing a wide range of I/O activities in an HPC cluster is important in maintaining mission-critical performance in a large-scale, multi-user, parallel storage system. Center-wide I/O traces can provide high-level information and fine-grained activities per application or per user running in the system. Studying such large-scale traces can provide helpful insights into the system. It can be used to develop predictive methods for making predictive decisions, adjusting scheduling policies, or providing decisions for the design of next-generation systems. However, sharing real-world I/O traces to expedite such research efforts leaves a few concerns; i) the cost of sharing the large traces is expensive due to this large size, and ii) privacy concern is an issue.We address such issues by building an end-to-end machine learn- ing (ML) workflow that can generate I/O traces for large-scale HPC applications. We leverage ML based feature selection and gener- ative models for I/O trace generation. The generative models are trained on I/O traces collected by the darshan I/O characterization tool over a period of one year. We present a two-step generation process consisting of two deep-learning models, called the feature generator and the trace generator. The combination of two-step generative models provides robustness by reducing the bias of the model and accounting for the stochastic nature of the I/O traces across different runs of an application. We evaluate the performance of the generative models and show that the two-step model can generate time-series I/O traces with less than 20% root mean square error.

Paul, Arnab↗
DOE OSTI2020

S/U Comparison Study with a Focus on USLs

Under a DOE Nuclear Criticality Safety Program (NCSP) task involving Analytical Methods, three Laboratories collaborated in a comparison of results obtained from Sensitivity/Uncertainty (S/U) packages relevant to validation of transport codes. The task involves Institut de Radioprotection et de Sûreté Nucléaire (IRSN), Los Alamos National Laboratory (LANL), and Oak Ridge National Laboratory (ORNL) comparing results of MORET 5/MACSENS V3.0, MCNP6.2/Whisper-1.1, and SCALE 6.2.3/TSUNAMI/USLSTATS respectively. All Monte Carlo transport code results utilize ENDF/B-VII.1. Four cases from the International Handbook of Evaluated Criticality Safety Benchmark Experiments (ICSBEP Handbook) were selected as application models: HEU-MET-FAST-013-001, HEU-SOL-THERM 001-008, PU-MET-FAST-022-001, and PU-SOL-THERM 001-001. Ultimately, comparison is made between Upper Subcritical Limits (USLs) obtained using each code package for each application case. Since differences exist in whether packages take into account margin of subcriticality (MOS), the USL may be computed using bias and bias uncertainty, also known as the calculational margin (CM) in ANSI/ANS 8.24. Application of portions of MOS to the USL for nuclear data uncertainty of and potential code margin is referred to as USL herein. In either case, additional MOS is considered for actual application cases.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Improving Photometric Redshift Estimates with Training Sample Augmentation

Abstract Large imaging surveys will rely on photometric redshifts (photo- z 's), which are typically estimated through machine-learning methods. Currently planned spectroscopic surveys will not be deep enough to produce a representative training sample for Legacy Survey of Space and Time (LSST), so we seek methods to improve the photo- z estimates that arise from nonrepresentative training samples. Spectroscopic training samples for photo- z 's are biased toward redder, brighter galaxies, which also tend to be at lower redshift than the typical galaxy observed by LSST, leading to poor photo- z estimates with outlier fractions nearly 4 times larger than for a representative training sample. In this Letter, we apply the concept of training sample augmentation, where we augment simulated nonrepresentative training samples with simulated galaxies possessing otherwise unrepresented features. When we select simulated galaxies with ( g - z ) color, i -band magnitude, and redshift outside the range of the original training sample, we are able to reduce the outlier fraction of the photo- z estimates for simulated LSST data by nearly 50% and the normalized median absolute deviation (NMAD) by 56%. When compared to a fully representative training sample, augmentation can recover nearly 70% of the degradation in the outlier fraction and 80% of the degradation in NMAD. Training sample augmentation is a simple and effective way to improve training samples for photo- z 's without requiring additional spectroscopic samples.

Moskowitz, Irene (ORCID:0000000222068589)↗
DOE OSTI2022

Assessment of dislocation reduction on 100 mm diameter bulk GaN grown by the NEAT method

X-ray topography measurements on a 100 mm diameter GaN boule grown by the Near Equilibrium AmmonoThermal method revealed an improvement in dislocation density from >1 × 10 6 cm -2 to between 2 × 10 5 and 5 × 10 5 cm -2, an improvement greater than two to five times from seed to growth. This data builds on previous X-ray diffraction and defect selective etching to quantify the reduction in defect density that is closely associated with increasing growth thickness. This result indicates that there is a pathway to further dislocation reduction by increasing growth thickness for GaN crystals including those of 100 mm or larger diameter. Further reduction of the dislocation density of large-area substrates will lead to GaN power devices with reduced leakage current under reverse bias and better device performance.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Monthly averages of ED2 model simulations initialized with airborne lidar structure, Jan 1981-Dec 2018, Brazilian Amazon

Deforestation and forest degradation (selective logging, fires, fragmentation) have impacted nearly 40% of the original extent of the Brazilian Amazon, and have markedly impacted forest structure across the region. To date, few studies analysed how shifts in forest structure from degradation influence the forest sensitivity to climate extremes, because of the complex interactions between forest structure and micro-environmental conditions. To address this knowledge gap, we carried out a series of simulations across the Brazilian Amazon using the Ecosystem Demography Model (ED2), using observed forest structure derived from 541 airborne lidar transects (375 ha each) and two scenarios representing forest recovery and expansion of degradation to investigate how shifts in forest structure impact ecosystem function under near-average and extreme climate conditions, as part of the manuscript Longo et al 2025 "Degradation and Deforestation Increase the Sensitivity of the Amazon Forest to Climate Extremes". This dataset provides the output results from the ED2 model simulations for the three simulations at monthly time scales, in NetCDF format. For all simulations, we used bias-corrected hourly reanalyses (WFDE5) for most meteorological drivers, except for precipitation, which was obtained from CHIRPS. The meteorological drivers used in the study span 38 years (Jan 1981–Dec 2018). The output results correspond to the last 38 years of simulation (one full cycle of meteorological drivers), in which ED2 simulations used static stand structure (i.e., the forest structure was held constant). The following files are provided:ED2_emean_Global_R004_BrAmaz_s1c0t0l0f0.nc. This corresponds to the Control simulation. The forest structure was obtained from the airborne lidar.ED2_emean_Global_R005_BrAmaz_s1c0t1l1f0.nc. This corresponds to the Degraded simulation. The forest structure was obtained from a spin-up simulation initialized with airborne lidar and a scenario that expanded deforestation and selective logging across the Amazon.ED2_emean_Global_R006_BrAmaz_s1c0t1l0f0.nc. This corresponds to the Recovery simulation. The forest structure was obtained from a spin-up simulation initialized with airborne lidar and a scenario that completely halted deforestation and degradation, allowing degraded forests to recover for 38 years.We also provide file ED2_zones_R004_BrAmaz_s1c0t0l0f0.nc, which classifies each grid cell into zones used in the reference manuscript: 1: Southeast. 2: South. 3: West. 4: Central. 5: Northeast. 6: North. 7: Northwest". Index 0 corresponds to grid cells excluded from sub-region analyses because they were dominated by flooded forests, deforestation, and naturally non-forest vegetation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Material-dependent photon ionizing radiation effects in Si and GaAs PIN diodes: A numerical investigation

We present a finite-element drift-diffusion-Poisson model in the Multiphysics Object-Oriented Simulation Environment (MOOSE) framework to compare the radiation response of silicon (Si) and gallium arsenide (GaAs) PIN diodes under high-energy photon irradiation. The model solves coupled carrier continuity and Poisson’s equations with Shockley-Read-Hall recombination, and is verified against standard analytical J-V behavior. Using a simplified 1D geometry with ideal Ohmic contacts, we quantify device response under forward and reverse bias with a 100 MeV photon flux. Under forward bias, Si exhibits markedly greater radiation sensitivity than GaAs, including larger increases in current density, stronger local field and carrier-product perturbations, and higher recombination. Under reverse bias, GaAs shows larger radiation-induced photocurrent and broader current-density peaks near junctions, indicating an advantage for photodetection. Integrated steady-state recombination is consistently higher in Si across voltages. Under periodic photon pulses, GaAs produces higher-amplitude photoresponse and settles more rapidly than Si. These results highlight material-dependent trade-offs for radiation-tolerant, high-speed optoelectronics and provide guidance for selecting PIN architectures in aerospace, nuclear, and high-energy physics environments.

36 MATERIALS SCIENCE↗
DOE OSTI2022

The effect of spiral arms on the Sérsic photometry of galaxies

The Sérsic profile is a widely used model for describing the surface brightness distribution of galaxies. Spiral galaxies, however, are qualitatively different from a Sérsic model. Aims. The goal of this study is to assess how accurately the total flux and half-light radius of a galaxy with spiral arms can be recovered when fitted with a Sérsic profile. Methods. I selected a sample of bulge-dominated galaxies with spiral arms. Using photometric data from the Hyper Suprime-Cam survey, I estimated the contribution of the spiral arms to their total flux. Then I generated simulated images of galaxies with similar characteristics, fitted them with a Sérsic model, and quantified the error on the determination of the total flux and half-light radius. Results. Spiral arms can introduce biases on the photometry of galaxies in a way that depends on the underlying smooth surface brightness profile, the location of the arms, and the depth of the photometric data. A set of spiral arms accounting for 10% of the flux of a bulge-dominated galaxy typically causes the total flux and the half-light radius to be overestimated by 15% and 30%, respectively. This bias, however, is much smaller if the galaxy is disk-dominated. Conclusions. Galaxies with a prominent bulge and a non-zero contribution from spiral arms are the most susceptible to biases in the total flux and half-light radius when fitted with a Sérsic profile. If photometric measurements with high accuracy are required, then measurements over finite apertures are to be preferred over global estimates of the flux.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Dark Energy Survey Year 3 results: magnification modelling and impact on cosmological constraints from galaxy clustering and galaxy–galaxy lensing

We study the effect of magnification in the Dark Energy Survey Year 3 analysis of galaxy clustering and galaxy–galaxy lensing, using two different lens samples: a sample of luminous red galaxies, redMaGiC, and a sample with a redshift-dependent magnitude limit, MagLim. We account for the effect of magnification on both the flux and size selection of galaxies, accounting for systematic effects using the Balrog image simulations. We estimate the impact of magnification on the galaxy clustering and galaxy–galaxy lensing cosmology analysis, finding it to be a significant systematic for the MagLim sample. We show cosmological constraints from the galaxy clustering autocorrelation and galaxy–galaxy lensing signal with different magnifications priors, finding broad consistency in cosmological parameters in ΛCDM and wCDM. However, when magnification bias amplitude is allowed to be free, we find the two-point correlation functions prefer a different amplitude to the fiducial input derived from the image simulations. We validate the magnification analysis by comparing the cross-clustering between lens bins with the prediction from the baseline analysis, which uses only the autocorrelation of the lens bins, indicating that systematics other than magnification may be the cause of the discrepancy. We show that adding the cross-clustering between lens redshift bins to the fit significantly improves the constraints on lens magnification parameters and allows uninformative priors to be used on magnification coefficients, without any loss of constraining power or prior volume concerns.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Collective Risk Ranking of Highway Segments on the Basis of Severity-Weighted Crash Rates

This study is intended to focus on the major factors affecting traffic crash rates and severity levels, in addition to identifying crash-prone locations (i.e., black spots) based on the two indicators. The available crash data for different road segments used for the analysis were obtained from the Washington state database provided by the Highway Safety Information System (HSIS) for the years 2006 to 2011. A Random Forest (RF) classifier was used to predict the outcome level of crash severity, while crash rates were predicted by applying RF regressor. Certain features were selected for each model besides the abstraction of new features to check if there are unobserved correlations affecting the independent variables, such as accounting for the number and weight of crashes within 1 km2 area by implementing the Getis-Ord Gi∗ index. Moreover, to calculate the collective risk (CR) score, crash rates were adjusted to incorporate crash severity weights (cost per severity type) and regression-to-the-mean (RTM) bias via Empirical Bayes (EB) method. Finally, segments were ranked according to their CR score.

Li, Dawei↗
DOE OSTI2023

The relationship between the incidence of X-ray selected AGN in nearby galaxies & star-formation rate

ABSTRACT We present the identification and analysis of an X-ray selected AGN sample that lie within the local (z < 0.35) galaxy population. From a parent sample of 22 079 MPA-JHU (based on SDSS DR8) galaxies, we identified 917 galaxies with central, excess X-ray emission (from 3XMM-DR7) likely originating from an AGN. We measured the host galaxies’ star formation rates and classified them as either star-forming or quiescent based on their position relative to main sequence of star formation. Only 72 per cent of the X-ray selected sample were identified as AGN using BPT selection; this technique is much less effective in quiescent hosts, only identifying 50 per cent of the X-ray AGN. We also calculated the growth rates of the black holes powering these AGN in terms of their specific accretion rate (∝ LX/M*) and found quiescent galaxies, on average, accrete at a lower rate than star-forming galaxies. Finally, we measured the sensitivity function of 3XMM so we could correct for observational bias and construct probability distributions as a function of accretion rate. AGN were found in galaxies across the full range of star formation rates ($\log _{10} \, \mathrm{SFR/M_\odot \ yr^{-1}} = -3\ \mathrm{to}\ 2$) in both star-forming and quiescent galaxies. The incidence of AGN was enhanced by a factor 2 (at a 3.5σ significance) in star-forming galaxies compared to quiescent galaxies of equivalent stellar mass and redshift, but we also found a significant population of AGN hosted by quiescent galaxies.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Accounting for Training Data Error in Machine Learning Applied to Earth Observations

Remote sensing, or Earth Observation (EO), is increasingly used to understand Earth system dynamics and create continuous and categorical maps of biophysical properties and land cover, especially based on recent advances in machine learning (ML). ML models typically require large, spatially explicit training datasets to make accurate predictions. Training data (TD) are typically generated by digitizing polygons on high spatial-resolution imagery, by collecting in situ data, or by using pre-existing datasets. TD are often assumed to accurately represent the truth, but in practice almost always have error, stemming from (1) sample design, and (2) sample collection errors. The latter is particularly relevant for image-interpreted TD, an increasingly commonly used method due to its practicality and the increasing training sample size requirements of modern ML algorithms. TD errors can cause substantial errors in the maps created using ML algorithms, which may impact map use and interpretation. Despite these potential errors and their real-world consequences for map-based decisions, TD error is often not accounted for or reported in EO research. Here we review the current practices for collecting and handling TD. We identify the sources of TD error, and illustrate their impacts using several case studies representing different EO applications (infrastructure mapping, global surface flux estimates, and agricultural monitoring), and provide guidelines for minimizing and accounting for TD errors. To harmonize terminology, we distinguish TD from three other classes of data that should be used to create and assess ML models: training reference data, used to assess the quality of TD during data generation; validation data, used to iteratively improve models; and map reference data, used only for final accuracy assessment. We focus primarily on TD, but our advice is generally applicable to all four classes, and we ground our review in established best practices for map accuracy assessment literature. EO researchers should start by determining the tolerable levels of map error and appropriate error metrics. Next, TD error should be minimized during sample design by choosing a representative spatio-temporal collection strategy, by using spatially and temporally relevant imagery and ancillary data sources during TD creation, and by selecting a set of legend definitions supported by the data. Furthermore, TD error can be minimized during the collection of individual samples by using consensus-based collection strategies, by directly comparing interpreted training observations against expert-generated training reference data to derive TD error metrics, and by providing image interpreters with thorough application-specific training. We strongly advise that TD error is incorporated in model outputs, either directly in bias and variance estimates or, at a minimum, by documenting the sources and implications of error. TD should be fully documented and made available via an open TD repository, allowing others to replicate and assess its use. To guide researchers in this process, we propose three tiers of TD error accounting standards. Finally, we advise researchers to clearly communicate the magnitude and impacts of TD error on map outputs, with specific consideration given to the likely map audience.

58 GEOSCIENCES↗