SEARCH · Search NASA

Results for “Building benchmarking models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Better bang for your buck? Comparing savings realization from ESPCs and direct-funded projects

Energy savings performance contracts (ESPCs) offer an opportunity to tremendously scale decarbonization projects, given their paid-from-savings premise. However, prospective customers still question whether ESPC is worth the effort and expense. This study evaluates the savings realization rates of ESPCs compared to direct-funded projects using ENERGY STAR Portfolio Manager (ESPM) benchmarking data. It compares normalized EUIs from ESPM before and after energy conservation projects implemented via ESPC or direct funding in roughly 450 federal buildings. This documented change, ideally a savings, can then be compared to the estimated savings from the energy conservation initiative in order to generate a rough realization rate. Preliminary results indicate a notably greater savings realization rate for the ESPC buildings (median = 105% of estimated savings) than those that underwent direct-funded projects (median = 46%). Because of a shortage of good quality data and the wide range in results, the difference is only significant at the p < 0.20 level. However, the higher savings realization of ESPCs corroborates the authors’ 2014 findings using a different comparison method. With continued soft government funding and regulatory impediments for climate change mitigation in the U.S., financed, paid-from-savings project models (e.g., ESPC, PACE, EaaS and others) certainly merit more attention. This study’s results are particularly compelling (and encouraging) given the country’s likely reliance on these vehicles to address existing building retrofits.

Earni, Shankar↗

DOE OSTI2020

OpenACC Profiling Support for Clang and LLVM using Clacc and TAU

Since its launch in 2010, OpenACC has evolved into one of the most widely used portable programming models for accelerators on HPC systems today. Clacc is a project funded by the US Exascale Computing Project (ECP) to bring OpenACC support for C and C++ to the popular Clang and LLVM compiler infrastructure. In this paper, we describe Clacc's support for the OpenACC Profiling Interface, a critical component of the OpenACC specification that standardizes an interface that profiling tools and libraries can depend upon across OpenACC implementations. As part of Clacc's general strategy to build OpenACC support upon OpenMP, we describe how Clacc builds OpenACC Profiling Interface support upon an extended version of OMPT. We then describe how a major profiling and tracing toolkit within ECP, the TAU Performance System, takes advantage of this support. We also describe TAU's selective instrumentation support for OpenACC. Finally, using Clacc and TAU, we present example visualizations for several SPEC ACCEL OpenACC benchmarks running on an IBM AC922 node, and we show that the associated performance overhead is negligible.

Coti, Camille↗

DOE OSTI2022

Nonlinear dimension reduction for surrogate modeling using gradient information

We introduce a method for the nonlinear dimension reduction of a high-dimensional function $u:{\mathbb{R}}^d\rightarrow{\mathbb{R}}$, $d\gg 1$. Our objective is to identify a nonlinear feature map $g:{\mathbb{R}}^d\rightarrow{\mathbb{R}}^m$, with a prescribed intermediate dimension $m\ll d$, so that $u$ can be well approximated by $f\circ g$ for some profile function $f:{\mathbb{R}}^m\rightarrow{\mathbb{R}}$. We propose to build the feature map by aligning the Jacobian $\nabla g$ with the gradient $\nabla u$, and we theoretically analyze the properties of the resulting $g$. Once $g$ is built, we construct $f$ by solving a gradient-enhanced least squares problem. Our practical algorithm uses a sample $\{{\textbf{x}}^{(i)},u({\textbf{x}}^{(i)}),\nabla u({\textbf{x}}^{(i)})\}_{i=1}^N$ and builds both $g$ and $f$ on adaptive downward-closed polynomial spaces, using cross validation to avoid overfitting. We numerically evaluate the performance of our algorithm across different benchmarks, and explore the impact of the intermediate dimension $m$. We show that building a nonlinear feature map $g$ can permit more accurate approximation of $u$ than a linear $g$, for the same input data set.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1987

A NASA initiative: Software engineering for reliable complex systems

The objective is the development of methods, technology, and skills that will enable NASA to cost-effectively specify, build, and manage reliable software which can evolve and be maintained over an extended period. The need for such software is rooted in the increasing integration of software and computing components into NASA systems. Current NASA Software Engineering expertise was applied toward some of the largest reliable systems including: shuttle launch; ground support; shuttle simulation; minor control; satellite tracking; and scientific data systems. Unfortunately, no theory exists for reliable complex software systems. NASA is seeking to fill this theoretical gap through a number of approaches. One such approach is to conduct research on theoretical foundations for managing complex software systems. It includes: communication models, new and modified paradigms, and life-cycle models. Another approach is research in the theoretical foundations for reliable software development and validation. It focuses upon formal specifications, programming languages, software engineering systems, software reuse, formal verification, and software safety. Further approaches involve benchmarking a NASA software environment, experimentation within the NASA context, evolution of present NASA methodology, and transfer of technology to the space station software support environment.

Holcomb, Lee B.↗

DOE OSTI2022

Out of Distribution Detection with Neural Network Anchoring

This is code to reproduce and build on OOD detection from the paper "Out of Distribution Detection with Neural Network Anchoring". Our goal here is to exploit heteroscedastic temperature scaling as a calibration strategy for out of distribution (OOD) detection. Heteroscedasticity here refers to the fact that the optimal temperature parameter for each sample can be different, as opposed to conventional approaches that use the same value for the entire distribution. To enable this, we propose a new training strategy called anchoring that can estimate appropriate temperature values for each sample, leading to state-of-the-art OOD detection performance across several benchmarks. Using NTK theory, we show that this temperature function estimate is closely linked to the epistemic uncertainty of the classifier, which explains its behavior. In contrast to some of the best-performing OOD detection approaches, our method does not require exposure to additional outlier datasets, custom calibration objectives, or model ensembling. Through empirical studies with different OOD detection settings - far OOD, near OOD, and semantically coherent OOD - we establish a highly effective OOD detection approach.

Thiagarajan, Jayaraman↗

DOE OSTI2023

Lightweight and effective tensor sensitivity for atomistic neural networks

Atomistic machine learning focuses on the creation of models that obey fundamental symmetries of atomistic configurations, such as permutation, translation, and rotation invariances. In many of these schemes, translation and rotation invariance are achieved by building on scalar invariants, e.g., distances between atom pairs. There is growing interest in molecular representations that work internally with higher rank rotational tensors, e.g., vector displacements between atoms, and tensor products thereof. Here, we present a framework for extending the Hierarchically Interacting Particle Neural Network (HIP-NN) with Tensor Sensitivity information (HIP-NN-TS) from each local atomic environment. Crucially, the method employs a weight tying strategy that allows direct incorporation of many-body information while adding very few model parameters. We show that HIP-NN-TS is more accurate than HIP-NN, with negligible increase in parameter count, for several datasets and network sizes. As the dataset becomes more complex, tensor sensitivities provide greater improvements to model accuracy. In particular, HIP-NN-TS achieves a record mean absolute error of 0.927 $\frac{\text{kcal}}{\text{mol}}$ for conformational energy variation on the challenging COMP6 benchmark, which includes a broad set of organic molecules. We also compare the computational performance of HIP-NN-TS to HIP-NN and other models in the literature.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2025

Counterpart identification and classification for eRASS1 and characterisation of the active galactic nuclei content

Context. Accurately accounting for the Active Galactic Nucleus (AGN) phase in galaxy evolution requires a large, clean AGN sample. This is now possible with SRG/eROSITA, which completed its first all-sky X-ray survey (eRASS1) on June 12, 2020. The public Data Release 1 (DR1, Jan 31, 2024) includes 930,203 sources from the western Galactic hemisphere. Aims. The data enable the selection of a large AGN sample and the discovery of rare sources. However, scientific return depends on accurate characterisation of the X-ray emitters, requiring high-quality multi-wavelength data. This paper presents the identification and classification of optical and infrared counterparts to eRASS1 sources. Methods. Counterparts to eRASS1 X-ray point sources were identified using Gaia DR3, CatWISE2020, and Legacy Survey DR10 (LS10) with the Bayesian NWAY algorithm and trained priors. Sources were classified as Galactic or extragalactic via a machine-learning model combining optical/IR and X-ray properties, trained on a reference sample. For extragalactic LS10 sources, photometric redshifts were computed using CIRCLEZ. Results. Within the LS10 footprint, all 656,614 eROSITA/DR1 sources have at least one possible optical counterpart; ∼570 000 are extragalactic and likely AGN. Half are new detections compared to AllWISE, Gaia, and Quaia AGN catalogues. Gaia and CatWISE2020 counterparts are less reliable, due to the survey’s shallowness and the limited amount of features available to assess the probability of being an X-ray emitter. In the Galactic plane, where the overdensity of stellar sources also increases the chance of associations, using conservative reliability cuts, we identified approximately 18 000 Gaia and 55 000 CatWISE2020 extragalactic sources. Conclusions. We have released three high-quality counterpart catalogues – plus the training and validation sets – as a benchmark for the field. These datasets have many applications, but in particular, they empower researchers to build AGN samples tailored for completeness and purity, accelerating the hunt for the Universe’s most energetic engines.

X-rays: general↗

DOE OSTI2021

Freight Analysis Framework Version 5 (FAF5) Base Year 2017 Data Development Technical Report

The Freight Analysis Framework (FAF) integrates data from a variety of sources to create a comprehensive national picture of freight movements among states and major metropolitan areas by all modes of transportation. The latest of this data series is FAF5, which is the fifth generation FAF and is benchmarked on Commodity Flow Survey (CFS) 2017. Except for FAF1 that provided estimates for truck, rail, and water tonnage for calendar year 1998, later generations of FAF (FAF2 through FAF5) were built based on their benchmark year CFS data, for 2002, 2007, 2012, and 2017 respectively. The FAF is produced under a partnership between Bureau of Transportation Statistics (BTS) and Federal Highway Administration (FHWA). As a major data product of the FAF program, the FAF regional database provides a national picture of freight flows to, from, and within the United States (among regions and states), by commodity and mode for the base year, as well as for forecasts up to 30 years into the future in a 5-year interval. Additional FAF data products also include FAF network flows database, where truck movements are routed onto the national highway network, estimates of annual projections, and synchronized historical data series. This report is a technical document prepared to describe the data sources and methodologies applied in the process of building the FAF5 base-year 2017 regional database, released as FAF5.0 in February 2021. This report offers a description of the diverse data sources and modeling methods used in constructing the base year FAF5 regional database. The FAF5 base-year database is used as the base for development of forecasts and for assignment of truck flows on highway network. Similarly, the FAF5 base-year database will be used as the base to generate FAF5 annual estimates. In addition to this report, users are encouraged to refer to the FAF5 User’s Guide, which provides basic information of the data, including definitions of the data attributes, information on how to access the data and tool, as well as detailed data dictionary and code tables.

42 ENGINEERING↗

DOE OSTI2022

Testing and Expertise for Marine Energy (TEAMER) Program Support - Numerical Modeling Assistance for iProTech's "PIP" WEC Device: Cooperative Research and Development Final Report, CRADA Number CRD-20-17303

Develop a time-domain, 3 degree of freedom numerical model of IProTech's PIP device, including power take-off (PTO), that can be used to develop control algorithms and establish a baseline technology performance level (TPL). This is important to IProTech because having a good numerical model allows IProTech to optimize design and control algorithms on an ongoing basis. A baseline TPL enables the PIP technology to be benchmarked against other wave energy converter (WEC) concepts, providing (i) justification (or not) for investment in wave tank model testing, (ii) identifying optimum design parameters and control algorithms for the test model, (iii) specifying test conditions and (iv) interpreting wave tank test results. An accredited WEC-Sim model of the PIP device is an essential tool for further development of the technology. 1) The development of numerical models of the PIP concept will enable iProTech to evaluate the potential performance of this concept, and to investigate how sensitive the device is to different variables within the system (e.g. hydraulic components, geometry dimensions, etc.) 2) The numerical models were developed with WEC-Sim and PTO-Sim -- the de-facto industry standard WEC numerical modeling tools. A detailed model of the system's hydraulic circuit was developed with PTO-Sim. However, WEC-Sim is based on linear hydrodynamics and has several limitations; validation against high fidelity models/physical scale models would help to build confidence in the results.

16 TIDAL AND WAVE POWER↗

NASA NTRSDate not supplied

Language Model For Earth Science: Exploring Potential Downstream Applications As Well As Current Challenges

The use of deep learning techniques to build transformer language models such as SciBERT and GPT3 have transformed the natural language technology (NLT) landscape. These new NLTs are being used in speech to text and vice versa, auto-mated text classification, sentiment analysis, topic modeling, text summarization, and cognitive assistants. While Earth science has no shortage of unstructured data such as journal and conference papers, little efforts have focused on harnessing NLTs for knowledge extraction and supporting the scientific process. This paper surveys the use of language models in different science. BERT-E, a new Earth science-specific language model, is presented. BERT-E is generated using a transfer learning solution. A language model that has already been trained for general Science (SciBERT) is fine-tuned using abstracts and full text extracted from various Earth science-related articles. A downstream keywords classification application is used for evaluation, and the use of BERT-E shows improved performance. The need to develop a robust set of benchmarks in evaluating the language model such as BERT-E is discussed. Finally, example applications are presented to inspire additional ideas for applications using domain-specific language models.

R Ramachandran↗

NASA NTRSDate not supplied

An Expanded Role for Python in Expediting System Simulation Development

The Python programming language has traditionally been used as a “scripting” language and is not generally recognized as a language for building system simulations, where C++, Java, and other compiled languages are typically used. While very powerful and gaining favor for a large set of programming tasks, Python is generally regarded as not having the speed to directly do the extensive numeric computations required in large-scale simulation. This paper aims to present an approach, supported by case studies, to re-think this assertion. Directly using Python in this new role has the potential to expedite an agile, spiral cycle of development of design-test-modify for systems simulation.

programming language↗

DOE OSTI2024

Towards interpretable Cryo-EM: disentangling latent spaces of molecular conformations

Molecules are essential building blocks of life and their different conformations (i.e., shapes) crucially determine the functional role that they play in living organisms. Cryogenic Electron Microscopy (cryo-EM) allows for acquisition of large image datasets of individual molecules. Recent advances in computational cryo-EM have made it possible to learn latent variable models of conformation landscapes. However, interpreting these latent spaces remains a challenge as their individual dimensions are often arbitrary. The key message of our work is that this interpretation challenge can be viewed as an Independent Component Analysis (ICA) problem where we seek models that have the property of identifiability. That means, they have an essentially unique solution, representing a conformational latent space that separates the different degrees of freedom a molecule is equipped with in nature. Thus, we aim to advance the computational field of cryo-EM beyond visualizations as we connect it with the theoretical framework of (nonlinear) ICA and discuss the need for identifiable models, improved metrics, and benchmarks. Moving forward, we propose future directions for enhancing the disentanglement of latent spaces in cryo-EM, refining evaluation metrics and exploring techniques that leverage physics-based decoders of biomolecular systems. Moreover, we discuss how future technological developments in time-resolved single particle imaging may enable the application of nonlinear ICA models that can discover the true conformation changes of molecules in nature. The pursuit of interpretable conformational latent spaces will empower researchers to unravel complex biological processes and facilitate targeted interventions. This has significant implications for drug discovery and structural biology more broadly. More generally, latent variable models are deployed widely across many scientific disciplines. Thus, the argument we present in this work has much broader applications in AI for science if we want to move from impressive nonlinear neural network models to mathematically grounded methods that can help us learn something new about nature.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2025

Using the ATLAS experiment software on heterogeneous resources

With the large dataset expected from 2030 onwards by the HL-LHC at CERN, the ATLAS experiment is reaching the limits of the current data processing model in terms of traditional CPU resources based on x86_64 architectures and an extensive program for software upgrades towards the HL-LHC has been set up. The ARM CPU architecture is becoming a competitive and energy efficient alternative. Accelerators like GPUs are available in any recent HPC. In the past years ATLAS has successfully ported its full data processing and simulation software framework Athena to ARM and has invested significant effort in porting parts of the reconstruction and simulation algorithms to GPUs. We report on the successful usage of the ATLAS experiment offline and online software framework Athena on ARM and GPUs through the PanDA workflow management system at various WLCG sites. Furthermore we report on performance optimizations of the builds for ARM CPUs and the GPU integration efforts. We will discuss performance comparisons of different ARM and x86_64 architectures on WLCG resources and Cloud compute providers like GCP and AWS using ATLAS productions workflows as used in the Hep-Score23 benchmark suite.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

NASA NTRS2010

Labyrinth Seal Flutter Analysis and Test Validation in Support of Robust Rocket Engine Design

High energy-density turbomachines, like the SSME turbopumps, utilize labyrinth seals, also referred to as knife-edge seals, to control leakage flow. The pressure drop for such seals is order of magnitude higher than comparable jet engine seals. This is aggravated by the requirement of tight clearances resulting in possible unfavorable fluid-structure interaction of the seal system (seal flutter). To demonstrate these characteristics, a benchmark case of a High Pressure Oxygen Turbopump (HPOTP) outlet Labyrinth seal was studied in detail. First, an analytical assessment of the seal stability was conducted using a Pratt & Whitney legacy seal flutter code. Sensitivity parameters including pressure drop, rotor-to-stator running clearances and cavity volumes were examined and modeling strategies established. Second, a concurrent experimental investigation was undertaken to validate the stability of the seal at the equivalent operating conditions of the pump. Actual pump hardware was used to construct the test rig, also referred to as the (Flutter Rig). The flutter rig did not include rotational effects or temperature. However, the use of Hydrogen gas at high inlet pressure provided good representation of the critical parameters affecting flutter especially the speed of sound. The flutter code predictions showed consistent trends in good agreement with the experimental data. The rig test program produced a stability threshold empirical parameter that separated operation with and without flutter. This empirical parameter was used to establish the seal build clearances to avoid flutter while providing the required cooling flow metering. The calibrated flutter code along with the empirical flutter parameter was used to redesign the baseline seal resulting in a flutter-free robust configuration. Provisions for incorporation of mechanical damping devices were introduced in the redesigned seal to ensure added robustness

El-Aini, Yehia↗

DOE OSTI2022

Model Assumptions and Data Characteristics: Impacts on Domain Adaptation in Building Segmentation

Studies on domain adaptation (DA) for remote sensing (RS) imagery analysis lack consistency in selection and description of evaluation scenarios. Without properly characterizing datasets, model assumptions, and evaluation scenarios, it is difficult to objectively compare DA methods and reach conclusions about their suitability across different applications. With this motivation, this work seeks to empirically assess to which extent the interaction between data characteristics and model assumptions influences the effectiveness of DA methods. Using the widely explored task of building footprint segmentation as a case study, we perform a large-scale study across over 200 DA scenarios that include variations across view angles, areas observed, and sensors used for data acquisition. Rather than adopting different model architectures or optimization criteria, we contrast the performances of two DA methods based on adversarial learning that differ only in their assumptions about source and target domains. Informed by metadata and data characteristics unveiled using traditional computer vision (CV) techniques as well as pretrained deep models, we provide a detailed meta-analysis of experiments highlighting the importance of accurately considering data assumptions for DA in RS segmentation tasks. As demonstrated by a “cherry-picking” exercise, different claims regarding which model is best could be made by selecting different subsets of evaluation scenarios. While well-calibrated assumptions can be beneficial, mismatching assumptions can lead to negative biases in DA applications. Furthermore, this study intends to motivate the community toward more consistent evaluation protocols while providing recommendations and insights toward creating novel benchmark datasets, documenting data characteristics, application-specific knowledge, and model assumptions.

42 ENGINEERING↗

DOE OSTI2023

Statistical Learning for Nonlinear Model Reduction from Local Simulations of Stochastic and Particle- and Agent-Based Systems

Stochastic physical systems across the sciences that have very high-dimensional state spaces, with a large number of fast degrees of freedom that force direct simulators to proceed by integration steps that are orders of magnitude smaller than events of interests (e.g., particle collisions). Examples range from molecular motion to dynamics of large populations of cells. A grand challenge in the simulation and understanding of such systems is the systematic construction of accurate, interpretable, reduced models, enabling faster simulations, revealing fundamental properties of the dynamics, and predicting phenomena of interest that the original simulator could not reached with sufficient accuracy or within a given computational budget. In this projected we developed novel statistical estimation/machine learning techniques for analyzing and building empirical reduced models for important families of high-dimensional stochastic systems, in particular: - we developed techniques for estimating interaction kernels in interacting particle- and agent-based systems, which are ubiquitous in Physics, Biology and many other sciences, given observed trajectories of the system; - we developed techniques for nonlinear model reduction for high-dimensional stochastic systems that have a small number of unknown, nonlinear slow variables, and a large number of fast modes, that are possibly of large magnitude, given observed short trajectories of the system in the form of bursts of trajectories from different initial conditions; - we developed novel techniques for estimating linear dynamical systems on graphs when both the dynamics and the underlying graph are unknown, and we have a sparse set of space-time observations; - we considered the problem of estimating an unknown nonlinear observation function of a standard process (e.g. Brownian motion), so that we can recognized if an observed dynamics is "just" a nonlinear version of a known dynamics; we also developed benchmarks for learning algorithms aimed at learning and classifying diffusion processes.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks

Electroconvection is a multiphysics problem involving coupling of the flow field with the electric field as well as the cation and anion concentration fields. Here, we use electroconvection as a benchmark problem to put forward a new data assimilation framework, the DeepM&Mnet, for simulating multiphysics and multiscale problems at speeds much faster than standard numerical methods using pre-trained neural networks. We first pre-train DeepONets that can predict independently each field, given general inputs from the rest of the fields of the coupled system. DeepONets can approximate nonlinear operators and are composed of two sub-networks, a branch net for the input fields and a trunk net for the locations of the output field. DeepONets, which are extremely fast, are used as building blocks in the DeepM&Mnet and form constraints for the multiphysics solution along with some sparse available measurements of any of the fields. We demonstrate the new methodology and document the accuracy of each individual DeepONet, and subsequently we present two different DeepM&Mnet architectures that infer accurately and efficiently 2D electroconvection fields for unseen electric potentials. Furthermore, the DeepM&Mnet framework is general and can be applied for building any complex multiphysics and multiscale models based on very few measurements using pre-trained DeepONets in a “plug-and-play” mode.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

DECOVALEX-2023: Task F1 Final Report

DECOVALEX-2023 Task F is a comparison of models and methods for post-closure performance assessment (PA) of a deep geologic repository for radioactive waste. The general aims of Task F are to build confidence in the models, methods, and software used for PA and to stimulate additional research and development in PA methodologies. The task objectives are to motivate development of PA modelling skills and capabilities, to examine the influence of model choices on calculated repository performance, and to compare the uncertainties introduced by model choices to other sources of uncertainty. Task F involves no actual experiment or site. It is a PA modelling exercise that requires the conceptual development of hypothetical repository designs and geologic settings. Because three of the teams were interested in salt and the rest of the teams were interested in crystalline rock, Task F was split into two branches: Task F1 for crystalline rock and Task F2 for salt. This report is for Task F1, crystalline rock. Teams from seven countries (Canada, Czech Republic, Germany, Korea, Sweden, Taiwan, and United States) participated in Task F1. The teams worked together to define the features, events, and processes of the reference case repository and established a set of performance measures. In addition, they defined a set of benchmark problems designed to test and compare modelling capabilities for fracture flow and transport at different scales. The repository design and benchmark problems are documented in a Task Specification that evolved over time as the group honed the specifications. The benchmark problems verified that each team can aptly model flow and transport in fractured media in 1-, 2-, and 3-dimensions. Two general approaches were used for the 3-dimensional benchmarks: discrete fracture network (DFN) and equivalent continuous porous medium (ECPM). DFN modelling involves explicit meshing of each fracture while ECPM modelling aims to capture the effective porosity and directional permeability of each cell in a space-filling mesh as affected by intersecting fractures. In some models, a combination of the two is used, i.e., DFN for large known fractures and ECPM for the rest of the domain. Transport is solved by using either the advection-dispersion equation or particle tracking. Although some variation is observed among model breakthrough curves in the benchmark problems, there is strong agreement in breakthrough behaviour up to at least the 75 th percentile for all benchmarks. At the 90 th percentile, breakthrough results show larger differences, suggesting several models retain substantially higher fractions of tracer in regions of slower moving water. In addition to the flow and transport benchmarks, several teams completed the source term benchmark, verifying capabilities for modelling radionuclide decay and ingrowth, waste package breach, instant release fractions, fuel matrix degradation rates, and radionuclide solubility limitations. The reference case is conceptualized as a generic spent fuel repository at a depth of 450 m in fractured crystalline rock. The repository has 50 parallel backfilled drifts, each with 50 deposition holes 6 m apart. Each deposition hole contains a 4-PWR waste package and bentonite buffer. The rock domain is 5 km in length, 2 km in width, and 1 km in depth. It has 6 deterministic fractured deformation zones and a multitude of stochastic fractures. Teams generally used the ECPM approach for the entire rock or a hybrid approach in which the deterministic fracture zones are modelled with a DFN and the rest of the rock is modelled by ECPM. Of the reference case problems specified, only the results of the initial reference case problem are compared in this report. The initial problem focuses on transport from the deposition holes to the surface, i.e., it neglects waste package performance. Tracers are released at all waste package locations at time zero and tracked for their releases to the near field and ground surface. The water fluxes calculated at the ground surface entry and exit regions of the domain are similar for all models except for two that have considerably lower fluxes. For tracer transport, large differences are observed among models in the magnitude of tracer transported. Much of the difference appears to be due to how the repository is implemented and hence the different degrees of repository simplification. Models that exclude the drifts, buffer, and backfill from the domain tend to show greater release of tracers and radionuclides from the repository. The initial study presented here indicates that major differences in modelling important processes within the repository (e.g., diffusion through buffer and backfill) can produce broadly different release and transport results, especially when those processes are excluded. Even for the models that included all specified features, events, and processes, the results show significant differences and demonstrate the importance of examining multiple modelling approaches in performance assessment. The differences in results observed in this study are expected to motivate teams to either increase complexity in future versions of the reference case models or to improve methods to account for the effects of simplified features and processes. Either way, future improvements in these models are expected to produce results that more closely agree.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗