Search NASA⌕ Search

SEARCH · Search NASA

Results for “Parallel file systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

PFLOTRAN User Manual: A Massively Parallel Reactive Flow and Transport Model for Describing Surface and Subsurface Processes

PFLOTRAN solves a system of generally nonlinear partial differential equations describing multi-phase, multicomponent and multiscale reactive flow and transport in porous materials. The code is designed to run on massively parallel computing architectures as well as workstations and laptops (e.g. Hammond et al., 2011). Parallelization is achieved through domain decomposition using the PETSc (Portable Extensible Toolkit for Scientific Computation) libraries for the parallelization framework (Balay et al., 1997). PFLOTRAN has been developed from the ground up for parallel scalability and has been run on up to 218 processor cores with problem sizes up to 2 billion degrees of freedom. Written in object oriented Fortran 90, the code requires the latest compilers compatible with Fortran 2003. At the time of this writing this requires gcc 4.7.x, Intel 12.1.x and PGC compilers. As a requirement of running problems with a large number of degrees of freedom, PFLOTRAN allows reading input data that is too large to fit into memory allotted to a single processor core. The current limitation to the problem size PFLOTRAN can handle is the limitation of the HDF5 file format used for parallel IO to 32 bit integers. Noting that 2 32 = 4; 294; 967; 296, this gives an estimate of the maximum problem size that can be currently run with PFLOTRAN. Hopefully this limitation will be remedied in the near future.

58 GEOSCIENCES↗

Quantum Monte Carlo Endstation for Petascale Computing

NCSU research group has been focused on accomplising the key goals of this initiative: establishing new generation of quantum Monte Carlo (QMC) computational tools as a part of Endstation petaflop initiative for use at the DOE ORNL computational facilities and for use by computational electronic structure community at large; carrying out high accuracy quantum Monte Carlo demonstration projects in application of these tools to the forefront electronic structure problems in molecular and solid systems; expanding the impact of QMC methods and approaches; explaining and enhancing the impact of these advanced computational approaches. In particular, we have developed quantum Monte Carlo code (QWalk, www.qwalk.org) which was significantly expanded and optimized using funds from this support and at present became an actively used tool in the petascale regime by ORNL researchers and beyond. These developments have been built upon efforts undertaken by the PI's group and collaborators over the period of the last decade. The code was optimized and tested extensively on a number of parallel architectures including petaflop ORNL Jaguar machine. We have developed and redesigned a number of code modules such as evaluation of wave functions and orbitals, calculations of pfaffians and introduction of backflow coordinates together with overall organization of the code and random walker distribution over multicore architectures. We have addressed several bottlenecks such as load balancing and verified efficiency and accuracy of the calculations with the other groups of the Endstation team. The QWalk package contains about 50,000 lines of high quality object-oriented C++ and includes also interfaces to data files from other conventional electronic structure codes such as Gamess, Gaussian, Crystal and others. This grant supported PI for one month during summers, a full-time postdoc and partially three graduate students over the period of the grant duration, it has resulted in 13 published papers, 15 invited talks and lectures nationally and internationally. My former graduate student and postdoc Dr. Michal Bajdich, who was supported byt this grant, is currently a postdoc with ORNL in the group of Dr. F. Reboredo and Dr. P. Kent and is using the developed tools in a number of DOE projects. The QWalk package has become a truly important research tool used by the electronic structure community and has attracted several new developers in other research groups. Our tools use several types of correlated wavefunction approaches, variational, diffusion and reptation methods, large-scale optimization methods for wavefunctions and enables to calculate energy differences such as cohesion, electronic gaps, but also densities and other properties, using multiple runs one can obtain equations of state for given structures and beyond. Our codes use efficient numerical and Monte Carlo strategies (high accuracy numerical orbitals, multi-reference wave functions, highly accurate correlation factors, pairing orbitals, force biased and correlated sampling Monte Carlo), are robustly parallelized and enable to run on tens of thousands cores very efficiently. Our demonstration applications were focused on the challenging research problems in several fields of materials science such as transition metal solids. We note that our study of FeO solid was the first QMC calculation of transition metal oxides at high pressures.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

LiverTox: Advanced QSAR and Toxicogeomic Software for Hepatotoxicity Prediction

YAHSGS LLC and Oak Ridge National Laboratory (ORNL) established a CRADA in an attempt to develop a predictive system using a pre-existing ORNL computational neural network and wavelets format. This was in the interest of addressing national needs for toxicity prediction system to help overcome the significant drain of resources (money and time) being directed toward developing chemical agents for commerce. The research project has been supported through an STTR mechanism and funded by the National Institute of Environmental Health Sciences beginning Phase I in 2004 (CRADA No. ORNL-04-0688) and extending Phase II through 2007 (ORNL NFE-06-00020). To attempt the research objectives and aims outlined under this CRADA, state-of-the-art computational neural network and wavelet methods were used in an effort to design a predictive toxicity system that used two independent areas on which to base the system’s predictions. These two areas were quantitative structure-activity relationships and gene-expression data obtained from microarrays. A third area, using the new Massively Parallel Signature Sequencing (MPSS) technology to assess gene expression, also was attempted but had to be dropped because the company holding the rights to this promising MPSS technology went out of business. A research-scale predictive toxicity database system called Multi-Intelligent System for Toxicogenomic Applications (MISTA) was developed and its feasibility for use as a predictor of toxicological activity was tested. The fundamental focus of the CRADA was an attempt and effort to operate the MISTA database using the ORNL neural network. This effort indicated the potential that such a fully developed system might be used to assist in predicting such biological endpoints as hepatotoxcity and neurotoxicity. The MISTA/LiverTox approach if eventually fully developed might also be useful for automatic processing of microarray data to predict modes of action. A technical paper describing the methods and technology used in the CRADA has been published. This paper was entitled “A Toxicity Evaluation and Predictive System Based on Neural Networks and Wavelets” and appeared in an American Chemical Society peer-reviewed publication this year (J. Chem. Inf. Model. 47: 676685, 2007). A patent application was filed but later abandoned.

59 BASIC BIOLOGICAL SCIENCES↗

DOE SciDAC’s Earth System Grid Center for Enabling Technologies Final Report for University of Southern California Information Sciences Institute

The mission of the Earth System Grid Federation (ESGF) is to provide the worldwide climate-research community with access to the data, information, model codes, analysis tools, and intercomparison capabilities required to make sense of enormous climate data sets. Its specific goals are to (1) provide an easy-to-use and secure web-based data access environment for data sets; (2) add value to individual data sets by presenting them in the context of other data sets and tools for comparative analysis; (3) address the specific requirements of participating organizations with respect to bandwidth, access restrictions, and replication; (4) ensure that the data are readily accessible through the analysis and visualization tools used by the climate research community; and (5) transfer infrastructure advances to other domain areas. For the ESGF, the U.S. Department of Energy’s (DOE’s) Earth System Grid Center for Enabling Technologies (ESG-CET) team has led international development and delivered a production environment for managing and accessing ultra-scale climate data. This production environment includes multiple national and international climate projects (such as the Community Earth System Model and the Coupled Model Intercomparison Project), ocean model data (such as the Parallel Ocean Program), observation data (Atmospheric Radiation Measurement Best Estimate, Carbon Dioxide Information and Analysis Center, Atmospheric Infrared Sounder, etc.), and analysis and visualization tools, all serving a diverse user community. These data holdings and services are distributed across multiple ESG-CET sites (such as ANL, LANL, LBNL/NERSC, LLNL/PCMDI, NCAR, and ORNL) and at unfunded partner sites, such as the Australian National University National Computational Infrastructure, the British Atmospheric Data Centre, the National Oceanic and Atmospheric Administration Geophysical Fluid Dynamics Laboratory, the Max Planck Institute for Meteorology, the German Climate Computing Centre, the National Aeronautics and Space Administration Jet Propulsion Laboratory, and the National Oceanic and Atmospheric Administration. The ESGF software is distinguished from other collaborative knowledge systems in the climate community by its widespread adoption, federation capabilities, and broad developer base. It is the leading source for present climate data holdings, including the most important and largest data sets in the globalclimate community, and—assuming its development continues—we expect it to be the leading source for future climate data holdings as well. Recently, ESG-CET extended its services beyond data-file access and delivery to include more detailed information products (scientific graphics, animations, etc.), secure binary data-access services (based upon the OPeNDAP Data Access Protocol), and server-side analysis. The latter capabilities allow users to request data subsets transformed through commonly used analysis and intercomparison procedures. As we transition from development activities to production and operations, the ESG-CET team is tasked with making data available to all users seeking to understand, process, extract value from, visualize, and/or communicate it to others—this is of course if funding continues at some level. This ongoing effort, though daunting in scope and complexity, would greatly magnify the value of numerical climate model outputs and climate observations for future national and international climate-assessment reports. The ESG-CET team also faces substantial technical challenges due to the rapidly increasing scale of climate simulation and observational data, which will grow, for example, from less than 50 terabytes for the last Intergovernmental Panel on Climate Change (IPCC) assessment to multiple Petabytes for the next IPCC assessment. In a world of exponential technological change and rapidly growing sophistication in climate data analysis, an infrastructure such as ESGF must constantly evolve if it is to remain relevant and useful.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

UPC++

UPC++ is a Partitioned Global Address Space (PGAS) extension for C++, with three main objectives: 1) to provide an object-oriented PGAS programming model in the context of the popular C++ language; 2) to add useful parallel programming idioms unavailable in UPC, such as asynchronous remote function invocation and multidimensional arrays, to support complex scientific applications; 3) to offer an easy on-ramp to PGAS programming through interoperability with other existing parallel programming systems (e.g., MPI, OpenMP, CUDA). UPC++ includes two main features beyond UPC: 1) One of the major limitations of UPC in the context of DOE application is the relatively weak notion of multidimensional arrays, which it inherited from C. To remedy this problem in UPC++, we included a multidimensional domain and array library based on that of Titanium. Titanium's library is similar to Chapel's dense and strided domains, as both were inspired by the dense and strided regions and arrays in ZPL. 2) Another feature provided by UPC++ but not by UPC is remote function invocation, which is inspired by Phalanx, X10, Habanero-C, and C++11. The user may start an asynchronous remote function invocation with the following syntax: "async(place)(function, args...);". UPC++ provides two programming constructs for specifying dynamic dependencies among tasks: 1) event-driven execution as in Phalanx and Habanero-C; 2) finish-async as in X10. We implemented UPC++ with a "compiler-free" approach using C++ templates and runtime libraries. The UPC++ front-end consists of a set of C++ header files that enables the C++ compiler to "translate" UPC++ features in a user program to runtime library calls. In particular, we use templates and operator overloading to customize the behavior of UPC++ types. This software package has been replaced with UPCC++ V 1.0

Amir Kamil, YiliZheng↗

DOE SciDAC's Earth System Grid Center for Enabling Technologies Final Report

The mission of the Earth System Grid Federation (ESGF) is to provide the worldwide climate-research community with access to the data, information, model codes, analysis tools, and intercomparison capabilities required to make sense of enormous climate data sets. Its specific goals are to (1) provide an easy-to-use and secure web-based data access environment for data sets; (2) add value to individual data sets by presenting them in the context of other data sets and tools for comparative analysis; (3) address the specific requirements of participating organizations with respect to bandwidth, access restrictions, and replication; (4) ensure that the data are readily accessible through the analysis and visualization tools used by the climate research community; and (5) transfer infrastructure advances to other domain areas. For the ESGF, the U.S. Department of Energy's (DOE's) Earth System Grid Center for Enabling Technologies (ESG-CET) team has led international development and delivered a production environment for managing and accessing ultra-scale climate data. This production environment includes multiple national and international climate projects (such as the Community Earth System Model and the Coupled Model Intercomparison Project), ocean model data (such as the Parallel Ocean Program), observation data (Atmospheric Radiation Measurement Best Estimate, Carbon Dioxide Information and Analysis Center, Atmospheric Infrared Sounder, etc.), and analysis and visualization tools, all serving a diverse user community. These data holdings and services are distributed across multiple ESG-CET sites (such as ANL, LANL, LBNL/NERSC, LLNL/PCMDI, NCAR, and ORNL) and at unfunded partner sites, such as the Australian National University National Computational Infrastructure, the British Atmospheric Data Centre, the National Oceanic and Atmospheric Administration Geophysical Fluid Dynamics Laboratory, the Max Planck Institute for Meteorology, the German Climate Computing Centre, the National Aeronautics and Space Administration Jet Propulsion Laboratory, and the National Oceanic and Atmospheric Administration. The ESGF software is distinguished from other collaborative knowledge systems in the climate community by its widespread adoption, federation capabilities, and broad developer base. It is the leading source for present climate data holdings, including the most important and largest data sets in the global-climate community, and - assuming its development continues - we expect it to be the leading source for future climate data holdings as well. Recently, ESG-CET extended its services beyond data-file access and delivery to include more detailed information products (scientific graphics, animations, etc.), secure binary data-access services (based upon the OPeNDAP protocol), and server-side analysis. The latter capabilities allow users to request data subsets transformed through commonly used analysis and intercomparison procedures. As we transition from development activities to production and operations, the ESG-CET team is tasked with making data available to all users seeking to understand, process, extract value from, visualize, and/or communicate it to others. This ongoing effort, though daunting in scope and complexity, will greatly magnify the value of numerical climate model outputs and climate observations for future national and international climate-assessment reports. The ESG-CET team also faces substantial technical challenges due to the rapidly increasing scale of climate simulation and observational data, which will grow, for example, from less than 50 terabytes for the last Intergovernmental Panel on Climate Change (IPCC) assessment to multiple Petabytes for the next IPCC assessment. In a world of exponential technological change and rapidly growing sophistication in climate data analysis, an infrastructure such as ESGF must constantly evolve if it is to remain relevant and useful. Regretfully, we submit our final report at the end of project funding. To continue to serve the climate-science community, we are currently seeking additional funding. Such funding would allow us to maintain and enhance ESGF production and operation of this vital endeavor of cataloging, serving, and analyzing ultra-scale climate science data. At this time, the entire ESG-CET team would like to take this opportunity to sincerely thank our funding agencies in the DOE Scientific Discovery through Advanced Computing (SciDAC) program and the Office of Biological and Environmental Research (OBER) - as well as our national and international collaborators, stakeholders, and partners - for allowing us to work with you and serve the community these past several years.

54 ENVIRONMENTAL SCIENCES↗

Computational and Experimental Benchmarking for Transient Fuel Testing

In 2018 the Transient Reactor Test (TREAT) Facility was restarted. This now-operational facility brings back world-class transient fuel testing capabilities to the U.S. Among a number of parallel efforts prior to the TREAT Facility’s restart a Department of Energy (DoE) Integrated Research Project (IRP) was funded for the purpose of benchmarking the TREAT with several goals in mind: To produce a comprehensive evaluation of existing TREAT Facility neutronics data using the next generation reactor core neutronics codes in accordance with established guidelines per the International Handbook of Evaluated Reactor Physics Benchmark Experiments (IRPhEP). To perform a benchmark using ready-use computational fluid dynamics tools against that of a historically collected sodium loop experiment collected in the TREAT. To design, develop and test a new experimental thermal hydraulic system of a representative TREAT Facility water flow loop. And benchmark the data collected against that produced via RELAP5-3D and TRACE. To develop a comprehensive instrumentation plan for the TREAT Facility by benchmarking performing of advanced in-pile instruments in the Massachusetts Institute of Technology Research Reactor (MITR) and the TREAT. The integrated team brought together collaborating members from three universities including Oregon State University (OSU), the University of Michigan (UM), and Massachusetts Institute of Technology (MIT); three national laboratories, Idaho National Laboratory, Argonne National Laboratory (ANL), and Oak Ridge National Laboratory (ORNL); and two industry partners, TerraPower (TP) and Harris Thermal Transfer Products (HTTP). A brief summary of accomplishments resulting from this IRP includes two filed patents, more than 40 conference papers, two published benchmark reports through IRPhEP, a new representative thermal hydraulic test facility alike the in-pile PWR water flow loop for TREAT and the first advanced instruments tested within the TREAT core upon its restart. The papers presented herein were developed and provide a summary of the significant technical accomplishments which align with the four project’s objectives. The collaborators has significant support from countless parties including the operational and technical staff at TREAT to ensure that their outcomes most impactfully aligned with program needs. Lastly the collaborators would like to acknowledge the fiscal support of the DoE’s Office of Nuclear Energy under work package identification number NU-15-OR-OSU- 0701-01. This report contains selected papers from the Computational and Experimental Benchmarking for Transient Fuel Testing Integrated Research Project.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

TH-AB-207A-05: A Fully-Automated Pipeline for Generating CT Images Across a Range of Doses and Reconstruction Methods

Purpose: To evaluate the robustness of CAD or Quantitative Imaging methods, they should be tested on a variety of cases and under a variety of image acquisition and reconstruction conditions that represent the heterogeneity encountered in clinical practice. The purpose of this work was to develop a fully-automated pipeline for generating CT images that represent a wide range of dose and reconstruction conditions. Methods: The pipeline consists of three main modules: reduced-dose simulation, image reconstruction, and quantitative analysis. The first two modules of the pipeline can be operated in a completely automated fashion, using configuration files and running the modules in a batch queue. The input to the pipeline is raw projection CT data; this data is used to simulate different levels of dose reduction using a previously-published algorithm. Filtered-backprojection reconstructions are then performed using FreeCT-wFBP, a freely-available reconstruction software for helical CT. We also added support for an in-house, model-based iterative reconstruction algorithm using iterative coordinate-descent optimization, which may be run in tandem with the more conventional recon methods. The reduced-dose simulations and image reconstructions are controlled automatically by a single script, and they can be run in parallel on our research cluster. The pipeline was tested on phantom and lung screening datasets from a clinical scanner (Definition AS, Siemens Healthcare). Results: The images generated from our test datasets appeared to represent a realistic range of acquisition and reconstruction conditions that we would expect to find clinically. The time to generate images was approximately 30 minutes per dose/reconstruction combination on a hybrid CPU/GPU architecture. Conclusion: The automated research pipeline promises to be a useful tool for either training or evaluating performance of quantitative imaging software such as classifiers and CAD algorithms across the range of acquisition and reconstruction parameters present in the clinical environment. Funding support: NIH U01 CA181156; Disclosures (McNitt-Gray): Institutional research agreement, Siemens Healthcare; Past recipient, research grant support, Siemens Healthcare; Consultant, Toshiba America Medical Systems; Consultant, Samsung Electronics.

60 APPLIED LIFE SCIENCES↗

SU-F-BRB-07: A Plan Comparison Tool to Ensure Robustness and Deliverability in Online-Adaptive Radiotherapy

Purpose: To develop a plan comparison tool that will ensure robustness and deliverability through analysis of baseline and online-adaptive radiotherapy plans using similarity metrics. Methods: The ViewRay MRIdian treatment planning system allows export of a plan file that contains plan and delivery information. A software tool was developed to read and compare two plans, providing information and metrics to assess their similarity. In addition to performing direct comparisons (e.g. demographics, ROI volumes, number of segments, total beam-on time), the tool computes and presents histograms of derived metrics (e.g. step-and-shoot segment field sizes, segment average leaf gaps). Such metrics were investigated for their ability to predict that an online-adapted plan reasonably similar to a baseline plan where deliverability has already been established. Results: In the realm of online-adaptive planning, comparing ROI volumes offers a sanity check to verify observations found during contouring. Beyond ROI analysis, it has been found that simply editing contours and re-optimizing to adapt treatment can produce a delivery that is substantially different than the baseline plan (e.g. number of segments increased by 31%), with no changes in optimization parameters and only minor changes in anatomy. Currently the tool can quickly identify large omissions or deviations from baseline expectations. As our online-adaptive patient population increases, we will continue to develop and refine quantitative acceptance criteria for adapted plans and relate them historical delivery QA measurements. Conclusion: The plan comparison tool is in clinical use and reports a wide range of comparison metrics, illustrating key differences between two plans. This independent check is accomplished in seconds and can be performed in parallel to other tasks in the online-adaptive workflow. Current use prevents large planning or delivery errors from occurring, and ongoing refinements will lead to increased assurance of plan quality.

60 APPLIED LIFE SCIENCES↗

Data for Kim et al., "Variations in the optical and molecular composition of dissolved organic matter exported from coastal wetlands"

Knowledge about sources and composition of marsh-derived dissolved organic matter (DOM) is critical for understanding the role of marshes in coastal biogeochemical cycling and the fate of marsh-derived DOM in the ocean. To investigate tidal variability in composition of marsh-derived DOM, Kim et al. examined the optical and molecular characteristics of hourly surface water samples at three tidal creeks in the Chesapeake Bay. Groundwater samples along the terrestrial landscape gradient as well as estuarine water from the adjacent estuary at each site were also collected to help resolve sources of surface water DOM. Samples were collected in summer 2024 at three sites – SWH: Sweet Hall Marsh, GCW: Kirkpatrick Marsh, and GWI: Goodwin Islands – which are part of synoptic sites in the Chesapeake Bay region of the COMPASS-FME (Coastal Observations, Mechanisms, and Predictions Across Systems and Scales - Field, Measurements, and Experiments) project. Surface water samples were collected hourly over a 48-hour period at each site. Groundwater and estuarine water samples were collected once. This dataset includes- Surface water depth and salinity- Dissolved organic carbon (DOC) and total dissolved nitrogen (TDN) concentrations- Optical indices and relative composition of parallel factor analysis (PARAFAC) components- High resolution mass spectrometry data.

54 ENVIRONMENTAL SCIENCES↗

Specifications and pre-production of n + -in-p large-format strip sensors fabricated in 6-inch silicon wafers, ATLAS18, for the Inner Tracker of the ATLAS Detector for High-Luminosity Large Hadron Collider

Abstract The ATLAS experiment is constructing new all-silicon inner tracking system for HL-LHC. The strip detectors cover the radial extent of 40 to 100 cm. A new approach is adopted to use p-type silicon material, making the readout in n + -strips, so-called n + -in-p sensors. This allows for enhanced radiation tolerance against an order of magnitude higher particle fluence compared to the LHC. To cope with varying hit rates and occupancies as a function of radial distance, there are two barrel sensor types, the short strips (SS) for the inner 2 and the long strips (LS) for the outer 2 barrel cylinders, respectively. The barrel sensors exhibit a square, 9.8 × 9.8 cm 2 , geometry, the largest possible sensor area from a 6-inch wafer. The strips are laid out in parallel with a strip pitch of 75.5 μm and 4 or 2 rows of strip segments. The strips are AC-coupled and biased via polysilicon resistors. The endcap sensors employ a “stereo-annulus” geometry exhibiting a skewed-trapezoid shapes with circular edges. They are designed in 6 unique shapes, R0 to R5, corresponding to progressively increasing radial extents and which allows them to fit within the petal geometry and the 6-inch wafer maximally. The strips are in fan-out geometry with an in-built rotation angle, with a mean pitch of approximately 75 μm and 4 or 2 rows of strip segments. The eight sensor types are labeled as ATLAS18xx where xx stands for SS, LS, and R0 to R5. According to the mechanical and electrical specifications, CAD files for wafer processing were laid out, following the successful designs of prototype barrel and endcap sensors, together with a number of optimizations. A pre-production was carried out prior to the full production of the wafers. The quality of the sensors is reviewed and judged excellent through the test results carried out by vendor. These sensors are used for establishing acceptance procedures and to evaluate their performance in the ATLAS collaboration, and subsequently for pre-production of strip modules and stave and petal structures.

Instruments & Instrumentation↗

Refining Processing Engines from SAPHIRE: Initialization of Fault Tree/Event Tree Solver

SAPHIRE has been extensively employed for over 35 years to model risk-important systems and quantify risk models. As a well-established and thoroughly documented Probabilistic Risk Assessment (PRA) tool, SAPHIRE has continuously tracked computational trends and received regular updates. Despite its ongoing evolution, there remains a need for further enhancements, particularly in dealing with the quantification of exceptionally large models. These improvements could take the form of algorithmic advancements, harnessing the power of parallel computing, and exploring the potential benefits of cloud computing solutions. Considering these aspirations, the notion of a remote solve option was introduced and subsequently evolved into a dedicated project within the SAPHIRE development team. A significant outcome of this initiative is SAPHSOLVE, an engine extracted from the SageRisk API designed specifically for remote solving capabilities. The ongoing project is nearing its culmination, marked by a series of discoveries that have brought undocumented aspects to light. Among these revelations is the intricacy of the input and output format for the SAPHSOLVE engine. This document serves the crucial purpose of meticulously delineating the precise formats for both input and output, as they form an indispensable foundation. The importance of documenting these formats cannot be overstated, as it is a pivotal step in facilitating rigorous testing and comparison. Whether it involves scrutinizing SAPHSOLVE results against those of the internal integrated solver or other external solvers, the ability to construct models or transform existing ones into a compatible SAPHSOLVE format is imperative. Chapter 1 offers a succinct introduction to both SAPHIRE and SAPHSOLVE, followed by Chapter 2 which outlines the roadmap for enhancing SAPHSOLVE. The core of this report is Chapter 3, which intricately elucidates the intricacies of the input and output file formats. To provide a tangible illustration of these formats, a rudimentary example has been compiled and is available in Appendix. SAPHSOLVE represents a novel external solving mechanism developed by the SAPHIRE team, although it has not yet reached the full spectrum of capabilities possessed by SAPHIRE's internal solver. However, the SAPHIRE team has set a comprehensive course for incorporating the functionalities of SAPHSOLVE. A comprehensive outlook on the future of SAPHSOLVE is expounded upon in Chapter 4.

97 MATHEMATICS AND COMPUTING↗

VINE-A NUMERICAL CODE FOR SIMULATING ASTROPHYSICAL SYSTEMS USING PARTICLES. I. DESCRIPTION OF THE PHYSICS AND THE NUMERICAL METHODS

We present a numerical code for simulating the evolution of astrophysical systems using particles to represent the underlying fluid flow. The code is written in Fortran 95 and is designed to be versatile, flexible, and extensible, with modular options that can be selected either at the time the code is compiled or at run time through a text input file. We include a number of general purpose modules describing a variety of physical processes commonly required in the astrophysical community and we expect that the effort required to integrate additional or alternate modules into the code will be small. In its simplest form the code can evolve the dynamical trajectories of a set of particles in two or three dimensions using a module which implements either a Leapfrog or Runge-Kutta-Fehlberg integrator, selected by the user at compile time. The user may choose to allow the integrator to evolve the system using individual time steps for each particle or with a single, global time step for all. Particles may interact gravitationally as N-body particles, and all or any subset may also interact hydrodynamically, using the smoothed particle hydrodynamic (SPH) method by selecting the SPH module. A third particle species can be included with a module to model massive point particles which may accrete nearby SPH or N-body particles. Such particles may be used to model, e.g., stars in a molecular cloud. Free boundary conditions are implemented by default, and a module may be selected to include periodic boundary conditions. We use a binary 'Press' tree to organize particles for rapid access in gravity and SPH calculations. Modules implementing an interface with special purpose 'GRAPE' hardware may also be selected to accelerate the gravity calculations. If available, forces obtained from the GRAPE coprocessors may be transparently substituted for those obtained from the tree, or both tree and GRAPE may be used as a combination GRAPE/tree code. The code may be run without modification on single processors or in parallel using OpenMP compiler directives on large-scale, shared memory parallel machines. We present simulations of several test problems, including a merger simulation of two elliptical galaxies with 800,000 particles. In comparison to the Gadget-2 code of Springel, the gravitational force calculation, which is the most costly part of any simulation including self-gravity, is {approx}4.6-4.9 times faster with VINE when tested on different snapshots of the elliptical galaxy merger simulation when run on an Itanium 2 processor in an SGI Altix. A full simulation of the same setup with eight processors is a factor of 2.91 faster with VINE. The code is available to the public under the terms of the Gnu General Public License.

79 ASTRONOMY AND ASTROPHYSICS↗

brulilo, Version 0.x

brulilo is a Python package for building and evolving thermonuclear reaction networks, such as those used in astrophysical explosion calculations. It is very much a current work in progress, and much of the intended functionality is still under development. The main goal of brulilo is an easy-to-use code for quick calculations or post-processing, as well as a learning tool to test various implementations of solvers on both CPUs and GPUs. Currently, brulilo parses an input text file that specifies the reactions to be included in the network. The notation of the inputs are in the typical astrophysical notation; i.e. hydrogen burning to deuterium in the sun, "p + p + e --> d + neutrino", is given as "p(p,)d." Each reaction is added to a list comprising the reaction network, and once completed, the network contains a list of all isotopes and reactions involved. brulilo then parses publicly available nuclear data (binding energy, spin, partition function data, etc.) for each isotope from the WebNucleo group's data at nucleo.ces.clemson.edu. Fits to reaction rate data for each reaction are obtained from the publicly available data provided by the Joint Institute for Nuclear Astrophysics' Reaclib database. Reverse reactions are calculated from detailed balance. The reaction network then consists of a sparse system of mathematically stiff ordinary differential equations (ODEs). Physically, "stiffness" here means that there are some reaction rates that are occurring much faster than others; the ratio of the fast to slow often being greater than 10^15 in astrophysics. Numerical integration of such systems typically involves implicit integration schemes, which brulilo will leverage from the scientific computing Python package, Numpy, as well as implement its own. There has recently been work from researchers at Oak Ridge National Lab who have found good results from applying quasi-equilibrium and quasi-asymptotic conditions to the stiff system of ODEs, which effectively remove some of the stiffness and allow for efficient explicit integration techniques to be used. The original intent of brulilo was to implement these stiffness-alleviating techniques with explicit integrators and compare the performance to traditional implicit integrations of the full stiff system. This is still underway, as the code is very much in an alpha-release state. Furthermore, explicit integrators are often much easier to parallelize than their implicit counterparts. brulilo will implement parallelization of these techniques, leveraging both the Python implementation of MPI, mpi4py, as well as highly parallelized versions targeted at GPUs with PyOpenCL and/or PyCUDA.

Malone, Chris↗

brulilo, Version 0.x

brulilo is a Python package for building and evolving thermonuclear reaction networks, such as those used in astrophysical explosion calculations. It is very much a current work in progress, and much of the intended functionality is still under development. The main goal of brulilo is an easy-to-use code for quick calculations or post-processing, as well as a learning tool to test various implementations of solvers on both CPUs and GPUs. Currently, brulilo parses an input text file that specifies the reactions to be included in the network. The notation of the inputs are in the typical astrophysical notation; i.e. hydrogen burning to deuterium in the sun, "p + p + e --> d + neutrino", is given as "p(p,)d." Each reaction is added to a list comprising the reaction network, and once completed, the network contains a list of all isotopes and reactions involved. brulilo then parses publicly available nuclear data (binding energy, spin, partition function data, etc.) for each isotope from the WebNucleo group's data at nucleo.ces.clemson.edu. Fits to reaction rate data for each reaction are obtained from the publicly available data provided by the Joint Institute for Nuclear Astrophysics' Reaclib database. Reverse reactions are calculated from detailed balance. The reaction network then consists of a sparse system of mathematically stiff ordinary differential equations (ODEs). Physically, "stiffness" here means that there are some reaction rates that are occurring much faster than others; the ratio of the fast to slow often being greater than 10^15 in astrophysics. Numerical integration of such systems typically involves implicit integration schemes, which brulilo will leverage from the scientific computing Python package, Numpy, as well as implement its own. There has recently been work from researchers at Oak Ridge National Lab who have found good results from applying quasi-equilibrium and quasi-asymptotic conditions to the stiff system of ODEs, which effectively remove some of the stiffness and allow for efficient explicit integration techniques to be used. The original intent of brulilo was to implement these stiffness-alleviating techniques with explicit integrators and compare the performance to traditional implicit integrations of the full stiff system. This is still underway, as the code is very much in an alpha-release state. Furthermore, explicit integrators are often much easier to parallelize than their implicit counterparts. brulilo will implement parallelization of these techniques, leveraging both the Python implementation of MPI, mpi4py, as well as highly parallelized versions targeted at GPUs with PyOpenCL and/or PyCUDA.

Malone, Chris↗

Flow Redirection and Induction in Steady State (FLORIS) Wind Plant Power Production Data Sets

This dataset contains turbine- and plant-level power outputs for 252,500 cases of diverse wind plant layouts operating under a wide range of yawing and atmospheric conditions. The power outputs were computed using the Gaussian wake model in NREL's FLOw Redirection and Induction in Steady State (FLORIS) model, version 2.3.0. The 252,500 cases include 500 unique wind plants generated randomly by a specialized Plant Layout Generator (PLayGen) that samples randomized realizations of wind plant layouts from one of four canonical configurations: (i) cluster, (ii) single string, (iii) multiple string, (iv) parallel string. Other wind plant layout parameters were also randomly sampled, including the number of turbines (25-200) and the mean turbine spacing (3D-10D, where D denotes the turbine rotor diameter). For each layout, 500 different sets of atmospheric conditions were randomly sampled. These include wind speed in 0-25 m/s, wind direction in 0 deg.-360 deg., and turbulence intensity chosen from low (6%), medium (8%), and high (10%). For each atmospheric inflow scenario, the individual turbine yaw angles were randomly sampled from a one-sided truncated Gaussian on the interval 0 deg.-30 deg. oriented relative to wind inflow direction. This random data is supplemented with a collection of yaw-optimized samples where FLORIS was used to determine turbine yaw angles that maximize power production for the entire plant. To generate this data, a subset of cases were selected (50 atmospheric conditions from 50 layouts each for a total of additional 2,500 cases) for which FLORIS was re-run with wake steering control optimization. The IEA onshore reference turbine, which has a 130 m rotor diameter, a 110 m hub height, and a rated power capacity of 3.4 MW was used as the turbine for all simulations. The simulations were performed using NREL's Eagle high performance computing system in February 2021 as part of the Spatial Analysis for Wind Technology Development project funded by the U.S. Department of Energy Wind Energy Technologies Office. The data was collected, reformatted, and preprocessed for this OEDI submission in May 2023 under the Foundational AI for Wind Energy project funded by the U.S. Department of Energy Wind Energy Technologies Office. This dataset is intended to serve as a benchmark against which new artificial intelligence (AI) or machine learning (ML) tools may be tested. Baseline AI/ML methods for analyzing this dataset have been implemented, and a link to their repository containing those models has been provided. The .h5 data file structure can be found in the GitHub repository under explore_wind_plant_data_h5.ipynb.

AI↗

Scalla: Structured Cluster Architecture for Low Latency Access

Scalla is a distributed low-latency file access system that incorporates novel techniques that minimize latency and maximize scalability over a large distributed system with a distributed namespace. Scalla's techniques have shown to be effective in nearly a decade of service for the high-energy physics community using commodity hardware and interconnects. We describe the two components used in Scalla that are instrumental in its ability to provide low-latency, fault-tolerant name resolution and load distribution, and enable its use as a high-throughput, low-latency communication layer in the Qserv system, the Large Synoptic Survey Telescope's (LSST's) prototype astronomical query system. Scalla arguably exceeded its three main design objectives: low latency, scaling, and recoverability. In retrospect, these objectives were met using a simple but effective design. Low latency was met by uniformly using linear or constant time algorithms in all high-use paths, avoiding locks whenever possible, and using compact data structures to maximize the memory caching efficiency. Scaling was achieved by architecting the system as a 64-ary tree. Nodes can be added easily and as the number of nodes increases, search performance increases at an exponential rate. Recoverability is inherent in that no permanent state information is maintained and whatever state information is needed it can be quickly constructed or reconstructed in real time. This allows dynamic changes in a cluster of servers with little impact on over-all performance or usability. Today, Scalla is being deployed in environments and for uses that were never conceived in 2001. This speaks well for the systems adaptability but the underlying reason is that the system can meet its three fundamental objectives at the same time.

99 GENERAL AND MISCELLANEOUS↗