Search NASA⌕ Search

SEARCH · Search NASA

Results for “scientific software testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Multi-component testing using HZ-PAN and AgZ-PAN Sorbents for OSPREY Model validation

In efforts to further develop the capability of the Off-gas SeParation and RecoverY (OSPREY) model, multi-component tests were completed using both HZ-PAN and AgZ-PAN sorbents. The primary purpose of this effort was to obtain multi-component xenon and krypton capacities for comparison to future OSPREY predicted multi-component capacities using previously acquired Langmuir equilibrium parameters determined from single component isotherms. Experimental capacities were determined for each sorbent using two feed gas compositions of 1000 ppmv xenon and 150 ppmv krypton in either a helium or air balance. Test temperatures were consistently held at 220 K and the gas flowrate was 50 sccm. Capacities were calculated from breakthrough curves using TableCurve® 2D software by Jandel Scientific. The HZ-PAN sorbent was tested in the custom designed cryostat while the AgZ-PAN was tested in a newly installed cooling apparatus. Previous modeling validation efforts indicated the OSPREY model can be used to effectively predict single component xenon and krypton capacities for both engineered form sorbents. Results indicated good agreement with the experimental and predicted capacity values for both krypton and xenon on the sorbents. Overall, the model predicted slightly elevated capacities for both gases which can be partially attributed to the estimation of the parameters and the uncertainty associated with the experimental measurements. Currently, OSPREY is configured such that one species adsorbs and one does not (i.e. krypton in helium). Modification of OSPREY code is currently being performed to incorporate multiple adsorbing species and non-ideal interactions of gas phase species with the sorbent and adsorbed phases. Once these modifications are complete, the sorbent capacities determined in the present work will be used to validate OSPREY multicomponent adsorption predictions.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

ExaWorks software development kit: a robust and scalable collection of interoperable workflows technologies

Scientific discovery increasingly requires executing heterogeneous scientific workflows on high-performance computing (HPC) platforms. Heterogeneous workflows contain different types of tasks (e.g., simulation, analysis, and learning) that need to be mapped, scheduled, and launched on different computing. That requires a software stack that enables users to code their workflows and automate resource management and workflow execution. Currently, there are many workflow technologies with diverse levels of robustness and capabilities, and users face difficult choices of software that can effectively and efficiently support their use cases on HPC machines, especially when considering the latest exascale platforms. We contributed to addressing this issue by developing the ExaWorks Software Development Kit (SDK). The SDK is a curated collection of workflow technologies engineered following current best practices and specifically designed to work on HPC platforms. We present our experience with (1) curating those technologies, (2) integrating them to provide users with new capabilities, (3) developing a continuous integration platform to test the SDK on DOE HPC platforms, (4) designing a dashboard to publish the results of those tests, and (5) devising an innovative documentation platform to help users to use those technologies. Our experience details the requirements and the best practices needed to curate workflow technologies, and it also serves as a blueprint for the capabilities and services that DOE will have to offer to support a variety of scientific heterogeneous workflows on the newly available exascale HPC platforms.

97 MATHEMATICS AND COMPUTING↗

Experiences with a Flexible User Research Process to Build Data Change Tools

Scientific software development processes are understood to be distinct from commercial software development practices due to uncertain and evolving states of scientific knowledge. Sustaining these software products is a recognized challenge, but under-examined is the usability and usefulness of such tools to their scientific end users. User research is a well-established set of techniques (e.g., interviews, mockups, usability tests) applied in commercial software projects to develop foundational, generative, and evaluative insights about products and the people who use them. Currently these approaches are not commonly applied and discussed in scientific software development work. The use of user research techniques in scientific environments can be challenging due to the nascent, fluid problem spaces of scientific work, varying scope of projects and their user communities, and funding/economic constraints on projects.In this paper, we reflect on our experiences undertaking a multi-method user research process in the Deduce project. The Deduce project is investigating data change to develop metrics, methods, and tools that will help scientists make decisions around data change. There is a lack of common terminology since the concept of systematically measuring and managing data change is under explored in scientific environments. To bridge this gap we conducted user research that focuses on user practices, needs, and motivations to help us design and develop metrics and tools for data change. This paper contributes reflections and the lessons we have learned from our experiences. We offer key takeaways for scientific software project teams to effectively and flexibly incorporate similar processes into their projects.

97 MATHEMATICS AND COMPUTING↗

SPEL: Software tool for Porting E3SM Land Model with OpenACC in a Function Unit Test Framework

Most high-end computers adopt hybrid architecture, porting a large-scale scientific code onto accelerators is necessary. The paper presents a generic method for porting large-scale scientific code onto accelerators using compiler directives within a modularized function unit test platform. We have implemented the method and designed a software tool (SPEL) to port the E3SM Land Model (ELM) onto the GPUs in the Summit computer. SPEL automatically generates GPU-ready test modules for all ELM functions, such as CanopyFlux, SoilTemperature, and EcosystemDynamics. SPEL breaks the ELM into a collection of standalone unit test programs for easy code verification and further performance improvement. We further optimize several ELM test modules with advanced techniques, including memory reduction, reconstructed parallel loops, and asynchronous GPU kernel launch. We hope our study will inspire new toolkit developments that expedite large-scale scientific code porting with compiler directives.

Schwartz, Peter↗

HPC-driven computational reproducibility in numerical relativity codes: a use case study with IllinoisGRMHD

Abstract Reproducibility of results is a cornerstone of the scientific method. Scientific computing encounters two challenges when aiming for this goal. Firstly, reproducibility should not depend on details of the runtime environment, such as the compiler version or computing environment, so results are verifiable by third-parties. Secondly, different versions of software code executed in the same runtime environment should produceconsistent numerical results for physical quantities. In this manuscript, we test the feasibility of reproducing scientific results obtained using theIllinoisGRMHDcode that is part of an open-source community software for simulation in relativistic astrophysics, theEinstein Toolkit. We verify that numerical results of simulating a single isolated neutron star withIllinoisGRMHDcan be reproduced, and compare them to results reported by the code authors in 2015. We use two different supercomputers: Expanse at SDSC, and Stampede2 at TACC. By compiling the source code archived along with the paper on both Expanse and Stampede2, we find thatIllinoisGRMHDreproduces results published in its announcement paper up to errors comparable to round-off level changes in initial data parameters. We also verify that a current version ofIllinoisGRMHDreproduces these results once we account for bug fixes which have occurred since the original publication.

Astronomy & Astrophysics↗

A Unit Testing Framework for Scientific Legacy Code

Large-scale scientific applications play important roles in supporting research. However, it is often very expensive and time-consuming to make changes to, maintain and evolve the scientific code due to its complexity and poor programming skills of researchers. Therefore, in order to visualize scientific code architecture to optimize software design, understand undocumented source code, and analyze software flow and functionality, we first introduce a unit testing framework (UTF). Then, because such infrastructure's performance is very crucial in practical use since the scientific legacy applications simulate instances in a long period of time, we improve the UTF by applying Message Passing based Parallelization and parallel I/O operations. Furthermore, due to the scientific code has enormous state data and the I/O capacity on the server is limited, we apply in situ data analysis method to encounter fewer resource limitations, and adopt signal processing to greatly reduce data transfer. Last, we demonstrated the correctness and high-efficiency of our framework for legacy Earth model on Titan supercomputer.

97 MATHEMATICS AND COMPUTING↗

Numerical Implementation of Time-Dependent Density Functional Theory for Extended Systems in Extreme Environments

In recent years, DFT-MD has been shown to be a useful computational tool for exploring the properties of WDM. These calculations achieve excellent agreement with shock compression experiments, which probe the thermodynamic parameters of the Hugoniot state. New X-ray Thomson Scattering diagnostics promise to deliver independent measurements of electronic density and temperature, as well as structural information in shocked systems. However, they require the development of new levels of theory for computing the associated observables within a DFT framework. The experimentally observable x-ray scattering cross section is related to the electronic density-density response function, which is obtainable using TDDFT - a formally exact extension of conventional DFT that describes electron dynamics and excited states. In order to develop a capability for modeling XRTS data and, more generally, to establish a predictive capability for first principles simulations of matter in extreme conditions, real-time TDDFT with Ehrenfest dynamics has been implemented in an existing PAW code for DFT-MD calculations. The purpose of this report is to record implementation details and benchmarks as the project advances from software development to delivering novel scientific results. Results range from tests that establish the accuracy, efficiency, and scalability of our implementation, to calculations that are verified against accepted results in the literature. Aside from the primary XRTS goal, we identify other more general areas where this new capability will be useful, including stopping power calculations and electron-ion equilibration.

97 MATHEMATICS AND COMPUTING↗

Performance Assessment of a New Gamma Spectroscopy Toolkit: Peak Doctor and the Spectral Nondestructive Assay Platform (SNAP) - 17397

High resolution gamma spectroscopy (HRGS) is a widely used technique for non-destructive characterization of nuclear materials, and assay of radioactive waste. The ability to accurately evaluate raw spectral data is an important step in producing high-quality results. A spectral analysis application known as Peak Doctor has been developed under a Cooperative Research and Development Agreement (CRADA) between LANL and Pajarito Scientific Corporation (PSC). The software has undergone several years of testing and validation by the Radioactive Waste Characterization Team of the LANL Waste Management Services Group (WM-SVS). PeakDoctor is designed to work with the Spectral Nondestructive Assay Platform (SNAP{sup TM}) to provide a complete HRGS analysis engine. SNAP{sup TM} performs model specific corrections for attenuation and geometry. These applications are the major components of a new toolkit designed primarily for portable far-field non-destructive assay. Peak Doctor performs a very accurate peak fit analysis with the goal of improving upon the capabilities of existing commercially available routines. It is designed to be user-configurable and can be customized to any given type of detector and any given type of spectrum. Peak Doctor was developed to resolve many of the inherent complexities in gamma ray spectra including overlapping peaks, complex multiplet regions, large step peaks, backscatter regions and Compton edges. Unlike most spectral analysis platforms, Peak Doctor does not rely on the energy calibration and detector resolution parameters that are saved by the operator in the spectrum file. Instead, key parameters are pre-defined for individual detectors. The software fits a continuum using a series of splines between a series of junctions or 'knots'. Fitting techniques are used to join the splines together to create a near-continuous multi-element curve. Optimizing the location of the knots is one of Peak Doctor's most powerful features. The knots are initially configured using a pre-defined list and are removed or added automatically during the fitting process. Knots are also created when new features are identified such as Step Peaks and Compton Edges. The user can also manually specify knots in order to improve the continuum fit. The initial fit is improved on by removing any outlier regions (which are converted to peaks) and the continuum fitting process is repeated. At this stage, an experienced user can work interactively with the fit by manually converting peak regions to continuum or vice versa. The final step is to fit a series of peaks above the continuum based on detector parameters. Single and multiplet peaks are then processed by either the tailing (piecewise) or non-tailing (Gaussian) Levenberg Marquardt non-linear peak fitting routine depending on the user defined selection. Only a rough knowledge of the detector characteristics (energy calibration and resolution) is required for the fitting process to proceed with good accuracy. Any peaks that are missed can be manually added by the user and will also be fitted. Discontinuities in the peak / continuum fit can be rapidly identified by reviewing the 'residuals scale' plot. For waste streams that are well characterized, it is often the case that routinely encountered spectra will have similar peaks and continuum features. For these cases a handy 'One Step' method is available that proceeds in an automated fashion without user interaction. Peak Doctor has been configured with a set of default spectrum types including plutonium / transuranic waste (including Weapons Grade Pu, Heat Source Pu, Am-241, Am-243 and Np-237), depleted or enriched uranium, nuclear power plant / intermediate level waste (with fission and activation products such as Cs-137 and Co-60) as well as Naturally Occurring Radioactive Materials (NORM). The user can easily add new spectrum types for specific waste streams or other types of radiological / special nuclear materials. The output from PeakDoctor (including net peak area, centroid energy and a list of minimum detectable activity levels at pre-defined regions) is used by SNAP{sup TM} to complete the data analysis process. Isotope activities are calculated using configurable template-driven geometry and attenuation correction methods. The software accounts for matrix and container wall effects, collimator / angular corrections, detector efficiency (based on calibration curves) and stand-off distance. SNAP{sup TM} uses published algorithms incorporating a series of decision points during its operation to allow the analyst to provide logical input to guide the software during the analysis of each unique spectrum. The Peak Doctor / SNAP{sup TM} toolkit has been extensively tested and validated under a variety of conditions using 'benchmark' reference spectra. Test scenarios reported include HPGe-based measurements of well characterized uranium and plutonium standards and radioactive sources with a variety of different geometries and matrix materials including surrogate drums and boxes. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Best practices in software development for robust and reproducible geoscientific models based on insights from the Global Carbon Budget's dynamic vegetation models

Computational models play an increasingly vital role in scientific research by enabling the numerical simulation of complex processes. Such models are also fundamental in geosciences. For instance, they offer critical insights into the impacts of global change on the Earth system today and in the future. Beyond their value as research tools, models are also software products and should therefore adhere to certain established software engineering standards. However, scientists are rarely trained as software developers, which can lead to potential deficiencies in software quality like unreadable, inefficient, or erroneous code. The complexity of models, coupled with their integration into broader workflows, also often makes it challenging to reproduce results, evaluate processes, and build upon them. In this paper, we review the state and current practices of the development processes of the state-of-the-art land surface models used by the Global Carbon Budget. We combine the experience of modelers from the respective research groups with the expertise of software engineers from tech companies to outline key principles and tools for improving software quality in research. We explore four main areas: (1) model testing and validation, (2) scientific, technical, and user documentation, (3) version control, continuous integration, and code review, and (4) the portability and reproducibility of workflows. Our review reveals that while modeling communities are incorporating many best practices, significant room for improvement remains in areas such as automated testing, automated documentation, and reproducibility. Therefore, we here identify and promote essential software engineering practices, including numerous examples of practices from within the community that can serve as guidelines for other models and could help streamline processes across the entire community. We conclude with an open-source example implementation of these principles, demonstrating portable and reproducible data flows, a continuous integration setup, and web-based visualizations. This example may serve as a practical resource for model developers, users, and all scientists engaged in scientific programming.

Gregor, Konstantin [Technical Univ. of Munich (Ger↗

Providing a Flexible and Comprehensive Software Stack Via Spack, an Extreme-Scale Scientific Software Stack, and Software Development Kits

To manage the complex demands of modern high-performance computing (HPC), software applications increasingly depend on software developed by other teams, often at other institutions. An HPC software ecosystem approach is required to support dependencies on third-party scientific software. An ecosystem approach provides layers of activity above the individual software product level that promote interoperability, quality improvement, porting, testing, and deployment. The U.S. Exascale Computing Project (ECP) developed its HPC software ecosystem using a three-pronged approach. First, the ECP adopted and invested in Spack, a package manager designed to handle complex HPC package dependencies. Second, the ECP created the Extreme Scale Scientific Software Stack, an effort that supports developing, deploying, and running scientific applications on HPC platforms. Third, the ECP supported software product communities, or software development kits, to develop and promote best practices, improve software interoperability, and other collaborative efforts. This article describes ECP contributions to HPC software ecosystem challenges.

97 MATHEMATICS AND COMPUTING↗

Remote Job Testing for the Neutron Science TeraGrid Gateway

Remote job execution gives neutron science facilities access to high performance computing such as the TeraGrid. A scientific community can use community software with a community certificate and account through a common interface of a portal. Results show this approach is successful, but with more testing and problem solving, we expect remote job executions to become more reliable.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Evaluation of Low Power ARM Processors for Monte Carlo Particle Transport: MCNP6 on the ARM Cortex-A8 - Paper 83

In the push towards exascale computing, the limiting factors are power requirements and heat dissipation. Using current technology, the power required to drive an exascale supercomputer would near 400-MW. Clearly, future computing hardware will have to be more energy efficient in order to achieve exascale performance. Most of the current research in advanced computing architectures is in the use of standard CPUs with accelerators. However, these accelerators are not generally supported by the traditional scientific programming languages (C, C++, and Fortran) and the scientific software must be tailored specifically to take advantage of the accelerators. Large scale scientific software (hundreds-of-thousand to millions of lines of code) can not be easily ported to the computing platform du jour. The widespread use of low power ARM based processors in the mobile computing market offer a low power alternative to accelerators that are also supported by the traditional scientific programming languages and programming paradigms such as MPI and OpenMP. Computing architectures for scientific computing are often assessed using standard benchmarks such as LINPACK benchmark or using suites of mini-applications that are greatly simplified stand-ins for scientific software. Often the real applications that will deployed on the scientific supercomputers are not ported or tested until delivery of the machines. In order to evaluate the use of ARM processors for scientific computing the Monte Carlo particle transport code, MCNP version 6 was compiled on the ARM Cortex-A8 and a set of benchmark problems were used to assess performance in comparison to an Intel Xeon E5-2670 and the low power Intel Atom N270. MCNP is a continuous energy Monte Carlo particle transport code in development at Los Alamos National Laboratory since 1977. MCNP is used to transport neutrons, photon, electrons, and charged particles using continuous energy interaction data. It is commonly used in fields of nuclear power, nuclear medicine, radiation protection, and high-energy physics through-out the world. MCNP version 6 is approximately 450 thousand lines of code, written primary in Fortran90, with some C and C++. MCNP supports distributed multiprocessing using MPI and threading with OpenMP. The ARM Cortex-A8 executes MCNP6 using about twice as much power as the Intel Xeon E5-2670 and about the same amount of power as the Intel Atom N270. However, these results are encouraging for the use of low-power ARM chips in future supercomputers. The ARM Cortex-A8 is a 32-bit CPU and it's NEON general-purpose SIMD (single-instruction multiple data) engine does not support double-precision floating-point arithmetic. More recent ARM processors (ARM Cortex-A53 and Cortex-A57) support 64-bit addressing and have NEON co-processors that support double-precision floating-point arithmetic. ARM claims that the A50 series delivers a performance increase of 15 times the Cortex-A8 performance for similar power usage. (authors)

61 RADIATION PROTECTION AND DOSIMETRY↗

ENHANCING SEISMIC CALIBRATION RESEARCH THROUGH SOFTWARE AUTOMATION AND SCIENTIFIC INFORMATION MANAGEMENT

The National Nuclear Security Administration (NNSA) Ground-Based Nuclear Explosion Monitoring Research and Development (GNEMRD) Program at LLNL continues to make significant progress enhancing the process of deriving seismic calibrations and performing scientific integration, analysis, and information management with software automation tools. Our tool efforts address the problematic issues of very large datasets and varied formats encountered during seismic calibration research. New information management and analysis tools have resulted in demonstrated gains in efficiency of producing scientific data products and improved accuracy of derived seismic calibrations. The foundation of a robust, efficient data development and processing environment is comprised of many components built upon engineered versatile libraries. We incorporate proven industry 'best practices' throughout our code and apply source code and bug tracking management as well as automatic generation and execution of unit tests for our experimental, development and production lines. Significant software engineering and development efforts have produced an object-oriented framework that provides database centric coordination between scientific tools, users, and data. Over a half billion parameters, signals, measurements, and metadata entries are all stored in a relational database accessed by an extensive object-oriented multi-technology software framework that includes stored procedures, real-time transactional database triggers and constraints, as well as coupled Java and C++ software libraries to handle the information interchange and validation requirements. Significant resources were applied to schema design to enable management of processing methods and station parameters, responses and metadata. This allowed for the development of merged ground-truth (GT) data sets compiled by the NNSA labs and AFTAC that include hundreds of thousands of events and tens of millions of arrivals. The schema design groundwork facilitated extensive quality-control and revalidation steps. In support of the GT merge effort, a comprehensive site merge process was also accomplished this year that included station site information for tens of thousands of entries from NNSA labs, AFTAC, NEIC, ISC, and IRIS. A core capability is the ability to rapidly select and present subsets of related signals and measurements to the researchers for analysis and distillation both visually (JAVA GUI client applications) and in batch mode (instantiation of multi-threaded applications on clusters of processors). RBAP Version 2 is one such example. Over the past year RBAP was significantly improved in capability and performance. A new role-based security model now allows fine-grain access control over all aspects of the tool's functions enabling researchers to share their work with others without fear of unintended parameter alterations. A new, faster and more reliable GIS mapping framework was added, as well as expanded powerful interactive plotting graphics. In addition, we implemented parent-child type projects to enhance calibration data management. Our specific automation methodology and tools improve the researchers ability to assemble quality-controlled research products for delivery into the NNSA Knowledge Base (KB). The software and scientific automation tasks provide the robust foundation upon which synergistic and efficient development of GNEMRD Program seismic calibration research may be built.

58 GEOSCIENCES↗

SABATH: Surrogate AI Benchmarking Applications' Testing Harness

SABATH provides benchmarking infrastructure for evaluating scientific ML/AI models. It contains support for scientific machine learning surrogates from external repositories. The software dependences are explicitly exposed in the surrogate model definition, which allows the use of advanced optimization, communication, and hardware features.

Luszczek, Piotr [Univ. of Tennessee, Knoxville, TN↗

epicsuite(EAS)

Software, Documentation, Tutorials, Testing data, Example data for processing, analysis, filtering, querying, visualization and otherwise transforming genomic data for scientific analysis and discovery.

Rogers, David H.↗

ComPort: Rigorous Testing Methods to Safeguard Software Porting (Final UW Report)

This report summarizes the University of Washington’s contributions to the ComPort project, Rigorous Testing Methods to Safeguard Software Porting, funded under the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research (ASCR), award number DE-SC0022081. UW’s work addressed numerics-driven correctness challenges arising in heterogeneous and rapidly evolving computing environments. Our efforts focused on developing interactive analysis tools, correctness-preserving rewriting systems, semantic-aware code-transformation infrastructure, and multi-language support for reproducible numerical experiments.

97 MATHEMATICS AND COMPUTING↗

A Python Object Oriented & Data Driven Testing Infrastructure for an Environmental Information Management System

Software systems such as TEIMS, the Taurus Environmental Information Management System evolve in size and complexity making manual testing no longer sufficient or feasible, becoming a performance and quality risk. TEIMS is a system of systems comprised of dynamic, data-driven web-based applications that support planning, operational monitoring, data collection, sample management, reporting and, scientific findings for LLNL’s Environmental Restoration Department (ERD). Selenium’s Integrated Development Environment (IDE) was previously used to support TEIMS testing. An approach with limited flexibility for future software development. As a student intern I was assigned the task to develop an object oriented testing infrastructure for TEIMS. A critical resource in this approach is the development of a catalogue of page elements. To accomplish this I created a comma separated value file, which is comprised of key identifiers based on page elements. Using the Python language and the Selenium Web driver I then developed the object mapping structure, which serves as a wrapper for exposure of each element allowing interaction with the tests and test suites. A master driver “Module” allowed the web driver to be shared among the objects and tests. I then developed a web-based graphical user interface (GUI) for test history, metrics and searching. This data driven approach provides flexibility and extensibility, allowing changes to the structure to be performed in a single place. The result is an object oriented/data driven infrastructure that allows tests to be simpler, flexible, and maintainable, with searchable results.

97 MATHEMATICS AND COMPUTING↗