SEARCH · Search NASA
Results for “Test Framework”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Testing contamination source identification methods for water distribution networks
In the event of contamination in a water distribution network (WDN), source identification (SI) methods that analyze sensor data can be used to identify the source location(s). Knowledge of the source location and characteristics are important to inform contamination control and cleanup operations. Various SI strategies that have been developed by researchers differ in their underlying assumptions and solution techniques. The following manuscript presents a systematic procedure for testing and evaluating SI methods. The performance of these SI methods is affected by various factors including the size of WDN model, measurement error, modeling error, time and number of contaminant injections, and time and number of measurements. This paper includes test cases that vary these factors and evaluates three SI methods on the basis of accuracy and specificity. The tests are used to review and compare these different SI methods, highlighting their strengths in handling various identification scenarios. These SI methods and a testing framework that includes the test cases and analysis tools presented in this paper have been integrated into EPA’s Water Security Toolkit (WST), a suite of software tools to help researchers and others in the water industry evaluate and plan various response strategies in case of a contamination incident. Lastly, a set of recommendations are made for users to consider when working with different categories of SI methods.
Analytic energy gradient of excited electronic state within TDDFT/MMpol framework: Benchmark tests and parallel implementation
The time-dependent density functional theory (TDDFT) has become the most popular method to calculate the electronic excitation energies, describe the excited-state properties, and perform the excited-state geometric optimization of medium and large-size molecules due to the implementation of analytic excited-state energy gradient and Hessian in many electronic structure software packages. To describe the molecules in condensed phase, one usually adopts the computationally efficient hybrid Quantum Mechanics/Molecular Mechanics (QM/MM) models. Here, we extend our previous work on the energy gradient of TDDFT/MM excited state to account for the mutual polarization effects between QM and MM regions, which is believed to hold a crucial position in the potential energy surface of molecular systems when the photoexcitation-induced charge rearrangement in the QM region is drastic. The implementation of a simple polarizable TDDFT/MM (TDDFT/MMpol) model in Q-Chem/CHARMM interface with both the linear response and the state-specific features has been realized. Several benchmark tests and preliminary applications are exhibited to confirm our implementation and assess the effects of different treatment of environmental polarization on the excited-state properties, and the efficiency of parallel implementation is demonstrated as well.
Exact Coulomb exchange calculations in the Skyrme-Hartree-Fock-BCS framework and tests of the Slater approximation
An exact treatment of the Coulomb interaction is performed within the Skyrme-Hartree-Fock/Bardeen-Cooper-Schrieffer approach for even-even nuclei ranging from light to superheavy nuclei. A test of the usual Slater approximation for the exchange part is carried out. The error made on the exchange term of the Coulomb interaction when using this approximation follows two schematic patterns. Beyond a decreasing behavior when increasing the mass number A, a more important structural effect has been found. The relative error ranges roughly from 0 to 8% being maximal for light closed proton (sub-)shell nuclei and minimal for open proton (sub-)shell nuclei.
Assessing Local Structure Motifs Using Order Parameters for Motif Recognition, Interstitial Identification, and Diffusion Path Characterization
Structure–property relationships form the basis of many design rules in materials science, including synthesizability and long-term stability of catalysts, control of electrical and optoelectronic behavior in semiconductors, as well as the capacity of and transport properties in cathode materials for rechargeable batteries. The immediate atomic environments (i.e., the first coordination shells) of a few atomic sites are often a key factor in achieving a desired property. Some of the most frequently encountered coordination patterns are tetrahedra, octahedra, body and face-centered cubic as well as hexagonal close packed-like environments. Here, we showcase the usefulness of local order parameters to identify these basic structural motifs in inorganic solid materials by developing classification criteria. We introduce a systematic testing framework, the Einstein crystal test rig, that probes the response of order parameters to distortions in perfect motifs to validate our approach. Subsequently, we highlight three important application cases. First, we map basic crystal structure information of a large materials database in an intuitive manner by screening the Materials Project (MP) database (61,422 compounds) for element-specific motif distributions. Second, we use the structure-motif recognition capabilities to automatically find interstitials in metals, semiconductor, and insulator materials. Our Interstitialcy Finding Tool (InFiT) facilitates high-throughput screenings of defect properties. Third, the order parameters are reliable and compact quantitative structure descriptors for characterizing diffusion hops of intercalants as our example of magnesium in MnO 2 -spinel indicates. Finally, the tools developed in our work are readily and freely available as software implementations in the pymatgen library, and we expect them to be further applied to machine-learning approaches for emerging applications in materials science.
Integrating CVMix into GOTM (v6.0): a consistent framework for testing, comparing, and applying ocean mixing schemes
Abstract. The General Ocean Turbulence Model (GOTM) is a one-dimensional water column model, including a set of state-of-the-art turbulence closure models, and has widely been used in various applications in the ocean modeling community. Here, we extend GOTM to include a set of newly developed ocean surface vertical mixing parameterizations of Langmuir turbulence via coupling with the Community Vertical Mixing Project (CVMix). A Stokes drift module is also implemented in GOTM to provide the necessary ocean surface waves information to the Langmuir turbulence parameterizations, as well as to facilitate future development and evaluation of new Langmuir turbulence parameterizations. In addition, a streamlined workflow with Python and Jupyter notebooks is also described, enabled by the newly developed and more flexible configuration capability of GOTM. The newly implemented Langmuir turbulence parameterizations are evaluated against theoretical scalings and available observations in four test cases, including an idealized wind-driven entrainment case and three realistic cases at Ocean Station Papa, the northern North Sea, and the central Baltic Sea, and compared with the existing general length scale scheme in GOTM. The results are consistent with previous studies. This development extends the capability of GOTM towards including the effects of ocean surface waves and provides useful toolsets for the ocean modeling community to further study the effects of Langmuir turbulence in a broader scope.
A common task framework for testing and evaluation at the Space Domain Awareness Tools, Applications, and Processing Lab
Explore the source record for details and available documents.
Who watches the watchers?: preventing fault in a fault tolerance library
The Scalable Checkpoint/Restart library (SCR) was developed and is used by researchers at Lawrence Livermore National Laboratory to provide a fast and efficient method of saving and recovering large applications during runtime on high-performance computing (HPC) systems. Though SCR protects other programs, up until June 2017, nothing was actively protecting SCR. The goal of this project was to automate the building and testing of this library on the varying HPC architectures on which it is used. Our methods centered around the use of a continuous integration tool called Bamboo that allowed for automation agents to be installed on the HPC systems themselves. These agents provided a way for us to establish a new and unique way to automate and customize the allocation of resources and running of tests with CMake’s unit testing framework, CTest, as well as integration testing scripts though an HPC package manager called Spack. These methods provided a parallel environment in which to test the more complex features of SCR. As a result, SCR is now automatically built and tested on several HPC architectures any time changes are made by developers to the library’s source code. The results of these tests are then communicated back to the developers for immediate feedback, allowing them to fix functionality of SCR that may have broken. Hours of developers’ time are now being saved from the tedious process of manually testing and debugging, which saves money and allows the SCR project team to focus their efforts towards development. Thus, HPC system users can use SCR in conjunction with their own applications to efficiently and effectively checkpoint and restart as needed with the assurance that SCR itself is functioning properly.
SUPERMASSIVE BLACK HOLES IN THE HIERARCHICAL UNIVERSE: A GENERAL FRAMEWORK AND OBSERVATIONAL TESTS
We present a simple framework for the growth and evolution of supermassive black holes (SMBHs) in the hierarchical structure formation paradigm, adopting the general idea that quasar activity is triggered in major mergers. In our model, black hole accretion is triggered during major mergers (mass ratio approx>0.3) of host dark matter halos. The successive evolution of quasar luminosities follows a universal light-curve (LC) form during which the growth of the SMBH is modeled self-consistently: an initial exponential growth at a constant Eddington ratio of order unity until it reaches the peak luminosity, followed by a power-law decay. Assuming that the peak luminosity correlates with the post-merger halo mass, we convolve the LC with the triggering rate of quasar activity to predict the quasar luminosity function (LF). Our model reproduces the observed LF at 0.5 < z < 4.5 for the full luminosity ranges probed by current optical and X-ray surveys. At z < 0.5, our model underestimates the LF at L {sub bol} < 10{sup 45} erg s{sup -1}, allowing room for the active galactic nuclei (AGNs) activity triggered by secular processes instead of major mergers. At z > 4.5, in order to reproduce the observed quasar abundance, the typical quasar hosts must shift to lower mass halos, and/or minor mergers can also trigger quasar activity. Our model reproduces both the observed redshift evolution and luminosity dependence of the linear bias of quasar/AGN clustering. Due to the scatter between instantaneous luminosity and halo mass, quasar/AGN clustering weakly depends on luminosity at low-to-intermediate luminosities; but the linear bias rises rapidly with luminosity at the high luminosity end and at high redshift. In our model, the Eddington ratio distribution is roughly lognormal, which broadens and shifts to lower mean values from high luminosity quasars (L {sub bol} approx> 10{sup 46} erg s{sup -1}) to low-luminosity AGNs (L {sub bol} approx< 10{sup 45} erg s{sup -1}), in good agreement with observations. The model predicts that the vast majority of approx>10{sup 8.5} M {sub sun} SMBHs were already in place by z = 1, and approx<50% of them were in place by z = 2; but the less massive (approx<10{sup 7} M {sub sun}) SMBHs were assembled more recently, likely more through secular processes than by major mergers-in accordance with the downsizing picture of SMBH assembly since the peak of bright quasar activity around z approx 2-3.
Modernization efforts for the R -Matrix code SAMMY [Abstract]
The R-Matrix code SAMMY is a widely used nuclear data evaluation code focused on the resolved range, which includes corrections for experimental effects. The code is still mostly written in Fortran 77, and uses a memory management system suitable for the time of its initial writing (1984). A modernization effort is under way to bring the code in-line with modern software development practices. A continuous-integration testing framework was added, automating the large existing set of test cases. It is run on every commit. The memory management was updated to current standard practices suitable for modern software analysis tools. The code can be obtained from https://code.ornl.gov/RNSD/SAMMY. The resonance parameters and covariance information are now stored in C++ objects shared by SAMMY and AMPX, the processing code that generates nuclear data libraries for SCALE. This allows for easier maintenance and access to the resonance parameters inside and outside of SAMMY. This feature is already used by accessing and changing parameters in memory in the Bayesian Monte Carlo Evaluation Framework for Cross Sections Nuclear Data and Integral Benchmark Experiments project, Further plans include the switch to the ENDF reading and writing routines in AMPX, as these routines are more robust, easier to maintain, and support more features. Of note here is support for the new GNDS format. Previously it wasn’t easy to share the full covariance matrix for evaluations containing more than one isotope due to limitations on the ENDF format; this is now supported in GNDS. The data are currently available in a binary SAMMY format and can be exported to GNDS to make them more widely available and sharable. The next step will be to use the same resonance processing code at 0K in AMPX and SAMMY as one of the available Reich-Moore R-Matrix formalism. The first step toward this goal is to isolate the reconstruction into a module that takes resonance parameters as its input and does not depend on SAMMY global parameters. This goal has been achieved and it should now be possible to more easily change the resonance formalism and add enhancements as the Phenomenological R-Matrix parameterization of direct, doorway, and compound nuclear reactions discussed elsewhere on this conference. This concerted modernization and enhancement effort provides multiple advantages to the nuclear data community. It will allow parameter optimization using enhanced formalisms, including experimental effects, that better match complex experimental data. Then those evaluated parameters can immediately be passed off to AMPX to be reconstructed with the exact same cross section model and be put into a data library for subsequent testing using SCALE and the Valid Benchmark suite or other suitable benchmark suites.
Development of a Microchannel High Temperature Recuperator for Fuel Cell Systems
This report summarizes the progress made in development of microchannel recuperators for high temperature fuel cell/turbine hybrid systems for generation of clean power at very high efficiencies. Both Solid Oxide Fuel Cell/Turbine (SOFC/T) and Direct FuelCell/Turbine (DFC/T) systems employ an indirectly heated Turbine Generator to supplement fuel cell generated power. The concept extends the high efficiency of the fuel cell by utilizing the fuel cell’s byproduct heat in a Brayton cycle. Features of the SOFC/T and DFC/T systems include: electrical efficiencies of up to 65% on natural gas, minimal emissions, reduced carbon dioxide release to the environment, simplicity in design, and potential cost competitiveness with existing combined cycle power plants. Project work consisted of candidate material selection from FuelCell Energy (FCE) and Pacific Northwest National Laboratory (PNNL) institutional databases as well as from industrial and academic literature. Candidate materials were then downselected and actual samples were tested under representative environmental conditions resulting in further downselection. A microchannel thermal-mechanical model was developed to calculate overall device cost to be later used in developing a final Tier 1 material candidate list. Specifications and operating conditions were developed for both SOFC/T and DFC/T systems. This development included system conceptualization and progression to process flow diagrams (PFD’s) including all major equipment. Material and energy balances were then developed for the two types of systems which were then used for extensive sensitivity studies that used high temperature recuperator (HTR) design parameters (e.g., operating temperature) as inputs and calculated overall system parameters (e.g., system efficiency). The results of the sensitivity studies determined the final HTR design temperatures, pressure drops, and gas compositions. The results also established operating conditions and specifications for all equipment in the SOFC/T and DFC/T systems. Capital cost and Cost of Electricity (COE) sensitivity analyses have been completed for MW-scale SOFC/T and DFC/T systems. Environmental testing consisted of 1000-hour and 2000-hour dry air oxidation testing on leading candidate materials, used to rank order and, in part, develop a final Tier 1 material candidate list. A thermal-mechanical model was subsequently used to provide material and manufacturing cost estimations for microchannel HTR’s to further refine the Tier 1 candidates. A capital cost and 20-year levelized cost of electricity (COE) was developed for a MW-scale version of the SOFC/T system concept as well as for a MW-scale version of the DFC/T system concept. Test frameworks were established for subsequent long-term materials stability testing, including oxidation resistance and mechanical strength. Mechanical strength testing was then carried out by a third-party test laboratory. Technology demonstration vehicles (TDV’s) were designed and fabricated. Several iterations of TDV’s were fabricated, each improved over the previous build as far as fabrication techniques. Two of three fabricated TDV’s were integrated with the TDV Test Facility for hot-testing at simulated operating conditions. The second of these two was successfully hot-tested for over 1000 hours at simulated temperature and pressure. Post-test leakdown assessment showed negligible leakage at benchtop conditions of 30 psig, a considerable improvement over the previous TDV’s.
A streamlined build system foundation for developing HPC software
BLT bundles custom CMake macros, unit testing frameworks for C++ and Fortran, and a set of smoke tests for common HPC dependencies. The combination of these three provides a foundation for quickly bootstrapping a CMale-based system for developing HPC softward.
Large scale maximum average power multiple inference on time-course count data with application to RNA-seq analysis
Not provided.
A Cybersecurity Event Simulation Tool and Platform
Digital Instrumentation and Control Systems (ICSs) have replaced analog control systems in nuclear power plants raising cybersecurity concerns. To study and understand the cybersecurity risks of nuclear power plants both high fidelity models of the plant physics and controllers must be created, and a framework to test and evaluate cyber security events must be established. A testing and evaluation framework of cybersecurity events consists of a method of interfering with control systems, a simulation of the plant network, and a network packet capture and recording tool. Sandia National Labs (SNL) in collaboration with the University of New Mexico’s Institute for Space and Nuclear Power Studies (UNM-ISNPS) is developing such a cybersecurity testing framework.
Pyctos
Pyctos is a concolic testing framework for standard, dynamically-typed Python. Pyctos is able to generate exhaustive test inputs reaching 100% coverage for a subset of pure Python in the absence of type annotations, even where modern fuzzers would fail. Pyctos's underlying reasoning engine is the CVC5 SMT solver, though Z3 is also supported. Pyctos also supports a growing subset of the standard library and some third-party libraries, such as NumPy.
Autonomous Closed-Loop Experimental Characterization and Dynamic Model Validation of a Scaled Underwater Kite
This paper presents the closed-loop experimental framework and dynamic model validation for a 1/12-scale underwater kite design. The pool-based tow testing framework described herein, which involves a fully actuated, closed-loop controlled kite and flexible tether, significantly expands upon the capabilities of any previously developed open-source framework for experimental underwater kite characterization. Specifically, the framework has allowed for the validation of three closed-loop flight control strategies, along with a critical comparison between dynamic model predictions and experimental results. In this paper, we provide a detailed presentation of the experimental tow system and kite setup, describe the control algorithms implemented and tested, and quantify the level of agreement between our multi-degree-of-freedom kite dynamic model and experimental data. Finally, we also present a sensitivity analysis that helps to identify the most influential parameters to kite performance and further explain the remaining mismatches between the model and data.
UnitTesterSG
UnitTesterSG is a unit testing framework that is designed for nested and/or scientific/engineering data structures. It is designed primarily for testing the outputs if a single function or simulation run by storing the expected results file such that comparisons to the stored output can be made with unit tests after the function or software has been edited. This module is designed to be able to unit test arrays, strings, as well as nested and deeply nested objects (including arrays with tuples inside, arrays with strings inside, etc.). The module is compatable with nested objects, and thus has a dependency on the nestedObjectsFunctions module within. The module takes *calculated* results and then compares them to *expected* results. If no expected results are available (or they do not match the calculated results) the module then offers to store the calculated results as expected results for next time.