Search NASA⌕ Search

SEARCH · Search NASA

Results for “Test Framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27

WearGP: A computationally efficient machine learning framework for local erosive wear predictions via nodal Gaussian processes

Computational fluid dynamics (CFD)-based wear predictions are computationally expensive to evaluate, even with a high-performance computing infrastructure. Thus, it is difficult to provide accurate local wear predictions in a timely manner. Data-driven approaches provide a more computationally efficient way to approximate the CFD wear predictions without running the actual CFD wear models. In this paper, a machine learning (ML) approach, termed WearGP, is presented to approximate the 3D local wear predictions, using numerical wear predictions from steady-state CFD simulations as training and testing datasets. The proposed framework is built on Gaussian process (GP) and utilized to predict wear in a much shorter time. The WearGP framework can be segmented into three stages. At the first stage, the training dataset is built by using a number of CFD simulations in the order of O (10 2 ). At the second stage, the data cleansing and data mining process are performed, where the nodal wear solutions are extracted from the solution database to build a training dataset. At the third stage, the wear predictions are made, using trained GP processes. Two CFD case studies including 3D slurry pump impeller and casing are used to demonstrate the WearGP framework, in which 144 training and 40 testing data points are used to train and test the proposed method, respectively. The numerical accuracy, computational effiency and effectiveness between the WearGP framework and CFD wear model for both slurry pump impellers and casings are compared. It is shown that the WearGP framework can achieve highly accurate results that are comparable with the CFD results, with a relatively small size training dataset, with a computational time reduction in an order of 10 5 to 10 6 .

42 ENGINEERING↗

Real-Sim Flexible Interface for X-in-the-loop Simulation (FIXS)

Real-Sim is a multi-resolution X-in-the-loop simulation framework to support inclusive testing and evaluation of CAVs before actual implementation in the real world. Real-Sim framework connects different simulation components and integrates various traffic and vehicle simulation tools, virtual environments, and XIL systems through a Flexible Interface for XIL Simulation (FIXS). Connections among different tools are transparent to users and handled by the interface behind the scenes. Co-simulation of various vehicle and traffic simulation tools with different XIL systems can be easily achieved and become a transparent "plug-and-play" process to users.

Shao, Yunli (0000000185287925)↗

HDTS 2017.0 Testing and verification document

This report is a continuation of the series of Hunter Dose Tracking System (HDTS) Quality Assurance documents including (Foley and Powell, 2010; Dixon, 2012). In this report we have created a suite of automated test cases and a system to analyze the results of those tests as well as documented the methodology to ensure the field system performs within specifications. The software test cases cover all of the functions and interactions of functions that are practical to test. With the developed framework, if software defects are discovered, it will be easy to create one or more test cases to reproduce the defect and ensure that code changes correct the defect. These tests con rm HDTS version 2017.0 performs according to its specifications and documentation and that its performance meets the needs of its users at the Savannah River Site.

97 MATHEMATICS AND COMPUTING↗

HDTS 2017.1 Testing and Verification Document

This report is a continuation of the series of Hunter Dose Tracking System (HDTS) Quality Assurance documents including (Foley and Powell, 2010; Dixon, 2012; Whiteside, 2017b). In this report we have created a suite of automated test cases and a system to analyze the results of those tests as well as documented the methodology to ensure the field system performs within specifications. The software test cases cover all of the functions and interactions of functions that are practical to test. With the developed framework, if software defects are discovered, it will be easy to create one or more test cases to reproduce the defect and ensure that code changes correct the defect.

97 MATHEMATICS AND COMPUTING↗

Emulation Framework for Distributed Large-Scale Systems Integration

Recent trends in systems engineering include integration of very large-scale systems, which entails significant challenges when they are geographically dispersed. In these scenarios, intelligent integration of distributed large-scale systems requires significant coordination among hardware elements as well as all software components. The approach of integrated systems (both computing platform and experimental equipment) for end-to-end orchestration is called federation. Virtual frameworks can aid in the testing, assessment, and implementation of a functional system of interconnected resources. We present an emulation framework that replicates the software environments of multi-site federations of computing systems and instruments. Our emulation framework allows systems engineers to reduce developmentcost and avoid disruptions to production infrastructure. Our framework was effectively used to develop and test software modules for various tasks including container orchestration and instrument access. For performance assessment, however, the emulated framework is severely limited in providing accurate network and IO measurements at 10 Gbps and higher data rates. The data transfer performance profiles estimated using these emulated measurements are usually inaccurate for high bandwidth and high latency connections, since emulation does not accurately reflect the critical network transport dynamics.We utilize measurements from a physical testbed with hardware network emulators to obtain data transfer profiles that closely match the expected profiles for the emulated federations. We show the effectiveness of our approach by an illustrative example of integrated (federated) multi-site ultra large-scale systems that are connected via high speed wide area networks.

Imam, Neena↗

Application of Numerical Atmospheric Models

In this study, a numerical modeling framework for simulating extreme storm events was established using the Weather Research and Forecasting (WRF) model. Such a framework is necessary for the derivation of engineering parameters such as probable maximum precipitation that are the cornerstone of large water-management infrastructure design. Here, this framework was built based on a heavy storm that occurred in Nashville, Tennessee (U.S.), in 2010, and verified using two other extreme storms. To achieve the optimal setup, several combinations of model resolutions, initial/boundary conditions (IC/BC), cloud microphysics, and cumulus parameterization schemes were evaluated using multiple metrics of precipitation characteristics. The evaluation suggests that WRF is most sensitive to the IC/BC option. Simulation generally benefits from finer resolutions up to 5 km. At the 15 km level, NCEP2 IC/BC produces better results, whereas NAM IC/BC performs best at the 5 km level. The recommended model configuration from this study is: NAM or NCEP2 IC/BC (depending on data availability), 15 km or 15–5 km nested grids, Morrison microphysics, and Kain-Fritsch cumulus schemes. Validation of the optimal framework suggests that these options are good starting choices for modeling extreme events similar to the test cases. This optimal framework is proposed in response to emerging engineering demands of extreme storm event forecasting and analyses for design, operations, and risk assessment of large water infrastructures.

Chen, Xiaodong↗

The combination of an Environmental Management System and Life Cycle Assessment at the territorial level

A framework to include a Life Cycle Assessment in the significance evaluation of the environmental aspects of an Environmental Management System has been studied for some industrial sectors, but there is a literature gap at the territorial level, where the indirect impact assessment is crucial. To overcome this criticality, our research proposes the Life Cycle Assessment as a framework to assess environmental aspects of public administration within an Environmental Management System applied at the territorial level. This research is structured in two parts: the design of a new methodological framework and the pilot application for an Italian municipality. The methodological framework designed supports Initial Environmental Analysis at the territorial level thanks to the results derived from the impact assessment phase. The pilot application in an Italian municipality EMAS registered demonstrates the applicability of the framework and its effectiveness in evaluating the environmental impact assessment for direct and indirect aspects. Through the discussion of the results, we underline the growing knowledge derived by this research in terms of the reproducibility and consistency of the criteria to define the significance of the direct and indirect environmental aspects for a local public administration. - Highlights: • The combination between Environmental Management System and LCA is studied. • A methodological framework is elaborated and tested at the territorial level. • Life Cycle Impact Assessment supports the evaluation of aspects significance. • The framework assures consistency of evaluation criteria on the studied territory.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Establishing a Numerical Modeling Framework for Hydrologic Engineering Analyses of Extreme Storm Events

In this study a numerical modeling framework for simulating extreme storm events was established using the Weather Research and Forecasting (WRF) model. Such a framework is necessary for the derivation of engineering parameters such as probable maximum precipitation that are the cornerstone of large water management infrastructure design. Here this framework was built based on a heavy storm that occurred in Nashville (USA) in 2010, and verified using two other extreme storms. To achieve the optimal setup, several combinations of model resolutions, initial/boundary conditions (IC/BC), cloud microphysics and cumulus parameterization schemes were evaluated using multiple metrics of precipitation characteristics. The evaluation suggests that WRF is most sensitive to IC/BC option. Simulation generally benefits from finer resolutions up to 5 km. At the 15km level, NCEP2 IC/BC produces better results, while NAM IC/BC performs best at the 5km level. Recommended model configuration from this study is: NAM or NCEP2 IC/BC (depending on data availability), 15km or 15km-5km nested grids, Morrison microphysics and Kain-Fritsch cumulus schemes. Validation of the optimal framework suggests that these options are good starting choices for modeling extreme events similar to the test cases. This optimal framework is proposed in response to emerging engineering demands of extreme storm events forecasting and analyses for design, operations and risk assessment of large water infrastructures.

atmospheric model↗

A Networked Microgrid Framework and Testbed for Communication, Controls, and Optimization Testing

This paper presents the development and experimental results of a networked AC microgrid testbed located at Oak Ridge National Laboratory. The testbed comprises two, 480V three-phase, four wire microgrids designed to operate standalone, grid-tied, or as a network of microgrids. This testbed represents both the state of the industry, by incorporating grid-assets commonly found in real microgrids, and the state of the art, as it is a platform to evaluate advanced controllers. The main elements of this networked microgrid testbed are presented in this paper including a Scenario Manager, local microgrid controls, and a networked microgrid control. The Scenario Manager has the objective of emulating real-world conditions. The local microgrid controller oversees standalone, grid-tied, or islanded operation. The microgrid control is a higher-level control that coordinates interaction between islanded microgrids. This paper delves into these controllers and validates their operation in the networked microgrid testbed showcasing the operational flexibility and advance control capabilities.

Ferrari Maglia, Max↗

Benchmark and Framework for Encouraging Research on Multi-Threaded Testing Tools

A problem that has been getting prominence in testing is that of looking for intermittent bugs. Multi-threaded code is becoming very common, mostly on the server side. As there is no silver bullet solution, research focuses on a variety of partial solutions. In this paper (invited by PADTAD 2003) we outline a proposed project to facilitate research. The project goals are as follows. The first goal is to create a benchmark that can be used to evaluate different solutions. The benchmark, apart from containing programs with documented bugs, will include other artifacts, such as traces, that are useful for evaluating some of the technologies. The second goal is to create a set of tools with open API s that can be used to check ideas without building a large system. For example an instrumentor will be available, that could be used to test temporal noise making heuristics. The third goal is to create a focus for the research in this area around which a community of people who try to solve similar problems with different techniques, could congregate.

Havelund, Klaus↗

A framework for evaluating power electronic systems for grid integration

As power electronic systems (PESs) grow increasingly complex, integrating and testing these systems in the development stages is becoming more crucial and more challenging. New modes and resource integration strategies are in development and must be tested before pilot demonstrations and mass production. This paper proposes a framework for testing of PES technologies including early validation with integration of simulated resources. The framework is demonstrated to work in two example configurations as proof of principle.

Dean, Ben↗

Statistical interpretation of the Amerithrax “morph” assay results

In the Amerithrax investigation PCR–based “morph assays” were used to link the anthrax letters with the RMR–1029 flask at USAMRIID. Quantitative data reported for several of these assays are not consistent with Poisson sampling statistics, but instead exhibit “Taylor's Law” behavior where the variance greatly exceeds the mean. A plausible statistical model for this behavior can explain the large number of observed negative and “inconclusive” findings, and implies a high likelihood that a repository sample could contain a “morph” mutant at concentrations well above the nominal detection limit but nonetheless give a negative or inconclusive test result. A Bayesian framework relates the assay results to the probability that a sample actually contains all four morph mutants, even though it tested negative for at least one. The analysis implies that the observed false negative rate actually does not significantly weaken the conclusion that the morph assays correctly excluded all but the stocks derived from RMR–1029 as possible sources of the letter powders, at least when the test results were unambiguous. In conclusion, these findings expand upon and resolve some of the issues cited in recent reviews, and indicate the importance of developing a rigorous statistical framework for interpreting “morph” assay data.

59 BASIC BIOLOGICAL SCIENCES↗

Open-Source Framework for Data Storage and Visualization of Real-Time Experiments

Digital real time simulators (DRTS) are increasingly being used for the evaluation of power hardware and controller hardware in the laboratory prior to field deployment. Although DRTS are capable of simulating large models in real-time, it is challenging to visualize the results of large models without overdrawing or confounding the viewer. This paper provides an open-source framework for users to visualize their DRTS-based hardware-in-the-loop (HIL) experimental results in real-time. This proposed framework can be used by experimental test beds that can push data through an internet protocol based network. The proposed framework includes three main components. First, it includes the DRTS that generates and pushes the data to a relay. Second, it includes an application that serves multiple purposes, from data storage, testing, and translation of the data to a publisher/subscriber protocol. Finally, it includes libraries and applications that can be used to visualize the data by subscribing to the relay. This framework is available in open source, and it is tested using the HIL platform developed for testing advanced distribution management systems.

advanced distribution management systems↗

Ecologically inspired metrics for transitioning to a sustainable and resilient circular economy with application to multilayer plastic films

Current Circular Economy (CE) frameworks applied to product chains exhibit notable shortcomings. These include neglecting the resilience and robustness of design, requiring detailed economic and environmental data for impact assessment, and relying on qualitative rather than quantitative metrics capturing certain CE design aspects. In the current contribution, we addressed these shortcomings by developing an Ecologically inspired (Eco-inspired) Framework using the mathematical foundations of Ecological Network Analysis (ENA). While ENA metrics have previously found application in designing circular economies, particularly in Industrial Symbiosis (IS) networks, our adaptation tailors these metrics for use in product-level CE, recognizing the inherent distinctions between product-level CE and IS. Our Eco-inspired Framework comprises three key categories to provide holistic and granular-level metrics for designing product-level CE. The first set of metrics assesses circularity and resource efficiency. The second set gauges network intensity and robustness as complementary indicators ensuring a CE is both sustainable and resilient. The third group of metrics evaluates enhancement potential of CE strategies through introducing quantitative metrics for measuring the degree of closed-loop strategies and average circularity level of a CE design. The three comprehensive set of indicators within the Eco-inspired Framework uniquely captures various facets of circular design, whether originating from technological innovations and recovery improvement at the end of life (EoL), shifts in human consumption patterns, alterations in product design, or changes in business models. The framework’s application is tested in designing a CE for multilayer Polyethylene-Polyamide (PE-PA) films. Using the Eco-inspired Framework, we identified the best strategy for designing a resilient and sustainable CE for PE-PA films. A diverse set of EoL strategies along with a reduction in product consumption can improve circularity and resilience by 650% and 255%, respectively, and mitigate greenhouse gas emissions by 90%. The framework minimizes trade-offs between sustainability and circularity goals and offers insights on how to enhance each strategy for achieving a resilient and sustainable CE for products.

54 ENVIRONMENTAL SCIENCES↗

Bayesian Analysis of Congruence of Core Genes in Prochlorococcus and Synechococcus and Implications on Horizontal Gene Transfer

It is often suggested that horizontal gene transfer is so ubiquitous in microbes that the concept of a phylogenetic tree representing the pattern of vertical inheritance is oversimplified or even positively misleading. “Universal proteins” have been used to infer the organismal phylogeny, but have been criticized as being only the “tree of one percent.” Currently, few options exist for those wishing to rigorously assess how well a universal protein phylogeny, based on a relative handful of well-conserved genes, represents the phylogenetic histories of hundreds of genes. Here, we address this problem by proposing a visualization method and a statistical test within a Bayesian framework. We use the genomes of marine cyanobacteria, a group thought to exhibit substantial amounts of HGT, as a test case. We take 379 orthologous gene families from 28 cyanobacteria genomes and estimate the Bayesian posterior distributions of trees – a “treecloud” – for each, as well as for a concatenated dataset based on putative “universal proteins.” We then calculate the average distance between trees within and between all treeclouds on various metrics and visualize this high-dimensional space with non-metric multidimensional scaling (NMMDS). We show that the tree space is strongly clustered and that the universal protein treecloud is statistically significantly closer to the center of this tree space than any individual gene treecloud. We apply several commonly-used tests for incongruence/HGT and show that they agree HGT is rare in this dataset, but make different choices about which genes were subject to HGT. Our results show that the question of the representativeness of the “tree of one percent” is a quantitative empirical question, and that the phylogenetic central tendency is a meaningful observation even if many individual genes disagree due to the various sources of incongruence.

59 BASIC BIOLOGICAL SCIENCES↗

Validating automated resonance evaluation with synthetic data

The integrity and precision of nuclear data are crucial for a broad spectrum of applications, from national security and nuclear reactor design to medical diagnostics, where the associated uncertainties can significantly impact outcomes. A substantial portion of uncertainty in nuclear data originates from the subjective biases in the evaluation process, a crucial phase in the nuclear data production pipeline. Recent advancements indicate that automation of certain routines can mitigate these biases, thereby standardizing the evaluation process and enhancing reproducibility. This research aims to provide a methodology, framework, and metrics for the validation of automated nuclear data evaluation software leveraging high-quality synthetic data that closely mimic real experimental observables. An introduced error metric provides a scale and intuitive measure of the evaluation quality by quantifying the estimate’s accuracy and performance across the specified energy range. Synthetic data provides access to experimental observables and underlying resonance parameters, enabling comparison of different evaluations. The methodology is demonstrated using Ta-181 isotope data in the resolved resonance region. The Automated Resonance Identification Subroutine (ARIS), which operates without prior resonance information, was used to test and showcase the framework’s capabilities utilizing the proposed error metrics. The results demonstrate the effectiveness of the proposed approach and framework for optimizing software parameters and testing hypotheses through “what-if” controlled experiments, such as modifying assumptions about experimental conditions or average resonance parameters.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

A plan for application system verification tests: The value of improved meteorological information, volume 1

The framework within which the Applications Systems Verification Tests (ASVTs) are performed and the economic consequences of improved meteorological information demonstrated is described. This framework considers the impact of improved information on decision processes, the data needs to demonstrate the economic impact of the improved information, the data availability, the methodology for determining and analyzing the collected data and demonstrating the economic impact of the improved information, and the possible methods of data collection. Three ASVTs are considered and program outlines and plans are developed for performing experiments to demonstrate the economic consequences of improved meteorological information. The ASVTs are concerned with the citrus crop in Florida, the cotton crop in Mississippi and a group of diverse crops in Oregon. The program outlines and plans include schedules, manpower estimates and funding requirements.

Source record↗

ASC Data Science Infrastructure Project June 2023 [Slides]

Data Science Infrastructure (DSI) Project offers a standardized but flexible way of storing data and associated metadata to support AI/ML and data intensive scientific workflows: Establish requirements via use case interviews; Develop DSI framework to support flexible, secure, and accessible data stores tailored to researcher needs; Augment existing institutional frameworks for code management, performance testing and file indexing via automated DSI interfaces; Deploy DSI framework to support key use cases in collaboration with domain scientists.

97 MATHEMATICS AND COMPUTING↗