Search NASA⌕ Search

SEARCH · Search NASA

Results for “test frameworks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Co-simulation of transactive energy markets: A framework for market testing and evaluation

The proliferation of distributed energy resources (DER)—and the ability to intelligently control these assets—is re-defining the electrical distribution system. As the number of controllable devices rapidly expands, grid operators must determine how to incorporate these assets while delivering reliable, equitable, and affordable electricity. One possible approach is to establish distribution-level electricity markets and allow devices/aggregations of devices to participate in price establishment. While this approach purports some of the same benefits as the highly successful wholesale electricity markets (i.e., open competition, efficient price discovery, reduced communication overhead), this needs to be researched and quantified via an analysis platform that models distribution-level markets at the appropriate fidelity. Specifically, the simultaneous evaluation of market performance, DER performance, DER bidding approaches, and distribution feeder power quality requires modeling that spans multiple technical areas. Co-simulation has emerged as a powerful tool in addressing this type of problem, where outputs depend on a range of underlying areas of expertise and associated models. In this paper we describe a solution, as implemented in the HELICS co-simulation platform, where we include (1) high fidelity house models, (2) intelligent bidding agents, (3) a modular market integration/design, and (4) a distribution feeder model. We then present a case study where we test two different market designs: (1) a pseudo-wholesale double-blind auction, and (2) an asynchronous matching market. In this work, the markets are run under two DER penetration levels and economic results are compared to full retail net energy metering and avoided cost net metering scenarios that bookend current approaches to remuneration of DER participation. We show the potential for transactive markets to provide increased value for most customers relative to net metering (and all customers relative to avoided cost scenarios) while decreasing costs for the utility.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2009

Towards a Framework for Generating Tests to Satisfy Complex Code Coverage in Java Pathfinder

We present work on a prototype tool based on the JavaPathfinder (JPF) model checker for automatically generating tests satisfying the MC/DC code coverage criterion. Using the Eclipse IDE, developers and testers can quickly instrument Java source code with JPF annotations covering all MC/DC coverage obligations, and JPF can then be used to automatically generate tests that satisfy these obligations. The prototype extension to JPF enables various tasks useful in automatic test generation to be performed, such as test suite reduction and execution of generated tests.

Staats, Matt↗
DOE OSTI2020

OpenCGRA: An Open-Source Unified Framework for Modeling,Testing, and Evaluating CGRAs

Coarse-grained reconfigurable arrays (CGRAs),loosely defined as arrays of functional units (e.g, adder, sub-tractor, multiplier, divider, or larger multi-operation units, butsmaller than a general-purpose core) interconnected through aNetwork-on-Chip, provide higher flexibility than domain-specificASIC accelerators while offering increased hardware efficiencywith respect to fine-grained reconfigurable devices, such as FieldProgrammable Gate Arrays (FPGAs). The fast evolving fieldsof machine learning and edge computing, which are seeing acontinuous flow of novel algorithms and larger models, makeCGRAs ideal target architectures to allow domain specializationwithout loosing too much generality. They also generally offerquicker and more effective reconfigurability than FPGAs, po-tentially allowing adaptation during actual algorithm execution,and implement a dataflow programming paradigm that adaptswell to these emerging workloads. Designing and generating aCGRA, however, still requires to define the type and number ofthe specific functional units, implement their interconnect andthe network topology, and perform its simulation and validation,given a variety of workloads of interest.In this paper, we propose OpenCGRA, a Python-based unifiedframework that integrates generation, modeling, testing and eval-uation for CGRAs. OpenCGRA is the first open-source integratedframework able to support the full top-to-bottom design flow forspecializing and implementing CGRAs: modeling at different ab-straction levels (functional level, cycle level, register-transfer level),generation, simulation, testing at different granularities (unit test-ing, integration testing, property-based testing), and characteriza-tion (area, power, and timing). OpenCGRAs will be made availableon GitHub.

CGRA, synthesis↗
NASA NTRS2019

A Distributed Hierarchical Framework for Autonomous Spacecraft Control

Future human space missions for exploring beyond low Earth orbit are in the conceptual design stage. One such mission describes a habitat in cis-lunar orbit that is visited by crew periodically, others describe missions to Mars. These missions have one important thing in common: the need for autonomy on the spacecraft. This need stems from the latency and bandwidth constraints on communications between the vehicle and ground control. A variable amount of autonomy may be necessary whether the spacecraft has crew on board or not. Spacecraft are complex systems that are engineered as a collection of subsystems. These subsystems work together to control the overall state of the spacecraft. As such, solutions that increase the autonomy of the spacecraft (called autonomous functions) should respect both the independence and interconnectedness of the spacecraft subsystems. This distributed and hierarchical approach to system monitoring and control is a key idea in the Modular Autonomous Systems Technology (MAST) framework. The MAST framework enables a component-based architecture that provides interfaces and structure to developing autonomous technologies. The framework enforces a distributed, hierarchical architecture for autonomous control systems across subsystems, systems, elements, and vehicles. An example autonomous system was implemented in this framework and tested using realistic spacecraft software and hardware simulations. This paper will discuss the framework, tests conducted, results, and future work.

Badger, Julia M.↗
DOE OSTI2020

A Multi-Instance learning Framework for Seismic Detectors

In this report, we construct and test a framework for fusing the predictions of a ensemble of seismic wave detectors. The framework is drawn from multi-instance learning and is meant to improve the predictive skill of the ensemble beyond that of the individual detectors. We show how the framework allows the use of multiple features derived from the seismogram to detect seismic wave arrivals, as well as how it allows only the most informative features to be retained in the ensemble. The computational cost of the "ensembling" method is linear in the size of the ensemble, allowing a scalable method for monitoring multiple features/transformations of a seismogram. The framework is tested on teleseismic and regional p-wave arrivals at the IMS (International Monitoring System) station in Warramunga, NT, Australia and the PNSU station in University of Utah's monitoring network.

58 GEOSCIENCES↗
DOE OSTI2019

Prototyping the BOPTEST Framework for Simulation-Based Testing of Advanced Control Strategies in Buildings

Advanced control strategies are becoming increasingly necessary in buildings in order to meet and balance requirements for energy efficiency, demand flexibility, and occupant comfort. Additional development and widespread adoption of emerging control strategies, however, ultimately require low implementation costs to reduce payback period and verified performance to gain control vendor, building owner, and operator trust. This is difficult in an already first-cost driven and risk-averse industry. Recent innovations in building simulation can significantly aid in meeting these requirements and spurring innovation at early stages of development by evaluating performance, comparing state-of-the-art to new strategies, providing installation experience, and testing controller implementations. This paper presents the development of a simulation framework consisting of test cases and software platform for the testing of advanced control strategies (BOPTEST - Building Optimization Performance Test). The objectives and requirements of the framework, components of a test case, and proposed software platform architecture are described, and the framework is demonstrated with a prototype implementation and example test case.

Blum, David↗
DOE OSTI2022

Resolve the Clear-Sky Continuous Diurnal Cycle of High-Resolution ECOSTRESS Evapotranspiration and Land Surface Temperature

Although numerous evapotranspiration (ET) estimates have been developed from remote sensing measurements, a diurnally resolved product with high spatial resolution (<100 m) is still lacking, which is critically needed for agricultural and ecological monitoring. The ECOsystem Spaceborne Thermal Radiometer Experiment on Space Station (ECOSTRESS) provides, for the first time, ET at 70 m resolution with diurnal samplings, enabling a step toward filling this data gap. This study developed the first framework to resolve the full diurnal cycle of ET and Land Surface Temperature (LST) from sporadic ECOSTRESS measurements. We first constructed 70 m diurnal LST utilizing a diurnal temperature cycle (DTC) model that fused ECOSTRESS and Geostationary Operational Environmental Satellite (GOES) LST. Next, we derived 70 m diurnal ET from the diurnally resolved LST, along with ancillary meteorological and reflectance data sets, using the Priestley-Taylor Jet Propulsion Laboratory (PT-JPL) algorithm. This framework was tested in two regions: A relatively homogeneous semiarid grassland landscape (centered at the US-Seg Ameriflux site) and a heterogeneous humid cropland landscape (centered at US-ARM). Our diurnally resolved LST and ET successfully reproduced the spatial patterns in the native ECOSTRESS measurements during overpasses for both landscapes (LST: r > 0.97, RMSE ~2°C; ET: r > 0.99, and RMSE = 1–7 W m -2 ). The constructed time series also well captured the temporal variability of the in situ measurements (especially under clear-sky conditions) at both US-Seg and US-ARM, albeit performance varying with availability and quality of ECOSTRESS samples. This study sets the stage for testing and applying our framework to broader climates, biomes, and landscapes toward eventually generating diurnally resolved 70 m global operational LST/ET products to enhance ecological/agricultural applications.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1972

Einstein on the firing line.

A theoretical framework for testing theories of gravitation is considered. This framework separates theories into those that are viable and those that are nonviable by sending each through a series of tests. Most of these tests compare the theory's predictions with the results of experiment, but one test judges each theory's completeness, self-consistency and agreement with Newtonian physics at lowest order. Only a few theories, including general relativity, are currently viable, that is, pass all five tests in the framework. Einstein's 1912 theory is also ruled out because it predicts only half the observed time-delay effect. Future tests proposed include improvements of old experiments and a number of new experiments.

Will, C. M.↗
NASA NTRS2003

A Framework for Intelligent Rocket Test Facilities with Smart Sensors

A long-term center goal at the John C. Stennis Space Center (SSC) is the formulation and implementation of a framework for an Intelligent Rocket Test Facility (IRTF), which incorporates distributed smart sensor elements. The IRTF is to provide reliable, high-confident measurements. Specific objectives include: 1. Definition of a framework and architecture that supports implementation of highly autonomous methodologies founded on basic physical principles and embedded knowledge. 2. Modeling of autonomous sensors and processes as self-sufficient, evolutionary elements. 3. Development of appropriate communications protocols to enable the complex interactions that must take place to allow timely and high-quality flow of of information among all the autonomous elements of the system. 4. Development of lab-scale prototypes of key system elements. Though our application is next-generation rocket test facilities, applications for the approach are much wider and include monitoring of shuttle launch operations, air and spacecraft operations and health monitoring, and other large-scale industrial system operations such as found in processing and manufacturing plans. Elements of prototype IRTF have been implemented in preparation for advanced development and validation using rocket test stand facilities as SSC. This work has identified issues that are important to further development of complex network and should be of interest to other working with sensor networks.

Figueroa, Fernando↗
DOE OSTI2023

End-to-end AI framework for interpretable prediction of molecular and crystal properties

We introduce an end-to-end computational framework that allows for hyperparameter optimization using the DeepHyper library, accelerated model training, and interpretable AI inference. The framework is based on state-of-the-art AI models including CGCNN, PhysNet, SchNet, MPNN, MPNN-transformer, and TorchMD-NET. We employ these AI models along with the benchmark QM9, hMOF, and MD17 datasets to showcase how the models can predict user-specified material properties within modern computing environments. We demonstrate transferable applications in the modeling of small molecules, inorganic crystals and nanoporous metal organic frameworks with a unified, standalone framework. We have deployed and tested this framework in the ThetaGPU supercomputer at the Argonne Leadership Computing Facility, and in the Delta supercomputer at the National Center for Supercomputing Applications to provide researchers with modern tools to conduct accelerated AI-driven discovery in leadership-class computing environments. We release these digital assets as open source scientific software in GitLab, and ready-to-use Jupyter notebooks in Google Colab.

36 MATERIALS SCIENCE↗
DOE OSTI2017

Bayesian Inversion of Seismic and Electromagnetic Data for Marine Gas Reservoir Characterization Using Multi-chain Markov Chain Monte Carlo Sampling

In this study we developed an efficient Bayesian inversion framework for interpreting marine seismic amplitude versus angle (AVA) and controlled source electromagnetic (CSEM) data for marine reservoir characterization. The framework uses a multi-chain Markov-chain Monte Carlo (MCMC) sampler, which is a hybrid of DiffeRential Evolution Adaptive Metropolis (DREAM) and Adaptive Metropolis (AM) samplers. The inversion framework is tested by estimating reservoir-fluid saturations and porosity based on marine seismic and CSEM data. The multi-chain MCMC is scalable in terms of the number of chains, and is useful for computationally demanding Bayesian model calibration in scientific and engineering problems. As a demonstration, the approach is used to efficiently and accurately estimate the porosity and saturations in a representative layered synthetic reservoir. The results indicate that the seismic AVA and CSEM joint inversion provides better estimation of reservoir saturations than the seismic AVA-only inversion, especially for the parameters in deep layers. The performance of the inversion approach for various levels of noise in observational data was evaluated – reasonable estimates can be obtained with noise levels up to 25%. Sampling efficiency due to the use of multiple chains was also checked and was found to have almost linear scalability.

Ren, Huiying↗
NASA NTRS2012

Nemesis Autonomous Test System

A generalized framework has been developed for systems validation that can be applied to both traditional and autonomous systems. The framework consists of an automated test case generation and execution system called Nemesis that rapidly and thoroughly identifies flaws or vulnerabilities within a system. By applying genetic optimization and goal-seeking algorithms on the test equipment side, a "war game" is conducted between a system and its complementary nemesis. The end result of the war games is a collection of scenarios that reveals any undesirable behaviors of the system under test. The software provides a reusable framework to evolve test scenarios using genetic algorithms using an operation model of the system under test. It can automatically generate and execute test cases that reveal flaws in behaviorally complex systems. Genetic algorithms focus the exploration of tests on the set of test cases that most effectively reveals the flaws and vulnerabilities of the system under test. It leverages advances in state- and model-based engineering, which are essential in defining the behavior of autonomous systems. It also uses goal networks to describe test scenarios.

Barltrop, Kevin J.↗
DOE OSTI2023

An open-source framework for simulation-based testing of buildings control strategies

Here this paper presents a simulation framework for evaluating building control strategies (BCSs), developed with VOLTTRON, an opensource platform that integrates data, devices, and systems for sensing and control applications, and is based on co-simulation interfaces. It realizes an integrated environment for both testing and deploying BCSs, thereby eliminating the need for having a dedicated implementation of BCS for testing. For the first time, this framework provides critical functionalities, including scalable communication management and time-drive simulation advance, i.e., advancing the simulation based on clock time. We applied this framework to evaluating two BCSs from ASHRAE Guideline 36-2008 and ASHRAE Standard 90.1-2019. The evaluation results show that those BCSs primarily benefit the heating operation by reducing gas usage but yield insignificant savings in electricity consumption. The results also emphasize the importance of tuning the parameters of the BCSs to achieve better performances.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2022

Framework for Estimating Performance and Associated Uncertainty for Modified Aircraft Configurations

Flight testing has been the historical standard for determining aircraft airworthiness - however, increases in the cost of flight testing and the accuracy of inexpensive CFD promote certification by analysis to reduce or replace flight testing. A framework is introduced to predict the performance in the special case of a modification to an existing, previously certified aircraft. This framework uses a combination of existing flight test or high fidelity data of the original aircraft as well as lower fidelity data of the original and modified configurations. Two methods are presented which estimate the model form uncertainty of the modified configuration, which is then used to conduct non-deterministic simulations. The framework is applied to an example aircraft system with simulated flight test data to demonstrate the ability to predict the performance and associated uncertainty of modified aircraft configurations. However, it is important that the models and methods used are applicable and accurate throughout the intended use domain. The factors and limitations of the framework are explored to determine the range of applicability of the framework. The effects of these factors on the performance and uncertainty results are demonstrated using the example aircraft system. The framework is then applied to NASA’s X-57 Maxwell and each of its modifications. The estimated performance and associated uncertainties are then compared to the airworthiness criteria to evaluate the potential of the framework as a component to the certification by analysis process.

Uncertainty Quantification↗
DOE OSTI2017

Testing of the SIDIS framework at Jefferson Lab Hall C using precise measurements of light meson electroproduction

Semi-inclusive deep inelastic scattering (SIDIS) has proved to be an important tool for unveiling the inner structure of the core of the atom, the nucleons and their basic constituents. Azimuthal spin asymmetries in polarized SIDIS are directly related to transverse momentum dependent parton distributions (TMDs) and fragmentation functions. The TMDs allow one to probe the 3D dynamical structure of partons inside the nucleon. Understanding the nature of the SIDIS process is essential for TMD studies, in particular at modest energies where deviations from the leading-order factorized picture may be a significant. Precise maps of the meson cross sections and their ratios at low transverse momentum and Jefferson Lab beam energies provide a stringent test of the use of the standard SIDIS framework in terms of factorized parton distributions convoluted with fragmentation functions. The magnetic spectrometers in Hall C at Jefferson Lab are well suited for such precision measurements of fully L/T separated cross sections and their ratios. The addition of neutral particle detection enables additional opportunities. Plans for upcoming experiments that will investigate the SIDIS process at the 12 GeV Jefferson Lab will be presented.

Kinney, Edward↗
NASA NTRS2022

Framework for Estimating Performance and Associated Uncertainty for Modified Aircraft Configurations

Flight testing has been the historical standard for determining aircraft airworthiness. However, increases in the cost of flight testing and the accuracy of inexpensive CFD encourage the adoption of certification by analysis to reduce or replace flight testing. A framework is introduced to predict the performance in the special case of a modification to an existing, previously certified aircraft. This framework uses a combination of existing flight tests or high fidelity data of the original aircraft as well as lower fidelity data from CFD or wind tunnel testing of the original and modified configurations to create 6-DOF flight dynamics models. Two methods are presented which generate an updated flight dynamics model and estimate the model form uncertainty for the modified aircraft configuration using knowledge of the original aircraft. This updated dynamics model and uncertainty estimate are then used to conduct non-deterministic simulations with wind turbulence included. The framework is applied to an example aircraft system to demonstrate the ability to predict the performance and associated model from the uncertainty of modified aircraft configurations.

uncertainty quantification↗