Search NASA⌕ Search

SEARCH · Search NASA

Results for “Test Framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,513 records · Page 84

Evaluation of Seismic Artificial Intelligence with Uncertainty

Artificial intelligence has transformed the seismic community with deep learning models (DLMs) that are trained to complete specific tasks within workflows. However, there is still a lack of robust evaluation frameworks for evaluating and comparing DLMs. Here, we address this gap by designing an evaluation framework that jointly incorporates two crucial aspects: performance uncertainty and learning efficiency. To target these aspects, we meticulously construct the training, validation, and test splits using a clustering method tailored to seismic data and enact an expansive training design to segregate performance uncertainty arising from stochastic training processes and random data sampling. The framework’s ability to guard against misleading declarations of model superiority is demonstrated through the evaluation of PhaseNet (Zhu and Beroza, 2018), a popular seismic phase picking DLM, under three training approaches. Our framework helps practitioners choose the best model for their problem and set performance expectations by explicitly analyzing model performance with uncertainty at varying budgets of training data.

58 GEOSCIENCES↗

Tetris-inspired detector with neural network for radiation mapping

Abstract Radiation mapping has attracted widespread research attention and increased public concerns on environmental monitoring. Regarding materials and their configurations, radiation detectors have been developed to identify the position and strength of the radioactive sources. However, due to the complex mechanisms of radiation-matter interaction and data limitation, high-performance and low-cost radiation mapping is still challenging. Here, we present a radiation mapping framework using Tetris-inspired detector pixels. Applying inter-pixel padding for enhancing contrast between pixels and neural networks trained with Monte Carlo (MC) simulation data, a detector with as few as four pixels can achieve high-resolution directional prediction. A moving detector with Maximum a Posteriori (MAP) further achieved radiation position localization. Field testing with a simple detector has verified the capability of the MAP method for source localization. Our framework offers an avenue for high-quality radiation mapping with simple detector configurations and is anticipated to be deployed for real-world radiation detection.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A Bayesian framework to investigate radiation reaction in strong fields

Recent experiments aiming to measure phenomena predicted by strong-field quantum electrodynamics (SFQED) have done so by colliding relativistic electron beams and high-power lasers. In such experiments, measurements of collision parameters are not always feasible. However, precise knowledge of these parameters is required to accurately test SFQED. Here, we present a novel Bayesian inference procedure that infers collision parameters that could not be measured on-shot. This procedure is applicable to all-optical non-linear Compton scattering experiments investigating radiation reaction. The framework allows multiple diagnostics to be combined self-consistently and facilitates the inclusion of known information pertaining to the collision parameters. Using this Bayesian analysis, the relative validity of the classical, quantum-continuous and quantum-stochastic models of radiation reaction was compared for several test cases, which demonstrates the accuracy and model selection capability of the framework and highlight its robustness if the experimental values of fixed parameters differ from their values in the models.

47 OTHER INSTRUMENTATION↗

Specificity and Transfer in Learning How to Follow Navigation Instructions

We report a series of experiments that use a navigation task in which instructions for navigating in a space displayed as grids on a computer screen are given to subjects who then attempt to follow them by mouse clicking on the grids. The navigation task was broken down into component dimensions (e.g., presentation mode of the instructions, length of the instructions, characteristics of the display, size of the grids, response type). For each task dimension, one condition was used at training and the same or another condition was used at test. Each task dimension was examined in terms of two measures. One measure provided an index of transfer (i.e., better performance at test than at training when test and training involved different conditions), and the other provided an index of specificity (i.e., better performance at test when training and test conditions were the same than when training and test conditions were different). By and large, these two indices were complementary, so there was evidence of either transfer or specificity but not both. For one dimension transfer but no specificity was evident, and for another dimension specificity but no transfer was evident. For the remaining dimensions, however, there was asymmetrical transfer, with transfer evident for some conditions and specificity evident for others. The findings are interpreted within the procedural reinstatement framework. They have practical implications concerning how to optimize training and how much fidelity to the testing situation is necessary when training.

Healy, Alice F.↗

An Erlang-Based Front End Framework for Accelerator Controls

We have developed a new front-end framework for the ACNET control system in Erlang. Erlang is a functional programming language developed for real-time telecommunications applications. The primary task of the front-end software is to connect the control system with drivers collecting data from individual field bus devices. Erlang's concurrency and message passing support have proven well-suited for managing large numbers of independent ACNET client requests for front-end data. Other Erlang features which make it particularly well-suited for a front-end framework include fault-tolerance with process monitoring and restarting, real-time response,and the ability to change code in running systems. Erlang's interactive shell and dynamic typing make writing and running unit tests an easy part of the development process. Erlang includes mechanisms for distributing applications which we will use for deploying our framework to multiple front-ends, along with a configured s et of device drivers. We've developed Erlang code to use Fermilab's TCLK event distribution clock and Erlang's interface to C/C++ allows hardware-specific driver access.

43 PARTICLE ACCELERATORS↗

DESCQA: An Automated Validation Framework for Synthetic Sky Catalogs

The use of high-quality simulated sky catalogs is essential for the success of cosmological surveys. The catalogs have diverse applications, such as investigating signatures of fundamental physics in cosmological observables, understanding the effect of systematic uncertainties on measured signals and testing mitigation strategies for reducing these uncertainties, aiding analysis pipeline development and testing, and survey strategy optimization. The list of applications is growing with improvements in the quality of the catalogs and the details that they can provide. Given the importance of simulated catalogs, it is critical to provide rigorous validation protocols that enable both catalog providers and users to assess the quality of the catalogs in a straightforward and comprehensive way. For this purpose, we have developed the DESCQA framework for the Large Synoptic Survey Telescope Dark Energy Science Collaboration as well as for the broader community. The goal of DESCQA is to enable the inspection, validation, and comparison of an inhomogeneous set of synthetic catalogs via the provision of a common interface within an automated framework. Here in this paper, we present the design concept and first implementation of DESCQA. In order to establish and demonstrate its full functionality we use a set of interim catalogs and validation tests. We highlight several important aspects, both technical and scientific, that require thoughtful consideration when designing a validation framework, including validation metrics and how these metrics impose requirements on the synthetic sky catalogs.

79 ASTRONOMY AND ASTROPHYSICS↗

Autonomous nondestructive evaluation of resistance spot welded joints

The application of non-destructive evaluation approaches has attracted strong interests in modern automotive industries. Here, we present an autonomous deep-computing framework to analyze raw videos from infrared systems and to predict weld nugget shape and size with unprecedented accuracy and speed. In a comprehensive training and testing experiment with 90 videos (seven sets of welding material stack-ups), a new method was developed to assemble sufficient datasets for neural network training. Our framework successfully predicts all the nugget shapes with F1 scores that range from 0.84 to 0.92. The total training time on Nvidia DGX station takes less than 10 min for each set of welding material stack-up. The real inference time of an individual dataset (with 30 video frames) takes about 0.005 s. The procedure and methods developed in the study can be applied to other image-based weld property prediction, as well as other manufacturing processes. Furthermore, our well-trained neural networks take limited memory resources (2.3 MB) and are suitable for embedded microprocessors for in-situ welding quality control as edge computing within an intelligent welding framework.

42 ENGINEERING↗

Database Design Strategies for Coordinated Simulation and Testing in Additive Manufacturing

The qualification and certification (Q&C) process presents a significant challenge for widespread adoption of additive manufacturing (AM) materials and processes for aerospace applications. A relational database framework will be presented as a tool for data curation of coordinated experimental and computational materials modeling research activities. A comparison of relational and hierarchical data structures in this domain will be emphasized through the evolution of a database design strategy. This framework’s mission is to support the advancement of computational materials-informed Q&C by providing the necessary data infrastructure to trace reliability and reproducibility measures through unified AM materials simulation and experimental testing. FAIR (findable, accessible, interoperable, and reusable) data will be highlighted as a necessary precursor for automation of specific actions, which ultimately reduces the time and expense burden for Q&C. The discussion will be mostly limited to back-end design elements, though a few front-end user experience examples will also be shared.

Qualification↗

Decoy selection for protein structure prediction via extreme gradient boosting and ranking

Background: Identifying one or more biologically-active/native decoys from millions of non-native decoys is one of the major challenges in computational structural biology. The extreme lack of balance in positive and negative samples (native and non-native decoys) in a decoy set makes the problem even more complicated. Consensus methods show varied success in handling the challenge of decoy selection despite some issues associated with clustering large decoy sets and decoy sets that do not show much structural similarity. Recent investigations into energy landscape-based decoy selection approaches show promises. However, lack of generalization over varied test cases remains a bottleneck for these methods. Results: We propose a novel decoy selection method, ML-Select, a machine learning framework that exploits the energy landscape associated with the structure space probed through a template-free decoy generation. The proposed method outperforms both clustering and energy ranking-based methods, all the while consistently offering better performance on varied test-cases. Moreover, ML-Select shows promising results even for the decoy sets consisting of mostly low-quality decoys. Conclusions: ML-Select is a useful method for decoy selection. This work suggests further research in finding more effective ways to adopt machine learning frameworks in achieving robust performance for decoy selection in template-free protein structure prediction.

59 BASIC BIOLOGICAL SCIENCES↗

Varying alpha and cosmic acceleration in Brans-Dicke-BSBM theory: stability analysis and observational tests

By integration of generalized BSBM and Brans-Dicke cosmological models, in this article, we investigate the theoretical framework of fine structure constant variation and current cosmic acceleration. We first develop a mathematical formalism to analyse the stability of the model. By employing observational data to constrain the model parameters, phase space is performed and the attractor solutions of the model are detected. We then examine the model against observational data such as observational Hubble parameter dataset and quasar absorption spectra. The results confirm current universe acceleration and also predicts fine structure constant variation. Furthermore, extrapolation of the best fitted model in high redshift (z > 15) illustrates a significantly larger variation of fine structure constant in earlier epoch of the universe.

79 ASTRONOMY AND ASTROPHYSICS↗

Testing self-energy embedding theory in combination with GW

In this work, we present a theoretical framework and implementation details for self-energy embedding theory (SEET) with the GW approximation for the treatment of weakly correlated degrees of freedom and configuration interactions solver for handing the strongly correlated degrees. On a series of molecular examples, for which the exact results are known within a given basis, we demonstrate that SEET(CI/GW) is a systematically improvable and well controlled method capable of giving accurate results and well behaved causal self-energies and Green's functions. We compare the theoretical framework of SEET(CI/GW) to that of GW+DMFT and comment on differences between these to approaches that aim to treat both strongly and weakly correlated simultaneously.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Fabrication and test of COMB dipole magnet with STAR® wires

Fermilab is working on superconducting accelerator magnet R&D under the framework of the U.S. MagnetDevelopment Program (USMDP). An integral part of that program is High Temperature Superconducting (HTS)accelerator magnet development to- demonstrate HTS self‐fields of 5 T or greater- be compatible with operation in hybrid LTS/HTS configuration• to probe fields beyond 16 T for future High Energy Physics (HEP) experiments

43 PARTICLE ACCELERATORS↗

Resilience Metrics Framework for Solar Photovoltaics

This presentation was given at the Photovoltaic Specialist Conference (PVSC) 54 in New Orleans, Louisiana. Photovoltaic (PV) systems are routinely exposed to extreme weather, including wind and hail storms. Historically, most systems have proven to be resilient to such events, but some storms have damaged PV systems, leading to physical and financial loss. Storm hardening measures and specific system attributes can reduce this risk. This work introduces a set of resilience metrics and a framework for quantifying, comparing, and predicting PV system resilience. The framework is divided into two parts: 1) predictive, attribute metrics based on site and component characteristics, and 2) impact metrics that assess post-storm performance. Metrics are weighted and aggregated, producing hazard-specific resilience scores. We derive damage functions from storm-impacted PV systems, establishing a baseline against which post-storm performance can be compared. This damage was widely variable across hail and wind intensities, and field hail damage was less than predicted by laboratory tests, suggesting that system features - in addition to storm conditions - influence damage likelihood. Finally, the metrics framework is demonstrated using three case studies of storm damaged PV systems. Although additional data are needed to create attribute specific damage functions and establish metric weights, this study presents a methodology for evaluating PV resilience and contributes new damage functions to the literature.

14 SOLAR ENERGY↗

Development of Control System Functional Capabilities within the IES Plug-and-Play Simulation Environment

The concept of an Integrated Energy System (IES) is meant to combine different energy technologies in synergistic ways to achieve a more secure and economical energy supply. The RAVEN-based HYBRID framework is used to find the optimal installed capacity and the optimal economical dispatch of each component of the IES. A new RAVEN (Risk Analysis Virtual ENvironment) plugin for grid and capacity optimization (HERON) has been developed for optimizing the production variables of the IES given the demand profile. Currently, only the limits that affect the production variables and their corresponding time rates of change are considered (explicit constraints). However, other variables are additionally subject to constraints, but the associated limits are not accounted for (implicit constraints). In particular, for the power dispatch problem, the optimization algorithm takes into account the limits on the electrical power output and the corresponding hourly power variations but does not consider other constraints on process variables whose response effects the service life of the IES. This report describes a scheme that allows accounting for implicit constraints without increasing the size of the optimization problem. The Reference Governor (RG) algorithm is traditionally used for enforcing state and control constraints by modifying the set-point trajectories supplied to the feedback regulators. In our application, the RG is coupled within an iterative loop with the HERON-power dispatcher to generate optimal trajectories that ensure the operational constraints are met. A data-driven procedure to derive a representation of the dynamics of the controlled system was developed. First, the variables that represented the state of the system are selected (PCA-based approach), and then state-space representation matrices are derived from the collected measurements (DMDc algorithm). A preliminary version of the developed workflow based on Linear Time Invariant matrices was assessed by adopting a two-unit test case. More sophisticated versions of this workflow foreseeing the on-line derivation of system matrices will be deployed in FY 2021. Finally, a “plug-and-play” library of controllers and state observers was developed in Dymola. Some aspects of the current configuration of the IES unit components, e.g., the encapsulation of the control schemes into dedicated blocks, are consistent with the “plug-and-play” philosophy. Other features, e.g., the system buses collecting the input and the output variables, are not. For this reason, once listed and described the limits of the current configuration, necessary modifications to the plant model interface are presented. As a test-case, the interfaces of the SES model in the RAVEN-based HYBRID framework were reworked accordingly, and two different control schemes were applied to the same plant model.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Target Detection via Cognitive Radars Using Change-Point Detection, Learning, and Adaptation

Many radar detection algorithms that assume a stationary environment (clutter) have been proposed and analyzed over the years. However, in practice, changes in the nonstationary environment can perturb the parameters of the clutter distribution, or even alter the clutter distribution family, which can greatly deteriorate the target detection capability. To avoid such potential performance degradation, cognitive radar systems are envisioned which are required to rapidly realize the nonstationarity, accurately learn the new characteristics of the environments, and adaptively update the detector. In this paper, aiming to develop a fully cognitive radar for target detection in nonstationary environments, we propose a unifying framework that integrates (i) change-point detection of clutter distributions by using a data-driven cumulative sum (CUSUM) algorithm and its extended version, (ii) learning/identification of clutter distribution by applying sparse theory and kernel density estimation methods, and (iii) adaptive target detection by automatically modifying the likelihood-ratio test and corresponding detection threshold. Further, with extensive numerical examples, we demonstrate the achieved improvements in detection performance due to the proposed framework in comparison to a nonadaptive case, an adaptive matched filter (AMF) method, and the clairvoyant case. Herein, we also use Wilcoxon rank-sum tests to evaluate the statistical significance of the performance improvements

42 ENGINEERING↗

Tectonics of Lakshmi Planum, Venus - Tests for Magellan

The origin of Lakshmi Planum and its surrounding mountain belts remains an important unresolved element in the global tectonic framework of Venus. From the perspective of gravity signature and potential driving forces, the mantle upwelling model is the simplest, and its principal failure, that it cannot produce radial shortening on the uplift periphery, may be resolved if the lithosphere is laterally heterogeneous. The preferred model consists of a hot mantle plume rising beneath a preexisting block of tessera. The lithosphere is weakened at this hotter and presumably thicker crust, and the outward near-surface flow is attenuated at the peripheral discontinuity in lithospheric strength. Crustal thickening and mountain belt formation occur there. Several criteria are proposed to test this 'tessera-plume' model together with its competitors at the higher resolution in both imaging and gravity afforded by the Magellan mission.

Grimm, Robert E.↗

AN INDUCTIVE MAPPING WITH CONVOLUTIONAL REPRESENTATIONS FOR HUMAN SETTLEMENT DETECTION: PRELIMINARY RESULTS

Undoubtedly, deep convolutional learning methods continue to improve performance in image-level classification for computer vision and remote sensing applications. However, the spatio-temporal nature of remote sensing imagery offers other interesting challenges in multiscale image understanding. Emerging opportunities include seeking fine-grained and neighborhood mapping with overhead imagery. Limitations due to lack of relevant scale ground-truth often mandates that these challenges are pursued as disjoint. We test this premise and explore representation spaces from a single deep convolutional network and their visualization to argue for a unified feature extraction framework. The objective is to utilize and re-purpose trained feature extractors without the need for network retraining on three remote sensing tasks i.e. superpixel mapping, pixel-level segmentation and semantic based image visualization. By leveraging the same convolutional feature extractors and viewing them as visual information extractors that encode different image representation spaces, we demonstrate surprising potential on multiscale experiments that incorporate edge-level details up to semantic-level informationWe test this premise and explore representation spaces from a single deep convolutional network and their visualization to argue for a novel unified feature extraction framework. The objective is to utilize and re-purpose trained feature extractors without the need for network retraining on three remote sensing tasks i.e. superpixel mapping, pixel-level segmentation and semantic based image visualization. By leveraging the same convolutional feature extractors and viewing them as visual information extractors that encode different image representation spaces, we demonstrate a preliminary inductive transfer learning potential on multiscale experiments that incorporate edge-level details up to semantic-level informationWe test this premise and explore representation spaces from a single deep convolutional network and their visualization to argue for a novel unified feature extraction framework. The objective is to utilize and re-purpose trained feature extractors without the need for network retraining on three remote sensing tasks i.e. superpixel mapping, pixel-level segmentation and semantic based image visualization. By leveraging the same convolutional feature extractors and viewing them as visual information extractors that encode different image representation spaces, we demonstrate a preliminary inductive transfer learning potential on multiscale experiments that incorporate edge-level details up to semantic-level information.

Lunga, Dalton↗