Search NASA⌕ Search

SEARCH · Search NASA

Results for “Automated Testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Seascape Interface Control Document (V. 2)

This paper serves as the Interface Control Document (ICD) for the Seascape automated test harness developed at Sandia National Laboratories. The primary purposes of the Seascape system are: (1) provide a place for accruing large, curated, labeled data sets useful for developing and evaluating detection and classification algorithms (including, but not limited to, supervised machine learning applications) (2) provide an automated structure for specifying, running and generating reports on algorithm performance. Seascape uses GitLab, Nexus, Solr, and Banana, open source codes, together with code written in the Python language, to automatically provision and configure computational nodes, queue up jobs to accomplish algorithms test runs against the stored data sets, gather the results and generate reports which are then stored in the Nexus artifact server.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Seascape Interface Control Document

This paper serves as the Interface Control Document (ICD) for the Seascape automated test harness developed at Sandia National Laboratories. The primary purposes of the Seascape system are: (1) provide a place for accruing large, curated, labeled data sets useful for developing and evaluating detection and classification algorithms (including, but not limited to, supervised machine learning applications) (2) provide an automated structure for specifying, running and generating reports on algorithm performance. Seascape uses GitLab, Nexus, Solr, and Banana, open source software, together with code written in the Python language, to automatically provision and configure computational nodes, queue up jobs to accomplish algorithms test runs against the stored data sets, gather the results and generate reports which are then stored in the Nexus artifact server.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Update to the Microcontroller Benchmark for Radiation Testing

LANL developed a benchmark of software code for radiation testing of microprocessors several years ago, and it was published under an open-source license on GitHub. Publishing the software is necessary for other researchers to adopt and implement this benchmark for radiation testing of other microprocessors to standardize test practices so that test data can be compared across different microprocessors. The original codes have been used several times by other organizations to test a wide range of microcontrollers and microprocessors. After several years of research, LANL is ready to update the benchmark. Changes include: 1. Addition of new codes that allow common software codes to be tested, 2. Addition of new codes that instrument more microprocessor circuitry, 3. Addition of input patterns that allow for a more compressive understanding of how the memory layout affects the sensitivity to radiation-induced faults and better use of automated test pattern generation standards, and 4. Modification of current codes for faster and more resilient detection, reporting and correction of radiation-induced faults. These codes have been tested by LANL researchers over the last few years, which has been published in the open literature. As the code base for the new benchmarks are stable, it is time to release the update to the GitHub repository, where the original codes were released.

Quinn, Heather↗
DOE OSTI2024

Field-Emission Properties of Vertically Aligned Carbon Nanotube Cathodes of Varying Geometries

This work characterizes the bulk emission properties of carbon nanotube (CNT) forest cathodes fabricated with various geometries. Geometries explored include dense nanotube forests of varying height grown on 5×5 mm silicon (Si) substrates and discrete, patterned CNT pillars fabricated using UV photolithography. Dense forests heights ranged from 526 μ m to 1.41 mm, with packing fraction for dense forest calculated to be 4e10 nanotubes/cm2 based on an average nanotube separation distance of 100 nm for fixed growth dense forests. Patterned sample micro pillar heights ranged from 47 to 393 μ m with pillar widths on tested samples ranging from 250 to 270 μ m . Properties explored include emission current, turn-on field, and emission current performance over time. A parallel plate electron beam diode with a 100- μ m A-K gap and an automated test apparatus were developed to provide a configurable experiment that provides accurate and repeatable measurements for dc, dc sweep, and timed performance testing. Operating voltages for the voltage sweeps spanned from 0 to −350 V. Furthermore, testing has shown evidence of a hysteresis effect on the emission current tied to the applied field history as well as shifting of the turn-on field magnitude throughout the testing period, suggesting a conditioning effect during use. Three separate emission regions in the I–V curves during sweep testing have also been observed. In the geometric study, dense forests and patterned samples were sweep tested up to a peak applied voltage of −250 V, with the taller samples generally performing better. Currents produced in the geometric study from the dense forest emitters ranged from 36.8 to 572.34 μ A , with current densities ranging from 15 to 2.29 mA/cm2. Currents produced from the patterned micropillar emitters ranged from 39.4 to 317.51 μ A , with current densities ranging from 67 to 3 mA/cm2. DC time testing s...

36 MATERIALS SCIENCE↗
DOE OSTI2025

Pyomo: Accidentally outrunning the bear

Pyomo is an open-source optimization modeling software that has undergone significant evolution since its inception in 2008. Pyomo has evolved to enhance flexibility, solver integration, and community engagement. Modern collaborative tools for open-source software have facilitated the development of new Pyomo functionality and improved our development process through automated testing and performance-tracking pipelines. However, Pyomo faces challenges typical of research software, including resource limitations and knowledge retention. The Pyomo team’s commitment to better development practices and community engagement reflects a proactive approach to these issues. We describe Pyomo’s development journey, highlighting both successes and failures, in the hopes that other open-source research software packages may benefit from our experiences.

automation↗
DOE OSTI2025

Descriptor: High Temporal Resolution Meteorological Data at Oak Ridge Reservation (ORR-HiResMet)

Access to continuous, quality assessed meteorological data is critical for understanding the climatology and atmospheric dynamics of a region. Research facilities like Oak Ridge National Laboratory (ORNL) rely on such data to assess site-specific climatology, model potential emissions, establish safety baselines, and prepare for emergency scenarios. To meet these needs, on-site towers at ORNL collect meteorological data at 15-minute and hourly intervals. However, data measurements from meteorological towers are affected by sensor sensitivity, degradation, lightning strikes, power fluctuations, glitching, and sensor failures, all of which can affect data quality. To address these challenges, we conducted a comprehensive quality assessment and processing of five years of meteorological data collected from ORNL at 15-minute intervals, including measurements of temperature, pressure, humidity, wind, and solar radiation. The time series of each variable was pre-processed and gap-filled using established meteorological data collection and cleaning techniques, i.e., the time series were subjected to structural standardization, data integrity testing, automated and manual outlier detection, and gap-filling. The data product and highly generalizable processing workflow developed in Python Jupyter notebooks are publicly accessible online. As a key contribution of this study, the evaluated 5-year data will be used to train atmospheric dispersion models that simulate dispersion dynamics across the complex ridge-and-valley topography of the Oak Ridge Reservation in East Tennessee.

Steckler, Morgan R. [Oak Ridge National Laboratory↗
DOE OSTI2024

CCSI Toolset 3.21 Release

CCSI Toolset 3.21 Release Highlights Parallelization support was added for Sequential Design of Experiments (SDOE) computations using Dask (preliminary). Input type dependent ordering capability was added to the SDOE module. With this implementation the user can specify the level of difficulty to change an input (Easy or Hard) and FOQUS will generate the appropriate ordered design depending on the input difficulty combination. Python version support was extended. FOQUS is now compatible with Python 3.8 through 3.12. Platforms used for automated testing were expanded to include macOS ARM (Apple Silicon). Updates to the FOQUS documentation to include information on how to set paths for SimSinter and TurbineLite. Turbine configuration section was added to Debugging Documentation.

AS↗
DOE OSTI2023

pvlib python: 2023 project update

pvlib python is a community-developed, open-source software toolbox for simulating the performance of solar photovoltaic (PV) energy components and systems. It provides reference implementations of over 100 empirical and physics-based models from the peer-reviewed scientific literature, including solar position algorithms, irradiance models, thermal models, and PV electrical models. In addition to individual low-level model implementations, pvlib python provides high-level workflows that chain these models together like building blocks to form complete “weather-to-power” photovoltaic system models. It also provides functions to fetch and import a wide variety of weather datasets useful for PV modeling. pvlib python has been developed since 2013 and follows modern best practices for open-source python software, with comprehensive automated testing, standards-based packaging, and semantic versioning. Its source code is developed openly on GitHub and releases are distributed via the Python Package Index (PyPI) and the conda-forge repository. pvlib python’s source code is made freely available under the permissive BSD-3 license. Here we (the project’s core developers) present an update on pvlib python, describing capability and community development since our 2018 publication (Holmgren, Hansen, & Mikofski, 2018).

14 SOLAR ENERGY↗
DOE OSTI2020

Data-driven Alignment of the Tracker for the Mu2e Experiment

An alignment of the Mu2e Tracker is investigated using the Mu2e Offline Software and Millepede-II. Simulated misalignments of Tracker planes were inserted between Monte-Carlo event generation and before reconstruction using randomly generated $(x,y,z)$-translations and rotations of the rigid-body planes in the detector model. Cosmic ray muons, with no magnetic field present in the Detector Solenoid, will follow a straight-line trajectory. A maximum-likelihood fit is used to reconstruct a track and determine its parameters. Tracks traversing at least four planes, with ten registered straw hits, and a maximum time residual of $20$ ns are selected to be used in an alignment sample. The cuts were chosen to discard tracks with low information content, such as those with high levels of multiple scattering or those with low numbers of measurements contributing to a single track fit. Millepede-II produces the resulting alignment constants according to a simultaneous least-squares fit of the alignment parameters and the track parameters for all input tracks. The improved alignment constants are used to repeat the process and recover an increased number of suitable tracks, further improving the estimate of the alignment constants in subsequent iterations. A performance study of the process was conducted on a misaligned Tracker in the case of translation-only, and translation plus rotational misalignments. In both cases, the correct alignment constants were recovered to within statistical error after three global fit iterations. Additionally, a continuous integration system was developed for the \texttt{Mu2e/Offline} GitHub repository, introducing a convenient automated testing process for all proposed changes (``pull requests"). The tests developed included a full build of the software, followed by tests of the validation routines, geometry overlap checking, and multi-threading mode. Additionally, code quality checks and low-statistics physics comparison checks a re also available.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2018

Teaching Computational Reproducibility for Neuroimaging

We describe a project-based introduction to reproducible and collaborative neuroimaging analysis. Traditional teaching on neuroimaging usually consists of a series of lectures that emphasize the big picture rather than the foundations on which the techniques are based. The lectures are often paired with practical workshops in which students run imaging analyses using the graphical interface of specific neuroimaging software packages. Our experience suggests that this combination leaves the student with a superficial understanding of the underlying ideas, and an informal, inefficient, and inaccurate approach to analysis. To address these problems, we based our course around a substantial open-ended group project. This allowed us to teach: (a) computational tools to ensure computationally reproducible work, such as the Unix command line, structured code, version control, automated testing, and code review and (b) a clear understanding of the statistical techniques used for a basic analysis of a single run in an MR scanner. The emphasis we put on the group project showed the importance of standard computational tools for accuracy, efficiency, and collaboration. The projects were broadly successful in engaging students in working reproducibly on real scientific questions. We propose that a course on this model should be the foundation for future programs in neuroimaging. We believe it will also serve as a model for teaching efficient and reproducible research in other fields of computational science.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2019

Studies of the FBK Low-Gain Avalanche Detectors for the CMS Endcap Timing Layer

The CMS MIP Timing Detector (MTD) will bring precision timing information as part of the Phase II upgrade program to prepare CMS for the HL-LHC. The MTD will provide hermetic timing coverage up to a pseudo-rapidity of |η| = 3, with time resolution better than 40 ps. In the high-radiation forward region from |η|=1.6 and 3.0, this achieved with the Endcap Timing Layer (ETL), based on silicon low gain avalanche detectors (LGADs). An extensive LGAD R&D program has been ongoing to characterize sensor performance to develop LGADs suitable for ETL. We discuss the details of the automated test beam DAQ environment developed for efficient characterization of prototype LGAD sensors at the Fermilab Test Beam Facility (FTBF), and the latest results for sensors produced by Fondazione Bruno Kessler (FBK) with a focus on characterization of radiation hardness and uniformity.

43 PARTICLE ACCELERATORS↗
DOE OSTI2019

MY EXES and EDM for Automated Remediation Data Quality Assessments - 19053

MY Electronic data exchange and Evaluation System (EXES) is a new web-based technology providing external admin access and full control to the Web-based EXES tool developed by GD for EPA for automated environmental remediation data quality assessments. EXES replaces most manual assessment processes, reducing remediation data review time and costs and ensuring that data are of known and documented quality. This tool is available from EPA to federal agencies and remediation firms and has successfully been used to support over 190 site investigations, 105 Remedial investigations and over 20 compliance monitoring efforts. The new MY EXES functionality has been developed for project managers and decision makers to setup projects, track receipt of environmental remediation monitoring data, assess data quality, and manage data. It is designed to be flexible and configurable to meet any government agency's data review and data management needs. It is accessible 24/7 through a secure portal. MY EXES efficiently evaluates analytical data regardless of the underlying method and data reporting requirements. It is a centralized project and data assessment tool that is scalable, flexible and configurable; improving data quality while reducing cost through the following: - Provides users with the ability to create a project and to specify the analytical methods corresponding to the data that will be submitted for that project. A project can be modified and managed anytime during the project's life cycle. - Allows users to manage a library of analytical methods that can be made available for association to their projects. - Users can specify the automated tests and assessment that needs to be performed on the analytical data deliverables based on specific requirements in the project's Quality Assurance Project Plan (QAPP). - Users specify: - The format of the incoming data deliverables from the laboratory; - The type and level of review services to be performed; - The assessment deliverables to be generated - output spreadsheets and reports; - Project owner sets up system access and permissions for members of the project team. - A faster (600%) and less expensive (85%) process than the traditional data usability/validation. The EXES Data Manager application (EDM) provides a centralized data repository to view and retrieve generated reports, spreadsheets, and other data deliverables as defined by the client, eliminating the need for data transfer between project team members. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2023

Implementation and test of an automated control hunting fault correction algorithm in a fault detection and diagnostics tool

Control hunting due to improper proportional–integral–derivative (PID) parameters in the building automation system (BAS) is one of the most common faults identified in commercial buildings. It can cause suboptimal performance and early failure of heating, ventilation, and air conditioning (HVAC) equipment. Commercial fault detection and diagnostics (FDD) software represents one of the fastest growing market segments in smart building technologies in the United States. Implementation of PID retuning procedures as an auto-correction algorithm and integration into FDD software has the potential to mitigate control hunting across a heterogeneous portfolio of buildings with different BAS in a scalable way. This paper presents the development, implementation, and field testing of an automated control hunting fault correction algorithm based on lambda tuning open-loop rules. The algorithm was developed in a commercial FDD software and successfully tested among nine variable air volume boxes in an office building in the United States. The paper shows the feasibility of using FDD tools to automatically correct control hunting faults, discusses scalability considerations, and proposes a path forward for the HVAC industry and academia to further improve this technology.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Improved Throughput and Analysis of Scratch Test Results via Automation and Machine Learning

A data analysis automation interface that incorporates machine learning (ML) has been developed to improve productivity, efficiency, and consistency in identifying and defining critical load values (or other values associated with optically identifiable characteristics) of a coating when a scratch test is performed. In this specific program, the machine learning component of the program has been trained to identify the Critical Load 2 (L C2 ) value by analyzing images of the scratch tracks created in each test. An optical examination of the scratch by a human operator is currently used to determine where this value occurs. However, the vagueness of the standard has led to varying interpretations and nonuniform usage by different operators at different laboratories where the test is implemented, resulting in multiple definitions of the desired parameter. Using a standard set of training and validation images to create the dataset, the critical load can be identified consistently amongst different laboratories using the automation interface without requiring the training of human operators. When the model was used in conjunction with an instrument manufacturer's scratch test software, the model produced accurate and repeatable results and defined L C2 values in as little as half of the time compared to a human operator. When combined with a program that automates other aspects of the scratch testing process usually conducted by a human operator, scratch testing and analysis can occur with little to no intervention from a human beyond initial setup and frees them to complete other work in the lab.

36 MATERIALS SCIENCE↗
DOE OSTI2024

MUPPET: An automated OpenMP mutation testing framework for performance optimization

MUPPET is a tool for OpenMP programs that identifies program modifications, called mutations, aimed at improving program performance. Existing performance optimization techniques, including profiling-based and auto-tuning techniques, fail to indicate program modifications at the source level thus preventing their portability across compilers. MUPPET aims to help HPC developers reason about performance defects and missed opportunities to improve performance at the source code level.

Parasyris, Konstantinos↗
DOE OSTI2025

Microreactor Automated Control System Test Bed Digital Architecture for Real-Time, Hardware-in-the-Loop Simulation

This work describes progress made towards the development of a real-time hardware-in-the-loop (HIL) test bed for non-nuclear testing of microreactor control schemes and failure modes. Non-nuclear testing is a crucial step in developing robust control algorithms for managing microreactor dynamics. The creation of an HIL simulation harnesses the realistic dynamics of physical analogue systems while additionally considering the challenges of variable communication delay. This collaborative effort between Oak Ridge National Laboratory and Idaho National Laboratory has resulted in a LabVIEW-based gRPC communication protocol which couples a TRANSFORM Modelica simulation of nuclear components to the ViBRANT physical hardware for realistic feedback and visual representation of control action in real time. A modular python client structure is developed to manage FMU-based Modelica simulation and real-time gRPC communication. HIL testing suggests that the modeled reactor with natural convection molten salt loop coolant configuration responds well to PID control of drum positioning for modulation of reactor core power, however, future efforts will be made to explore the added thermal inertial delay of system level control and downstream demand changes. Development of this platform with a generalized methodology provides a foundation for exploring a variety of reactor configurations and failure modes in rapid order to provide insight into the most effective avenues of study for further research and development.

McConnell, Jono [ORNL] (ORCID:0000000238984741)↗
DOE OSTI2021

Automated Cyber Security Testing Platform for Industrial Control Systems

Nuclear Power Plants (NPPs) are a complex system of coupled physics controlled by a network of Programmable Logic Controllers (PLCs). These PLCs communicate process data across the network to coordinate control actions with each other and inform the operators of process variables and control decisions. Networking the PLCs allows more effective process control and provides the operator more information which results in more efficient plant operation. This interconnectivity creates new security issues, as operators have more access to the plant controls, so will bad actors. As plant networks become more digitized and encompass more sophisticated controllers, the network surface exposed to cyber interference grows. Understanding the dynamics of these coupled systems of physics, control logic, and network communications is critical to their protection. The research into the cybersecurity of the Operational Technologies of NPPs is developing and requires a platform that can allow high fidelity physics simulations to interact with digital networks of controllers. This will require three main components: a network simulation environment, a physics simulator, and virtual PLCs (vPLC) that represent typical industry hardware. A platform that incorporates these three components to provide the most accurate representation of actual NPP networks and controllers is developed in this paper.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗