Search NASA⌕ Search

SEARCH · Search NASA

Results for “software metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

Paramedir: A Tool for Programmable Performance Analysis

Performance analysis of parallel scientific applications is time consuming and requires great expertise in areas such as programming paradigms, system software, and computer hardware architectures. In this paper we describe a tool that facilitates the programmability of performance metric calculations thereby allowing the automation of the analysis and reducing the application development time. We demonstrate how the system can be used to capture knowledge and intuition acquired by advanced parallel programmers in order to be transferred to novice users.

Jost, Gabriele↗

Distributed Engine Control Empirical/Analytical Verification Tools

NASA's vision for an intelligent engine will be realized with the development of a truly distributed control system featuring highly reliable, modular, and dependable components capable of both surviving the harsh engine operating environment and decentralized functionality. A set of control system verification tools was developed and applied to a C-MAPSS40K engine model, and metrics were established to assess the stability and performance of these control systems on the same platform. A software tool was developed that allows designers to assemble easily a distributed control system in software and immediately assess the overall impacts of the system on the target (simulated) platform, allowing control system designers to converge rapidly on acceptable architectures with consideration to all required hardware elements. The software developed in this program will be installed on a distributed hardware-in-the-loop (DHIL) simulation tool to assist NASA and the Distributed Engine Control Working Group (DECWG) in integrating DCS (distributed engine control systems) components onto existing and next-generation engines.The distributed engine control simulator blockset for MATLAB/Simulink and hardware simulator provides the capability to simulate virtual subcomponents, as well as swap actual subcomponents for hardware-in-the-loop (HIL) analysis. Subcomponents can be the communication network, smart sensor or actuator nodes, or a centralized control system. The distributed engine control blockset for MATLAB/Simulink is a software development tool. The software includes an engine simulation, a communication network simulation, control algorithms, and analysis algorithms set up in a modular environment for rapid simulation of different network architectures; the hardware consists of an embedded device running parts of the CMAPSS engine simulator and controlled through Simulink. The distributed engine control simulation, evaluation, and analysis technology provides unique capabilities to study the effects of a given change to the control system in the context of the distributed paradigm. The simulation tool can support treatment of all components within the control system, both virtual and real; these include communication data network, smart sensor and actuator nodes, centralized control system (FADEC full authority digital engine control), and the aircraft engine itself. The DECsim tool can allow simulation-based prototyping of control laws, control architectures, and decentralization strategies before hardware is integrated into the system. With the configuration specified, the simulator allows a variety of key factors to be systematically assessed. Such factors include control system performance, reliability, weight, and bandwidth utilization.

DeCastro, Jonathan↗

Image Correlation Pattern Optimization for Micro-Scale In-Situ Strain Measurements

The accuracy and precision of digital image correlation (DIC) is a function of three primary ingredients: image acquisition, image analysis, and the subject of the image. Development of the first two (i.e. image acquisition techniques and image correlation algorithms) has led to widespread use of DIC; however, fewer developments have been focused on the third ingredient. Typically, subjects of DIC images are mechanical specimens with either a natural surface pattern or a pattern applied to the surface. Research in the area of DIC patterns has primarily been aimed at identifying which surface patterns are best suited for DIC, by comparing patterns to each other. Because the easiest and most widespread methods of applying patterns have a high degree of randomness associated with them (e.g., airbrush, spray paint, particle decoration, etc.), less effort has been spent on exact construction of ideal patterns. With the development of patterning techniques such as microstamping and lithography, patterns can be applied to a specimen pixel by pixel from a patterned image. In these cases, especially because the patterns are reused many times, an optimal pattern is sought such that error introduced into DIC from the pattern is minimized. DIC consists of tracking the motion of an array of nodes from a reference image to a deformed image. Every pixel in the images has an associated intensity (grayscale) value, with discretization depending on the bit depth of the image. Because individual pixel matching by intensity value yields a non-unique scale-dependent problem, subsets around each node are used for identification. A correlation criteria is used to find the best match of a particular subset of a reference image within a deformed image. The reader is referred to references for enumerations of typical correlation criteria. As illustrated by Schreier and Sutton and Lu and Cary systematic errors can be introduced by representing the underlying deformation with under-matched shape functions. An important implication, as discussed by Sutton et al., is that in the presence of highly localized deformations (e.g., crack fronts), error can be reduced by minimizing the subset size. In other words, smaller subsets allow the more accurate resolution of localized deformations. Contrarily, the choice of optimal subset size has been widely studied and a general consensus is that larger subsets with more information content are less prone to random error. Thus, an optimal subset size balances the systematic error from under matched deformations with random error from measurement noise. The alternative approach pursued in the current work is to choose a small subset size and optimize the information content within (i.e., optimizing an applied DIC pattern), rather than finding an optimal subset size. In the literature, many pattern quality metrics have been proposed, e.g., sum of square intensity gradient (SSSIG), mean subset fluctuation, gray level co-occurrence, autocorrelation-based metrics, and speckle-based metrics. The majority of these metrics were developed to quantify the quality of common pseudo-random patterns after they have been applied, and were not created with the intent of pattern generation. As such, it is found that none of the metrics examined in this study are fit to be the objective function of a pattern generation optimization. In some cases, such as with speckle-based metrics, application to pixel by pixel patterns is ill-conditioned and requires somewhat arbitrary extensions. In other cases, such as with the SSSIG, it is shown that trivial solutions exist for the optimum of the metric which are ill-suited for DIC (such as a checkerboard pattern). In the current work, a multi-metric optimization method is proposed whereby quality is viewed as a combination of individual quality metrics. Specifically, SSSIG and two auto-correlation metrics are used which have generally competitive objectives. Thus, each metric could be viewed as a constraint imposed upon the others, thereby precluding the achievement of their trivial solutions. In this way, optimization produces a pattern which balances the benefits of multiple quality metrics. The resulting pattern, along with randomly generated patterns, is subjected to numerical deformations and analyzed with DIC software. The optimal pattern is shown to outperform randomly generated patterns.

Bomarito, G. F.↗

Advanced Diagnostic and Prognostic Testbed (ADAPT) Testability Analysis Report

As system designs become more complex, determining the best locations to add sensors and test points for the purpose of testing and monitoring these designs becomes more difficult. Not only must the designer take into consideration all real and potential faults of the system, he or she must also find efficient ways of detecting and isolating those faults. Because sensors and cabling take up valuable space and weight on a system, and given constraints on bandwidth and power, it is even more difficult to add sensors into these complex designs after the design has been completed. As a result, a number of software tools have been developed to assist the system designer in proper placement of these sensors during the system design phase of a project. One of the key functions provided by many of these software programs is a testability analysis of the system essentially an evaluation of how observable the system behavior is using available tests. During the design phase, testability metrics can help guide the designer in improving the inherent testability of the design. This may include adding, removing, or modifying tests; breaking up feedback loops, or changing the system to reduce fault propagation. Given a set of test requirements, the analysis can also help to verify that the system will meet those requirements. Of course, a testability analysis requires that a software model of the physical system is available. For the analysis to be most effective in guiding system design, this model should ideally be constructed in parallel with these efforts. The purpose of this paper is to present the final testability results of the Advanced Diagnostic and Prognostic Testbed (ADAPT) after the system model was completed. The tool chosen to build the model and to perform the testability analysis with is the Testability Engineering and Maintenance System Designer (TEAMS-Designer). The TEAMS toolset is intended to be a solution to span all phases of the system, from design and development through health management and maintenance. TEAMS-Designer is the model-building and testability analysis software in that suite.

Ossenfort, John↗

Development of a Method for Shape Optimization for a Gas Turbine Fuel Injector Design Using Metal-Additive Manufacturing

Adjoint shape optimization has enabled physics-based optimal designs for aerodynamic surfaces. Additive manufacturing (AM) makes it possible to manufacture complex shapes. However, there has been a gap between optimal and manufacturable surfaces due to the inherent limitations of commercial computational fluid dynamics (CFD) codes to implement geometric constraints during adjoint computation. In such cases, the design sensitivities are exported and used to perform constrained shape modifications using parametric information stored in computer aided design (CAD) files to satisfy manufacturability constraints. However, modifying the design using adjoint methods in CFD solvers and performing constrained shape modification in CAD can lead to inconsistencies due to different shape parameterization schemes. This paper describes a method to enable the simultaneous optimization of the fluid domain and impose AM manufacturability constraints, resolving one of the key issues of geometry definition for isogeometric analysis. Similar to a grid convergence study, the proposed method verifies the consistencies between shape parameterization techniques present within commercial CAD and CFD software during mesh movement as a part of the adjoint shape optimization routine. By identifying the appropriate parameters essential to a shape optimization study, the error metric between the different parameterization techniques converges to demonstrate sufficient consistencies for justifiable exchange of data between CAD and CFD. For the identified shape optimization parameters, the error metric to measure the deviation between the two parameterization schemes lies within the AM laser-powder bed fusion (L-PBF) process tolerance. Additionally, comparison for subsequent objective function calculations between iterations of the optimization loop showed acceptable differences within 1% variation between the modified geometries obtained using the two parameterization schemes. This method provides justification for the use of multiphysics guided adjoint design sensitivities computed in CFD software to perform shape modifications in CAD to incorporate AM manufacturability constraints during the shape optimization loop such that optimal designs are also additively manufacturable.

33 ADVANCED PROPULSION SYSTEMS↗

IDAES-PSE Software Tools for Optimizing Energy Systems and Market Interactions

Modern power grids coordinate electricity production and consumption via multi-scale wholesale energy markets. Historically, levelized cost metrics were the de facto standard for techno-eco-nomic analyses of energy systems and comparison of technology options. However, these metrics neglect the complexity of energy infrastructure including the time-varying value of electricity. An emerging alternative is multi-period optimization, which considers the locational marginal price of electricity as input data (parameters). In this work, we present a general interface for multi-period optimization with time-varying energy prices to facilitate rapid analysis and comparison of potential energy systems models. The PriceTakerModel class is written in the IDAES-PSE platform and allows users to generate a multi-period, price-taker model instance, as well as automatically generate common operational constraints for their model, such as start-up and shutdown. We show this interface successfully generates multi-period price-taker models, facilitates model discrimination, and aids in analyzing various technologies for deployment in unique energy markets.

Laky, Daniel↗

Support for life-cycle product reuse in NASA's SSE

The Software Support Environment (SSE) is a software factory for the production of Space Station Freedom Program operational software. The SSE is to be centrally developed and maintained and used to configure software production facilities in the field. The PRC product TTCQF provides for an automated qualification process and analysis of existing code that can be used for software reuse. The interrogation subsystem permits user queries of the reusable data and components which have been identified by an analyzer and qualified with associated metrics. The concept includes reuse of non-code life-cycle components such as requirements and designs. Possible types of reusable life-cycle components include templates, generics, and as-is items. Qualification of reusable elements requires analysis (separation of candidate components into primitives), qualification (evaluation of primitives for reusability according to reusability criteria) and loading (placing qualified elements into appropriate libraries). There can be different qualifications for different installations, methodologies, applications and components. Identifying reusable software and related components is labor-intensive and is best carried out as an integrated function of an SSE.

Shotton, Charles↗

DECIDER

This software offers methods and functions for building failure detectors for deep image classification models with the aid of vision-language models and LLMs. It includes functionalities for training baseline image classifiers, debiasing classifiers using vision-language models and LLMs, evaluating failure between models along with baselines. Developed using PyTorch, this software is compatible with standard neural network architectures used for imaging data. Additionally, it provides capabilities to compute evaluation metrics for assessing the performance and quality of the detectors.

Narayanaswamy, Vivek Sivaraman↗

NuGraph2: A Graph Neural Network for Neutrino Event Reconstruction

Neutrino experiments are set to probe some of the most important open questions in physics, from CP violation and the nature of dark matter. The technology of choice for many of these experiments is the liquid argon time projection chamber (LArTPC). In current LArTPC experiments, reconstruction performance often represents a limiting factor for the sensitivity. New developments are therefore needed to unlock the full potential of LArTPC experiments. NuGraph2 is a state of the art Graph Neural Network for reconstruction of data in LArTPC experiments. NuGraph2 utilizes a heterogeneous graph structure, with separate subgraphs of 2D nodes (hits in each plane) connected across planes via 3D nodes (space points). The model provides a consistent description of the neutrino interaction across all planes. NuGraph2 is a multi-purpose network, with a common message-passing attention engine connected to multiple decoders with different classification or regression tasks. These include the classification of detector hits according to the particle type that produced them (semantic segmentation) and the separation of hits from the neutrino interaction from hits due to noise or cosmic-ray background. Additional decoders are being developed, performing tasks such as the regression of the neutrino interaction vertex position. Performance results will be presented based on publicly available samples from MicroBooNE. These include both physics performance metrics, achieving 95% accuracy for semantic segmentation and 98% classification of neutrino hits, as well as computational metrics for training and for inference on CPU or GPU. The status of the NuGraph integration in the LArSoft software framework will be presented, as well as initial studies about model interpretability and injection of domain knowledge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

The NASA Technical Report Server

The National Aeronautics and Space Act of 1958 established NASA and charged it to "provide for the widest practicable and appropriate dissemination of information concerning its activities and the results thereof." The search for innovative methods to distribute NASA's information lead a grass-roots team to create the NASA Technical Report Server (NTRS), which uses the World Wide Web and other popular Internet-based information systems as search engines. The NTRS is an inter-center effort which provides uniform access to various distributed publication servers residing on the Internet. Users have immediate desktop access to technical publications from NASA centers and institutes. The NTRS is comprised of several units, some constructed especially for inclusion in NTRS, and others that are existing NASA publication services that NTRS reuses. This paper presents the NTRS architecture, usage metrics, and the lessons learned while implementing and maintaining the service. The NTRS is largely constructed with freely available software running on existing hardware. NTRS builds upon existing hardware and software, and the resulting additional exposure for the body of literature contained ensures that NASA's institutional knowledge base will continue to receive the widest practicable and appropriate dissemination.

Nelson, Michael L.↗

Propulsion Diagnostic Method Evaluation Strategy (ProDiMES) User's Guide

This report is a User's Guide for the Propulsion Diagnostic Method Evaluation Strategy (ProDiMES). ProDiMES is a standard benchmarking problem and a set of evaluation metrics to enable the comparison of candidate aircraft engine gas path diagnostic methods. This Matlab (The Mathworks, Inc.) based software tool enables users to independently develop and evaluate diagnostic methods. Additionally, a set of blind test case data is also distributed as part of the software. This will enable the side-by-side comparison of diagnostic approaches developed by multiple users. The Users Guide describes the various components of ProDiMES, and provides instructions for the installation and operation of the tool.

Simon, Donald L.↗

NuGraph2: A Graph Neural Network for Neutrino Event Reconstruction

Neutrino experiments are set to probe some of the most important open questions in physics, from CP violation and the nature of dark matter. The technology of choice for many of these experiments is the liquid argon time projection chamber (LArTPC). In current LArTPC experiments, reconstruction performance often represents a limiting factor for the sensitivity. New developments are therefore needed to unlock the full potential of LArTPC experiments. NuGraph2 is a state of the art Graph Neural Network for reconstruction of data in LArTPC experiments. NuGraph2 utilizes a heterogeneous graph structure, with separate subgraphs of 2D nodes (hits in each plane) connected across planes via 3D nodes (space points). The model provides a consistent description of the neutrino interaction across all planes. NuGraph2 is a multi-purpose network, with a common message-passing attention engine connected to multiple decoders with different classification or regression tasks. These include the classification of detector hits according to the particle type that produced them (semantic segmentation) and the separation of hits from the neutrino interaction from hits due to noise or cosmic-ray background. Additional decoders are being developed, performing tasks such as the regression of the neutrino interaction vertex position. Performance results will be presented based on publicly available samples from MicroBooNE. These include both physics performance metrics, achieving 95% accuracy for semantic segmentation and 98% classification of neutrino hits, as well as computational metrics for training and for inference on CPU or GPU. The status of the NuGraph integration in the LArSoft software framework will be presented, as well as initial studies about model interpretability and injection of domain knowledge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NuGraph2: A Graph Neural Network for Neutrino Event Reconstruction

Neutrino experiments are set to probe some of the most important open questions in physics, from CP violation and the nature of dark matter. The technology of choice for many of these experiments is the liquid argon time projection chamber (LArTPC). In current LArTPC experiments, reconstruction performance often represents a limiting factor for the sensitivity. New developments are therefore needed to unlock the full potential of LArTPC experiments. NuGraph2 is a state of the art Graph Neural Network for reconstruction of data in LArTPC experiments [https://arxiv.org/abs/2403.11872]. NuGraph2 utilizes a heterogeneous graph structure, with separate subgraphs of 2D nodes (hits in each plane) connected across planes via 3D nodes (space points). The model provides a consistent description of the neutrino interaction across all planes. NuGraph2 is a multi-purpose network, with a common message-passing attention engine connected to multiple decoders with different classification or regression tasks. These include the classification of detector hits according to the particle type that produced them (semantic segmentation) and the separation of hits from the neutrino interaction from hits due to noise or cosmic-ray background. Additional decoders are being developed, performing tasks such as the regression of the neutrino interaction vertex position. Performance results will be presented based on publicly available samples from MicroBooNE. These include both physics performance metrics, achieving 95% accuracy for semantic segmentation and 98% classification of neutrino hits, as well as computational metrics for training and for inference on CPU or GPU. The status of the NuGraph integration in the LArSoft software framework will be presented, as well as initial studies about model interpretability and injection of domain knowledge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

A Quantum Volume Metric for Qudit Quantum Computers

Qudit-based quantum computers, such as the high-Q superconducting radio frequency resonator cavities being pursued by the SQMS Center hosted at Fermilab, may provide computational advantages relative to qubit-based machines for some problems. However, the novel system’s hardware and software will present different errors that must be understood and mitigated, along with other unforeseen inefficiencies across the entire stack. It is essential to benchmark these qudit platforms against existing systems. We propose such a benchmark by generalizing the qubit-based quantum volume metric to qudit devices. In particular, we show that the performance depends on the available gateset used to compile unitary operations, as well as realistic noise sources.

Quantum Computing↗

Modeling defect trends for iterative development

The Employment of Defects (EoD) approach to measuring and analyzing defects seeks to identify and capture trends and phenomena that are critical to managing software quality in the iterative software development lifecycle at JPL.

modeling defects metrics↗

JUSTIFI: Software for Improving Performance Objectives via Energy Efficiency

With growing energy supply concerns and rising costs, energy efficiency is a critical component of industrial energy resilience and competitiveness by directly reducing energy operating costs. Energy efficiency projects in manufacturing also yield valuable benefits to other key metrics, such as improved quality, reduced maintenance costs, improved safety, decreased pollution, and enhanced productivity. However, it is difficult to receive approval for energy efficiency projects, so implementation rates are low, even when meeting capital project payback period criteria. The inclusion and quantification of non-energy benefits (NEBs) in the decision-making process for energy efficiency projects can improve the overall financial payback period while demonstrating a positive impact on the firm's key performance metrics and business strategy. Despite their significant financial and strategic value, NEBs are rarely factored into decision-making due to lack of tools to effectively identify and quantify them. Therefore, a comprehensive and integrative approach is needed for the rapidly evolving energy landscape. To address these challenges, through funding from U.S. Department of Energy, our new assessment methodology integrates common continuous improvement six sigma concepts, such as the DMAIC process, and a protocol of guiding questions, into energy efficiency assessments to identify NEBs. We have also developed open-source software, JUSTIFI, to guide users through this process, data collection, and quantification. It is designed to be used concurrently with DOE energy system analysis software suite, MEASUR. Our methodology and tools inform energy assessors, firm engineering, decision makers, and workforce seeking to increase energy resilience and to maximize benefits aligned with performance metrics.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Dynamic IT Security Database and Analytics for Launch Control Systems Software

During the Summer 2020 session, I worked with intern Destani S. Van Arsdalen of EGS Software. Together, we co-created a tool to aid the dynamic investigation, updated over time,of the security compliance of LCS COTS and open source software. We originally planned touse spreadsheet software for management and analysis, but through this exploratoryproject, chose to use Python and JSON after receiving feedback on our project’s current anddesired capabilities at that time.At first, the project was solely designed to help on-board new COTS software, based on aquestionnaire that could be filled out for each software package. This, combined with usingthe spreadsheet application’s web-query capabilities to fetch information from the NVD,allowed presentation and analytics cells to automatically populate as elements of themanually-filled questionnaire changed. While this system was promising, we decided tochange technologies for a few reasons. In the spreadsheet, single cells could not hold complexdata like arrays and objects. The automatic population of cells and dynamic updates made itdifficult to manage and add new features. And finally, it had limited extensibility sinceadding new software required significant understanding of how both the spreadsheet wasconstructed, and the more obscure, proprietary scripting languages packaged with it.The pivot to a standard computer science database language of JSON, aided by thescripting capabilities of Python, greatly helped to improve the project’s functionality. First,and most importantly, the script’s import and analysis of database data is easilyreproducible. Additional data analysis can be modularly added without requiringmodification of the script and is capable of routine scheduling. The revised process can besplit into three parts. First, the conversion of LCS asset and software documentation into theJSON hierarchical database format. Second, the merging of this database with the NVD,forming a new data structure, using CPEs of the CVE object as a linking element betweenthem. And third, the automatically performed analytics and analysis of the combined data,in a modular and extensible format, to produce better informed business decisions. The outputted graphs, for example, are automatically generated by the Python script inconnection with the combined database. This allows updated graphs and any analytics to be re-rendered automatically following updates to the LCS’s initial asset documentation. Afinal report can then be programmatically and easily constructed from these sources to allow fully reproducible metrics for heavily evidenced risk management decisions.

it↗

Integrated Spatial, Spectral, & Temporal Optical Reflectance System for Precision Occupancy & Location Sensing to Improve Building Energy Efficiency

Buildings consume approximately 35% of the electricity used in the U.S. and building owners can significantly reduce this energy use by providing services like heating, electrical power and lighting only when people are present. The ARPAe funded program titled “INTEGRATED SPATIAL, SPECTRAL, & TEMPORAL OPTICAL REFLECTANCE SYSTEM FOR PRECISION OCCUPANCY & LOCATION SENSING TO IMPROVE BUILDING ENERGY EFFICIENCY” demonstrates how a low cost sensor technology developed for measuring distances can be used to count and locate occupants with a high degree of precision with a very low error rates. This platform tells a building control system where occupants are located (but not who they are) so that energy consuming services can be provided only when the services are needed by building occupants. The original proof of concept involved using low cost, commercially available time-of-flight (TOF) sensors that measure distance, but the performance of these existing sensors was lacking, as they could not operate properly in the presence of sunlight, which blinded the simple TOF sensors and limited their utility in buildings. This project proposed a powerful new class of TOF sensors that used state-of-the-art integrated circuit (IC) fabrication processes that combined advanced photonics with conventional silicon chip circuitry for improved sensor performance. An equally important part of this project was to find ways to maximize occupant count and location accuracy while using the fewest number of sensors possible, in order to keep costs low. By using building blueprints to create digital twins of commercial building spaces, the team developed new algorithms to maximize occupant count and tracking accuracy by properly locating the minimum number of sensors at just the right spots in the building. This capability not only minimizes system costs but also simplified sensor installation and system commissioning. Our simulations of our sensor networks for a range of commercial floorplan designs demonstrated that our installed cost target of $0.08/sqft was attainable, though not fully demonstrated during the project. Finally, we noted that the TOF sensor concept could provide a valuable role in health and eldercare by tracking patients without the need for worn sensors and would be useful for fall detection and other patient safety metrics, including tracking healthcare/patient interactions. We feel that, when fully developed, this new class of sophisticated TOF sensors and support software will be a powerful new approach to improving building energy efficiency based on occupant centric control platforms and will also open new levels of patient safety in healthcare operations. To realize this potential, the team formed the Troy Sensor Company LLC to oversee licensing of the programs patents and continue to seek commercialization of this program’s activity sensing technologies.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗