Search NASA⌕ Search

SEARCH · Search NASA

Results for “software flaw detection”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Using Neural Architecture Search for Improving Software Flaw Detection in Multimodal Deep Learning Models

Software flaw detection using multimodal deep learning models has been demonstrated as a very competitive approach on benchmark problems. In this work, we demonstrate that even better performance can be achieved using neural architecture search (NAS) combined with multimodal learning models. We adapt a NAS framework aimed at investigating image classification to the problem of software flaw detection and demonstrate improved results on the Juliet Test Suite, a popular benchmarking data set for measuring performance of machine learning models in this problem domain.

97 MATHEMATICS AND COMPUTING↗

Using Neural Architecture Search for Improving Software Flaw Detection in Multimodal Deep Learning Models

Software flaw detection using multimodal deep learning models has been demonstrated as a very competitive approach on benchmark problems. In this work, we demonstrate that even better performance can be achieved using neural architecture search (NAS) combined with multimodal learning models. We adapt a NAS framework aimed at investigating image classification to the problem of software flaw detection and demonstrate improved results on the Juliet Test Suite, a popular benchmarking data set for measuring performance of machine learning models in this problem domain.

97 MATHEMATICS AND COMPUTING↗

Multimodal Deep Learning for Flaw Detection in Software Programs

We explore the use of multiple deep learning models for detecting flaws in software programs. Current, standard approaches for flaw detection rely on a single representation of a software program (e.g., source code or a program binary). We illustrate that, by using techniques from multimodal deep learning, we can simultaneously leverage multiple representations of software programs to improve flaw detection over single representation analyses. Specifically, we adapt three deep learning models from the multimodal learning literature for use in flaw detection and demonstrate how these models outperform traditional deep learning models. We present results on detecting software flaws using the Juliet Test Suite and Linux Kernel.

97 MATHEMATICS AND COMPUTING↗

Joint Analysis of Program Data Representations using Machine Learning for Improved Software Assurance and Development Capabilities

We explore the use of multiple deep learning models for detecting flaws in software programs. Current, standard approaches for flaw detection rely on a single representation of a software program (e.g., source code or a program binary). We illustrate that, by using techniques from multimodal deep learning, we can simultaneously leverage multiple representations of software programs to improve flaw detection over single representation analyses. Specifically, we adapt three deep learning models from the multimodal learning literature for use in flaw detection and demonstrate how these models outperform traditional deep learning models. We present results on detecting software flaws using the Juliet Test Suite and Linux Kernel.

97 MATHEMATICS AND COMPUTING↗

Optimizing Probability of Detection Point Estimate Demonstration

Probability of detection (POD) analysis is used in assessing reliably detectable flaw size in nondestructive evaluation (NDE). MIL-HDBK-18231and associated mh18232POD software gives most common methods of POD analysis. Real flaws such as cracks and crack-like flaws are desired to be detected using these NDE methods. A reliably detectable crack size is required for safe life analysis of fracture critical parts. The paper provides discussion on optimizing probability of detection (POD) demonstration experiments using Point Estimate Method. POD Point estimate method is used by NASA for qualifying special NDE procedures. The point estimate method uses binomial distribution for probability density. Normally, a set of 29 flaws of same size within some tolerance are used in the demonstration. The optimization is performed to provide acceptable value for probability of passing demonstration (PPD) and achieving acceptable value for probability of false (POF) calls while keeping the flaw sizes in the set as small as possible.

Koshti, Ajay M.↗

Low Temperature SQUID for NDE Applications

We have developed a low temperature SuperConducting Quantum Interference Device - SQUID measurement system for detection of defects deep under the surface of aluminum structures using eddy current techniques. The system uses a two dimensional planar inducer with two different excitation frequencies to induce a current in the sample. We have developed a data analysis software program that enabled us to distinguish between round defects (holes), straight defects (slots) and slots close to holes simulating cracks starting from rivets in aluminum structures. We were able to detect defects that are 8mm below the surface. We have also measured the change in phase of the detected signal as a function of depth of the defect. This relationship can be used to determine the depth of hidden flaws. Using this analysis software with the high temperature SQUID system at NASA Langley we were able to detect slots close to holes in layered aluminum sample.

Wincheski, Buzz↗

Reliably Detectable Flaw Size for NDE Methods that Use Calibration

Probability of detection (POD) analysis is used in assessing reliably detectable flaw size in nondestructive evaluation (NDE). MIL-HDBK-1823 and associated mh1823 POD software gives most common methods of POD analysis. In this paper, POD analysis is applied to an NDE method, such as eddy current testing, where calibration is used. NDE calibration standards have known size artificial flaws such as electro-discharge machined (EDM) notches and flat bottom hole (FBH) reflectors which are used to set instrument sensitivity for detection of real flaws. Real flaws such as cracks and crack-like flaws are desired to be detected using these NDE methods. A reliably detectable crack size is required for safe life analysis of fracture critical parts. Therefore, it is important to correlate signal responses from real flaws with signal responses form artificial flaws used in calibration process to determine reliably detectable flaw size.

Koshti, Ajay M.↗

Development of a Navigator and Imaging Techniques for the Cryogenic Dark Matter Search Detectors

This project contributes to the detection of flaws in the germanium detectors for the Cryogenic Dark Matter Search (CDMS) experiment. Specifically, after imaging the detector surface with a precise imaging and measuring device, they developed software to stitch the resulting images together, applying any necessary rotations, offsets, and averaging, to produce a smooth image of the whole detector that can be used to detect flaws on the surface of the detector. These images were also tiled appropriately for the Google Maps API to use as a navigation tool, allowing viewers to smoothly zoom and pan across the detector surface. Automated defect identification can now be implemented, increasing the scalability of the germanium detector fabrication.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Machine Learning Enabled Sensor Fusion for In-Situ Defect Detection in Laser Powder Bed Fusion

Laser Powder Bed Fusion (L-PBF) Additive Manufacturing (AM) is among the metal 3D printing technologies most broadly adopted by the manufacturing industry. The current industry qualification paradigm for critical-application L-PBF parts relies heavily on expensive non-destructive inspection techniques such as X-Ray Computed Tomography (XCT), which significantly limits the use-cases of L-PBF. In situ monitoring of the process promises a less expensive alternative to ex situ testing, but existing sensor technologies and data analysis techniques struggle to detect sub-surface flaws (e.g., porosity and cracking) on production-scale L-PBF printers. RTX Technologies Research Center (RTRC) has licensed ORNL’s Peregrine software package – a printer- and camera-agnostic data analytics tool designed specifically for detecting process anomalies using in situ data collected during powder bed printing. The goal of this project was to feed temporally rich, multi-modal sensor data, including visible light, integrated near infrared (NIR), and spatially mapped co-axial melt pool thermal emission data into Peregrine to enable detection of subsurface flaws. XCT data was used as ground truth training data to allow Peregrine’s deep learning algorithms to recognize anomalies in these complex data streams in both test artifacts and industrially relevant geometries. Completion of this program has seen the successful implementation of multi-modal, multi-layer sensor data footprints for training of machine learning models in Peregrine. Flaws detected in XCT data have been successfully detected directly from this in situ data footprint, and initial analyses of the in situ probability-of-detection has been conducted, showing performance levels commensurate with traditional non-destructive evaluation (NDE) methods. The in situ monitoring methodology was then applied to an industrially relevant component that was using post-build NDE, highlighting the utility of the proposed method for hard-to-inspect AM components. As a direct result of this program, two journal manuscripts [1], [2] have been published in Additive Manufacturing, with additional manuscripts planned following program completion.

36 MATERIALS SCIENCE↗

Study of guided wave transmission through complex junction in sodium cooled reactor

Ultrasonic guided wave techniques are seen as suitable candidates for the inspection of welded structures within sodium cooled fast reactors (SFR), as the long range propagation of guided waves without amplitude attenuation can overcome the accessibility problem due to the liquid sodium. In the context of the development of the Advanced Sodium Test Reactor for Industrial Demonstration (ASTRID), the French Atomic Commission (CEA) investigates non-destructive testing techniques based on guided wave propagation. In this work, guided wave NDT methods are applied to control the integrity of welds located in a junction-type structure welded to the main vessel. The method presented in this paper is based on the analysis of scattering matrices peculiar to each expected defect, and takes advantage of the multi-modal and dispersive characteristics of guided wave generation. In a simulation study, an algorithm developed using the CIVA software is presented. It permits selecting appropriate incident modes to optimize detection and identification of expected flawed configurations. In the second part of this paper, experimental results corresponding to a first validation step of the simulation results are presented. The goal of the experiments is to estimate the effectiveness of the incident mode selection in plates. The results show good agreement between experience and simulation. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Electronic Nose: Evaluation of Kamina Prototype Unit

The Kamina, Sam and Cyranose electronic nose systems were evaluated and partially trained. Much work was performed on the Kamina as it has the ability to respond to low (less than 10 ppb) concentrations of hydrazine compounds. We were able to tell the difference between Hydrazine (Hz) and Monomethylhydrazine (MMH) in standard clean humid air. We were able to detect MMH in reduced pressure (1/3 atm) at about 250 ppb, however the training set was to far from the real situation to be useful now. Various engineering and usability aspects of both the noses was noted, especially the software. One serious physical engineering flaw was remedied in the Kamina system. A gas flow manifold was created for the Sam system. Different chips were evaluated for the Kamina system. It is still unclear if they can be exchanged without retraining the software.The Sam Detect commercial unit was evaluated for solvent detection and evaluation. It was able to successfully identify some solvents. The Cyranose, was observed and evaluated for two days. It has the ability to detect gasses in the 100 parts per million level but not the 10 parts per billion level. It is very sensitive to humidity changes; there is software to partially handle this.

Schattke, Nathan↗

DETECTION OF LACK OF FUSION WELD DEFECTS BY RADIOGRAPHY

In this work, radiography was employed as the NDT technique for detection of flaws in circumferential girth welds of steel pipelines used in offshore installations in the petroleum industry. The kind of defect specifically focused was lack of fusion. It is currently accepted in the literature that radiography is not as sensitive as ultrasonics to detect lack of fusion defects. Unfortunately, the radiographic inspection can barely detect lack of fusion and only when it is associated to inclusions and voids of considerable size. However, in a previous article ('Reliability of radiographic inspection of steel pipeline girth welds', QNDE Conference, 2007), the authors showed that it is possible to detect lack of fusion defects if, in the radiographic tests, the angle of incidence is the same angle that the weld bevel makes with the test piece surface, which means lowering the angle of disorientation between the flaw and the radiographic beam. However, no concerns were made to sizing the defects. Computational simulation was used with XRSIM software to establish the optimal radiographic parameters to reach the lower limit for detection for this kind of defect.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

First Annual Report on Pulsed Thermal Tomography Nondestructive Evaluation of Additively Manufactured Reactor Materials and Components: Pulsed Thermal Tomography Nondestructive Examination of Additively Manufactured Reactor Materials and Components

Additive manufacturing (AM, or 3D printing) for commercial nuclear energy applications is an emerging method for cost-efficient manufacturing aimed at replacing aging nuclear reactor parts and reducing costs for new construction. Because of the geometry of metallic structures of interest for nuclear applications, which consist of planar primitives with no symmetry of revolution, limited options are available for nondestructive evaluation (NDE), either during or post manufacturing. Known material flaws in AM include low-density regions consisting of non-sintered powder, which have to be detected to ensure the safety of long-term performance nuclear reactor components. As a solution to NDE of AM, we are developing pulsed thermal tomography (PTT) models and depth inversion algorithms for 3D imaging. PTT has many advantages because the method is non-contact and allows for in-service, NDE of AM nuclear reactor parts. By analyzing transients of surface temperature response due to internal thermal resistances, one can obtain 3D reconstructions of material effusivity using a unique inversion algorithm developed at Argonne. This report provides results of preliminary performance evaluation of PTT capability in detection of flaws in metallic structures, as well imaging complex geometry AM structures. Performance of PTT was investigated through modeling and experiments, Computer models of PTT were developed using COMSOL Heat Transfer module. Defects were modeled as cylindrical flat bottom holes (FBH), which is a common model of calibrated material flaws in thermal tomography experiments. Materials considered in this study include stainless steel 316 (SS316), stainless steel 304 (SS304), and Inconel 718. Theoretical analyses were conducted to validate inversion of simulated PTT data with COMSOL for a plate. Subsequently, 3D reconstructions were performed on COMSOL simulations for FBH, revealing a decrease in spatial resolution over depth due to thermal diffusion. The results of this study show that the performance of the inversion algorithm for detecting smaller defects depends strongly on the depth of the defect as well as the incident heat flux. The size of detectable defect was estimated by fitting a Gaussian function to surface temperature profile. The criteria for detectability was taken as 20mK noise equivalent temperature difference (NETD), which is currently the sensitivity limit of high-performance infrared cameras. It was determined through computer simulations that the smallest detectable FBH in SS316 has a 50µm diameter and is located 0.5mm below the plate surface. Preliminary experimental studies were performed to evaluate PTT performance in detecting calibrated flaws using metallic specimens with AM defects simulated as drilled-in FBH’s. High strength Stainless Steel 316 and Inconel 718 alloys were considered, as well as lower grade Stainless Steel 304, Nickel 200, and Hastelloy C276. Specimens investigated in this report consisted of approximately 1/4in-thick plates made out of these alloys using conventional manufacturing methods. The diameters of FBH’s varied from 1mm to 8mm, and their depths below the plate flat surface varied between 1mm and 6mm. The size of the smallest FBH was limited to 1mm because conventional mechanical drills were used for creating the holes. PTT imaging results have shown that 1mm-diameter FBH located 1mm and 2mm below the surface were detectable. Larger size FBH were detectable at greater depth. For example, 6mm-diameter FBH could be detected at 8mm depth. Image contrast varied slightly between the specimens, with the best reconstructions obtained in SS316 and C276 plates. Finally, PTT capability in imaging several AM structures fabricated from Inconel 718 (IN718) powder feedstock was investigated. The AM structures have complex geometry, but do not have calibrated internal defects. The objective is to determine PTT settings, such as the total integration time, for imaging of representative AM structures. A new FLIR X8500sc IR camera with higher spatial resolution (1280x1024 at 181Hz frame rate) was integrated into the PTT laboratory setup. Software tools were developed to ensure compatibility of new camera files with legacy effusivity reconstruction codes. The new camera was used to obtain 3-D reconstructions of AM IN718 nozzle plate and a particle filter plate. It was shown that PTT can scan through 2/3in-thick Inconel 718 plates in approximately 15s. Several modes of 3-D data visualization were explored, including using ImageJ and MATLAB software packages. The results demonstrate the capability of PTT in 3-D imaging of complex geometry AM structures. Intermediate results demonstrated in this report will be developed further to investigate detection of flaws in AM specimens.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Assessing Reliability of NDE Flaw Detection Using Smaller Number of Demonstration Data Points

The paper provides an engineering analysis approach for assessing reliability of NDE flaw detection using smaller number of demonstration data points. It explores dependence of probability of detection (POD), probability of false positive (POF), on contrast-to-noise ratio, and net decision threshold-to-noise ratio in a simulated data; and draws some generically applicable inferences to devise the approach. ASTM nondestructive evaluation standards provide requirements on signal-to-noise ratio and/or contrast-to-noise ratio in order to provide reliable flaw detection and limit false positive calls. POD analysis of inspection test data results in an estimated flaw size, denoted by 𝑎90/95. This flaw size has 90% POD and minimum 95% confidence. POF is also estimated in the analysis. POD demonstration requires specimens with flaws of known size. In many situations, it is very expensive to produce the large number of flaws required for the POD analysis. In some situations, only real flaws can truly represent the flaws for demonstration. Real flaws of correct size and location in part configuration specimen may be difficult to produce, if not impossible. Here, an engineering analysis approach is devised using simulation to assess reliability of NDE technique when a limited number of flaws are available for demonstration. In this simulation, a technique is considered reliable, if it provides flaw detectability size equal to or better than the theoretical 𝑎90𝑡ℎ used in simulation and also provides a POF less than or equal to a chosen value. The paper uses simulated signal response versus flaw size data to devise the approach. Linear correlation is used between the signal response data and flaw size. POD software mh1823 uses generalized linear model (GLM) in POD analysis after transforming the flaw size and signal response, if needed, using logarithm. Therefore, this approach is in agreement with the linear signal correlation used in mh1823. Using the POD analysis of data, generic conditions on contrast-to-noise ratio and net decision threshold-to-noise ratio are derived for reliable flaw detection. In order to assess technique reliability using the engineering approach, signal response-to-flaw size correlation about the flaw size of concern is needed. In addition, measurement of noise is also needed. If the technique meets the above requirements, assumption of linear signal-to-flaw size correlation and conditions on noise, then the technique can be assessed using this analysis as it fits the underlying POD model used here. The approach is conservative and is designed to provide a larger flaw size compared to the POD approach. Such NDE technique assessment approach, although, not as rigorous as POD, can be cost effective if the larger flaw size can be tolerated. Typically, this is a situation for all quality control NDE inspections. Here, an NDE technique needs to be reliable and 𝑎90/95 is not estimated, but the assessed flaw size is assumed to be larger than the unknown a90 due to conservative factors or margins. Applicability of the approach for assessing reliability of flaw detection in x-ray radiography and 2D imaging in general is also explored.

Koshti, Ajay M.↗

Research and Development of Automated Eddy Current Testing for Composite Overwrapped Pressure Vessels

Eddy current testing (ET) was used to scan bare metallic liners used in the fabrication of composite overwrapped pressure vessels (COPVs) for flaws which could result in premature failure of the vessel. The main goal of the project was to make improvements in the areas of scan signal to noise ratio, sensitivity of flaw detection, and estimation of flaw dimensions. Scan settings were optimized resulting in an increased signal to noise ratio. Previously undiscovered flaw indications were observed and investigated. Threshold criteria were determined for the system software's flaw report and estimation of flaw dimensions were brought to an acceptable level of accuracy. Computer algorithms were written to import data for filtering and a numerical derivative filtering algorithm was evaluated.

Carver, Kyle L.↗

Demonstrating Advanced Sensors for In-Situ Monitoring Towards Qualification of Nuclear Relevant Components

The U.S. Department of Energy’s Office of Nuclear Energy Advanced Materials and Manufacturing Technologies (AMMT) program is pursuing qualification of laser powder bed fusion (LPBF) components for nuclear applications. A major focus of this effort is the use of in situ process monitoring and machine learning–based tools to establish real-time quality assurance. The primary objective of this report is to identify and evaluate the most relevant in situ sensor systems for LPBF, and to document the deployment of these systems across platforms critical to the AMMT program. This work demonstrates how in situ monitoring can detect process anomalies, track geometry-dependent flaws, and identify limiting combinations of processing parameters—particularly those related to energy density and complex geometries (e.g., overhanging structures). To support this goal, a diverse suite of sensor modalities was evaluated across LPBF platforms, including visible and near-infrared (NIR) imaging, fringe projection profilometry, long-wavelength infrared (LWIR) thermography, and high-speed photodiode/pyrometry systems. These sensor streams were integrated with Peregrine, a machine-agnostic software platform that, among other capabilities, can generate real-time process anomaly classification. This report documents sensor deployments on multiple AMMT flagship platforms, including the Concept Laser M2 and Renishaw AM400/AM250 systems. Calibration builds with complex, flaw-prone geometries such as unsupported overhangs, stepped features, and thin walls, were used to evaluate how well Peregrine and its associated sensors could detect process anomalies and other instabilities under varied energy densities. It will be shown how Peregrine reliably identifies common process anomalies such as recoater streaking, superelevation, etc., and can be used in post-build analysis for anomaly spatial distributions throughout the build height to better understand the impact of geometry and processing parameter choice on the build. This work demonstrates measurable progress toward the vision that components can be born-qualified by establishing a real-time monitoring framework, identifying limiting process conditions, and laying the foundation for sensor fusion–enabled prediction pipelines that are scalable across platforms and applicable to nuclear-relevant components.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗