SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
The Employment of Defects (EoD) approach to measuring and analyzing defects seeks to identify and capture trends and phenomena that are critical to managing software quality in the iterative software development lifecycle at JPL.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
The multi-scale and complex process of printing additively manufactured (AM) parts can have unexpected, but predictable, build conditions that result in material microstructure variability. In this work, we describe a fully parallel reduced order modeling approach that has been developed to evaluate the evolution of AM processes, termed the AM moment measure method. This method couples the known sequence of the AM process with a physically informed nearest neighbors’ calculation to map the conditions of a part-scale build. The result is a map of the build that is derived directly from build files or in-situ process monitoring sensors. The methodology and terminology of the approach will be described, and computed build maps will be calculated and compared for various laser powder bed fusion (LPBF) builds of Ti-6Al-4V. Such comparative results develop understanding of how the sequential process actions can affect the LPBF-AM build quality and microstructure variability.
Explore the source record for details and available documents.
The purpose of health monitoring systems is to detect failures or defects for increased safety and performance and to provide on-condition maintenance with reduced costs. The problems associated with health monitoring systems include high rates of false alarms and missed failures, which make monitoring an unreliable and costly task. The reason for this is that unaccounted variations invalidate signal modeling assumptions. Our approach was to focus on vibration monitoring of rotating components. We analyzed baseline signals to determine statistical variations, identify and model factors that influence vibrations (pre-production vs. post-production variations), determine hit and false alarm rates with baseline flight data, model and predict effects of defects and variations on vibrations, and develop algorithms and metrics for failure and anomaly detection in the presence of variations.
Components fabricated using the powder bed fusion laser beam metallic (PBF-LB/M) additive manufacturing (AM) process comprise a multitude of sequential weld passes. Qualifying materials for components in critical applications requires a thorough understanding of pertinent processing-microstructure-property relationships. Porosity defects have a strong adverse effect on the mechanical properties of a material. The morphology of porosity defects created by lack of fusion in PBF-LB/M often exhibit high aspect ratios that can act as stress concentrators and become crack initiation sites. The occurrence of spatter induced porosity is studied using synchronized AM model-based process metrics and porosity measured by high-resolution X-ray computed tomography. A novel AM process metric is introduced to quantify the relative exposure to spatter ejecta impacts and incorporation throughout the sequence of the PBF-LB/M process. Further characterization using optical metallography and scanning electron microscopy indicated that both keyhole and lack of fusion porosity correlated statistically with the spatter ejecta impact field of the PBF-LB/M process. Both keyhole and lack of fusion porosity can be predicted by the spatter exposure process metric, which can be exploited to control the fusion sequence.
A great deal of effort is now being devoted to the study, analysis, prediction, and minimization of software maintenance expected cost, long before software is delivered to users or customers. It has been estimated that, on the average, the effort spent on software maintenance is as costly as the effort spent on all other software costs. Software design methods should be the starting point to aid in alleviating the problems of software maintenance complexity and high costs. Two aspects of maintenance deserve attention: (1) protocols for locating and rectifying defects, and for ensuring that noe new defects are introduced in the development phase of the software process; and (2) protocols for modification, enhancement, and upgrading. This article focuses primarily on the second aspect, the development of protocols to help increase the quality and reduce the costs associated with modifications, enhancements, and upgrades of existing software. This study developed parsimonious models and a relative complexity metric for complexity measurement of software that were used to rank the modules in the system relative to one another. Some success was achieved in using the models and the relative metric to identify maintenance-prone modules.
Components fabricated using the powder bed fusion laser beam metallic (PBF-LB/M)additive manufacturing process are the result of a multitude of weld passes conducted sequentially. Qualifying components for aerospace applications requires a thorough understanding of the process-structure-properties relationships. Porosity defects are known to have a strong adverse effect on the mechanical properties of a component. In particular, porosity defects created by lack of fusion have high aspect ratio morphologies leading to stress concentrations that become crack initiation sites. In the present work, the occurrence of spatter induced lack of fusion porosity was studied using synchronized in-situ process monitoring, additive manufacturing model-based process metrics, and high-resolution X-ray computed tomography. The results show that lack of fusion porosity is statistically correlated with unremoved welding spatter ejecta of the PBF-LB/M process and process metrics related to the hatching strategy.
Components fabricated using the powder bed fusion laser beam metallic (PBF) additive manufacturing (AM) process comprise a multitude of sequential weld passes. Porosity defects resulting from weld pass inconsistencies are a concern for PBF materials and have a strong adverse effect on the mechanical properties. In an effort to understand and avoid spatter induced porosity, this work aims to provide a novel computational technique that can be used to quantify and predict the sequence-sensitive impact of spatter upon the PBF process stability and consequential porosity in as-printed material. A novel spatter impact AM model-based process metric (AM-PM) is introduced to model and quantify the impact that spatter has on the PBF process. The occurrence of spatter induced porosity is studied using thermal rise, lack of fusion, and spatter impact AM-PMs synchronized with porosity measured by high-resolution X-ray computed tomography. Six cylindrical specimens were designed to compare the AM-PM correlations with porosity across two different laser powers and three hatch scanning strategies. A point field based computational approach is shown to be effective for testing the influence of the different AM-PMs and interrogating the physical process conditions underlying the porosity formation. Further, characterization using optical metallography and scanning electron microscopy indicated that both keyhole and lack of fusion porosity correlated with the spatter impact AM-PM, which can be exploited to predict and control spatter related porosity by the hatch strategy. Efforts to predict porosity composition within PBF material should incorporate the PBF process disturbances that can result from spatter. The spatter impact AM-PM can be readily incorporated into the development of defect prediction models at the part scale.
It is often the case in software projects that when schedule and budget resources are limited, the Verification and Validation (V&V) activities suffer. Fewer V&V activities can be afforded and moreover, short-term challenges can result in V&V activities being scaled back or dropped altogether. As a result, too often the default solution is to save activities for improving software quality until too late in the life-cycle, relying on late-term code inspections followed by thorough testing activities to reduce defect counts to acceptable levels. As many project managers realize, however, this is a resource-intensive way of achieving the required quality for software. The Full Life-cycle Defect Management Assessment Initiative, funded by NASA s Office of Safety and Mission Assurance under the Software Assurance Research Program, aims to address these problems by: Improving the effectiveness of early life-cycle V&V activities to make their benefits more attractive to team leads. Specifically, we focus on software inspection, a proven method that can be applied to any software work product, long before executable code has been developed; Better communicating this effectiveness to software development teams, along with suggestions for parameters to improve in the future to increase effectiveness; Analyzing the impact of early life-cycle V&V on the effectiveness and cost required for late life-cycle V&V activities, such as testing, in order to make the tradeoffs more apparent. This white paper reports on an initial milestone in this work, the development of a preliminary model of inspection effectiveness across multiple NASA Centers. This model contributes toward reaching our project goals by: Allowing an examination of inspection parameters, across different types of projects and different work products, for an analysis of factors that impact defect detection effectiveness. Allowing a comparison of this NASA-specific model to existing recommendations in the literature regarding how to plan effective inspections. Forming a baseline model which can be extended to incorporate factors describing: the numbers and types of defects that are missed by inspections; how such defects flow downstream through software development phases; how effectively they can be caught by testing activities in the late stages of development. The model has been implemented in a prototype web-enabled decision-support tool which allows developers to enter their inspection data and receive feedback based on a comparison against the model. The tool also allows users to access reusable materials (such as checklists) from projects included in the baseline. Both the tool itself and the model underlying it will continue to be extended throughout the remainder of this initiative. As results of analyzing inspection effectiveness for defect containment are determined, they can be shared via the tool and also via updates to existing training courses on metrics and software inspections. Moreover, the tool will help satisfy key CMMI requirements for the NASA Centers, as it will enable NASA to take a global view across peer review results for various types of projects to identify systemic problems. This analysis can result in continuous improvements to the approach to verification.
This study benchmarks several FSS techniques and reports several studies where a large set metrics were reduced to a handful with little loss of detection accuracy. This result raises the possibility that software defect detection may be much simpler than previously believed.
Over the years many engineering disciplines have developed, including chemical, electronic, etc. Common to all engineering disciplines is the use of rigor, models, metrics, and predefined methodologies. Recently, a new engineering discipline has appeared on the scene, called software engineering. For over thirty years computer software has been developed and the track record has not been good. Software development projects often miss schedules, are over budget, do not give the user what is wanted, and produce defects. One estimate is there are one to three defects per 1000 lines of deployed code. More and more systems are requiring larger and more complex software for support. As this requirement grows, the software development problems grow exponentially. It is believed that software quality can be improved by applying engineering principles. Another compelling reason to bring the engineering disciplines to software development is productivity. It has been estimated that productivity of producing software has only increased one to two percent a year in the last thirty years. Ironically, the computer and its software have contributed significantly to the industry-wide productivity, but computer professionals have done a poor job of using the computer to do their job. Engineering disciplines and methodologies are now emerging supported by software tools that address the problems of software development. This paper addresses some of the current software engineering methodologies as a backdrop for the general evaluation of computer assisted software engineering (CASE) tools from actual installation of and experimentation with some specific tools.
Ultrasonic damage detection and characterization is commonly used in nondestructive evaluation (NDE) of aerospace composite components. In recent years there has been an increased development of guided wave based methods. In real materials and structures, these dispersive waves result in complicated behavior in the presence of complex damage scenarios. Model-based characterization methods utilize accurate three dimensional finite element models (FEMs) of guided wave interaction with realistic damage scenarios to aid in defect identification and classification. This work describes an inverse solution for realistic composite damage characterization by comparing the wavenumber-frequency spectra of experimental and simulated ultrasonic inspections. The composite laminate material properties are first verified through a Bayesian solution (Markov chain Monte Carlo), enabling uncertainty quantification surrounding the characterization. A study is undertaken to assess the efficacy of the proposed damage model and comparative metrics between the experimental and simulated output. The FEM is then parameterized with a damage model capable of describing the typical complex damage created by impact events in composites. The damage is characterized through a transdimensional Markov chain Monte Carlo solution, enabling a flexible damage model capable of adapting to the complex damage geometry investigated here. The posterior probability distributions of the individual delamination petals as well as the overall envelope of the damage site are determined.
The presentation will provide a brief overview of the SQI measurement program as well as describe each of these models and how they are currently being used in supporting JPL project, task and software managers to estimate and plan future software systems and subsystems.
NASA Langley Research Center (LaRC) is applying flow forming technology to aerospace aluminum alloys for the manufacture of integrally stiffened cylinders that meet aircraft fuselage or launch vehicle specifications. High-strength 2xxx series alloys, in various temper/product form conditions, are being compared with medium-strength, very formable AA 6061 as the benchmark. Screening of candidate materials traditionally relies on empirical forming trials, but more recently on metrics derived from constant nominal strain rate tensile testing. In this work, the established hierarchy of formability metrics is adopted as the baseline; tensile area reduction, work hardening exponent, and modulus of resilience. The suite of metrics can be expanded based on phenomenological and mechanistic considerations of plastic flow instabilities to incorporate anisotropy ratio, rate of work hardening, and dynamic strain aging effects. Separate research thrusts at LaRC are aimed at formulating a protocol for scalable production of fully formed, defect-free components: 1) quantifying the extended set of formability metrics from constant and jump strain rate tensile tests with digital image correlation (DIC); 2) using statistically designed experiments on a vertical spin/flow forming facility (WF VUD-600®) to isolate the effect of convenient process variables, such as axial feed rate and thickness reduction; and 3) predicting the effect of feed ratio (roller translation relative to mandrel rotation speeds) on material response via process modeling using the DEFORM® simulation software. Current results will be discussed in terms of correlating intrinsic material properties with the bulk workability characteristics to successfully fabricate cylinders with complex cross-sections.
In the increasingly sophisticated world of high-definition flat screen monitors and television screens, image clarity and the elimination of distortion are paramount concerns. As the devices that reproduce images become more and more sophisticated, so do the technologies that verify their accuracy. By simulating the manner in which a human eye perceives and interprets a visual stimulus, NASA scientists have found ways to automatically and accurately test new monitors and displays. The Spatial Standard Observer (SSO) software metric, developed by Dr. Andrew B. Watson at Ames Research Center, measures visibility and defects in screens, displays, and interfaces. In the design of such a software tool, a central challenge is determining which aspects of visual function to include while accuracy and generality are important, relative simplicity of the software module is also a key virtue. Based on data collected in ModelFest, a large cooperative multi-lab project hosted by the Optical Society of America, the SSO simulates a simplified model of human spatial vision, operating on a pair of images that are viewed at a specific viewing distance with pixels having a known relation to luminance. The SSO measures the visibility of foveal spatial patterns, or the discriminability of two patterns, by incorporating only a few essential components of vision. These components include local contrast transformation, a contrast sensitivity function, local masking, and local pooling. By this construction, the SSO provides output in units of "just noticeable differences" (JND) a unit of measure based on the assumed smallest difference of sensory input detectable by a human being. Herein is the truly amazing ability of the SSO, while conventional methods can manipulate images, the SSO models human perception. This set of equations actually defines a mathematical way of working with an image that accurately reflects the way in which the human eye and mind behold a stimulus. The SSO is intended for a wide variety of applications, such as evaluating vision from unmanned aerial vehicles, measuring visibility of damage to aircraft and to the space shuttles, predicting outcomes of corrective laser eye surgery, inspecting displays during the manufacturing process, estimating the quality of compressed digital video, evaluating legibility of text, and predicting discriminability of icons or symbols in a graphical user interface.
To identify best practices for the improvement of software engineering on projects, NASA's Offices of Chief Engineer (OCE) and Safety and Mission Assurance (OSMA) formed a team led by Heather Rarick and Sally Godfrey to conduct this benchmarking study. The primary goals of the study are to identify best practices that: Improve the management and technical development of software intensive systems; Have a track record of successful deployment by aerospace industries, universities [including research and development (R&D) laboratories], and defense services, as well as NASA's own component Centers; and Identify candidate solutions for NASA's software issues. Beginning in the late fall of 2010, focus topics were chosen and interview questions were developed, based on the NASA top software challenges. Between February 2011 and November 2011, the Benchmark Team interviewed a total of 18 organizations, consisting of five NASA Centers, five industry organizations, four defense services organizations, and four university or university R and D laboratory organizations. A software assurance representative also participated in each of the interviews to focus on assurance and software safety best practices. Interviewees provided a wealth of information on each topic area that included: software policy, software acquisition, software assurance, testing, training, maintaining rigor in small projects, metrics, and use of the Capability Maturity Model Integration (CMMI) framework, as well as a number of special topics that came up in the discussions. NASA's software engineering practices compared favorably with the external organizations in most benchmark areas, but in every topic, there were ways in which NASA could improve its practices. Compared to defense services organizations and some of the industry organizations, one of NASA's notable weaknesses involved communication with contractors regarding its policies and requirements for acquired software. One of NASA's strengths was its software assurance practices, which seemed to rate well in comparison to the other organizational groups and also seemed to include a larger scope of activities. An unexpected benefit of the software benchmarking study was the identification of many opportunities for collaboration in areas including metrics, training, sharing of CMMI experiences and resources such as instructors and CMMI Lead Appraisers, and even sharing of assets such as documented processes. A further unexpected benefit of the study was the feedback on NASA practices that was received from some of the organizations interviewed. From that feedback, other potential areas where NASA could improve were highlighted, such as accuracy of software cost estimation and budgetary practices. The detailed report contains discussion of the practices noted in each of the topic areas, as well as a summary of observations and recommendations from each of the topic areas. The resulting 24 recommendations from the topic areas were then consolidated to eliminate duplication and culled into a set of 14 suggested actionable recommendations. This final set of actionable recommendations, listed below, are items that can be implemented to improve NASA's software engineering practices and to help address many of the items that were listed in the NASA top software engineering issues. 1. Develop and implement standard contract language for software procurements. 2. Advance accurate and trusted software cost estimates for both procured and in-house software and improve the capture of actual cost data to facilitate further improvements. 3. Establish a consistent set of objectives and expectations, specifically types of metrics at the Agency level, so key trends and models can be identified and used to continuously improve software processes and each software development effort. 4. Maintain the CMMI Maturity Level requirement for critical NASA projects and use CMMI to measure organizations developing software for NASA. 5.onsolidate, collect and, if needed, develop common processes principles and other assets across the Agency in order to provide more consistency in software development and acquisition practices and to reduce the overall cost of maintaining or increasing current NASA CMMI maturity levels. 6. Provide additional support for small projects that includes: (a) guidance for appropriate tailoring of requirements for small projects, (b) availability of suitable tools, including support tool set-up and training, and (c) training for small project personnel, assurance personnel and technical authorities on the acceptable options for tailoring requirements and performing assurance on small projects. 7. Develop software training classes for the more experienced software engineers using on-line training, videos, or small separate modules of training that can be accommodated as needed throughout a project. 8. Create guidelines to structure non-classroom training opportunities such as mentoring, peer reviews, lessons learned sessions, and on-the-job training. 9. Develop a set of predictive software defect data and a process for assessing software testing metric data against it. 10. Assess Agency-wide licenses for commonly used software tools. 11. Fill the knowledge gap in common software engineering practices for new hires and co-ops.12. Work through the Science, Technology, Engineering and Mathematics (STEM) program with universities in strengthening education in the use of common software engineering practices and standards. 13. Follow up this benchmark study with a deeper look into what both internal and external organizations perceive as the scope of software assurance, the value they expect to obtain from it, and the shortcomings they experience in the current practice. 14. Continue interactions with external software engineering environment through collaborations, knowledge sharing, and benchmarking.