SEARCH · Search NASA

Results for “Software Assurance Research program”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

63 records · Page 4

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

Genesis Solar Wind – Capture, Return, Curate and Analyze: Looking Backward and Creating a Timeline

Introduction: In 1997 NASA’S Discovery Program selected the Genesis mission proposal to return solar wind samples to Earth for laboratory analyses. Principal Investigator Donald S. Burnett and the science team defined the purity of collector materials and ability to analyze solar wind composition to the precision required for planetary science. As a small mission, focused on a well-defined science goal, yet needing careful attention to engineering details, the communication among scientists and engineers, nurtured by Don Burnett, was exceptional. Genesis Mission and Curation Legacy: Genesis, as the first U. S. spacecraft to return astromaterial samples since Apollo, not only integrated the mission planning and flight teams, but also the science and sample curation teams during the mission development period. Since Genesis is a sample return mission, the Science Team was essential in certifying the collectors (sample containers for solar atoms). From inception, Genesis established mission funding for returned sample curation. JSC was lead in contamination control during mission preparation, including establishment of an ISO 4 cleanroom facility and use of ultrapure water (UPW) for cleaning flight hardware (and, as it turned out, for cleaning collectors after the mishap). Reliable, fast communication among scientists, engineers and curators at the hands-on level established deep respect among team members and efficient decision-making. JSC’s 50-years of astromaterial sample curation provided experienced sample processors onsite during recovery in Utah (a deep bench for emergency response). Post-recovery curation included iterative collaboration with science sample users to clean or verify cleanliness of samples. The science legacy from Genesis is addressed by Burnett and Jurewicz, this volume. In The Beginning: After Apollo sample return, Burnett and Marcia Neugebauer at JPL began discussing a solar wind sample return, with Neugebauer arguing that separate collection of solar wind regimes was essential science. By 1992 a solar wind sample return mission was presented at a workshop, and by 1994 a mission was proposed named Suess-Urey. The mission was re-proposed under a new name GENESIS and selected in 1997. Susan Niebur captured the Genesis mission history and stories, from high level management documents and from many interviews with participants [2]. Her account lets readers glimpse personality of participants in quotations from interviews. Need and Scope for Detailed Technical Timeline: A timeline constructed from lower level task documents has been initiated to document the resources and skills actually used, as well as task sequence or concurrency. Timelines for high level mission events are captured in two documents [1] [2] and for detailed re-entry events in [3]. A detailed technical timeline for Genesis mission and curation activities will provide data points for lower level tasks, such as ISO 4 curation facility construction time, preparation for nominal sample field recovery, mishap recovery, and UPW expansion. Changes in technology context 1990-2024: Semiconductor technologies were easily accessible in the U.S.A. (1990-1999), and the Genesis team used those resources for cleanroom design and UPW system expansion. Image documentation was changing from film to digital during cleanroom construction and payload cleaning (1997-2001). Engineering design was done using computer aided design proprietary software, making more difficult the archiving of payload configuration and materials. Email of documents, tracked delivery service and virtual meeting capability greatly improved communication efficiency. Information sources – Pre-launch mission preparation: Examples of mission science, engineering and contamination control are collector purity testing, payload design/fabrication and ISO 4 cleanroom construction. Information on timing of these activities comes from facility readiness reviews, management reviews, shipping documents, procurement documents, test reports, travel documents, laboratory logs, Quality Assurance documents, dates on images, participant notebooks and emails. Information sources – Sample return re-entry and field recovery activities: Information comes from event timelines produced by Mid-Air Recovery team, Lockheed team lead notes and from chase video, JPL Quality Assurance. Information also comes from images and logbooks from UTTR cleanroom operations and from curatorial documents. Information sources – Resulting science and sample cleaning processes: Agendas from the annual gatherings of the science team initially trace testing for collector purity/cleanliness, and after sample recovery, include collector cleaning and cleanliness assessment. Post-recovery documents include curatorial orders and procedures, sample allocation documents and LPSC abstracts. Timeline Objectives: A simple spreadsheet timeline with headers DATE, EVENT, PEOPLE, COMMENT, INFORMATION SOURCE has been initiated and currently has over 90 entries. While this is not definitive historical research, it is a quick look at the evolution of Genesis curation with pointers to documents or people with information. Engineers for future missions may find useful points of comparison for development of facilities. References:[1] Genesis Mission Reference Document, (2011) JPL D-62382.[2] Niebur S. M., edited by Brown D. W. (2023) NASA’s Discovery Program: The First 20 Years of Competitive Planetary Exploration, NASA-SP-2023-4238.[3] Genesis Mishap Investigation Board Report, Vol. 1 (July 2005).

solar wind↗

NASA NTRSDate not supplied

Students Solving Problems for ISS and Beyond: Inspiring the Next-Generation

Students Solving Problems for ISS and Beyond: Inspiring the Next-Generation Session Title: Students Solving Problems for ISS and Beyond: Inspiring the Next-Generation Session Description: NASA High school students United with NASA to Create Hardware (HUNCH) mission is to empower and inspire students through a Project-Based Learning program where 7-12 grade students learn 21st century skills and can launch their careers through participation in the design and fabrication of real-world valued products for NASA. With six different tracks consisting of Design & Prototype, Culinary Challenge, Softgoods, Precision Machining, Software, and Video Challenge, students are given the opportunity to create solutions for the International Space Station (ISS), the Moon, and beyond. Many projects are requested by the Crew to help ease living conditions, giving students the opportunity to make an impact on the lives of Astronauts. Other projects come directly from NASA and its partners. Join our session to learn how NASA is working with teachers across the country to mentor the next generation of scientists and engineers to solve some of NASA’s greatest challenges. Learning Outcomes: 1. Describe the NASA HUNCH Program, including the program objectives, goals, and strategic partnerships for middle school and high school outreach and advocacy efforts. 2. Identify the innovative strategies NASA is using to work with middle and high school students to solve real-world problems 3. Use the knowledge gained to inspire the next generation of scientists and engineers Session Track: Advocacy & Outreach Specialized Focus Area: Women in Government and Military Learning Level: Foundational Session Format: Listen & Learn Speaker Qualifications: 1. Deboshri Sadhukhan • Current Job Title: Deputy Project Manager • Topic Experience (years of experience related to proposed topic): 1-5 years • Biography: Deboshri Sadhukhan is an engineer for NASA Glenn Research Center (GRC). She has worked on numerous projects — from International Space Station (ISS) fluid technologies, to Orion European Service Module propulsion, to planetary science missions and other game-changing technologies. Her roles have ranged from Project Manager to System Safety Lead. She currently serves as a GRC Regional Mentor for the High school students United with NASA to Create Hardware (HUNCH) program. She also serves as Deputy Project Manager for an ISS payload and Safety & Mission Assurance Lead for a Radioisotope Power Systems project. In these roles, she oversees each phase of a project from beginning to end and provides leadership to increase the reliability, maintainability and system safety of hardware and personnel throughout the system life cycle. She holds a Bachelor of Science in Electrical Engineering from The University of Akron. 2. Nancy Hall • Current Job Title: Project Manager • Topic Experience: 20+ years • Biography: Nancy Rabel Hall earned a B.S. degree in Space Sciences from Florida Institute of Technology and a M.S. degree in Mechanical Engineering from the University of Toledo. She has been at NASA Glenn for over 30 years. She has led several International Space Station experiments that studied how the behavior of fluids and fluid systems behave differently in microgravity as compared to here on Earth. She is also the High school students United with NASA to Create Hardware (HUNCH) project manager, a program that allows students to design and fabricate hardware and softgoods for NASA as well as participate in a culinary and video challenge. She enjoys talking to the public and students about the work being done at NASA as well as showing students how math and science can be fun. She is an amateur radio operator, enjoys playing golf, and reading science fiction and fantasy books.

HUNCH↗

NASA NTRS2013

Evolution of International Space Station Program Safety Review Processes and Tools

The International Space Station Program at NASA is constantly seeking to improve the processes and systems that support safe space operations. To that end, the ISS Program decided to upgrade their Safety and Hazard data systems with 3 goals: make safety and hazard data more accessible; better support the interconnection of different types of safety data; and increase the efficiency (and compliance) of safety-related processes. These goals are accomplished by moving data into a web-based structured data system that includes strong process support and supports integration with other information systems. Along with the data systems, ISS is evolving its submission requirements and safety process requirements to support the improved model. In contrast to existing operations (where paper processes and electronic file repositories are used for safety data management) the web-based solution provides the program with dramatically faster access to records, the ability to search for and reference specific data within records, reduced workload for hazard updates and approval, and process support including digital signatures and controlled record workflow. In addition, integration with other key data systems provides assistance with assessments of flight readiness, more efficient review and approval of operational controls and better tracking of international safety certifications. This approach will also provide new opportunities to streamline the sharing of data with ISS international partners while maintaining compliance with applicable laws and respecting restrictions on proprietary data. One goal of this paper is to outline the approach taken by the ISS Progrm to determine requirements for the new system and to devise a practical and efficient implementation strategy. From conception through implementation, ISS and NASA partners utilized a user-centered software development approach focused on user research and iterative design methods. The user-centered approach used on the new ISS hazard system utilized focused user research and iterative design methods employed by the Human Computer Interaction Group at NASA Ames Research Center. Particularly, the approach emphasized the reduction of workload associated with document and data management activities so more resources can be allocated to the operational use of data in problem solving, safety analysis, and recurrence control. The methods and techniques used to understand existing processes and systems, to recognize opportunities for improvement, and to design and review improvements are described with the intent that similar techniques can be employed elsewhere in safety operations. A second goal of this paper is to provide and overview of the web-based data system implemented by ISS. The software selected for the ISS hazard systemMission Assurance System (MAS)is a NASA-customized vairant of the open source software project Bugzilla. The origin and history of MAS as a NASA software project and the rationale for (and advantages of) using open-source software are documented elsewhere (Green, et al., 2009).

Ratterman, Christian D.↗

NASA NTRS2018

Development of a Ground Test and Analysis Protocol for NASA's NextSTEP Phase 2 Habitation Concepts

The NASA Next Space Technologies for Exploration Partnerships (NextSTEP) program is a public-private partnership model that seeks commercial development of deep space exploration capabilities to support human spaceflight missions around and beyond cislunar space. NASA first issued the Phase 1 NextSTEP Broad Agency Announcement to U.S. industries in 2014, which called for innovative cislunar habitation concepts that leveraged commercialization plans for low-Earth orbit. These habitats will be part of the Deep Space Gateway (DSG), the cislunar space station planned by NASA for construction in the 2020s. In 2016, Phase 2 of the NextSTEP program selected five commercial partners to develop ground prototypes. A team of NASA research engineers and subject matter experts (SMEs) have been tasked with developing the ground-test protocol that will serve as the primary means by which these Phase 2 prototypes will be evaluated. Since 2008, this core test team has successfully conducted multiple spaceflight analog mission evaluations utilizing a consistent set of operational tools, methods, and metrics to enable the iterative development, testing, analysis, and validation of evolving exploration architectures, operations concepts, and vehicle designs. The purpose of implementing a similar evaluation process for the Phase 2 Habitation Concepts is to consistently evaluate different commercial partner ground prototypes to provide data-driven, actionable recommendations for Phase 3. This paper describes the process by which the ground test protocol was developed and the objectives, methods, and metrics by which the NextSTEP Phase 2 Habitation Concepts will be rigorously and systematically evaluated. The protocol has been developed using both a top-down and bottom-up approach. Top-down development began with the Human Exploration and Operations Mission Directorate (HEOMD) exploration objectives and ISS Exploration Capability Study Team (IECST) candidate flight objectives. Strategic questions and associated rationales, derived from these candidate architectural objectives, provide the framework by which the ground-test protocol will address the DSG stack elements and configurations, systems and subsystems, and habitation, science, and EVA functions. From these strategic questions, high-level functional requirements for the DSG were drafted and associated ground-test objectives and analysis protocols were established. Bottom-up development incorporated objectives from NASA SMEs in autonomy, avionics and software, communication, environmental control and life support systems, exercise, extravehicular activity, exploration medical operations, guidance navigation and control, human factors and behavioral performance, human factors and habitability, logistics, Mission Control Center operations, power, radiation, robotics, safety and mission assurance, science, simulation, structures, thermal, trash management, and vehicle health. Top-down and bottom-up objectives were integrated to form overall functional requirements - ground-test objectives and analysis mapping. From this mapping, ground-test objectives were organized into those that will be evaluated through inspection, demonstration, analysis, subsystem standalone testing, and human-in-the-loop (HITL) testing. For the HITL tests, mission-like timelines, procedures, and flight rules have been developed to directly meet ground test objectives and evaluate specific functional requirements. Data collected from these assessments will be analyzed to determine the acceptability of habitation element configurations and the combinations of capabilities that will result in the best habitation platform to be recommended by the test team for Phase 3.

Gernhardt, Michael L.↗

NASA NTRS2016

Clinical Outcome Metrics for Optimization of Robust Training

Introduction: The emphasis of this research is on the Human Research Program (HRP) Exploration Medical Capability's (ExMC) "Risk of Unacceptable Health and Mission Outcomes Due to Limitations of In-Flight Medical Capabilities." Specifically, this project aims to contribute to the closure of gap ExMC 2.02: We do not know how the inclusion of a physician crew medical officer quantitatively impacts clinical outcomes during exploration missions. The experiments are specifically designed to address clinical outcome differences between physician and non-physician cohorts in both near-term and longer-term (mission impacting) outcomes. Methods: Medical simulations will systematically compare success of individual diagnostic and therapeutic procedure simulations performed by physician and non-physician crew medical officer (CMO) analogs using clearly defined short-term (individual procedure) outcome metrics. In the subsequent step of the project, the procedure simulation outcomes will be used as input to a modified version of the NASA Integrated Medical Model (IMM) to analyze the effect of the outcome (degree of success) of individual procedures (including successful, imperfectly performed, and failed procedures) on overall long-term clinical outcomes and the consequent mission impacts. The procedures to be simulated are endotracheal intubation, fundoscopic examination, kidney/urinary ultrasound, ultrasound-guided intravenous catheter insertion, and a differential diagnosis exercise. Multiple assessment techniques will be used, centered on medical procedure simulation studies occurring at 3, 6, and 12 months after initial training (as depicted in the following flow diagram of the experiment design). Discussion: Analysis of procedure outcomes in the physician and non-physician groups and their subsets (tested at different elapsed times post training) will allow the team to 1) define differences between physician and non-physician CMOs in terms of both procedure performance (pre-IMM analysis) and overall mitigation of the mission medical impact (IMM analysis); 2) refine the procedure outcome and clinical outcome metrics themselves; 3) refine or develop innovative medical training products and solutions to maximize CMO performance; and 4) validate the methods and products of this experiment for operational use in the planning, execution, and quality assurance of the CMO training process The team has finalized training protocols and developed a software training/testing tool in collaboration with Butler Graphics (Detroit, MI). In addition to the "hands on" medical procedure modules, the software includes a differential diagnosis exercise (limited clinical decision support tool) to evaluate the diagnostic skills of participants. Human subject testing will occur over the next year.

Ebert, D.↗

NASA NTRSDate not supplied

Lunar Search & Rescue Applications of Lunar GNSS

Accurate lunar navigation and timing knowledge provides for the development of safety-critical services in the cislunar and lunar surface domain. Currently under development, the Goddard Space Flight Center’s (GSFC) Search and Rescue Mission Office is investigating and integrating search and rescue (SAR) capability into planned and future lunar communication and navigation interfaces. Lunar Search and Rescue (LunaSAR) development has a stated end-goal for assured, reliable, and timely indication of distress events for a wide variety of lunar surface users, including government-sponsored, commercial, and international users. LunaSAR performance requirements are modelled after the current terrestrial Cospas-Sarsat distress notification system, leveraging an internationally robust global navigation satellite system (GNSS) ecosystem as a core element of survivor locating capability. This presentation will discuss NASA’s work to develop user-focused distress messaging capabilities including infusion of example sensor data for triggering of automated distress alerts coupled with location-tagging. Additionally, the presentation will examine overall message structures, rotating fields for use in bi-directional distress messaging, and specific use cases based on NASA’s lunar exploration and lunar communication relay architectures. Modelling and simulation of LunaSAR use by individual lunar explorers will be discussed, based on notional industry and government design reference missions and mission considerations. Results from GSFC-funded Internal Research and Development (IRAD) efforts will be detailed, including successful distress message formulation simulating the ingestion of example legacy space suit telemetry fields. Hardware-in-the-loop testing using high-reliability software defined radio (SDR) modules serve as an example of IRAD successes and the framework for technical requirements. Architectural development and technical evolution from 2020 to 2021 included alignment of LunaSAR distress waveforms with ongoing NASA LunaNet interoperability development, as well as engagement with NASA Lunar Spectrum authorities for allocation of UHF-band distress frequencies on the lunar surface. S-Band and UHF-band transmission characteristics will be detailed, along with band-specific applications of each emission type. Additionally, examples of ingestion and formatting of GNSS signals (using historical terrestrial National Marine Electronics Association-formatted GNSS data) will be detailed, underscoring lunar user needs for a common lunar GNSS receiver output message framework. Maturity and ability to support evolving lunar exploration goals has been demonstrated and will be detailed, with maturity gaps such as position, navigation, and timing (PNT) and lunar reference frames identified within the context of distress message generation. Provision of LunaSAR services for lunar surface users represents a new era of ensured safety for lunar explorers and builds off of forty years of the Cospas-Sarsat program, underscoring the importance of lunar GNSS for safety-critical applications and growing interest in safe, reliable lunar surface operations. Enabled by new GNSS systems being developed by government and industry partners, NASA will continue to evolve and integrate lunar GNSS types into distress message generation, with a focus on compact and efficient message transmission over various lunar communication links. When fielded, LunaSAR will be the first dedicated search and rescue notification system employed on another celestial body. Robust lunar navigation and timing services form the core of LunaSAR capabilities, allowing for system syncing with time-dominant sensors, and high-accuracy location of those in distress while engaged in lunar surface activities.

Search and Rescue↗

NASA NTRS2004

Future of Fuel Savings

Using automation to free up controllers for more strategic management of air traffic is one approach being studied by NASA as it seeks to boost airspace system capacity and efficiency, thereby saving fuel. Heinz Erzberger, a NASA Ames Research Center senior scientist, says the Advanced Airspace Concept (AAC) has been studied for several years. It could increase efficiency 15% by providing optimal routes that cut airlines direct operating costs. A 25% increase in landings on existing runways could follow an important benefit. AAC is one of the efforts to be reviewed by the Joint Planning and Development Organization, an FAA-led initiative by six federal agencies to redesign the U.S. air transportation system by 2025. The main goal is to triple air traffic capacity within 20 years to avert the sort of gridlock that would make fuel consumption only one of many travel nightmares. The automated system approach would allow aircraft to fly optimal trajectories. A trajectory would be defined in the standard three dimensions and eventually include the fourth, time. The management of air traffic by the data-linked exchange of trajectories would start at high altitude and eventually move down to lower altitudes. The automated concept is an outgrowth of the type of tools developed by NASA for use by FAA controllers in managing traffic flows over the years, including ones that optimize routings for the best fuel burn. But AAC would push automation further to reduce workload so controllers can focus on "solving strategic control problems, managing traffic flow during changing weather and ... other unusal events." One key component, the automated trajectory server (ATS), is a ground systems that would rely on software to manage flight path requests from aircrews and controllers. But, Erzberger acknowledges, "The FAA's current plan for upgrades to air traffic services does not include [allowing] the future ground system to issue separation-critical clearances of trajectory changes autonomously to aircraft via data link without explicit approval of a controller," as the AAC proposes. The AAC enables pilots or controllers to data link requests for a trajectory change to the ATS for approval after they are deconflicted with the paths of other aircraft. To divert around storms, for example, pilots could data link their trajectory preference to the ATS. Since several aircraft might request similar routes, the computer would then have to suggest alternatives. This could be accomplished without pilot-controller radio calls, a big bottleneck now. The ATS would have a built-in conflict monitor to call for a resolution (turn, climb or descend), when loss of separation is likely in 1-20 min. The AAC system would reduce controller errors by 90%, according to NASA Ames estimates. The AAC would have a back-up program to assure separation-Tactical Separation Assurance (TSAFE). It s designed to detect short-term traffic conflicts within 3-4 min. of loss of separation. The last line of defense would still be provided by traffic alert & collision avoidance systems (TCAS).

Hughes, David↗

NASA NTRS2005

Formal Safety Certification of Aerospace Software

In principle, formal methods offer many advantages for aerospace software development: they can help to achieve ultra-high reliability, and they can be used to provide evidence of the reliability claims which can then be subjected to external scrutiny. However, despite years of research and many advances in the underlying formalisms of specification, semantics, and logic, formal methods are not much used in practice. In our opinion this is related to three major shortcomings. First, the application of formal methods is still expensive because they are labor- and knowledge-intensive. Second, they are difficult to scale up to complex systems because they are based on deep mathematical insights about the behavior of the systems (t.e., they rely on the "heroic proof"). Third, the proofs can be difficult to interpret, and typically stand in isolation from the original code. In this paper, we describe a tool for formally demonstrating safety-relevant aspects of aerospace software, which largely circumvents these problems. We focus on safely properties because it has been observed that safety violations such as out-of-bounds memory accesses or use of uninitialized variables constitute the majority of the errors found in the aerospace domain. In our approach, safety means that the program will not violate a set of rules that can range for the simple memory access rules to high-level flight rules. These different safety properties are formalized as different safety policies in Hoare logic, which are then used by a verification condition generator along with the code and logical annotations in order to derive formal safety conditions; these are then proven using an automated theorem prover. Our certification system is currently integrated into a model-based code generation toolset that generates the annotations together with the code. However, this automated formal certification technology is not exclusively constrained to our code generator and could, in principle, also be integrated with other code generators such as RealTime Workshop or even applied to legacy code. Our approach circumvents the historical problems with formal methods by increasing the degree of automation on all levels. The restriction to safety policies (as opposed to arbitrary functional behavior) results in simpler proof problems that can generally be solved by fully automatic theorem proves. An automated linking mechanism between the safety conditions and the code provides some of the traceability mandated by process standards such as DO-178B. An automated explanation mechanism uses semantic markup added by the verification condition generator to produce natural-language explanations of the safety conditions and thus supports their interpretation in relation to the code. It shows an automatically generated certification browser that lets users inspect the (generated) code along with the safety conditions (including textual explanations), and uses hyperlinks to automate tracing between the two levels. Here, the explanations reflect the logical structure of the safety obligation but the mechanism can in principle be customized using different sets of domain concepts. The interface also provides some limited control over the certification process itself. Our long-term goal is a seamless integration of certification, code generation, and manual coding that results in a "certified pipeline" in which specifications are automatically transformed into executable code, together with the supporting artifacts necessary for achieving and demonstrating the high level of assurance needed in the aerospace domain.

Denney, Ewen↗

NASA NTRS2013

NASA Software Engineering Benchmarking Study

To identify best practices for the improvement of software engineering on projects, NASA's Offices of Chief Engineer (OCE) and Safety and Mission Assurance (OSMA) formed a team led by Heather Rarick and Sally Godfrey to conduct this benchmarking study. The primary goals of the study are to identify best practices that: Improve the management and technical development of software intensive systems; Have a track record of successful deployment by aerospace industries, universities [including research and development (R&D) laboratories], and defense services, as well as NASA's own component Centers; and Identify candidate solutions for NASA's software issues. Beginning in the late fall of 2010, focus topics were chosen and interview questions were developed, based on the NASA top software challenges. Between February 2011 and November 2011, the Benchmark Team interviewed a total of 18 organizations, consisting of five NASA Centers, five industry organizations, four defense services organizations, and four university or university R and D laboratory organizations. A software assurance representative also participated in each of the interviews to focus on assurance and software safety best practices. Interviewees provided a wealth of information on each topic area that included: software policy, software acquisition, software assurance, testing, training, maintaining rigor in small projects, metrics, and use of the Capability Maturity Model Integration (CMMI) framework, as well as a number of special topics that came up in the discussions. NASA's software engineering practices compared favorably with the external organizations in most benchmark areas, but in every topic, there were ways in which NASA could improve its practices. Compared to defense services organizations and some of the industry organizations, one of NASA's notable weaknesses involved communication with contractors regarding its policies and requirements for acquired software. One of NASA's strengths was its software assurance practices, which seemed to rate well in comparison to the other organizational groups and also seemed to include a larger scope of activities. An unexpected benefit of the software benchmarking study was the identification of many opportunities for collaboration in areas including metrics, training, sharing of CMMI experiences and resources such as instructors and CMMI Lead Appraisers, and even sharing of assets such as documented processes. A further unexpected benefit of the study was the feedback on NASA practices that was received from some of the organizations interviewed. From that feedback, other potential areas where NASA could improve were highlighted, such as accuracy of software cost estimation and budgetary practices. The detailed report contains discussion of the practices noted in each of the topic areas, as well as a summary of observations and recommendations from each of the topic areas. The resulting 24 recommendations from the topic areas were then consolidated to eliminate duplication and culled into a set of 14 suggested actionable recommendations. This final set of actionable recommendations, listed below, are items that can be implemented to improve NASA's software engineering practices and to help address many of the items that were listed in the NASA top software engineering issues. 1. Develop and implement standard contract language for software procurements. 2. Advance accurate and trusted software cost estimates for both procured and in-house software and improve the capture of actual cost data to facilitate further improvements. 3. Establish a consistent set of objectives and expectations, specifically types of metrics at the Agency level, so key trends and models can be identified and used to continuously improve software processes and each software development effort. 4. Maintain the CMMI Maturity Level requirement for critical NASA projects and use CMMI to measure organizations developing software for NASA. 5.onsolidate, collect and, if needed, develop common processes principles and other assets across the Agency in order to provide more consistency in software development and acquisition practices and to reduce the overall cost of maintaining or increasing current NASA CMMI maturity levels. 6. Provide additional support for small projects that includes: (a) guidance for appropriate tailoring of requirements for small projects, (b) availability of suitable tools, including support tool set-up and training, and (c) training for small project personnel, assurance personnel and technical authorities on the acceptable options for tailoring requirements and performing assurance on small projects. 7. Develop software training classes for the more experienced software engineers using on-line training, videos, or small separate modules of training that can be accommodated as needed throughout a project. 8. Create guidelines to structure non-classroom training opportunities such as mentoring, peer reviews, lessons learned sessions, and on-the-job training. 9. Develop a set of predictive software defect data and a process for assessing software testing metric data against it. 10. Assess Agency-wide licenses for commonly used software tools. 11. Fill the knowledge gap in common software engineering practices for new hires and co-ops.12. Work through the Science, Technology, Engineering and Mathematics (STEM) program with universities in strengthening education in the use of common software engineering practices and standards. 13. Follow up this benchmark study with a deeper look into what both internal and external organizations perceive as the scope of software assurance, the value they expect to obtain from it, and the shortcomings they experience in the current practice. 14. Continue interactions with external software engineering environment through collaborations, knowledge sharing, and benchmarking.

Rarick, Heather L.↗