Understanding the nature of software evolution
In this paper, we analyze the measurements of structural evolution and fault counts obtained from the JPL flight software technology development effort.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
In this paper, we analyze the measurements of structural evolution and fault counts obtained from the JPL flight software technology development effort.
Reuse of products, processes and knowledge will be the key to enable the software industry to achieve the dramatic improvement in productivity and quality required to satisfy the anticipated growing demand. Although experience shows that certain kinds of reuse can be successful, general success has been elusive. A software life-cycle technology which allows broad and extensive reuse could provide the means to achieving the desired order-of-magnitude improvements. The scope of a comprehensive framework for understanding, planning, evaluating and motivating reuse practices and the necessary research activities is outlined. As a first step towards such a framework, a reuse-enabling software evolution environment model is introduced which provides a basis for the effective recording of experience, the generalization and tailoring of experience, the formalization of experience, and the (re-)use of experience.
In developing a software system, we would like to estimate the way in which the fault content changes during its development, as well determine the locations having the highest concentration of faults.
In developing a software system, we would like to estimate the way in which the fault content changes during its development, as well determine the locations having the highest concentration of faults. In the phases prior to test, however, there may be very little direct information regarding the number and location of faults. This lack of direct information requires developing a fault surrogate from which the number of faults and their location can be estimated. We develop a fault surrogate based on changes in the fault index, a synthetic measure which has been successfully used as a fault surrogate in previous work. We show that changes in the fault index can be used to estimate the rates at which faults are inserted into a system between successive revisions. We can then continuously monitor the total number of faults inserted into a system, the residual fault content, and identify those portions of a system requiring the application of additional fault detection and removal resources.
All software systems of significant size and longevity eventually undergo changes to their basic architectural structure. Such changes may be prompted by evolving requirements, changing technology, or other reasons. Whatever the cause, software architecture evolution is commonplace in real world software projects. Recently, software architecture researchers have begun to study this phenomenon in depth. However, this work has suffered from problems of validation; research in this area has tended to make heavy use of toy examples and hypothetical scenarios and has not been well supported by real world examples. To help address this problem, I describe an ongoing effort at the Jet Propulsion Laboratory to re-architect the Advanced Multimission Operations System (AMMOS), which is used to operate NASA's deep-space and astrophysics missions. Based on examination of project documents and interviews with project personnel, I describe the goals and approach of this evolution effort and then present models that capture some of the key architectural changes. Finally, I demonstrate how approaches and formal methods from my previous research in architecture evolution may be applied to this evolution, while using languages and tools already in place at the Jet Propulsion Laboratory.
The growth in computer hardware performance, coupled with reduced energy requirements, has led to a rapid expansion of the resources available to software systems, driving them towards greater logical abstraction, flexibility, and complexity. This shift in focus from compacting functionality into a limited field towards developing layered, multi-state architectures in a grand field has both driven and been driven by the history of embedded processor design in the robotic spacecraft industry.The combinatorial growth of interprocess conditions is accompanied by benefits (concurrent development, situational autonomy, and evolution of goals) and drawbacks (late integration, non-deterministic interactions, and multifaceted anomalies) in achieving mission success, as illustrated by the case of the Mars Reconnaissance Orbiter. Approaches to optimizing the benefits while mitigating the drawbacks have taken the form of the formalization of requirements, modular design practices, extensive system simulation, and spacecraft data trend analysis. The growth of hardware capability and software complexity can be expected to continue, with future directions including stackable commodity subsystems, computer-generated algorithms, runtime reconfigurable processors, and greater autonomy.
The National Fire Protection Association, with support from the Department of Energy, executed a multi-year initiative to develop, enhance, and disseminate Distributed Energy Resources Safety Training (DERST) tools for U.S. emergency responders. As Distributed Energy Resources (DER)—such as solar photovoltaics, battery energy storage systems (ESS), electric vehicles (EVs), and associated infrastructure—become increasingly prevalent, the NFPA identified a critical need for up-to-date standardized, accessible, and effective safety training tailored for the fire service and related public safety professionals. The project delivered a comprehensive suite of educational resources to improve responders’ abilities to safely manage DER-related incidents. This included: • Revised Modular Training Courses: Updated classroom-based DER safety courses, now modular and accessible nationwide through fire academies and the North American Fire Training Directors (NAFTD) network. • Live Burn Testing & Research: A full-scale controlled burn of a DER-equipped residential structure provided real-world data and insights, forming the basis for updated best practices. • A Gamified Simulation Tool – Firefighters Incident Response Simulation Tool (FIRST): A first-of-its-kind, multiplayer, scenario-based simulation using the Unreal Engine 5.0 to train responders in a realistic virtual, multi-DER incident environment. • Field Familiarization Software Tools & Prop Guide: Digital DER field familiarization evolutions software guide and a prop development manual to support field-based DER training exercises, enhancing responders' hands-on familiarity with DER infrastructure and collaboration on virtual incident responses. • National Dissemination Strategy: Strategic partnerships with NAFTD, Vector Solutions, and others enabled wide-scale distribution, with over 5,000 departments accessing resources and 1,100+ departments adopting the simulator in the first seven months. Also provided a web portal for easy access to all training and simulation programs developed under this grant for the U.S. responder community. Key findings from the project—particularly from the burn test—led to paradigm shifts in fire response tactics. For example, traditional approaches to garage fires may be hazardous if DERs are present, due to explosive off gassing and thermal runaway risks. The new training emphasizes scene assessment, stand-off approaches, thermal imaging verification, and careful post-incident cooling of DER components to prevent reignition. This initiative has had a significant national impact, raising awareness, enhancing preparedness, and supporting safer DER incident response practices. Significant engagement from the media, public safety organizations, and PBS coverage has further amplified the reach and adoption of NFPA’s DER safety training, tools, and simulations.
The Space Telecommunications Radio System (STRS) project investigated various Software Defined Radio (SDR) architectures for Space. An STRS architecture has been selected that separates the STRS operating environment from its various waveforms and also abstracts any specialized hardware to limit its effect on the operating environment. The design supports software evolution where new functionality is incorporated into the radio. Radio hardware functionality has been moving from hardware based ASICs into firmware and software based processors such as FPGAs, DSPs and General Purpose Processors (GPPs). Use cases capture the requirements of a system by describing how the system should interact with the users or other systems (the actors) to achieve a specific goal. The Unified Modeling Language (UML) is used to illustrate the Use Cases in a variety of ways. The Top Level Use Case diagram shows groupings of the use cases and how the actors are involved. The state diagrams depict the various states that a system or object may be in and the transitions between those states. The sequence diagrams show the main flow of activity as described in the use cases.
Much attention has been given to the climate dynamics and habitable boundaries of synchronously rotating planets around low mass stars. However, other rotational states are possible, including spin–orbit resonant configurations, particularly when higher eccentricity orbits can be maintained in a system. Additionally, the oscillating strain as a planet moves from periastron to apoastron results in friction and tidal heating, which can be an important energy source. Here, we simulate the climate of ocean-covered planets near the inner edge of the habitable zone around M to solar stars with the NASA GISS ROCKE-3D general circulation model, and leverage the planetary evolution software package, VPLanet, to calculate tidal heating rates for Earth-sized planets orbiting 2600 and 3000 K stars. This study is the first to use a 3D general circulation model that implements tidal heating to investigate habitability for multiple resonant states. We find that for reference experiments without tidal heating, the resonant state has little impact on the radial position of the inner edge because for a given stellar flux, higher-order states tend to be warmer than synchronous rotators, but for a given temperature, have drier upper atmospheres. However, when strong tidal heating is present, the rotational component implies a strong dependence of habitable conditions on the system evolution and rotational state. Since tidal and stellar heating both decrease rapidly with orbital distance, this results in a compact orbital width separating temperate and uninhabitable climates. We summarize these results and also compare ROCKE-3D to previously published simulations of the inner edge.
The use of a software toolkit and development methodology that supports software reuse is described. The toolkit includes source-code-level library modules and stand-alone tools which support such tasks as data reformatting and report generation, simple relational database applications, user interfaces, tactical planning, strategic planning and documentation. The current toolkit is written in C and supports applications that run on IBM-PC's under DOS and UNlX-based workstations under OpenLook and Motif. The toolkit is fully integrated for building scheduling systems that reuse AI knowledge base technology. A typical scheduling scenario and three examples of applications that utilize the reuse toolkit will be briefly described. In addition to the tools themselves, a description of the software evolution and reuse methodology that was used is presented.
This reference guide for developers of intelligent monitoring systems is based on lessons learned by developers of the DEcision Support SYstem (DESSY), an expert system that monitors Space Shuttle telemetry data in real time. DESSY makes inferences about commands, state transitions, and simple failures. It performs failure detection rather than in-depth failure diagnostics. A listing of rules from DESSY and cue cards from DESSY subsystems are included to give the development community a better understanding of the selected model system. The G-2 programming tool used in developing DESSY provides an object-oriented, rule-based environment, but many of the principles in use here can be applied to any type of monitoring intelligent system. The step-by-step instructions and examples given for each stage of development are in G-2, but can be used with other development tools. This guide first defines the authors' concept of real-time monitoring systems, then tells prospective developers how to determine system requirements, how to build the system through a combined design/development process, and how to solve problems involved in working with real-time data. It explains the relationships among operational prototyping, software evolution, and the user interface. It also explains methods of testing, verification, and validation. It includes suggestions for preparing reference documentation and training users.
Explore the source record for details and available documents.
Design is a complex activity whose purpose is to construct an artifact which satisfies a set of constraints and requirements. However the design process is not well understood. The software design and evolution process is the focus of interest, and a three dimensional software development space organized around a decision-making paradigm is presented. An initial instantiation of this model called 3DPM(sub p) which was partly implemented, is presented. Discussion of the use of this model in software reuse and process management is given.
Not Available
The evolution of software assurance is discussed. The definition and implementation of standards are considered. It is recommended that requirements be clarified at the start of a project. The need for quality assurance in hardware is identified as the coming trend in the production of high cost single units which call for eradication of all errors during the early stages of development. The need to apply quality assurance throughout the whole mission is stressed. The dangers of overpricing product assurance services is stressed.
Software-Only-Simulations have been an emerging but quickly developing field of study throughout NASA. The NASA Independent Verification Validation (IVV) Independent Test Capability (ITC) team has been rapidly building a collection of simulators for a wide range of NASA missions. ITC specializes in full end-to-end simulations that enable developers, VV personnel, and operators to test-as-you-fly. In four years, the team has delivered a wide variety of spacecraft simulations that have ranged from low complexity science missions such as the Global Precipitation Management (GPM) satellite and the Deep Space Climate Observatory (DSCOVR), to the extremely complex missions such as the James Webb Space Telescope (JWST) and Space Launch System (SLS).This paper describes the evolution of ITCs technologies and processes that have been utilized to design, implement, and deploy end-to-end simulation environments for various NASA missions. A comparison of mission simulators are discussed with focus on technology and lessons learned in complexity, hardware modeling, and continuous integration. The paper also describes the methods for executing the missions unmodified flight software binaries (not cross-compiled) for verification and validation activities.
A long range program is described whose ultimate purpose is to make possible the production of software in NASA within predictable schedule and budget constraints and with major characteristics such as size, run-time, and correctness predictable within reasonable tolerances. As part of the program a pilot NASA computer center will be chosen to apply software development and management techniques systematically and determine a set which is effective. The techniques will be developed by a Technology Group, which will guide the pilot project and be responsible for its success. The application of the technology will involve a sequence of NASA programming tasks graduated from simpler ones at first to complex systems in late phases of the project. The evaluation of the technology will be made by monitoring the operation of the software at the users' installations. In this way a coherent discipline for software design, production maintenance, and management will be evolved.
Twenty-five years of spacecraft onboard computer development have resulted in a better understanding of the requirements for effective, efficient, and fault tolerant flight computer systems. Lessons from eight flight programs (Gemini, Apollo, Skylab, Shuttle, Mariner, Voyager, and Galileo) and three reserach programs (digital fly-by-wire, STAR, and the Unified Data System) are useful in projecting the computer hardware configuration of the Space Station and the ways in which the Ada programming language will enhance the development of the necessary software. The evolution of hardware technology, fault protection methods, and software architectures used in space flight in order to provide insight into the pending development of such items for the Space Station are reviewed.