Software Development Cost: How Much? You Sure? Technical Summary
This viewgraph presentation reviews cost models to properly estimate the cost of software development.
machine learning↗
SEARCH · Search NASA
Results for “software effort estimating tool”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Ask Pete, software planning and estimation through project characterization
Ask Pete, was developed by NASA to provide a tool for integrating the estimation and planning activities for a software development effort. It incorporates COCOMO II estimating with NASA's software development practices and IV&V criteria to characterize a project. This characterization is then used to generate estimates and tailored planning documents.
Project characterization↗
Putting it all together: software planning, estimating and assessment for a successful project
This paper describes a coherent approach and accompanying tool support that addresses the challenges of large software efforts.
software planning risk management software estimat↗
ASCot, the NASA Analogy Software Cost Tool Suite: expanding our estimation horizons
The NASA Analogy Software Costing Tool Suite (ASCoT) consists of a cluster-based analogy estimator for estimating software development effort, a K-Nearest Neighbors (KNN) analogy estimator for estimating effort and delivered lines of code, a simple regression-based cost estimating relationship (CER) model that estimates cost in dollars, and a probabilistic version of COCOMO II. In this paper we document the analogy algorithms as well as summarize the results of the performance of the KNN and the principle components (PCA) cluster analogy models. KNN performance is assessed by varying the number of inputs and number of neighbors. Four different clustering methods: K-means, Spectral Clustering, Hierarchical Clustering, and Principle Components Analysis (PCA), and their respective evaluation criterion are described in detail. The comparative performance of all four estimation models is assessed using magnitude of relative error (MRE) measurements.
Menzies, Tim↗
Lessons learned in deploying software estimation technology and tools
Developing a software product involves estimating various project parameters. This is typically done in the planning stages of the project when there is much uncertainty and very little information. Coming up with accurate estimates of effort, cost, schedule, and reliability is a critical problem faced by all software project managers. The use of estimation models and commercially available tools in conjunction with the best bottom-up estimates of software-development experts enhances the ability of a product development group to derive reasonable estimates of important project parameters. This paper describes the experience of the IBM Software Solutions (SWS) Toronto Laboratory in selecting software estimation models and tools and deploying their use to the laboratory's product development groups. It introduces the SLIM and COSTAR products, the software estimation tools selected for deployment to the product areas, and discusses the rationale for their selection. The paper also describes the mechanisms used for technology injection and tool deployment, and concludes with a discussion of important lessons learned in the technology and tool insertion process.
Panlilio-Yap, Nikki↗
Software Development Cost Estimation Executive Summary
Identify simple fully validated cost models that provide estimation uncertainty with cost estimate. Based on COCOMO variable set. Use machine learning techniques to determine: a) Minimum number of cost drivers required for NASA domain based cost models; b) Minimum number of data records required and c) Estimation Uncertainty. Build a repository of software cost estimation information. Coordinating tool development and data collection with: a) Tasks funded by PA&E Cost Analysis; b) IV&V Effort Estimation Task and c) NASA SEPG activities.
data mining↗
Deliverable 6.7-Final Technical Report: Development Summary and Evaluation of the Solar Uncertainty Integrator (SUNI) Software
The Data Quality and Uncertainty Integration Project was a three-year effort to address stakeholder needs for assessing solar radiation resource data quality based on existing tools for estimating radiometer measurement uncertainties and assessing post-measurement data quality. The annual research objectives for the project addressed a logical progression of effort needed to achieve the ultimate project goal of developing the Solar Uncertainty Integrator (SUNI) software. This final technical report summarizes the development process for achieving these key research objectives and addresses the outreach and code development efforts in the final year of the project to develop a new solar irradiance data uncertainty integration software package.
14 SOLAR ENERGY↗
ASCoT 3: Nonlinear Principal Components Analysis and Uncertainty Quantification in Early Concept Spacecraft Flight Software Cost Estimation
For mission planners and evaluators alike, value in cost models comes from a mean or median prediction, an understanding of the uncertainty on that prediction, and an understanding of model performance. Here we apply advanced statistical and machine learning methods to spacecraft flight software cost, effort, and SLOC estimation, and present the results in the latest version of the Analogy Software Cost Tool (ASCoT). We present in- and out-of-sample performance metrics for our models, each of which incorporate some amount of epistemic uncertainty. ASCoT, hosted on the One NASA Cost Engineering (ONCE) database via the Online NASA Space Estimation Tool (ONSET), was first showcased in 2016 as a number of analogy-based models and methods (kNN and Clustering) to support early project formulation. This ASCoT update improves upon the previous analogic methods by incorporating uncertainty in the data transformations. In particular, we use a Nonlinear Principal Components Analysis (NLPCA) to deal with ordinal data.
Robotic Spacecraft↗
COSTMODL: An automated software development cost estimation tool
The cost of developing computer software continues to consume an increasing portion of many organizations' total budgets, both in the public and private sector. As this trend develops, the capability to produce reliable estimates of the effort and schedule required to develop a candidate software product takes on increasing importance. The COSTMODL program was developed to provide an in-house capability to perform development cost estimates for NASA software projects. COSTMODL is an automated software development cost estimation tool which incorporates five cost estimation algorithms including the latest models for the Ada language and incrementally developed products. The principal characteristic which sets COSTMODL apart from other software cost estimation programs is its capacity to be completely customized to a particular environment. The estimation equations can be recalibrated to reflect the programmer productivity characteristics demonstrated by the user's organization, and the set of significant factors which effect software development costs can be customized to reflect any unique properties of the user's development environment. Careful use of a capability such as COSTMODL can significantly reduce the risk of cost overruns and failed projects.
Roush, George B.↗
Failure Assessment
Three questions to which software developers want accurate, precise answers are "How can the software system fail?", "mat bad things will happen if the software fails?t', and "How many failures will the software experience?". Numerous techniques have been devised to answer these questions; three of the best known are: 1) Software Fault Tree Analysis (SFTA) 2) Software Failure Modes, Effects, and Criticality Analysis (SFMECA 3) Software Fault/Failure Modeling. SFTA and SFMECA have been successfully used to analyze the flight software for a number of robotic planetary exploration missions, including Galileo, Cassini, and Deep Space 1. Given the increasing interest in reusing software components from mission to mission, one of us has developed techniques for reusing the corresponding portions of the SFTA and SFMECA, reducing the effort required to conduct these analyses. SFTA has also been shown to be effective in analyzing the security aspects of software systems; intrusion mechanisms and effects can easily be modeled using these techniques. The Bi- Directional Safety Analysis (BDSA) method combines a forward search (similar to SFMECA) from potential failure modes to their effects, with a backward search (similar to SFTA) from feasible hazards to the contributing causes of each hazard. BDSA offers an efficient way to identify latent failures. Recent work has extended BDSA to product-line applications such as flight-instrumentation displays and developed tool support for the reuse of the failure-analysis artifacts within a product line. BDSA has also been streamlined to support those projects having tight cost and/or schedule constraints for their failure analysis efforts. We discuss lessons learned from practice, describe available tools, and identi@ some future directions for the topic. A substantial amount of research has been devoted to estimating the number of failures that a software system will experience during test and operations, as well as the number of faults that have been inserted into that system during its development. One of us has found that the amount of structural change to a system during its development is strongly related to the number of faults inserted into it. Using techniques requiring no additional effort on the part of the development organization, the required measurements of structural evolution can be easily obtained from a development effort's configuration management system and readily transformed into an estimate of fault content. So far, structure-fault relationships have been identified for source code; current work seeks to examine artifacts available earlier in the lifecycle to determine if similar relationships between structure and fault content can be found. In particular, relationships between requirements change requests and the number of faults inserted into the implemented system would provide a significant improvement in our ability to control software quality during the early development phases.
fault tree↗
Automated Estimation Of Software-Development Costs
COSTMODL is automated software development-estimation tool. Yields significant reduction in risk of cost overruns and failed projects. Accepts description of software product developed and computes estimates of effort required to produce it, calendar schedule required, and distribution of effort and staffing as function of defined set of development life-cycle phases. Written for IBM PC(R)-compatible computers.
Roush, George B.↗
An approach to software baseline generation
A current Data & Analysis Center for Software (DACS) effort to develop software baselines is summarized. This baseline effort is an on-going activity; that is, the baselines are meant to be updated as new software data becomes available. The information presented and processed was organized to make periodic updating a much simpler task. A baseline, for this effort, consists of an estimation of any characteristic of a software project that is helpful to a developer, manager, or monitor to manage, control, or influence a software product. The objective of these baselines is to provide a tool for aiding software developers in their daily work. Baselines were synthesized from an empirical dataset provided by the Software Engineering Laboratory at NASA Goddard Space Flight Center (NASA/SEL). These data were selected because the data collection effort developed at the NASA/SEL is the most thorough and complete available.
Romeu, J. L.↗
The NASA Analogy Software Cost Model: A Web-Based Cost Analysis Tool
This paper provides an overview of the many new features and algorithm updates in the release of the NASA Analogy Software Cost Tool (ASCoT). ASCoT is a web-based tool that provides a suite of estimation tools to support early lifecycle NASA Flight Software analysis. ASCoT employs advanced statistical methods such as Cluster Analysis to provide an analogy based estimate of software delivered lines of code and development effort, a regression based Cost Estimating Relationships (CER) model that estimates cost (dollars), and a COCOMO II based estimate. The ASCoT algorithms are designed to primarily work with system level inputs such as mission type (earth orbiter vs. planetary vs. rover), the number of instruments, and total mission cost. This allows the user to supply a minimal number of mission-level parameters which are better understood early in the life-cycle, rather than a large number of complex inputs.
Hihn, Jairus↗
Integrated In-Situ Resource Utilization Modeling of a Lunar Water Processing System
A key element of achieving a sustained surface presence, such as defined in NASA’s Artemis plan, is In-Situ Resource Utilization (ISRU). ISRU is the practice of using local resources to provide mission consumables that reduce system launch mass requirements, and regenerate resources (chiefly, water and oxygen) for propulsion and life support supporting both Lunar and Martian missions. ISRU systems require multiple complex processes, such as excavation, chemical reactors, and electrolysis subsystems that must operate in harmony to optimize the overall system process from beginning to end. The Mission Analysis and Integration Tool (MAIT) connects individual subsystem models into a customized, flexible framework for the purpose of technology downselect, optimization, and end-to-end process planning. MATLAB was favorable to use as the main software integration tool due to its ability to communicate with a vast number of other programming languages and makes up the backbone of data flow between inputs and outputs to the subsystem models. MAIT initially evaluated a suite of technologies, including but not limited to, the water processing Lunar Auger Dryer for ISRU (LADI) system and an oxygen extraction carbothermal reduction process. With individual models consolidated, the MAIT tool generated over 60,000 cases during its parametric sweeps; these system iterations produced valuable insight into the optimal LADI geometry for minimizing heater energy demands, estimating carbothermal reactor and radiator mass relationships, and calculated the power dynamics of the electrolysis unit. Efforts to update the MAIT tool are ongoing to accommodate and scale ISRU technologies supporting the Space Technology Mission Directorate’s (STMD) commercialization strategy, and increase the MAIT software capability to handle a wide array of ISRU system models beyond the Lunar environment, e.g. production of propellant for a Martian lander.
Avery Carlson↗
A survey of program slicing for software engineering
This research concerns program slicing which is used as a tool for program maintainence of software systems. Program slicing decreases the level of effort required to understand and maintain complex software systems. It was first designed as a debugging aid, but it has since been generalized into various tools and extended to include program comprehension, module cohesion estimation, requirements verification, dead code elimination, and maintainence of several software systems, including reverse engineering, parallelization, portability, and reuse component generation. This paper seeks to address and define terminology, theoretical concepts, program representation, different program graphs, developments in static slicing, dynamic slicing, and semantics and mathematical models. Applications for conventional slicing are presented, along with a prognosis of future work in this field.
Beck, Jon↗
Technology Infusion of CodeSonar into the Space Network Ground Segment (RII07): Software Assurance Symposium Technical Summary
Presents a source code analysis tool (CodeSonar) for use in the Space Network Ground Segment. The Space Network requires 99.9% proficiency and 97.0% availability of systems. Software has historically accounted for an annual average of 28% of the Space Network loss of availability and proficiency. CSCI A and CSCI B account for 42% of the previous eight months of software data loss. The technology infusion of CodeSonar into the Space Network Ground segment is meant to aid in determining the impact of the technology on the project both in the expenditure of effort and the technical results of the technology. Running a CodeSonar analysis and performing a preliminary review of the results averaged 3.5 minutes per finding (approximately 20 hours total). An additional 40 hours is estimated to analyze the 37 findings deemed too complex for the initial review. Using CodeSonar's tools to suppress known non-problems, delta tool runs will not repeat findings that have been marked as non-problems, further reducing the time needed for review. The 'non-interesting' finding rate of 70% is a large number, but filtering, search, and detailed contextual features of CodeSonar reduce the time per finding. Integration of the tool into the build process may also provide further savings by preventing developers from having to configure and operate the tool separately. These preliminary results show the tool to be easy to use and incorporate into the engineering process. These findings also provide significant potential improvements in proficiency and availability on the part of the software. As time-to-fix data become available a better cost trade can be made on person hours saved versus tool cost. Selective factors may be necessary to determine where best to apply CodeSonar to balance cost and benefits.
Benson, Markland J.↗
NDARC - NASA Design and Analysis of Rotorcraft
The NASA Design and Analysis of Rotorcraft (NDARC) software is an aircraft system analysis tool that supports both conceptual design efforts and technology impact assessments. The principal tasks are to design (or size) a rotorcraft to meet specified requirements, including vertical takeoff and landing (VTOL) operation, and then analyze the performance of the aircraft for a set of conditions. For broad and lasting utility, it is important that the code have the capability to model general rotorcraft configurations, and estimate the performance and weights of advanced rotor concepts. The architecture of the NDARC code accommodates configuration flexibility, a hierarchy of models, and ultimately multidisciplinary design, analysis, and optimization. Initially the software is implemented with low-fidelity models, typically appropriate for the conceptual design environment. An NDARC job consists of one or more cases, each case optionally performing design and analysis tasks. The design task involves sizing the rotorcraft to satisfy specified design conditions and missions. The analysis tasks can include off-design mission performance calculation, flight performance calculation for point operating conditions, and generation of subsystem or component performance maps. For analysis tasks, the aircraft description can come from the sizing task, from a previous case or a previous NDARC job, or be independently generated (typically the description of an existing aircraft). The aircraft consists of a set of components, including fuselage, rotors, wings, tails, and propulsion. For each component, attributes such as performance, drag, and weight can be calculated; and the aircraft attributes are obtained from the sum of the component attributes. Description and analysis of conventional rotorcraft configurations is facilitated, while retaining the capability to model novel and advanced concepts. Specific rotorcraft configurations considered are single-main-rotor and tail-rotor helicopter, tandem helicopter, coaxial helicopter, and tiltrotor. The architecture of the code accommodates addition of new or higher-fidelity attribute models for a component, as well as addition of new components.
Analysis↗
NDARC NASA Design and Analysis of Rotorcraft - Input, Appendix 2
The NASA Design and Analysis of Rotorcraft (NDARC) software is an aircraft system analysis tool that supports both conceptual design efforts and technology impact assessments. The principal tasks are to design (or size) a rotorcraft to meet specified requirements, including vertical takeoff and landing (VTOL) operation, and then analyze the performance of the aircraft for a set of conditions. For broad and lasting utility, it is important that the code have the capability to model general rotorcraft configurations, and estimate the performance and weights of advanced rotor concepts. The architecture of the NDARC code accommodates configuration exibility, a hierarchy of models, and ultimately multidisciplinary design, analysis, and optimization. Initially the software is implemented with low-fidelity models, typically appropriate for the conceptual design environment. An NDARC job consists of one or more cases, each case optionally performing design and analysis tasks. The design task involves sizing the rotorcraft to satisfy specified design conditions and missions. The analysis tasks can include off-design mission performance calculation, flight performance calculation for point operating conditions, and generation of subsystem or component performance maps. For analysis tasks, the aircraft description can come from the sizing task, from a previous case or a previous NDARC job, or be independently generated (typically the description of an existing aircraft). The aircraft consists of a set of components, including fuselage, rotors, wings, tails, and propulsion. For each component, attributes such as performance, drag, and weight can be calculated; and the aircraft attributes are obtained from the sum of the component attributes. Description and analysis of conventional rotorcraft configurations is facilitated, while retaining the capability to model novel and advanced concepts. Specific rotorcraft configurations considered are single-main-rotor and tail-rotor helicopter, tandem helicopter, coaxial helicopter, and tilt-rotor. The architecture of the code accommodates addition of new or higher-fidelity attribute models for a component, as well as addition of new components.
Appendix↗