Studies in software cost model behavior: do we really understand cost model performance
Explore the source record for details and available documents.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
No abstract available
Explore the source record for details and available documents.
The kinetics of the redistribution of dichlorosilane and trichlorosilane vapor over a tertiary amine ion exchange resin catalyst were investigated. The hydrogenation of SiCl4 to form HSiCl3 and the direct synthesis of H2SiCl2 from HCl gas and metallurgical silicon metal were also studied. The purification of SiH4 using activated carbon adsorbent was studied along with a process for storing SiH4 absorbed on carbon. The latter makes possible a higher volumetric efficiency than compressed gas storage. A mini-plant designed to produce ten pounds per day of SiH4 is described.
Extended operation of a small process-development unit routinely produced high quality silane in 97+% yield from dichlorosilane. The production rate was consistent with design loadings for the fractionating column and for the redistribution reactor. A glass fluid-bed reactor was constructed for room temperature operation. The behavior of a bed of silcon particles was observed as a function of various feedstocks, component configurations, and operating conditions. For operating modes other than spouting, the bed behaved in an erratic and unstable manner. A method was developed for casting molten silicon powder into crack-free solid pellets for process evaluation. The silicon powder was melted and cast into thin walled quartz tubes that sacrificially broke on cooling.
A quartz fluid bed reactor capable of operating at temperatures of up to 1000 C was designed, constructed, and successfully operated. During a 30 minute experiment, silane was decomposed within the reactor with no pyrolysis occurring on the reactor wall or on the gas injection system. A hammer mill/roller-crusher system appeared to be the most practical method for producing seed material from bulk silicon. No measurable impurities were detected in the silicon powder produced by the free space reactor, using the cathode layer emission spectroscopic technique. Impurity concentration followed by emission spectroscopic examination of the residue indicated a total impurity level of 2 micrograms/gram. A pellet cast from this powder had an electrical resistivity of 35 to 45 ohm-cm and P-type conductivity.
Progress from October 1, 1977, through December 31, 1977, is reported in the design of the 50 MT/year experimental facility for the preparation of high purity silicon by the zinc vapor reduction of silicon tetrachloride in a fluidized bed of seed particles to form a free flowing granular product.
Silicon epitaxy analysis of silane produced in the Process Development Unit operating in a completely integrated mode consuming only hydrogen and metallurgical silicon resulted in film resistivities of up to 120 ohms cm N type. Preliminary kinetic studies of dichlorosilane disproportionation in the liquid phase have shown that 11.59% SiH4 is formed at equilibrium after 12 minutes contact time at 56 C. The fluid-bed reactor was operated continuously for 48 hours with a mixture of one percent silane in helium as the fluidizing gas. A high silane pyrolysis efficiency was obtained without the generation of excessive fines. Gas flow conditions near the base of the reactor were unfavorable for maintaining a bubbling bed with good heat transfer characteristics. Consequently, a porous agglomerate formed in the lower portion of the reactor. Dense coherent plating was obtained on the silicon seed particles which had remained fluidizied throughout the experiment.
The presence of copper promotes a more rapid approach to the steady stete operating condition and results in a more consistent reactor effluent composition. The average kinetic and equilibrium yield are unchanged. Incoloy has been identified as the preferred choice of material of construction for the hydrogenation reactor although certain metallurgical changes were noted in samples exposed to the H2/HCl atmosphere at 500 C which indicate the need for more testing.
In most cost estimating applications at the NASA Langley Research Center (LaRC), it is desirable to present predicted cost as a range of possible costs rather than a single predicted cost. A cost risk analysis generates a range of cost for a project and assigns a probability level to each cost value in the range. Constructing a cost risk curve requires a good estimate of the expected cost of a project. It must also include a good estimate of expected variance of the cost. Many cost risk analyses are based upon an expert's knowledge of the cost of similar projects in the past. In a common scenario, a manager or engineer, asked to estimate the cost of a project in his area of expertise, will gather historical cost data from a similar completed project. The cost of the completed project is adjusted using the perceived technical and economic differences between the two projects. This allows errors from at least three sources. The historical cost data may be in error by some unknown amount. The managers' evaluation of the new project and its similarity to the old project may be in error. The factors used to adjust the cost of the old project may not correctly reflect the differences. Some risk analyses are based on untested hypotheses about the form of the statistical distribution that underlies the distribution of possible cost. The usual problem is not just to come up with an estimate of the cost of a project, but to predict the range of values into which the cost may fall and with what level of confidence the prediction is made. Risk analysis techniques that assume the shape of the underlying cost distribution and derive the risk curve from a single estimate plus and minus some amount usually fail to take into account the actual magnitude of the uncertainty in cost due to technical factors in the project itself. This paper addresses a cost risk method that is based on parametric estimates of the technical factors involved in the project being costed. The engineering process parameters are elicited from the engineer/expert on the project and are based on that expert's technical knowledge. These are converted by a parametric cost model into a cost estimate. The method discussed makes no assumptions about the distribution underlying the distribution of possible costs, and is not tied to the analysis of previous projects, except through the expert calibrations performed by the parametric cost analyst.
For early design concepts, the conventional approach to cost is normally some kind of parametric weight-based cost model. There is now ample evidence that this approach can be misleading and inaccurate. By the nature of its development, a parametric cost model requires historical data and is valid only if the new design is analogous to those for which the model was derived. Advanced aerospace vehicles have no historical production data and are nowhere near the vehicles of the past. Using an existing weight-based cost model would only lead to errors and distortions of the true production cost. This report outlines the development of a process-based cost model in which the physical elements of the vehicle are costed according to a first-order dynamics model. This theoretical cost model, first advocated by early work at MIT, has been expanded to cover the basic structures of an advanced aerospace vehicle. Elemental costs based on the geometry of the design can be summed up to provide an overall estimation of the total production cost for a design configuration. This capability to directly link any design configuration to realistic cost estimation is a key requirement for high payoff MDO problems. Another important consideration in this report is the handling of part or product complexity. Here the concept of cost modulus is introduced to take into account variability due to different materials, sizes, shapes, precision of fabrication, and equipment requirements. The most important implication of the development of the proposed process-based cost model is that different design configurations can now be quickly related to their cost estimates in a seamless calculation process easily implemented on any spreadsheet tool. In successive sections, the report addresses the issues of cost modeling as follows. First, an introduction is presented to provide the background for the research work. Next, a quick review of cost estimation techniques is made with the intention to highlight their inappropriateness for what is really needed at the conceptual phase of the design process. The First-Order Process Velocity Cost Model (FOPV) is discussed at length in the next section. This is followed by an application of the FOPV cost model to a generic wing. For designs that have no precedence as far as acquisition costs are concerned, cost data derived from the FOPV cost model may not be accurate enough because of new requirements for shape complexity, material, equipment and precision/tolerance. The concept of Cost Modulus is introduced at this point to compensate for these new burdens on the basic processes. This is treated in section 5. The cost of a design must be conveniently linked to its CAD representation. The interfacing of CAD models and spreadsheets containing the cost equations is the subject of the next section, section 6. The last section of the report is a summary of the progress made so far, and the anticipated research work to be achieved in the future.
Thermoelectric energy recovery systems require cost-optimized designs with high thermal and thermoelectric (TE) performance to surmount common commercialization barriers controlling the acceptance of thermoelectric generator (TEG) systems. Recent work has addressed this requirement with new analytic tools and paradigms that allow integrated TEG cost-performance analysis and optimization. New work herein has identified the existence of optimized TE element design (i.e., TE element length) for minimizing TEG cost and provides design guidelines associated with critical thermal and electrical contact resistance effects and TE manufacturing sensitivities. Once optimum hot-side heat flux criteria are satisfied, this work highlights the tradeoffs between interface resistance effects and manufacturing cost effects in determining optimum-cost TE element lengths, and then quantifies the increase in optimum-cost TE lengths as manufacturing costs become more sensitive to TE element lengths. Optimized TE element design criteria and key non-dimensional design parameters (i.e., [𝐶′′′LTE/𝐶′′], [TE/con], [con/(TE,aveLTE)]) driving the design are presented and explained showing the interdependencies between manufacturing cost parameters and thermal and electrical interface impacts on integrated TE cost-performance analysis and design. Comprehensive TEG system cost-performance relationships clearly demonstrate the two parameters that most greatly impact the TEG system cost are heat exchanger costs ($/(W/K)) and TE hot side heat flux. Reducing heat exhanger cost from $1/(W/K) to $0.5/(W/K) can decrease TEG system cost by $2/W to $4/W depending on TE hot side heat fluxes from 4-10 W/cm2. The impact of TE hot side heat flux on TEG system cost is larger; reducing TE system cost by factors of 2-4 as TE hot side heat flux increases from 4 W/cm2 to high levels around 18 W/cm2. The levels of thermal contact resistance and electrical contact resistance required to even approach the critical TEG system cost level of $1/W are characterized by con = 1.0 x 10-10 -m2 and r = 0.1, respectively. This not only lowers TEG costs, but it also makes the TEG system costs less sensitive to TE manufacturing dependences on TE element length. New design paradigms and relationships create holistic, integrated TE performance-cost models that enhance understanding of crucial interrelationships between component costs, TE design parameters and material properties, heat exchanger design parameters, interfacial heat flux, and now thermal and electrical interface effects in minimizing TEG system costs.
Space Transportation Avionics hardware and software cost has traditionally been estimated in Phase A and B using cost techniques which predict cost as a function of various cost predictive variables such as weight, lines of code, functions to be performed, quantities of test hardware, quantities of flight hardware, design and development heritage, complexity, etc. The output of such analyses has been life cycle costs, economic benefits and related data. The major objectives of Cost Estimation and Benefits analysis are twofold: (1) to play a role in the evaluation of potential new space transportation avionics technologies, and (2) to benefit from emerging technological innovations. Both aspects of cost estimation and technology are discussed here. The role of cost analysis in the evaluation of potential technologies should be one of offering additional quantitative and qualitative information to aid decision-making. The cost analyses process needs to be fully integrated into the design process in such a way that cost trades, optimizations and sensitivities are understood. Current hardware cost models tend to primarily use weights, functional specifications, quantities, design heritage and complexity as metrics to predict cost. Software models mostly use functionality, volume of code, heritage and complexity as cost descriptive variables. Basic research needs to be initiated to develop metrics more responsive to the trades which are required for future launch vehicle avionics systems. These would include cost estimating capabilities that are sensitive to technological innovations such as improved materials and fabrication processes, computer aided design and manufacturing, self checkout and many others. In addition to basic cost estimating improvements, the process must be sensitive to the fact that no cost estimate can be quoted without also quoting a confidence associated with the estimate. In order to achieve this, better cost risk evaluation techniques are needed as well as improved usage of risk data by decision-makers. More and better ways to display and communicate cost and cost risk to management are required.
The Space Transfer Vehicle (STV) Concepts and Requirements Study cost estimate and program planning analysis is presented. The cost estimating technique used to support STV system, subsystem, and component cost analysis is a mixture of parametric cost estimating and selective cost analogy approaches. The parametric cost analysis is aimed at developing cost-effective aerobrake, crew module, tank module, and lander designs with the parametric cost estimates data. This is accomplished using cost as a design parameter in an iterative process with conceptual design input information. The parametric estimating approach segregates costs by major program life cycle phase (development, production, integration, and launch support). These phases are further broken out into major hardware subsystems, software functions, and tasks according to the STV preliminary program work breakdown structure (WBS). The WBS is defined to a low enough level of detail by the study team to highlight STV system cost drivers. This level of cost visibility provided the basis for cost sensitivity analysis against various design approaches aimed at achieving a cost-effective design. The cost approach, methodology, and rationale are described. A chronological record of the interim review material relating to cost analysis is included along with a brief summary of the study contract tasks accomplished during that period of review and the key conclusions or observations identified that relate to STV program cost estimates. The STV life cycle costs are estimated on the proprietary parametric cost model (PCM) with inputs organized by a project WBS. Preliminary life cycle schedules are also included.
Spacecraft control centers have evolved from dedicated, single-mission or single mission-type support to multi-mission, service-oriented support for operating a variety of mission types. At the same time, available money for projects is shrinking and competition for new missions is increasing. These factors drive the need for an accurate and flexible model to support estimating service costs for new or extended missions; the cost model in turn drives the need for an accurate and efficient approach to service cost analysis. The National Aeronautics and Space Administration (NASA) Huntsville Operations Support Center (HOSC) at Marshall Space Flight Center (MSFC) provides operations services to a variety of customers around the world. HOSC customers range from launch vehicle test flights; to International Space Station (ISS) payloads; to small, short duration missions; and has included long duration flagship missions. The HOSC recently completed a detailed analysis of service costs as part of the development of a complete service cost model. The cost analysis process required the team to address a number of issues. One of the primary issues involves the difficulty of reverse engineering individual mission costs in a highly efficient multi-mission environment, along with a related issue of the value of detailed metrics or data to the cost model versus the cost of obtaining accurate data. Another concern is the difficulty of balancing costs between missions of different types and size and extrapolating costs to different mission types. The cost analysis also had to address issues relating to providing shared, cloud-like services in a government environment, and then assigning an uncertainty or risk factor to cost estimates that are based on current technology, but will be executed using future technology. Finally the cost analysis needed to consider how to validate the resulting cost models taking into account the non-homogeneous nature of the available cost data and the decreasing flight rate. This paper presents the issues encountered during the HOSC cost analysis process, and the associated lessons learned. These lessons can be used when planning for a new multi-mission operations center or in the transformation from a dedicated control center to multi-center operations, as an aid in defining processes that support future cost analysis and estimation. The lessons can also be used by mature service-oriented, multi-mission control centers to streamline or refine their cost analysis process.
Spacecraft control centers have evolved from dedicated, single-mission or single missiontype support to multi-mission, service-oriented support for operating a variety of mission types. At the same time, available money for projects is shrinking and competition for new missions is increasing. These factors drive the need for an accurate and flexible model to support estimating service costs for new or extended missions; the cost model in turn drives the need for an accurate and efficient approach to service cost analysis. The National Aeronautics and Space Administration (NASA) Huntsville Operations Support Center (HOSC) at Marshall Space Flight Center (MSFC) provides operations services to a variety of customers around the world. HOSC customers range from launch vehicle test flights; to International Space Station (ISS) payloads; to small, short duration missions; and has included long duration flagship missions. The HOSC recently completed a detailed analysis of service costs as part of the development of a complete service cost model. The cost analysis process required the team to address a number of issues. One of the primary issues involves the difficulty of reverse engineering individual mission costs in a highly efficient multimission environment, along with a related issue of the value of detailed metrics or data to the cost model versus the cost of obtaining accurate data. Another concern is the difficulty of balancing costs between missions of different types and size and extrapolating costs to different mission types. The cost analysis also had to address issues relating to providing shared, cloud-like services in a government environment, and then assigning an uncertainty or risk factor to cost estimates that are based on current technology, but will be executed using future technology. Finally the cost analysis needed to consider how to validate the resulting cost models taking into account the non-homogeneous nature of the available cost data and the decreasing flight rate. This paper presents the issues encountered during the HOSC cost analysis process, and the associated lessons learned. These lessons can be used when planning for a new multi-mission operations center or in the transformation from a dedicated control center to multi-center operations, as an aid in defining processes that support future cost analysis and estimation. The lessons can also be used by mature serviceoriented, multi-mission control centers to streamline or refine their cost analysis process.