Benjamin B Leser | Search NASA

NASA NTRS2019

Blind Validation Study of Parametric Cost Estimation Tool SEER-H for NASA Space Missions

One of the primary parametric cost modeling tools used by NASA and the aerospace industry to estimate the development and production cost of future spacecraft hardware is SEER-H by Galorath. To date, no independent validation of this tool for cost estimation of space missions has been reported in the literature. In the present validation study, cost estimators used SEER-H to estimate the cost of twelve different past NASA science missions. The estimators were prevented from knowing the actual cost of the missions in an effort to minimize cognitive biases. The point estimates of SEER-H had an average error of 23%, median error of -0.3%, and a standard deviation of 43%. Nine of the twelve mission's actual costs fell within the 80% confidence interval of SEER's probabilistic estimates. Several factors independent of SEER that may have affected the accuracy of the results have been identified and are discussed; these include: uncertainty in the technical data used for the estimates, the methods used to estimate uncertainty in spacecraft component mass and numbers of prototypes, and the experience of the estimators.

SEER-H↗

NASA NTRS2018

Blind Study Validating Parametric Costing Tools PRICE TruePlanning and SEER-H for NASA Science Missions

Two of the primary parametric costing tools used to estimate the development and production cost of future spacecraft hardware are PRICE TruePlanning - Space Missions Catalog by PRICE Systems, and SEER-H by Galorath. These are standard tools used by NASA and industry to estimate the cost of new aerospace hardware. However, no independent verification of the accuracy of these tools is publicly available. Both PRICE Systems and Galorath have completed internal validation studies of their parametric cost estimating tools; however, they only provided the results of the studies and did not detail the exact methods used to perform the validation. In the present study, cost estimators used PRICE TruePlanning and SEER-H to estimate the cost of twelve different past NASA science missions. The estimators were prevented from knowing the actual cost of the missions in an effort to minimize cognitive biases. In the present study, SEER had an average error of 23%, median error of -0.3%, with a standard deviation of 43%. PRICE had an average error of 52%, median error of 50%, and standard deviation of 45%. There were several factors independent of PRICE and SEER which may have affected the accuracy of the results in the present study including: uncertainty in the technical data used for the estimates, the methods used to estimate uncertainty in spacecraft component mass and numbers of prototypes, and the experience of the estimators.

Paul D Friz↗