Search NASA⌕ Search

SEARCH · Search NASA

Results for “Ranking methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 667 records · Page 37
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

DEEPEN 3D PFA Favorability Models and 2D Favorability Maps at Newberry Volcano

DEEPEN stands for DE-risking Exploration of geothermal Plays in magmatic ENvironments. Part of the DEEPEN project involved developing and testing a methodology for a 3D play fairway analysis (PFA) for multiple play types (conventional hydrothermal, superhot EGS, and supercritical). This was tested using new and existing geoscientific exploration datasets at Newberry Volcano. This GDR submission includes images, data, and models related to the 3D favorability and uncertainty models and the 2D favorability and uncertainty maps. The DEEPEN PFA Methodology is based on the method proposed by Poux et al. (2020), which uses the Leapfrog Geothermal software with the Edge extension to conduct PFA in 3D. This method uses all available data to build a 3D geodata model which can be broken down into smaller blocks and analyzed with advanced geostatistical methods. Each data set is imported into a 3D model in Leapfrog and divided into smaller blocks. Conditional queries can then be used to assign each block an index value which conditionally ranks each block's favorability, from 0-5 with 5 being most favorable, for each model (e.g., lithologic, seismic, magnetic, structural). The values between 0-5 assigned to each block are referred to as index values. The final step of the process is to combine all the index models to create a favorability index. This involves multiplying each index model by a given weight and then summing the resulting values. The DEEPEN PFA Methodology follows this approach, but split up by the specific geologic components of each play type. These components are defined as follows for each magmatic play type: 1. Conventional hydrothermal plays in magmatic environments: Heat, fluid, and permeability 2. Superhot EGS plays: Heat, thermal insulation, and producibility (the ability to create and sustain fractures suitable for and EGS reservoir) 3. Supercritical plays: Heat, supercritical fluid, pressure seal, and producibility (the proper permeability and pressure conditions to allow production of supercritical fluid) More information on these components and their development can be found in Kolker et al., 2022. For the purposes of subsurface imaging, it is easier to detect a permeable fluid-filled reservoir than it is to detect separate fluid and permeability components. Therefore, in this analysis, we combine fluid and permeability for conventional hydrothermal plays, and supercritical fluid and producibility for supercritical plays. More information on this process is described in the following sections. We also project the 3D favorability volumes onto 2D surfaces for simplified joint interpretation, and we incorporate an uncertainty component. Uncertainty was modeled using the best approach for the dataset in question, for the datasets where we had enough information to do so. Identifying which subsurface parameters are the least resolved can help qualify current PFA results and focus future efforts in data collection. Where possible, the resulting uncertainty models/indices were weighted using the same weights applied to the respective datasets, and summed, following the PFA methodology above, but for uncertainty. There are two different versions of the Leapfrog model and associated favorability models: - v1.0: The first release in June 2023 - v2.1: The second release, with improvements made to the earthquake catalog (included additional identified events, removed duplicate events), to the temperature model (fixed a deep BHT), and to the index models (updated the seismicity-heat source index models for supercritical and EGS, and the resistivity-insulation index models for all three play types). Also uses the jet color map rather than the magma color map for improved interpretability. - v2.1.1: Updated to include v2.0 uncertainty results (see below for uncertainty model versions) There are two different versions of the associated uncertainty models: - v1.0: The first release in June 2023 - v2.0: The se...

15 GEOTHERMAL ENERGY↗
DOE OSTI2024

DEEPEN: Final 3D PFA Favorability Models and 2D Favorability Maps at Newberry Volcano

Part of the DEEPEN (DE-risking Exploration of geothermal Plays in magmatic ENvironments) project involved developing and testing a methodology for a 3D play fairway analysis (PFA) for multiple play types (conventional hydrothermal, superhot EGS, and supercritical). This was tested using new and existing geoscientific exploration datasets at Newberry Volcano. This GDR submission includes images, data, and models related to the 3D favorability and uncertainty models and the 2D favorability and uncertainty maps. The DEEPEN PFA Methodology, detailed in the journal article below, is based on the method proposed by Poux & O'brien (2020), which uses the Leapfrog Geothermal software with the Edge extension to conduct PFA in 3D. This method uses all available data to build a 3D geodata model which can be broken down into smaller blocks and analyzed with advanced geostatistical methods. Each data set is imported into a 3D model in Leapfrog and divided into smaller blocks. Conditional queries can then be used to assign each block an index value which conditionally ranks each block's favorability, from 0-5 with 5 being most favorable, for each model (e.g., lithologic, seismic, magnetic, structural). The values between 0-5 assigned to each block are referred to as index values. The final step of the process is to combine all the index models to create a favorability index. This involves multiplying each index model by a given weight and then summing the resulting values. The DEEPEN PFA Methodology follows this approach, but split up by the specific geologic components of each play type. These components are defined as follows for each magmatic play type: 1. Conventional hydrothermal plays in magmatic environments: Heat, fluid, and permeability 2. Superhot EGS plays: Heat, thermal insulation, and producibility (the ability to create and sustain fractures suitable for and EGS reservoir) 3. Supercritical plays: Heat, supercritical fluid, pressure seal, and producibility (the proper permeability and pressure conditions to allow production of supercritical fluid) More information on these components and their development can be found in Kolker et al., (2022). For the purposes of subsurface imaging, it is easier to detect a permeable fluid-filled reservoir than it is to detect separate fluid and permeability components. Therefore, in this analysis, we combine fluid and permeability for conventional hydrothermal plays, and supercritical fluid and producibility for supercritical plays. We also project the 3D favorability volumes onto 2D surfaces for simplified joint interpretation, and we incorporate an uncertainty component. Uncertainty was modeled using the best approach for the dataset in question, for the datasets where we had enough information to do so. Identifying which subsurface parameters are the least resolved can help qualify current PFA results and focus future efforts in data collection. Where possible, the resulting uncertainty models/indices were weighted using the same weights applied to the respective datasets, and summed, following the PFA methodology above, but for uncertainty.

15 GEOTHERMAL ENERGY↗
DOE OSTI2015

Validation and quantification of uncertainty in coupled climate models using network analysis

We developed a fast, robust and scalable methodology to examine, quantify, and visualize climate patterns and their relationships. It is based on a set of notions, algorithms and metrics used in the study of graphs, referred to as complex network analysis. This approach can be applied to explain known climate phenomena in terms of an underlying network structure and to uncover regional and global linkages in the climate system, while comparing general circulation models outputs with observations. The proposed method is based on a two-layer network representation, and is substantially new within the available network methodologies developed for climate studies. At the first layer, gridded climate data are used to identify ‘‘areas’’, i.e., geographical regions that are highly homogeneous in terms of the given climate variable. At the second layer, the identified areas are interconnected with links of varying strength, forming a global climate network. The robustness of the method (i.e. the ability to separate between topological distinct fields, while identifying correctly similarities) has been extensively tested. It has been proved that it provides a reliable, fast framework for comparing and ranking the ability of climate models of reproducing observed climate patterns and their connectivity. We further developed the methodology to account for lags in the connectivity between climate patterns and refined our area identification algorithm to account for autocorrelation in the data. The new methodology based on complex network analysis has been applied to state-of-the-art climate model simulations that participated to the last IPCC (International Panel for Climate Change) assessment to verify their performances, quantify uncertainties, and uncover changes in global linkages between past and future projections. Network properties of modeled sea surface temperature and rainfall over 1956–2005 have been constrained towards observations or reanalysis data sets, and their differences quantified using two metrics. Projected changes from 2051 to 2300 under the scenario with the highest representative and extended concentration pathways (RCP8.5 and ECP8.5) have then been determined. The network of models capable of reproducing well major climate modes in the recent past, changes little during this century. In contrast, among those models the uncertainties in the projections after 2100 remain substantial, and primarily associated with divergences in the representation of the modes of variability, particularly of the El Niño Southern Oscillation (ENSO), and their connectivity, and therefore with their intrinsic predictability, more so than with differences in the mean state evolution. Additionally, we evaluated the relation between the size and the ‘strength’ of the area identified by the network analysis as corresponding to ENSO noting that only a small subset of models can reproduce realistically the observations.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1981

Some analysis on the diurnal variation of rainfall over the Atlantic Ocean

Data collected from the GARP Atlantic Tropical Experiment (GATE) was examined. The data were collected from 10,000 grid points arranged as a 100 x 100 array; each grid covered a 4 square km area. The amount of rainfall was measured every 15 minutes during the experiment periods using c-band radars. Two types of analyses were performed on the data: analysis of diurnal variation was done on each of grid points based on the rainfall averages at noon and at midnight, and time series analysis on selected grid points based on the hourly averages of rainfall. Since there are no known distribution model which best describes the rainfall amount, nonparametric methods were used to examine the diurnal variation. Kolmogorov-Smirnov test was used to test if the rainfalls at noon and at midnight have the same statistical distribution. Wilcoxon signed-rank test was used to test if the noon rainfall is heavier than, equal to, or lighter than the midnight rainfall. These tests were done on each of the 10,000 grid points at which the data are available.

Gill, T.↗
NASA NTRS1990

Neural network approaches versus statistical methods in classification of multisource remote sensing data

Neural network learning procedures and statistical classificaiton methods are applied and compared empirically in classification of multisource remote sensing and geographic data. Statistical multisource classification by means of a method based on Bayesian classification theory is also investigated and modified. The modifications permit control of the influence of the data sources involved in the classification process. Reliability measures are introduced to rank the quality of the data sources. The data sources are then weighted according to these rankings in the statistical multisource classification. Four data sources are used in experiments: Landsat MSS data and three forms of topographic data (elevation, slope, and aspect). Experimental results show that two different approaches have unique advantages and disadvantages in this classification application.

Benediktsson, Jon A.↗
NASA NTRS2001

Global Positioning System Satellite Selection Method

The satellite selection method as utilized by the spaceborne Global Positioning System receiver provides navigational solutions and is designed for use in low Earth orbit. The satellite selection method is a robust algorithm that can be used a GPS receiver to select appropriate GPS satellites for use in calculating point solutions or attitude solutions. The method is takes into account the difficulty of finding a particular GPS satellite phase code, especially when the search range in greatly increased due to Doppler shifts introduced into the carrier frequency. The method starts with an update of the antenna pointing and spacecraft vectors to determine the antenna backplane direction. Next, the GPS satellites that will potentially be in view of the antenna are ranked on a list, whereby the list is generated based on the estimated attitude and position of each GPS satellite. Satellites blocked by the Earth are not entered on this list. A second list is created, whereby the GPS satellites are ranked according to their desirability for use in attitude determination. GPS satellites are ranked according to their orthogonality to the antenna backplane, and according to geometric dilution of precision considerations. After the lists are created, the channels of the spaceborne GPS receiver are assigned to various GPS satellites for acquisition and lock. Preliminary Doppler frequencies for searching are assigned to the various channels.

Frederick A Niles↗
DOE OSTI2022

Cybersecurity vulnerability mitigation framework

Systems, methods, and computer media for mitigating cybersecurity vulnerabilities of systems are provided herein. A current cybersecurity maturity of a system can be determined based on maturity criteria. The maturity criteria can be ranked based on importance. Solution candidates for increasing the cybersecurity maturity of the system can be determined based on the ranking. The solution candidates specify cybersecurity levels for the maturity criteria. A present state value reflecting the current cybersecurity maturity of the system can be calculated. For the solution candidates, an implementation state value and a transition state value can be determined. The implementation state value represents implementation of the maturity levels of the solution candidate, and the transition state value represents a transition from the present state value to the implementation state value. Based on the transition state values, a solution candidate can be selected for the system, and the system can be modified accordingly.

Gourisetti, Sri Nikhil Gupta↗
DOE OSTI2025

Predicting receptor-ligand pairing preferences in plant-microbe interfaces via molecular dynamics and machine learning

Microbiome assembly, structure, and dynamics significantly influence plant health. Secreted microbial signaling molecules initiate and mediate symbiosis by binding to structurally compatible plant receptors. For example, lipo-chitooligosaccharides (LCOs), produced by nitrogen-fixing rhizobial bacteria and various fungi, are recognized by plant lysin motif receptor-like kinases (LysM-RLKs), which activate the common symbiotic pathway. Accurately predicting these molecular interactions could reveal complementary signatures underlying the initial stages of endosymbiosis. Despite the breakthrough in protein-ligand structure prediction with deep learning-based tools, such as AlphaFold3, the large size and highly flexible nature of signaling compounds like LCOs present major challenges for detailed structural characterization and binding-affinity prediction. Typical structure-/physics-based methods of ligand virtual screening are designed for small, drug-like molecules, often rely on high-resolution, experimentally determined structures of the protein receptors, and rarely achieve sufficient sampling to obtain converged thermodynamic quantities with large ligands. In this study, we developed a hybrid molecular dynamics/machine learning (MD/ML) approach capable of predicting binding affinity rankings with high accuracy in systems involving large, flexible ligands, despite limited experimental structural information. Using coarse initial structural models, the predictions using the MD/ML workflow achieved strong alignment with experimental trends, particularly in the top-affinity tier for four legume LysM-RLKs (LYR3) binding to LCOs and a chitooligosaccharide. Furthermore, the MD-based conformation selection protocol provided critical structural insights into substrate specificity and binding mechanisms. This study demonstrates a powerful method to screen for challenging cognate ligand-receptors and advance our understanding of the molecular basis of microbial colonization in plants.

Lipo-chitooligosaccharides↗
DOE OSTI2018

An Assessment of GCM Performance at a Regional Scale Using a Score-Based Method

A multicriteria score-based method was developed to assess the performances of 18 general circulation models (GCMs) in the study region from 1970 to 2005. The results indicate the following. (1) GCMs simulate temperature better than rainfall. The temporal and spatial distributions of simulated temperature performed well compared with those from the observations. In comparison to temperature, the spatial distribution of simulated precipitation performed poorly. Most of the GCMs underestimated temperature and overestimated precipitation. (2) The Grubbs test was used to detect anomalous moving changes in the rank score (RS) results; the inm-cm4 and ipsl-cm5b-lr models were rejected when simulating temperature, while the bnu-esm and canesm2 models performed poorly when simulating precipitation. (3) Adding or removing any criterion does not significantly influence the RS result, which indicates that the multicriteria score-based method is robust. The advantages of using multicriteria score-based method to assess GCMs performance were demonstrated, and this method also provides a more comprehensive assessment when compared with the single-criterion method. The multicriteria method could replace other criteria as the research requirements and could be easily extended to different study regions; the results could be used for better informed regional climate change impact analyses.

Shi, Fangxin↗
DOE OSTI2023

Robust inference of ecosystem soil water stress from eddy covariance data

Eddy covariance data are invaluable for determining ecosystem water use strategies under soil water stress. However, existing stress inference methods require numerous subjective data processing and model specification assumptions whose effect on the inferred soil water stress signal is rarely quantified. These uncertainties may confound the stress inference and the generalization of ecosystem water use strategies across multiple sites and studies. In this research, we quantify the sensitivity of soil water stress signals inferred from eddy covariance data to the prevailing data and modeling assumptions (i.e., their robustness) to compile a comprehensive list of sites with robust soil water stress signals and assess the performance of current stress inference methods. To accomplish this, we identify the most prevalent assumptions from the literature and perform a digital factorial experiment to extract probability distributions of plausible soil water stress signals and model performance at 151 FLUXNET2015 and AmeriFlux-FLUXNET sites. Here, we develop a new framework that summarizes these probability distributions to classify and rank the robustness of each site’s soil water stress signal, which we display with a user-friendly heat map. We estimate that only 5%–36% of sites exhibit a robust soil water stress signal due to deficient model performance and poorly constrained ecosystem water use parameters. We also find that the lack of robustness is site-specific, which undermines grouping stress signals by broad ecosystem categories or comparing results across studies with differing assumptions. Lastly, existing stress inference methods appear better suited for eddy covariance sites with grass/annual vegetation. Our findings call for more careful and consistent inference of ecosystem water stress from eddy covariance data.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Global Sensitivity Analysis of a Reactive Transport Model for Mineral Scale Formation During Hydraulic Fracturing

Injection of water-based hydraulic fracturing fluid (HFF) into tight shale gas/oil formations can increase formation permeability and enhance production rates, but this process frequently causes mineral scale formation that can occlude pore space and hinder flow. To identify the most important factors that control the formation of mineral scales, we applied a novel global sensitivity analysis method—distance-based generalized sensitivity analysis (DGSA)—to a reactive transport model (RTM) that was previously built and calibrated to simulate precipitation of barite [BaSO4] and iron (hydr)oxide [Fe(OH) 3 ] in shale matrices and on fracture surfaces. Reactive transport simulations were run with model parameters randomly sampled based on assigned uncertainties. Modeling results for barite and Fe(OH)3 formation were clustered using machine-learning algorithms. A list of ranked critical input parameters was obtained after statistical quantification of cumulative distribution functions of input parameters. We found that barite formation is most sensitive to the rate of sulfate ion generation, which is determined by the pyrite dissolution rate coefficient and oxidant availability. In addition, barite formation is sensitive to the initial amounts of barite in HFF and shale, followed by barite thermodynamics/kinetics. For Fe(OH) 3 formation, the ranked factors are Fe(OH)3 precipitation rate coefficients, initial HFF pH, initial Fe(OH) 3 amount in HFF, and oxidant availability. Overall, our results provide insights into managing mineral scale formation during hydraulic fracturing to enhance production. Meanwhile, this study serves as an example of global sensitivity analysis of RTMs using the efficient, straightforward, and open-source DGSA method.

58 GEOSCIENCES↗
DOE OSTI2019

CT-based machine learning model to predict the Fuhrman nuclear grade of clear cell renal cell carcinoma

Purpose: To predict the Fuhrman grade of clear cell renal cell carcinoma (ccRCC) with a machine learning classifier based on single- or three-phase computed tomography (CT) images. Materials and methods: Patients with pathologically proven ccRCC from February 1, 2009 to September 31, 2018 who were not treated were retrospectively collected for machine learning-based analysis. The texture features were extracted and ranked from precontrast phase (PCP), corticomedullary phase (CMP), nephrographic phase (NP) and three-phase CT images, and open-source gradient boosting from the decision tree library of CatBoost was used to establish a machine learning classifier to differentiate low- from high-grade ccRCC. The performances of machine learning classifiers based on features from single- and three-phase CT images were compared with each other. Results: A total of 231 patients with 232 pathologically proven ccRCC lesions were retrospectively collected. 35, 36, 41, and 22 Features were extracted and ranked from PCP, CMP, NP, and three-phase CT images, respectively. The machine learning model based on three-phase CT images [area under the ROC curve (AUC) = 0.87] achieved the best diagnostic performance for differentiating low- from high-grade ccRCC, followed by single-phase NP (AUC = 0.84), CMP (AUC = 0.80), and PCP images (AUC = 0.82). Conclusion: Machine learning classifiers can be promising noninvasive techniques to differentiate low- and high-Fuhrman nuclear grade ccRCC, and classifiers based on three-phase CT images are superior to those based on features from each single phase.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2022

Branches, quivers, and ideals for knot complements

We generalize the F K invariant, i.e. Z^ for the complement of a knot K in the 3-sphere, the knots-quivers correspondence, and A-polynomials of knots, and find several interconnections between them. We associate an F K invariant to any branch of the A-polynomial of K and we work out explicit expressions for several simple knots. We show that these F K invariants can be written in the form of a quiver generating series, in analogy with the knots-quivers correspondence. We discuss various methods to obtain such quiver representations, among others using R-matrices. We generalize the quantum a-deformed A-polynomial to an ideal that contains the recursion relation in the group rank, i.e. in the parameter a, and describe its classical limit in terms of the Coulomb branch of a 3d-5d theory. We also provide t-deformed versions. Furthermore, we study how the quiver formulation for closed 3-manifolds obtained by surgery leads to the superpotential of 3d N = 2 theory T[M 3 ] and to the data of the associated modular tensor category MTC[M 3 ].

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Spectrum of chronic kidney disease in China: A national study based on hospitalized patients from 2010 to 2015

ABSTRACT Aim To investigate the spectrum of chronic kidney disease (CKD) in China. Methods We used a large national in‐patient database covering 878 class three hospitals and involving 64.7 million adult patients in China from 2010 to 2015. The class 3 hospital in China is ranked as the top tier of medical system in China with at least 500 beds and the accreditation from health authorities. The specific causes of CKD were extracted from the International Classification of Diseases‐10 codes of discharge diagnoses. Results A total of 4.5% of hospitalized patients (1.8 million) were identified as CKD, with an increased percentage from 2010 (3.7%) to 2015 (4.7%). Increasing trends of diabetic kidney disease and hypertensive nephropathy were observed from 2010 to 2015 (19.5% vs 24.3% and 11.5% vs 15.9%, respectively), especially for urban residents from north China. The proportion of obstructive nephropathy also increased gradually (10.3% in 2010 vs 15.6% in 2015) and constituted another important cause of CKD for patients, especially for those from south China and rural residents. Conclusion The spectrum of CKD is changing in China, with variations over time and geographic regions, which has implication regarding developing the prevention strategy of CKD.

Huang, Yu‐Ming↗
DOE OSTI2020

Computationally Efficient Decompositions of Oblique Projection Matrices

Oblique projection matrices arise in problems in weighted least squares, signal processing, and optimization. While these matrices can be potentially very large, their low-rank structure can be exploited for efficient computation. Here, we propose fast and scalable algorithms for computing their eigendecomposition and singular value decomposition (SVD). Numerical experiments that compare our proposed approaches to existing methods, including randomized SVD, are presented. In addition, we test their accuracy on linear systems from equality constrained optimization problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A Mixed-Method Design Approach for Empirically Based Selection of Unbiased Data Annotators

Implicit bias embedded in the annotated data is by far the greatest impediment in the effectual use of supervised machine learning models in tasks involving race, ethics, and geopolitical polarization. For societal good and demonstrable positive impact on wider society, it is paramount to carefully select data annotators and rigorously validate the annotation process. Current approaches to selecting annotators are not sufficiently grounded in scientific principles and are limited at the policy-guidance level, thereby rendering them unusable for machine learning practitioners. This work proposes a new approach based on the mixed-methods design that is functional, adaptable, and simpler to implement in selecting unbiased annotators for any machine learning problem. By demonstrating it on a real-world geopolitical problem, we also identified and ranked key inane profile characteristics towards an empirically-based selection of unbiased data annotators.

Thakur, Gautam Malviya↗
NASA NTRS1980

Evaluation of the cyclic behavior of aircraft turbine disk alloys, part 2

Several nickel-base aircraft turbine disk superalloys were evaluated at 650 C for resistance to fatigue crack initiation and propagation under cyclic and cyclic/dwell conditions. Controlled strain low cycle fatigue (LCF) and controlled load crack propagation tests were performed and results utilized to provide a direct comparison among the alloys. Tests were performed on selected alloys to evaluate the effects of hold times, mean stresses, stress-dwell cycle types, inert environment, and contractor test methods. At the lower total strain ranges of interest, the alloys exhibited generally increasing initiation life with increasing tensile strength for both cyclic (0.33 Hz) and cyclic/dwell (900-sec hold per cycle) conditions. Rank order of the alloys by LCF initiation life changed substantially at higher strain ranges, approaching the rank order expected from monotonic tensile ductilities. The effect of the 900 sec (15 min) hold time fatigue life varied significantly from alloy to alloy. Generally, the higher-strength, finer-grained alloys exhibited more significant reductions in fatigue life due to the dwell. The effects of mean strain were found to be negligible and the effects of mean stress were pronounced. At high strain ranges the mean stress was near zero and did not contribute to reduction in life. At low strain ranges, however, mean stresses were large and significant reductions in LCF lives occurred.

Cowles, B. A.↗
NASA NTRS2007

The Business Case for Automated Software Engineering

Adoption of advanced automated SE (ASE) tools would be more favored if a business case could be made that these tools are more valuable than alternate methods. In theory, software prediction models can be used to make that case. In practice, this is complicated by the 'local tuning' problem. Normally. predictors for software effort and defects and threat use local data to tune their predictions. Such local tuning data is often unavailable. This paper shows that assessing the relative merits of different SE methods need not require precise local tunings. STAR 1 is a simulated annealer plus a Bayesian post-processor that explores the space of possible local tunings within software prediction models. STAR 1 ranks project decisions by their effects on effort and defects and threats. In experiments with NASA systems. STARI found one project where ASE were essential for minimizing effort/ defect/ threats; and another project were ASE tools were merely optional.

model evaluation↗