Results for “Heuristic Evaluation” — Page 6

NASA NTRS1994

Integrating Design and Manufacturing for a High Speed Civil Transport Wing

The aerospace industry is currently addressing the problem of integrating design and manufacturing. Because of the difficulties associated with using conventional, procedural techniques and algorithms, it is the authors' belief that the only feasible way to integrate the two concepts is with the development of an appropriate Knowledge-Based System (KBS). The authors propose a methodology for an aircraft producibility assessment, including a KBS, that addresses both procedural and heuristic aspects of integrating design and manufacturing of a High Speed Civil Transport (HSCT) wing. The HSCT was chosen as the focus of this investigation since it is a current NASA/aerospace industry initiative full of technological challenges involving many disciplines. The paper gives a brief background of selected previous supersonic transport studies followed by descriptions of key relevant design and manufacturing methodologies. Georgia Tech's Concurrent Engineering/Integrated Product and Process Development methodology is discussed with reference to this proposed conceptual producibility assessment. Evaluation criteria are presented that relate pertinent product and process parameters to overall product producibility. In addition, the authors' integration methodology and reasons for selecting a KBS to integrate design and manufacturing are presented in this paper. Finally, a proposed KBS is given, as well as statements of future work and overall investigation objectives.

Marx, William J.↗

NASA NTRS1991

Parallel algorithms for placement and routing in VLSI design

The computational requirements for high quality synthesis, analysis, and verification of very large scale integration (VLSI) designs have rapidly increased with the fast growing complexity of these designs. Research in the past has focused on the development of heuristic algorithms, special purpose hardware accelerators, or parallel algorithms for the numerous design tasks to decrease the time required for solution. Two new parallel algorithms are proposed for two VLSI synthesis tasks, standard cell placement and global routing. The first algorithm, a parallel algorithm for global routing, uses hierarchical techniques to decompose the routing problem into independent routing subproblems that are solved in parallel. Results are then presented which compare the routing quality to the results of other published global routers and which evaluate the speedups attained. The second algorithm, a parallel algorithm for cell placement and global routing, hierarchically integrates a quadrisection placement algorithm, a bisection placement algorithm, and the previous global routing algorithm. Unique partitioning techniques are used to decompose the various stages of the algorithm into independent tasks which can be evaluated in parallel. Finally, results are presented which evaluate the various algorithm alternatives and compare the algorithm performance to other placement programs. Measurements are presented on the parallel speedups available.

Brouwer, Randall Jay↗

NASA NTRS2001

Isolating Curvature Effects in Computing Wall-Bounded Turbulent Flows

The flow over the zero-pressure-gradient So-Mellor convex curved wall is simulated using the Navier-Stokes equations. An inviscid effective outer wall shape, undocumented in the experiment, is obtained by using an adjoint optimization method with the desired pressure distribution on the inner wall as the cost function. Using this wall shape with a Navier-Stokes method, the abilities of various turbulence models to simulate the effects of curvature without the complicating factor of streamwise pressure gradient can be evaluated. The one-equation Spalart-Allmaras turbulence model overpredicts eddy viscosity, and its boundary layer profiles are too full. A curvature-corrected version of this model improves results, which are sensitive to the choice of a particular constant. An explicit algebraic stress model does a reasonable job predicting this flow field. However, results can be slightly improved by modifying the assumption on anisotropy equilibrium in the model's derivation. The resulting curvature-corrected explicit algebraic stress model possesses no heuristic functions or additional constants. It lowers slightly the computed skin friction coefficient and the turbulent stress levels for this case (in better agreement with experiment), but the effect on computed velocity profiles is very small.

Rumsey, Christopher L.↗

NASA NTRS1989

A two-phase methodology for technology selection and system design

A two-phase methodology that can be used to guide R&D managers in the evaluation and selection of competing technologies is presented. Deterministic multiattribute utility theory is used in the first phase to rank the technological alternatives; the example presented involves the evaluation of electric and hybrid passenger vehicles. In all, 39 individuals from eight automotive firms were interviewed to assess their risk preferences and attitudes toward the vehicle design. In the second phase, the decision-maker must allocate a fixed amount of resources to different projects for the technology selected, some of which may be undertaken in parallel, to maximize a given measure of performance. When parallel funding is pursued the best outcome is chosen. The problem is formulated as a probabilistic network and solved heuristically using Monte Carlo simulation. Results are presented for two decision-makers and three budget options. In each case, the heuristic finds the optimal allocation of funds.

Bard, Jonathan F.↗

NASA NTRSDate not supplied

Qualitative Data Coding of User Experience with an Urban Air Mobility Fleet Manager Interface

The NASA Aeronautics Research Mission Directorate created the High Density Vertiplex (HDV) project to integrate and evaluate a prototype Urban Air Mobility (UAM) ecosystem. Part of the HDV testing environment included a prototype operator user interface called the Fleet Manager Interface (FMI). In 2023, HDV conducted flight testing with the FMI during which a user experience (UX) study was performed to assess the quality of UX and elicit design recommendations. As a result, a large database of open-ended, qualitative responses was generated and then coded using a new qualitative data coding technique called Directive String Coding, which used a blended coding approach to generate actionable heuristics that stakeholders (e.g., researchers, UI designers, software developers) could use to answer specific research questions and make future design decisions. The key themes that arose from the coded responses showed that the UX was pleasant, system notifications should be more salient, and information across multiple screens should be integrated into a central display. As the Fleet Manger operational role is still being defined, it is vital to increase our understanding of the tools and capabilities needed for such a role. The results from HDV work will eventually feed into standards for vertiport operations that will enable safe and scalable UAM operations in the United States.

high density vertiplex↗

NASA NTRSDate not supplied

Qualitative Data Coding of User Experience with an Urban Air Mobility Fleet Manager Interface

The NASA Aeronautics Research Mission Directorate created the High Density Vertiplex (HDV) project to integrate and evaluate a prototype Urban Air Mobility (UAM) ecosystem. Part of the HDV testing environment included a prototype operator user interface called the Fleet Manager Interface (FMI). In 2023, HDV conducted flight testing with the FMI during which a user experience (UX) study was performed to assess the quality of UX and elicit design recommendations. As a result, a large database of open-ended, qualitative responses was generated and then coded using a new qualitative data coding technique called Directive String Coding, which used a blended coding approach to generate actionable heuristics that stakeholders (e.g., researchers, UI designers, software developers) could use to answer specific research questions and make future design decisions. The key themes that arose from the coded responses showed that the UX was pleasant, system notifications should be more salient, and information across multiple screens should be integrated into a central display. As the Fleet Manger operational role is still being defined, it is vital to increase our understanding of the tools and capabilities needed for such a role. The results from HDV work will eventually feed into standards for vertiport operations that will enable safe and scalable UAM operations in the United States.

high density veriplex↗

NASA NTRS1990

Evaluation and selection of assembly plans

Two criteria are introduced for the evaluation and selection of assembly plans. The first criterion is to maximize the number of different sequences encompassed by the assembly plan. The second criterion is to maximize the amount of parallelism (i.e., simultaneity) that is possible in the execution of the assembly tasks. While the metrics corresponding to the criteria used in previous work can be expressed as a sum of terms, each being a function of a task or a state, the metrics corresponding to the criteria introduced are more complex functions of the whole assembly plan. An algorithm that performs a heuristic search for the best assembly plan over the AND/OR graph representation of assembly plans introduced in previous work is presented. Admissible heuristics for each of the two criteria are presented.

Homem De Mello, L. S.↗

NASA NTRS2013

Determining Component Probability using Problem Report Data for Ground Systems used in Manned Space Flight

During the shuttle era NASA utilized a failure reporting system called the Problem Reporting and Corrective Action (PRACA) it purpose was to identify and track system non-conformance. The PRACA system over the years evolved from a relatively nominal way to identify system problems to a very complex tracking and report generating data base. The PRACA system became the primary method to categorize any and all anomalies from corrosion to catastrophic failure. The systems documented in the PRACA system range from flight hardware to ground or facility support equipment. While the PRACA system is complex, it does possess all the failure modes, times of occurrence, length of system delay, parts repaired or replaced, and corrective action performed. The difficulty is mining the data then to utilize that data in order to estimate component, Line Replaceable Unit (LRU), and system reliability analysis metrics. In this paper, we identify a methodology to categorize qualitative data from the ground system PRACA data base for common ground or facility support equipment. Then utilizing a heuristic developed for review of the PRACA data determine what reports identify a credible failure. These data are the used to determine inter-arrival times to perform an estimation of a metric for repairable component-or LRU reliability. This analysis is used to determine failure modes of the equipment, determine the probability of the component failure mode, and support various quantitative differing techniques for performing repairable system analysis. The result is that an effective and concise estimate of components used in manned space flight operations. The advantage is the components or LRU's are evaluated in the same environment and condition that occurs during the launch process.

Monaghan, Mark W.↗

NASA NTRS2017

Improvement of Automated POST Case Success Rate Using Support Vector Machines

During early conceptual design of complex systems, concept down selection can have a large impact upon program life-cycle cost. Therefore, any concepts selected during early design will inherently commit program costs and affect the overall probability of program success. For this reason it is important to consider as large a design space as possible in order to better inform the down selection process. For conceptual design of launch vehicles, trajectory analysis and optimization often presents the largest obstacle to evaluating large trade spaces. This is due to the sensitivity of the trajectory discipline to changes in all other aspects of the vehicle design. Small deltas in the performance of other subsystems can result in relatively large fluctuations in the ascent trajectory because the solution space is non-linear and multi-modal. In order to help capture large design spaces for new launch vehicles, the authors have performed previous work seeking to automate the execution of the industry standard tool, Program to Optimize Simulated Trajectories (POST). This work initially focused on implementation of analyst heuristics to enable closure of cases in an automated fashion, with the goal of applying the concepts of design of experiments (DOE) and surrogate modeling to enable near instantaneous throughput of vehicle cases.3 As noted in [4] work was then completed to improve the DOE process by utilizing a graph theory based approach to connect similar design points.

Zwack, Mathew R.↗

NASA NTRSDate not supplied

A Hybrid Approach to Labeling Datasets in Earth Science Publications

NASA Data Centers provide the public with thousands of datasets that result in published papers, reports, and conference proceedings. Collecting accurate metrics on usage of these datasets is key to connecting different areas of knowledge and evaluating the datasets’ impact. While most of the datasets have Digital Object Identifiers (DOIs) assigned, most publications do not cite them hampering the automated search of these publications. Instead, articles mention attributes like organization, instrument, mission, variable, or a publication describing the dataset. Often only domain experts can deduce the dataset that was used in the publication text. The lack of a citation slows the spread of information and reduces the research’s impact. With thousands of papers produced each year, an automated means of labeling datasets is critical. This paper explores a hybrid approach of heuristics and a Natural Language Processing (NLP) Named Entity Recognition (NER) model to find and label the datasets used within Earth Science papers. Heuristics are used to produce the labelled sentences and any potential dataset candidates that can be derived from a sentence. The heuristic labels the sentences with the names of mission, instrument, re-analysis models, and science keywords taken from the Global Change Master Directory (GCMD) ontology. Additionally, it uses those labels to generate the dataset citation candidates. If the mission, instrument, and variable are sufficient to create the citation for the dataset the citation and the label the domain expert reviews the output without going through the NLP model. If the extracted label is not sufficient to label the dataset on its own, the sentence and its associated dataset labels will be inputted into the NER model. The model outputs the labeled sentence and the potential dataset candidates with their associated probabilities. The domain expert then reviews the NER model’s output and the correct labels are determined. The newly labelled papers can then be used as additional training data. This creates an iterative process for the approach to continuously improve. Because all the possible mentions are gathered by the model, the domain expert can quickly and easily label the papers resulting in large time savings.

Jacob Atkins↗

NASA NTRS2017

Improvement of Automated POST Case Success Rate Using Support Vector Machines

During early conceptual design of complex systems, concept down selection can have a large impact upon program life-cycle cost. Therefore, any concepts selected during early design will inherently commit program costs and affect the overall probability of program success. For this reason it is important to consider as large a design space as possible in order to better inform the down selection process. For conceptual design of launch vehicles, trajectory analysis and optimization often presents the largest obstacle to evaluating large trade spaces. This is due to the sensitivity of the trajectory discipline to changes in all other aspects of the vehicle design. Small deltas in the performance of other subsystems can result in relatively large fluctuations in the ascent trajectory because the solution space is non-linear and multi-modal [1]. In order to help capture large design spaces for new launch vehicles, the authors have performed previous work seeking to automate the execution of the industry standard tool, Program to Optimize Simulated Trajectories (POST). This work initially focused on implementation of analyst heuristics to enable closure of cases in an automated fashion, with the goal of applying the concepts of design of experiments (DOE) and surrogate modeling to enable near instantaneous throughput of vehicle cases [2]. Additional work was then completed to improve the DOE process by utilizing a graph theory based approach to connect similar design points [3]. The conclusion of the previous work illustrated the utility of the graph theory approach for completing a DOE through POST. However, this approach was still dependent upon the use of random repetitions to generate seed points for the graph. As noted in [3], only 8% of these random repetitions resulted in converged trajectories. This ultimately affects the ability of the random reps method to confidently approach the global optima for a given vehicle case in a reasonable amount of time. With only an 8% pass rate, tens or hundreds of thousands of reps may be needed to be confident that the best repetition is at least close to the global optima. However, typical design study time constraints require that fewer repetitions be attempted, sometimes resulting in seed points that have only a handful of successful completions. If a small number of successful repetitions are used to generate a seed point, the graph method may inherit some inaccuracies as it chains DOE cases from the non-global-optimal seed points. This creates inherent noise in the graph data, which can limit the accuracy of the resulting surrogate models. For this reason, the goal of this work is to improve the seed point generation method and ultimately the accuracy of the resulting POST surrogate model. The work focuses on increasing the case pass rate for seed point generation.

Zwack, Matthew R.↗

NASA NTRS1989

Interset: A natural language interface for teleoperated robotic assembly of the EASE space structure

A teleoperated robot was used to assemble the Experimental Assembly of Structures in Extra-vehicular activity (EASE) space structure under neutral buoyancy conditions, simulating a telerobot performing structural assembly in the zero gravity of space. This previous work used a manually controlled teleoperator as a test bed for system performance evaluations. From these results several Artificial Intelligence options were proposed. One of these was further developed into a real time assembly planner. The interface for this system is effective in assembling EASE structures using windowed graphics and a set of networked menus. As the problem space becomes more complex and hence the set of control options increases, a natural language interface may prove to be beneficial to supplement the menu based control strategy. This strategy can be beneficial in situations such as: describing the local environment, maintaining a data base of task event histories, modifying a plan or a heuristic dynamically, summarizing a task in English, or operating in a novel situation.

Boorsma, Daniel K.↗

NASA NTRS1999

Double Resonances and Spectral Scaling in the Weak Turbulence Theory of Rotating and Stratified Turbulence

In rotating turbulence, stably stratified turbulence, and in rotating stratified turbulence, heuristic arguments concerning the turbulent time scale suggest that the inertial range energy spectrum scales as k(exp -2). From the viewpoint of weak turbulence theory, there are three possibilities which might invalidate these arguments: four-wave interactions could dominate three-wave interactions leading to a modified inertial range energy balance, double resonances could alter the time scale, and the energy flux integral might not converge. It is shown that although double resonances exist in all of these problems, they do not influence overall energy transfer. However, the resonance conditions cause the flux integral for rotating turbulence to diverge logarithmically when evaluated for a k(exp -2) energy spectrum; therefore, this spectrum requires logarithmic corrections. Finally, the role of four-wave interactions is briefly discussed.

Rubinstein, Robert↗

NASA NTRS1988

NESSUS (Numerical Evaluation of Stochastic Structures Under Stress)/EXPERT: Bridging the gap between artificial intelligence and FORTRAN

The development of a probabilistic structural analysis methodology (PSAM) is described. In the near-term, the methodology will be applied to designing critical components of the next generation space shuttle main engine. In the long-term, PSAM will be applied very broadly, providing designers with a new technology for more effective design of structures whose character and performance are significantly affected by random variables. The software under development to implement the ideas developed in PSAM resembles, in many ways, conventional deterministic structural analysis code. However, several additional capabilities regarding the probabilistic analysis makes the input data requirements and the resulting output even more complex. As a result, an intelligent front- and back-end to the code is being developed to assist the design engineer in providing the input data in a correct and appropriate manner. The type of knowledge that this entails is, in general, heuristically-based, allowing the fairly well-understood technology of production rules to apply with little difficulty. However, the PSAM code, called NESSUS, is written in FORTRAN-77 and runs on a DEC VAX. Thus, the associated expert system, called NESSUS/EXPERT, must run on a DEC VAX as well, and integrate effectively and efficiently with the existing FORTRAN code. This paper discusses the process undergone to select a suitable tool, identify an appropriate division between the functions that should be performed in FORTRAN and those that should be performed by production rules, and how integration of the conventional and AI technologies was achieved.

Fink, Pamela K.↗

NASA NTRS2017

Evaluation of Integrated Demand Management Looking into Strategic & Tactical Flow Management

The motivation behind Integrated Demand Management (IDM) research is to explore possible improvements to United States National Airspace System (NAS) performance that could be realized through procedural integration of strategic traffic flow management capabilities, such as the Collaborative Trajectory Options Program (CTOP), and tactical capabilities, such as Time Based Flow Management (TBFM). An initial IDM concept for clear weather operations was developed and evaluated for potential benefits, including efficiency, delay reduction, predictability and throughput, and to identify any major issues that might represent a showstopper for a fielded application. Newark Liberty International Airport (EWR) arrival operations provided a use case for concept development. EWR uses miles-in-trail (MIT) metering to regulate demand into TBFM during high volume operations, and short-haul flights are often penalized with excessive, last-minute ground delays when the overhead stream is saturated. IDM addresses this problem by replacing MIT conditioning with CTOP to better manage the demand delivery to the TBFM entry points. A quasi-real time high-fidelity simulation that would normally involve participants was conducted using heuristic-based procedures that mimicked operators behaviors instead. Five total conditions were compared: two baseline conditions with MIT delivery to TBFM entry points using two different TBFM settings; and three IDM conditions: one with airborne speed control using an Required Time of Arrival (RTA) capability, a second without RTA, and a third with no wind forecast errors. Results suggest that the IDM concept can deliver traffic more efficiently by shifting the delays from airborne to ground for both RTA and non-RTA conditions, while maintaining a target throughput rate. The results also suggest that with good predictability of airport capacity, excessive TBFM ground delay can be minimized by applying more strategic CTOP delay, increasing predictability for the airline operators. Overall, the results indicate that the implementation of an IDM concept under clear weather conditions can improve NAS system performance. Future IDM research aims to expand the concept to address demandcapacity imbalance d severe weather.

Time-Based Flow Management (TBFM)↗

NASA NTRS1989

REDEX: The ranging equipment diagnostic expert system

REDEX, an advanced prototype expert system that diagnoses hardware failures in the Ranging Equipment (RE) at NASA's Ground Network tracking stations is described. REDEX will help the RE technician identify faulty circuit cards or modules that must be replaced, and thereby reduce troubleshooting time. It features a highly graphical user interface that uses color block diagrams and layout diagrams to illustrate the location of a fault. A semantic network knowledge representation technique was used to model the design structure of the RE. A catalog of generic troubleshooting rules was compiled to represent heuristics that are applied in diagnosing electronic equipment. Specific troubleshooting rules were identified to represent additional diagnostic knowledge that is unique to the RE. Over 50 generic and 250 specific troubleshooting rules have been derived. REDEX is implemented in Prolog on an IBM PC AT-compatible workstation. Block diagram graphics displays are color-coded to identify signals that have been monitored or inferred to have nominal values, signals that are out of tolerance, and circuit cards and functions that are diagnosed as faulty. A hypertext-like scheme is used to allow the user to easily navigate through the space of diagrams and tables. Over 50 graphic and tabular displays have been implemented. REDEX is currently being evaluated in a stand-alone mode using simulated RE fault scenarios. It will soon be interfaced to the RE and tested in an online environment. When completed and fielded, REDEX will be a concrete example of the application of expert systems technology to the problem of improving performance and reducing the lifecycle costs of operating NASA's communications networks in the 1990's.

Luczak, Edward C.↗

NASA NTRS1989

REDEX - The ranging equipment diagnostic expert system

REDEX, an advanced prototype expert system that diagnoses hardware failures in the Ranging Equipment (RE) at NASA's Ground Network tracking stations is described. REDEX will help the RE technician identify faulty circuit cards or modules that must be replaced, and thereby reduce troubleshooting time. It features a highly graphical user interface that uses color block diagrams and layout diagrams to illustrate the location of a fault. A semantic network knowledge representation technique was used to model the design structure of the RE. A catalog of generic troubleshooting rules was compiled to represent heuristics that are applied in diagnosing electronic equipment. Specific troubleshooting rules were identified to represent additional diagnostic knowledge that is unique to the RE. Over 50 generic and 250 specific troubleshooting rules have been derived. REDEX is implemented in Prolog on an IBM PC AT-compatible workstation. Block diagram graphics displays are color-coded to identify signals that have been monitored or inferred to have nominal values, signals that are out of tolerance, and circuit cards and functions that are diagnosed as faulty. A hypertext-like scheme is used to allow the user to easily navigate through the space of diagrams and tables. Over 50 graphic and tabular displays have been implemented. REDEX is currently being evaluated in a stand-alone mode using simulated RE fault scenarios. It will soon be interfaced to the RE and tested in an online environment. When completed and fielded, REDEX will be a concrete example of the application of expert systems technology to the problem of improving performance and reducing the lifecycle costs of operating NASA's communications networks in the 1990s.

Luczak, Edward C.↗

NASA NTRSDate not supplied

Planning Bias: Planning as a Source of Sampling Bias

Many data-driven planning methods are trained on data generated by planners. It is well known that many statistical learning methods are sensitive to sampling bias, and yet there has been little or no attention to planning as a sampling method and its role in introducing sampling bias into planner-generated training data. Recently, it has been demonstrated that A**,* in the presence of problems with variable heuristic error, prefers some solutions over other equally cost-optimal solutions. But, as we discuss in this paper, mitigation may not be as simple as resolving arbitrary tie-breaking by sampling from ties uniformly at random. In this paper, we formalize an intuition of planning bias. We focus on problems which output a single solution. Diverse planning only complicates the problem by generalizing it to bias in the set of sets; we show how it is subject to bias in the single solution. We make some useful observations about deterministic algorithms in contrast to non-deterministic algorithms. We explain how information entropy may be a good way to measure planning bias, and discuss some issues in evaluating practical approaches to measurement. We address the intuition that uniform random tiebreaking should mitigate bias; and sketch a novel approach to constructing an appropriate random distribution for duplicate detection during forward search for unbiased A*. Finally, we suggest directions for future work.

Planning Scheduling Algorithms↗