Search NASA⌕ Search

SEARCH · Search NASA

Results for “evaluation process”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

Assessing Advanced Technology in CENATE

PNNL's Center for Advanced Technology Evaluation (CENATE) is a new U.S. Department of Energy center whose mission is to assess and facilitate access to emerging computing technology. CENATE is assessing a range of advanced technologies, from evolutionary to disruptive. Technologies of interest include the processor socket (homogeneous and accelerated systems), memories (dynamic, static, memory cubes), motherboards, networks (network interface cards and switches), and input/output and storage devices. CENATE is developing a multi-perspective evaluation process based on integrating advanced system instrumentation, performance measurements, and modeling and simulation. We show evaluations of two emerging network technologies: silicon photonics interconnects and the Data Vortex network. CENATE's evaluation also addresses the question of which machine is best for a given workload under certain constraints. We show a performance-power tradeoff analysis of a well-known machine learning application on two systems.

Tallent, Nathan R.↗

Comparative Assessment of Data-driven Process Models in Health Information Technology

Process mining for conformance analysis consists of comparing a reference process model against a data-driven process model generated via log files from information technology systems. However, in the absence of a complete reference process model, we found no suggested approaches in the literature to address the need for evaluating process conformance among different healthcare facilities to assess standardization of care. Our goal is to find similarities and dissimilarities in data-driven process models among US Veterans Health Administration (VHA) facilities that can be indicative of patient safety issues. Our hypothesis was that the analysis would not produce statistically significant differences in outcome. We present a unique implementation of conformance analysis in process mining that consists of combining process mining, process mapping and statistical metrics. We illustrate our approach by applying it to the analysis of two clinical radiology order process models generated from healthcare data provided by two similar facilities in the VHA. The comparative assessment showed that about 70% of the orders completed successfully and 30% were not completed due to policy and duplications. Our analysis found a good statistical correlation between both facilities, as the Spearman’s correlation coefficient between facilities for the frequency of cases per total hours was 0.87879, for the frequency of cases by state transition was 0.79702 and for the throughput time per state transition was 0.63582. Additional statistical analyses using the Mann-Whitney U test and the root mean square error both produced values that were not significant. The foregoing approach validated our hypothesis by demonstrating a good statistical correlation of data describing the flow of clinical radiology orders absent a credible reference model. Finding good agreement between both facilities was important in confirming that the clinical orders flow in a similar manner, suggesting standardization of care.

97 MATHEMATICS AND COMPUTING↗

Special Observing Period (SOP) data for the Year of Polar Prediction site Model Intercomparison Project (YOPPsiteMIP)

The rapid changes occurring in the polar regions require an improved understanding of the processes that are driving these changes. At the same time, increased human activities such as marine navigation, resource exploitation, aviation, commercial fishing, and tourism require reliable and relevant weather information. One of the primary goals of the World Meteorological Organization's Year of Polar Prediction (YOPP) project is to improve the accuracy of numerical weather prediction (NWP) at high latitudes. During YOPP, two Canadian “supersites” were commissioned and equipped with new ground-based instruments for enhanced meteorological and system process observations. Additional pre-existing supersites in Canada, the United States, Norway, Finland, and Russia also provided data from ongoing long-term observing programs. These supersites collected a wealth of observations that are well suited to address YOPP objectives. In order to increase data useability and station interoperability, novel Merged Observatory Data Files (MODFs) were created for the seven supersites over two Special Observing Periods (February to March 2018 and July to September 2018). All observations collected at the supersites were compiled into this standardized NetCDF MODF format, simplifying the process of conducting pan-Arctic NWP verification and process evaluation studies. This paper describes the seven Arctic YOPP supersites, their instrumentation, data collection and processing methods, the novel MODF format, and examples of the observations contained therein. MODFs comprise the observational contribution to the model intercomparison effort, termed YOPP site Model Intercomparison Project (YOPPsiteMIP). All YOPPsiteMIP MODFs are publicly accessible via the YOPP Data Portal (Whitehorse: https://doi.org/10.21343/a33e-j150, Huang et al., 2023a; Iqaluit: https://doi.org/10.21343/yrnf-ck57, Huang et al., 2023b; Sodankylä: https://doi.org/10.21343/m16p-pq17, O'Connor, 2023; Utqiagvik: https://doi.org/10.21343/a2dx-nq55, Akish and Morris, 2023c; Tiksi: https://doi.org/10.21343/5bwn-w881, Akish and Morris, 2023b; Ny-Ålesund: https://doi.org/10.21343/y89m-6393, Holt, 2023; and Eureka: https://doi.org/10.21343/r85j-tc61, Akish and Morris, 2023a), which is hosted by MET Norway, with corresponding output from NWP models.

54 ENVIRONMENTAL SCIENCES↗

Advances in metals processing

Research on metals processing being conducted to develop improved forming and joining methods with the potential of reducing the weight and cost of future aerospace structures is discussed. The approach followed is to assess the state of the art for fabricating a given structural system, define candidate methods for improving processing, evaluate the merits of each, fabricate and test subelement components, and then scale up the process to demonstrate validity. The development and the state of the art of weldbrazing, superplastic forming (SPF), superplastic forming and codiffusion bonding and superplastic forming and weldbrazing for titanium and the SPF of aluminum is discussed.

Thomas T. Bales↗

Aviation System Analysis Capability Executive Assistant Design

In this technical document, we describe the design developed for the Aviation System Analysis Capability (ASAC) Executive Assistant (EA) Proof of Concept (POC). We describe the genesis and role of the ASAC system, discuss the objectives of the ASAC system and provide an overview of components and models within the ASAC system, and describe the design process and the results of the ASAC EA POC system design. We also describe the evaluation process and results for applicable COTS software. The document has six chapters, a bibliography, three appendices and one attachment.

Roberts, Eileen↗

Multi phenomena melt pool sensor data fusion for enhanced process monitoring of laser powder bed fusion additive manufacturing

Finding actionable trends in laser-based metal additive manufacturing process monitoring data is challenging owing to the diversity and complexity of the underlying physical interactions. A single monitoring solution that captures a particular process phenomenon, such as a photodiode that tracks melt pool intensity, is not alone capable of evaluating process stability or detecting flaw formation with sufficient precision for routine application in industry. In this work, to improve flaw detection performance, we adopted a data fusion approach that captures multiple process phenomena. To demonstrate this, we acquired data from laser powder bed fusion (LPBF) builds of cylindrical specimens produced with different laser spot sizes, emulating defocusing due to process faults such as thermal lensing. The resulting specimens had porosity of varying types and severity, quantified by post-build non-destructive X-ray computed tomography, Archimedes density measurements, and destructive metallographic characterization. During the build, the melt pool state was monitored with two coaxial high-speed video cameras and a temperature field imaging system. Physically intuitive low-level melt pool signatures, such as melt pool temperature, shape and size, and spatter intensity were extracted from this high-dimensional, image-based sensor data. These process signatures were subsequently used as input features in relatively simple machine learning models, such as a support vector machine, which were trained to detect laser defocusing, and in addition, predict porosity type and severity. The results show that the data fusion approach significantly enhanced system performance by reducing the overall false positive rate from ~ 0.1 to ~ 0.001 without sacrificing the true positive rate (~0.90). These results were at par with a black-box, deep machine learning approach (convolutional neural network).

36 MATERIALS SCIENCE↗

Distinguishing Charged Lepton Flavor Violation Scenarios with Inelastic 𝜇 → 𝑒 Conversion

The Mu2e and COMET experiments are expected to improve existing limits on charged lepton flavor violation (CLFV) by roughly 4 orders of magnitude. 𝜇 → 𝑒 conversion experiments are typically optimized for electrons produced without nuclear excitation, as this maximizes the electron energy and minimizes backgrounds from the free decay of the muon. Here we argue that Mu2e and COMET will be able to extract additional constraints on CLFV from inelastic 𝜇 → 𝑒 conversion, given the 27 Al target they have chosen and backgrounds they anticipate. We describe CLFV scenarios in which inelastic CLFV can induce measurable distortions in the near-endpoint spectrum of conversion electrons, including cases where certain contributing operators cannot be probed in elastic 𝜇 → 𝑒 conversion. We extend the nonrelativistic EFT treatment of elastic 𝜇 → 𝑒 conversion to include the new nuclear operators needed for the inelastic process, evaluate the associated nuclear response functions, and describe several new-physics scenarios where the inelastic process can provide additional information on CLFV.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Bringing Automated Fault Detection and Diagnostics Tools for HVAC&R Into the Mainstream

Heating, ventilation, and air-conditioning (HVAC) systems consume over 5 quads of energy annually, representing 30% of energy consumption in the U.S. commercial buildings. Additionally, commercial refrigeration (R) systems add about 1 quad to commercial buildings energy consumption. Most HVAC systems operate with one or more faults that result in increased energy consumption. Fault detection and diagnostics (FDD) tools have been developed to address this national issue, and many tools are commercially available. FDD tools have the potential to save considerable energy for an existing commercial rooftop unit (RTU) and refrigeration systems. These devices can be used for both retro commissioning and, when faults are addressed, continuous commissioning. However, there appears to be multiple market barriers for this technology. Although there are efforts to develop FDD tool standards, currently there are no standards and methods to define functions, capabilities, accuracy, and reliability of FDD tools in the field. Moreover, most of the commercial FDD tools have not been verified in the field independently. This paper presents a comprehensive approach for bringing HVAC FDD tools into the mainstream. The approach involves demonstrating ten commercially available FDD tools installed at ten different sites, independent testing and evaluation of the FDD tools, communication with various stakeholders, identifying and assessing market barriers, creating a process evaluation methodology, and assisting utility companies in developing incentive programs. Furthermore, the preliminary baseline results from the case study demonstrate how the use of an independent monitoring system (IMS) can be used for ground-truth in evaluating FDD tools in the field.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Evaluation of thermal network correction program using test temperature data

An evaluation process to determine the accuracy of a computer program for thermal network correction is discussed. The evaluation is required since factors such as inaccuracies of temperatures, insufficient number of temperature points over a specified time period, lack of one-to-one correlation between temperature sensor and nodal locations, and incomplete temperature measurements are not present in the computer-generated information. The mathematical models used in the evaluation are those that describe a physical system composed of both a conventional and a heat pipe platform. A description of the models used, the results of the evaluation of the thermal network correction, and input instructions for the thermal network correction program are presented.

Ishimoto, T.↗

Measurement/Evaluation Techniques and Nuclear Data Associated with Fission of 239Pu by Fission Spectrum Neutrons

This Panel was chartered to review and assess new evaluations of work on fission product data, as well as the evaluation process used by the two U.S. nuclear weapons physics laboratories. The work focuses on fission product yields resulting from fission spectrum neutrons incident on plutonium, and includes data from measurements that had not been previously published as well as new or revised fission product cumulative yield data, and related quantities such as Q values and R values. This report documents the Panel's assessment of the work presented by Los Alamos National Laboratory (LANL) and Lawrence Livermore National Laboratory (LLNL). Based on the work presented we have seven key observations: (1) Experiments conducted in the 1970s at LANL, some of which were performed in association with a larger, NIST-led, program, have recently been documented. A preliminary assessment of this work, which will be referred to in this document as ILRR-LANL, shows it to be technically sound. (2) LLNL has done a thorough, unbiased review and evaluation of the available literature and is in the process of incorporating the previously unavailable LANL data into its evaluation of key fission product yields. The results of the LLNL effort, which includes a preliminary evaluation of the ILRR-LANL data, have been documented. (3) LANL has also conducted an evaluation of fission product yields for fission spectrum neutrons on plutonium including a meta-analysis of benchmark data as part of a planned upgrade to the ENDF/B compilation. We found that the approach of using meta-analysis provides valuable additional insight for evaluating the sparse data sets involved in this assessment. (4) Both laboratories have provided convincing evidence for energy dependence in the fission product yield of {sup 147}Nd produced from the bombardment of {sup 239}Pu with fission spectrum neutrons over an incident neutron energy range of 0.2 to 1.9 MeV. (5) Consistent, complete, and explicit treatment of both systematic and statistical uncertainties, including correlations, are critical to the assessment of both the experimental measurements (due to variations between experimental techniques, irradiation conditions, calibration procedures, etc.), and the evaluation of those experiments to extract fundamental nuclear data. A clear example of the importance of uncertainty analysis is in the justification for energy-dependent {sup 147}Nd fission product yield, where the magnitude of the effect is comparable to the uncertainties of the individual fission product yield measurements. Both LANL and LLNL are committed to the inclusion of full uncertainty analysis in their evaluations. (6) The Panel reviewed in detail two methods for determining/evaluating fission product yields from which fission assessments can be made: the K factor method and high-resolution gamma spectroscopy (both described more fully in Sections 3 and 4). The panel concluded that fission product yields, and thus fission assessments, derived using either approach are equally valid, provided that the data were obtained from well understood, direct fission measurements and that the key underlying calibrations and/or data are valid for each technique. (7) The Panel found the process of peer review of the two complementary but independent methods to be an extremely useful exercise. Although work is still ongoing and the numbers presented to the Panel may change slightly, both groups are now in much better agreement on not just one, but four key fission product yields. The groups also have a better appreciation of the strengths and weaknesses of each other's methods.

42 ENGINEERING↗

Automated Fiber Placement Manufactured Composites for Science Applications

Science instruments with large collecting areas that maintain dimensional stability, such as James Webb Space Telescope and Wide Field Space Telescope, help achieve next generation science advancements. Composite materials often used for science applications include high modulus fibers in cyanate ester matrices to result in dimensionally stable structures with low contamination. Hand lay-up fabrication is the most common approach for science instrument structures. Automated Fiber Placement (AFP) using intermediate modulus fibers is commonplace in aircraft production reducing manufacturing time and increasing quality and consistency. AFP manufacturing for future large science instruments can similarly reduce costs and increase reliability. However, high modulus fibers are more prone to damage than intermediate modulus fibers. This study investigates the manufacturing viability of M55J/RS3C (Tencate) slit tape material using AFP processing. Tencate provides slit tape materials. NASA Langley Research Center (LaRC) manufactured hand layup and AFP lay-up laminates under room temperature for initial trials, Marshall Space Flight Center (MSFC) manufactured AFP laminates under room temperature and elevated temperature conditions to evaluate processing affects. Goddard Space Flight Center (GSFC) tests and evaluates tension and Coefficient of Thermal Expansion (CTE) properties by hand lay-up and AFP slit tape automated manufacturing for large science applications. These results show processing material warm reduces process induced fiber fracture; leading to stiffness and CTE properties consistent with hand lay-up, while observing a slight degradation in tensile strength.

Segal, Kenneth N.↗

Criteria for Determination of Material Control and Accountability System Effectiveness

The Nevada Test Site (NTS) is a test bed for implementation of the Safeguards First Principles Initiative (SFPI), a risk-based approach to Material Control & Accountability (MC&A) requirements. The Comprehensive Assessment of Safeguards Strategies (COMPASS) model is used to determine the effectiveness of MC&A systems under SFPI. Under this model, MC&A is divided into nine primary elements. Each element is divided into sub-elements. Then each sub-element is assigned two values, effectiveness and contribution, that are used to calculate the rating. Effectiveness is a measure of subelement implementation and how well it meets requirements. Contribution is a relative measure of the importance, and functions as a weighting factor. The COMPASS model provides the methodology for calculation of sub-element and element ratings, but not the actual criteria. Each site must develop its own criteria. For the rating to be meaningful, the effectiveness criteria must be objective and based on explicit, measurable criteria. Contribution (weights) must reflect the importance within the MC&A program. This paper details the NTS approach to system effectiveness and contribution values, and will cover the following: the basis for the ratings, an explanation of the contribution “weights,” and the objective, performance based effectiveness criteria. Finally, the evaluation process will be described.

61 RADIATION PROTECTION AND DOSIMETRY↗

Preliminary Process and Market Evaluation: Better Buildings Neighborhood Program

This document presents the preliminary process and market evaluation of the U.S. Department of Energy’s (DOE) Better Buildings Neighborhood Program (BBNP). The document was prepared in partial fulfillment of a contract with the Lawrence Berkeley National Laboratory to conduct a comprehensive program assessment of BBNP. BBNP is one of many programs funded through the American Recovery and Reinvestment Act of 2009. The funding initially targeted city and state governments, local neighborhood associations, nonprofits, colleges and universities, utilities, and financial institutions through the competitive Energy Efficiency and Conservation Block Grant program Retrofit Ramp-Up Solicitation. Initially DOE made 25 awards to local governmental or nonprofit organizations in amounts ranging from $1.2 to $40 million. Nine similar grantees from the Formula Energy Efficiency and Conservation Block Grant program and seven from the State Energy Program solicitation resulted in 41 Better Building grantees, with award allocations of $508,302,786 in funding for Better Buildings projects.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Computer-Based Tools for Evaluating Graphical User Interfaces

The user interface is the component of a software system that connects two very complex system: humans and computers. Each of these two systems impose certain requirements on the final product. The user is the judge of the usability and utility of the system; the computer software and hardware are the tools with which the interface is constructed. Mistakes are sometimes made in designing and developing user interfaces because the designers and developers have limited knowledge about human performance (e.g., problem solving, decision making, planning, and reasoning). Even those trained in user interface design make mistakes because they are unable to address all of the known requirements and constraints on design. Evaluation of the user inter-face is therefore a critical phase of the user interface development process. Evaluation should not be considered the final phase of design; but it should be part of an iterative design cycle with the output of evaluation being feed back into design. The goal of this research was to develop a set of computer-based tools for objectively evaluating graphical user interfaces. The research was organized into three phases. The first phase resulted in the development of an embedded evaluation tool which evaluates the usability of a graphical user interface based on a user's performance. An expert system to assist in the design and evaluation of user interfaces based upon rules and guidelines was developed during the second phase. During the final phase of the research an automatic layout tool to be used in the initial design of graphical inter- faces was developed. The research was coordinated with NASA Marshall Space Flight Center's Mission Operations Laboratory's efforts in developing onboard payload display specifications for the Space Station.

Moore, Loretta A.↗

Aviation System Analysis Capability Executive Assistant Development

In this technical document, we describe the development of the Aviation System Analysis Capability (ASAC) Executive Assistant (EA) Proof of Concept (POC) and Beta version. We describe the genesis and role of the ASAC system, discuss the objectives of the ASAC system and provide an overview of components and models in the ASAC system, and describe the design process and the results of the ASAC EA POC and Beta system development. We also describe the evaluation process and results for applicable COTS software. The document has seven chapters, a bibliography, and two appendices.

Roberts, Eileen↗

CROSS: A GDSS for the Evaluation and Prioritization of Engineering Support Requests and Advanced Technology Projects at NASA

Objective evaluation and prioritization of engineering support requests (ESRs) is a difficult task at the Kennedy Space Center (KSC) Shuttle Project Engineering Office. The difficulty arises from the complexities inherent in the evaluation process and the lack of structured information. The purpose of this project is to implement the consensus ranking organizational support system (CROSS), a multiple criteria decision support system (DSS) developed at KSC that captures the decision maker's beliefs through a series of sequential, rational, and analytical processes. CROSS utilizes the analytic hierarchy process (AHP), subjective probabilities, entropy concept, and maximize agreement heuristic (MAH) to enhance the decision maker's intuition in evaluation ESRs. Some of the preliminary goals of the project are to: (1) revisit the structure of the ground systems working team (GWST) steering committee, (2) develop a template for ESR originators to provide more comple and consistent information to the GSWT steering committee members to eliminate the need for a facilitator, (3) develop an objective and structured process for the initial screening of ESRs, (4) extensive training of the stakeholders and the GWST steering committee to eliminate the need for a facilitator, (5) automate the process as much as possible, (6) create an environment to compile project success factor data on ESRs and move towards a disciplined system that could be used to address supportability threshold issues at the KSC, and (7) investigate the possibility of an organization-wide implementation of CROSS.

Tavana, Madjid↗

Dynamic Analysis of Capture Devices for Momentum Exchange with Tethers

One of the significant challenges in developing a momentum exchange / electrodynamic reboost tether system is in the analysis and design of the capture device and its effects on the overall dynamics of the system. The goal of this work is to develop appropriate tether momentum exchange models that can simulate and evaluate the requirements of such a system, and be used to create specifications on the design of a capture device. This report briefly describes dynamic model development, simulation of the momentum exchange process, evaluation of dynamic effects of errors in the momentum exchange process, and the development of guidelines in selecting dynamic properties in the design of a capture device.

Stephen Canfield↗

Evaluation of the Neutron Data Standards

With the need for improving existing nuclear data evaluations, (e.g., ENDF/B-VIII.0 and JEFF-3.3 releases) the first step was to evaluate the standards for use in such a library. This new standards evaluation made use of improved experimental data and some developments in the methodology of analysis and evaluation. In addition to the work on the traditional standards, this work produced the extension of some energy ranges and includes new reactions that are called reference cross sections. Since the effort extends beyond the traditional standards, it is called the neutron data standards evaluation. This international effort has produced new evaluations of the following cross section standards: the H(n,n), 6 Li(n,t), 10 B(n,α), 10 B(n,α 1 γ), nat C(n,n), Au(n,γ), 235 U(n,f) and 238 U(n,f). Also in the evaluation process the 238 U(n,γ) and 239 Pu(n,f) cross sections that are not standards were evaluated. Evaluations were also obtained for data that are not traditional standards: the Maxwellian spectrum averaged cross section for the Au(n,γ) cross section at 30 keV; reference cross sections for prompt γ-ray production in fast neutron-induced reactions; reference cross sections for very high energy fission cross sections; the 252 Cf spontaneous fission neutron spectrum and the 235 U prompt fission neutron spectrum induced by thermal incident neutrons; and the thermal neutron constants. The data and covariance matrices of the uncertainties were obtained directly from the evaluation procedure.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗