Search NASA⌕ Search

SEARCH · Search NASA

Results for “Usability evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Offshore Oil and Gas Infrastructure Reuse for CCS: Opportunities and Challenges in the U.S. Gulf of Mexico

The United States Gulf of Mexico has historically been a significant oil and gas producing region and continues to produce hydrocarbons today. As such, significant investments have been made to install oil and gas infrastructure such as pipelines, platforms, and wells. As these assets reach the end of their useful life or production associated with the assets is curtailed, the assets become a liability for both the owner/operators and the U.S. and state governments responsible for regulating the assets. There are costs and risks associated with safely decommissioning the assets, and in some cases, the assets may be abandoned (e.g., the owner is no longer a going concern). In parallel, carbon capture and storage (CCS) technology has developed in the United States and interest and incentives for CCS projects have accelerated in recent years, with many public announcements of CCS projects in development in the U.S. Gulf of Mexico region. The region is particularly well-suited to CCS due to the high density of industry on the coastline (i.e., CO 2 sources) and favorable geology in the near-offshore region. CCS also requires the use of many of the same assets required for oil and gas production (pipelines, platforms, wells) and the oil and gas industry has direct experience with CO 2 transport and injection (i.e., EOR). Therefore, the combination of existing infrastructure in close proximity to potential CCS project sites appears to provide a potential “win-win” solution for re-use of existing oil and gas assets. This paper presents a high-level evaluation of oil and gas infrastructure in the Gulf of Mexico with the goals of identifying the scale of the opportunity (e.g., how much existing infrastructure is realistically re-usable), identifying gaps in knowledge/policy/information required to evaluate existing infrastructure for re-use, and proposing key actions that could be pursued to further the assessment of existing infrastructure re-use for CCS.

Infrastructure, reuse, re-purpose, oil and gas, pi↗
DOE OSTI2022

Guidelines for Publicly Archiving Terrestrial Model Data to Enhance Usability, Intercomparison, and Synthesis

Scientific communities are increasingly publishing data to evaluate, accredit, and build on published research. However, guidelines for curating data for publication are sparse for model-related research, limiting the usability of archived simulation data. In particular, there are no established guidelines for archiving data related to terrestrial models that simulate land processes and their coupled interactions with climate. Terrestrial modelers have a unique set of challenges when publishing data due to the diversity of scientific domains, research questions, and the types and scales of simulations. Researchers in the U.S. Department of Energy’s (DOE) projects use a variety of multiscale models to advance robust predictions of terrestrial and subsurface ecosystem processes. Here, we synthesize archiving needs for data associated with different DOE models, and provide guidelines for publishing terrestrial model data components following FAIR (Findable, Accessible, Interoperable, Reusable) principles. The guidelines recommend archiving model inputs and testing data used in final simulation runs along with associated codes, workflow scripts, and metadata in public repositories. Researchers should consider archiving model outputs if they are within the storage limits of the repository. We also provide considerations for how to bundle files into different data publications with citable digital object identifiers. Finally, we identify repository features and tools that would enable storage and reuse of model data. Given the diversity of DOE terrestrial models, these guidelines are transferable to other model types and will enable efficient reuse of simulation data for purposes such as model intercomparisons, initialization, benchmarking, synthesis, and comparisons with field observations.

58 GEOSCIENCES↗
DOE OSTI2020

A User-Centered Design Study in Scientific Visualization Targeting Domain Experts

The development of usable visualization solutions is essential for ensuring both their adoption and effectiveness. User-centered design principles, which involve users throughout the entire development process, have been shown to be effective in numerous information visualization endeavors. We describe how we applied these principles in scientific visualization over a two year collaboration to develop a hybrid in situ/post hoc solution tailored towards combustion researcher needs. Lastly, we examine the importance of user-centered design and lessons learned over the design process in an effort to aid others seeking to develop effective scientific visualization solutions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Employing MACS/ViBRANT as a Surrogate MARVEL Reactor for Startup Reactivity Tuning and Supervisory Control Processes

Advanced nuclear reactors are a key part of the future of nuclear energy both in the United States and globally. They offer unique benefits for various energy-demanding applications, including use in remote locations, compact size, modular manufacturing, remote monitoring, low and/or variable power rating operation, and reliance on novel technologies to enhance operational safety. To achieve economic feasibility, advanced reactors must significantly reduce their workforces in comparison with the current fleet. Achieving this reduction will occur through reducing staff workloads using technology to achieve autonomous or semi-autonomous operations, demonstrated by comprehensive testing and validation activities. These operations will require both software and hardware platforms during the design and testing phases. While simulations are useful during the design phase, their performance can significantly deviate during actual deployment on hardware. This report presents the outcomes of a collaborative technical initiative between the U.S. Department of Energy (DOE) Microreactor Program (MRP) and Advanced Sensors and Instrumentation (ASI) Program. The collaboration utilized the Microreactor Automated Control System (MACS) hardware platform to bridge the gap between theoretical reactor design and actual startup and control operations. Two key use cases were investigated: facilitating the startup testing period and demonstrating supervisory control. The first use case details the key Microreactor Applications Research Validation and Evaluation (MARVEL) reactor startup physics testing activities conducted using the MACS platform. These activities included drum worth measurements, shutdown margin assessment, temperature feedback analysis, and scram time evaluation, as well as unique testing that would apply to the MARVEL reactor to demonstrate the testing methodologies in a low-risk environment. The MACS platform, serving as a surrogate representation of the MARVEL reactor, proved instrumental in performing these tests. The exercise revealed aspects that led to optimized processes, refined hardware design, and enhanced base software capabilities. By maturing methods and technologies in this manner, the initiative promises to reduce wasted time in the actual on-site reactor deployment effort, thereby saving significant time and resources. The second use case focuses on the development and implementation of supervisory control methods aimed at managing core tilt, which can result from asymmetrical operations or manufacturing imperfections in fuel rods or reactivity control devices. A key objective was to assess and compare the use of artificial intelligence (AI) for supervisory control. The effort aimed to define the role of supervisory control to enhance performance without risking control instability. This effort explored three distinct approaches: rules-based (RB) methods, optimization techniques, and reinforcement learning (RL) algorithms. Each approach was evaluated for its ease of implementation, its usability, and its effectiveness in responding to asymmetries in neutron flux. Comparative analysis of these approaches provided valuable insights into their applicability and effectiveness, offering a robust framework for advanced reactor operations. Together, these two use cases highlight the potential of hardware test beds to help streamline the design, operation, and control of advanced nuclear reactors. This collaborative effort underscores the importance of continued innovation and experimentation in achieving the next generation of safe, reliable, and economically viable nuclear energy solutions.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

A Brief Survey on High Performance Computing Systems Power Management

This paper provides a survey of software-based power management techniques in High Performance Computing (HPC) systems. Seven existing power management and monitoring tools and frameworks are discussed. These are: Variorum, dynamic energy-performance optimizer (DEPO), Powersched, Bull Dynamic Power Optimizer (BDPO), Energy Aware Runtime (EAR), Global Extensible Open Power Manager (GEOPM), and PoLiMEr. Each of these tools is evaluated based on hardware abstraction, optimization methods, usability, and experimental validation. This survey highlights the diversity of approaches in managing energy efficiency, from vendor-neutral APIs to algorithm-driven power capping, and dynamic frequency adjustments. Given that energy requirements for large computational systems is increasing quickly, the importance of integrating these tools into existing HPC environments and the need for further research in this rapidly evolving field is also discussed.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Dispatch Manager for NEML2 Constitutive Model Calculations Embedded in MOOSE

This report describes the extended capabilities of the NEML2 constitutive modeling library, including a flexible and efficient work dispatching system designed to leverage both CPU and GPU resources. This enhancement addresses one of the primary computational challenges in large-scale simulations: the ability to distribute and execute batches of material model evaluations across heterogeneous computing devices. The new dispatch system introduces a modular set of dispatcher and scheduler classes that coordinate the flow of data and execution between devices. The dispatcher is responsible for efficiently packaging work, managing device-specific memory operations, and synchronizing results. This modularity allows for extensibility, making it straightforward to integrate additional computing backends in the future. From an implementation standpoint, the dispatcher system interfaces seamlessly with NEML2's existing models. They handle device-aware tensor operations, optimize memory transfers, and support asynchronous execution when applicable. This design ensures that batches of material points can be evaluated concurrently, substantially improving throughput compared to previous single-device or serial implementations. These improvements not only enhance the raw performance of NEML2 but also improve its usability in multiscale and high-fidelity simulations, where the simultaneous evaluation of large material point batches is critical. Benchmarks included in the report demonstrate the system’s scalability, highlighting its effectiveness when leveraging modern GPU architectures.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

A Visual Designer of Layer‐wise Relevance Propagation Models

Abstract Layer‐wise Relevance Propagation (LRP) is an emerging and widely‐used method for interpreting the prediction results of convolutional neural networks (CNN). LRP developers often select and employ different relevance backpropagation rules and parameters, to compute relevance scores on input images. However, there exists no obvious solution to define a “best” LRP model. A satisfied model is highly reliant on pertinent images and designers' goals. We develop a visual model designer, named as VisLRPDesigner, to overcome the challenges in the design and use of LRP models. Various LRP rules are unified into an integrated framework with an intuitive workflow of parameter setup. VisLRPDesigner thus allows users to interactively configure and compare LRP models. It also facilitates relevance‐based visual analysis with two important functions: relevance‐based pixel flipping and neuron ablation. Several use cases illustrate the benefits of VisLRPDesigner. The usability and limitation of the visual designer is evaluated by LRP users.

Huang, Xinyi↗
DOE OSTI2021

A Visual Designer of Layer-wise Relevance Propagation Models

Layer-wise Relevance Propagation (LRP) is an emerging and widely-used method for interpreting the prediction results of convolutional neural networks (CNN). LRP developers often select and employ different relevance backpropagation rules and parameters, to compute relevance scores on input images. However, there exists no obvious solution to define a "best" LRP model. A satisfied model is highly reliant on pertinent images and designers' goals. We develop a visual model designer, named as VisLRPDesigner, to overcome the challenges in the design and use of LRP models. Various LRP rules are unified into an integrated framework with an intuitive workflow of parameter setup. VisLRPDesigner thus allows users to interactively configure and compare LRP models. It also facilitates relevance-based visual analysis with two important functions: relevance-based pixel flipping and neuron ablation. Several use cases illustrate the benefits of VisLRPDesigner. The usability and limitation of the visual designer is evaluated by LRP users.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Re-Evaluating Virtual Reality Manipulation Techniques for Precise Alignment of Complex 3D Objects

Prior research has developed a number of manipulation techniques that can achieve precise object placement in virtual reality, but studies of these techniques typically use simple objects. We conducted a study comparing two existing techniques, (AMP-IT and WISDOM), during alignment of objects with complex geometry to evaluate the potential influence of geometric complexity on performance, usability, workload and preference. Our findings indicate that participants had faster completion times and higher trial completion rates with AMP-IT on high-precision alignment tasks, contrary to earlier findings that used simple objects. Yet WISDOM is still preferred and considered more usable, despite increased workload and poorer performance, exposing participants' willingness to trade objective performance for comfort during use.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Automated Data Review of Analytical Laboratory Results at Los Alamos National Laboratory - 20299

Newport News Nuclear BWXT-Los Alamos, LLC (N3B) collects samples in support of the U.S. Department of Energy's (DOE) Office of Environmental Management (EM) Los Alamos Legacy Cleanup Contract (LLCC). N3B receives and reviews over 1.6 million sample data points annually in support of various ongoing environmental monitoring and remediation projects of the LLCC. N3B must demonstrate and document that reported external analytical laboratory data produced for the LLCC are of sufficient quality to fulfill their intended purpose and to support defensible decision making as described in EPA QA/G4 Guidance for the Data Quality Objectives Process 1994. In 2018, N3B assumed management of the LLCC along with the Environmental Information Management (EIM) database that contains all historical and current environmental data associated with the LLCC. The entire EIM database is shared between N3B, Triad National Security, LLC (Triad), and New Mexico Environment Department (NMED). These three parties jointly manage the database, its configuration, and changes / updates. All environmental data that are entered into EIM are updated and available, on a daily basis, in the linked public database Intellus New Mexico (Intellus). The quality and defensibility of the environmental data generated from sampling activities is a key component of an effective remediation process. Providing quality data is accomplished through a data assessment process that includes examination, verification, and validation. Examination is the assessment of completeness of the deliverables, identification of any reporting errors, and determining the usability of the data based on the laboratory's evaluation of its data as described in the case narrative received with the data. Verification consists of an evaluation of the Electronic Data Deliverables (EDD) data report to determine the extent to which the external analytical laboratories met method and contract-specific quality control and reporting requirements. Validation consists of determining the data quality and the extent to which the external analytical laboratories accurately and completely reported all sample and quality control results and satisfied all contract requirements. EIM contains an automatic Data Validation Module which performs automated data review (DVM ADR). DVM ADR is a tool to assist in the validation process. When DVM ADR is used in conjunction with manual examination of sample data packages, the combination of the two will meet and exceed the requirements of verification. N3B recognized an opportunity for process improvement, focusing on DVM ADR configuration and enhancements in EIM. Testing EIM's configuration provided proof of the DVM ADR's capabilities and flexibility to accurately perform routine data checks based on analytical methods and regulatory requirements. In addition, the DVM ADR module was improved through enhancements for all analytes, particularly upgrades for radiochemistry data. Extensive testing of the DVM ADR module occurred using EDDs from actual laboratory analyses on the EIM testing site. During this process, N3B manipulated EDD information to verify that the actual outcomes matched the expected outcomes. The results of this testing were shared with the database architects, and configuration improvements were identified to address these results. During this process, N3B identified that the radiochemical DVM ADR capabilities were underutilized, and so enhanced the DVM ADR functionality with respect to radioanalytical assessment. N3B environmental data uploads to Intellus on a daily basis from EIM, once the analytical data undergoes examination and verification. As such, it is important to have a high level of confidence in the quality and defensibility of the data. The process of manual examination, along with the DVM ADR, in conjunction with full validation of a percentage the data specified through the Data Quality Objectives greatly increases efficiency of data review and confidence level of the quality of the data, and gives the project managers, governmental offices, and the public expedited access to high-quality data. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2023

A Comparison of Three Types of Computer-Based Procedures: An Experiment Using the Rancor Microworld Simulator

The nuclear power industry has historically used paper-based procedures, but a shift towards computer-based procedures (CBPs) has the potential to reduce human errors, alleviate mental workload, and improve work performance. Twenty-seven participants were randomly assigned to one of three CBP types using the Rancor Microworld Simulator, and each performed two different types of operational scenarios (startup and loss of feedwater). The three CBP types varied in levels of digitalization. It was hypothesized that there would be less favorable impressions of the most basic procedures, and that these would demonstrate lower usability than types with greater digitalization. Overall, our predictions were partially supported with some interesting caveats, specifically with some performance benefits for CBPs that provided indicators but not embedded controls. We discuss our findings in terms of optimal levels of digitalization/automation within nuclear operations and suggest pathways for future directions.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Trusted Simulation: Considering Model Quality in the Context of User Trust

A high‐quality simulation model should help its users to easily and appropriately calibrate their trust in the model. Traditional evaluation metrics such as validation and robustness are necessary but insufficient for this task. Trust calibration depends on factors like the model's transparency, applicability to intended use, usability, reputation, and consideration of potential bias. This article proposes a framework for designing and evaluating system dynamics models by considering factors that contribute to the proper calibration of user trust. This framework takes inspiration from trusted artificial intelligence, broadening our traditional concept of model quality and explicitly focusing on what users need to consider a model trustworthy and to understand the model's relevance to its intended purpose. The trusted simulation framework can improve our integration of model quality activities throughout the modeling process, leading to more impactful and better‐targeted model design, development, and evaluation.

Naugle, Asmeret Bier [Sandia National Laboratories↗
DOE OSTI2020

Paving the Way for Net Zero Energy Codes through Performance Based Approaches

The prescriptive path is the most widely used approach for commercial code compliance in the United States. However, to achieve near-term net zero energy performance that many stakeholders in the buildings industry desire, it is clear that energy codes will need to transition from prescriptive to performance-based approaches. However, that transition is not without potential pitfalls. Among the concerns expressed by stakeholders are added complexity and cost, lack of confidence in energy modeling results, gamesmanship on the part of applicants, lack or qualified reviewers, challenges in promoting de-carbonization, and the inequity of trading long lived envelope efficiency for short lived measures such as building controls. This paper will discuss multiple approaches being evaluated and implemented by Pacific Northwest National Laboratory to improve the usability of the performance-based approach for both code compliance and beyond code programs to meet aggressive energy savings targets. A 'system performance' approach provides a simpler approach compared to whole building performance, while keeping tradeoffs limited to specific building systems. A simplified whole-building performance-based approach for small commercial buildings can cost effectively achieve deeper savings for buildings that typically follow the prescriptive compliance approach. Improved reporting and verification processes applied to the traditional performance path can provide greater confidence in simulation results and facilitate adoption of performance-based approaches. Prescriptive packages provide prescriptive solutions while targeting specific performance thresholds. This paper discusses the progress being made with each of these approaches and provides examples of their implementation in energy codes and beyond code programs.

Performance Based Codes, Appendix G, PRM, TSPR, Sy↗
DOE OSTI2020

Experiences with a Flexible User Research Process to Build Data Change Tools

Scientific software development processes are understood to be distinct from commercial software development practices due to uncertain and evolving states of scientific knowledge. Sustaining these software products is a recognized challenge, but under-examined is the usability and usefulness of such tools to their scientific end users. User research is a well-established set of techniques (e.g., interviews, mockups, usability tests) applied in commercial software projects to develop foundational, generative, and evaluative insights about products and the people who use them. Currently these approaches are not commonly applied and discussed in scientific software development work. The use of user research techniques in scientific environments can be challenging due to the nascent, fluid problem spaces of scientific work, varying scope of projects and their user communities, and funding/economic constraints on projects.In this paper, we reflect on our experiences undertaking a multi-method user research process in the Deduce project. The Deduce project is investigating data change to develop metrics, methods, and tools that will help scientists make decisions around data change. There is a lack of common terminology since the concept of systematically measuring and managing data change is under explored in scientific environments. To bridge this gap we conducted user research that focuses on user practices, needs, and motivations to help us design and develop metrics and tools for data change. This paper contributes reflections and the lessons we have learned from our experiences. We offer key takeaways for scientific software project teams to effectively and flexibly incorporate similar processes into their projects.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Centralized and Decentralized Distributed Energy Resource Access Control Implementation Considerations.

A global transition to power grids with high penetrations of renewable energy generation is being driven in part by rapid installations of distributed energy resources (DER). New DER equipment includes standardized IEEE 1547-2018 communication interfaces and proprietary communications capabilities. Interoperable DER provides new monitoring and control capabilities. The existence of multiple entities with different roles and responsibilities within the DER ecosystem makes the Access Control (AC) mechanism necessary. In this paper, we introduce and compare two novel architectures, which provide a Role-Based Access Control (RBAC) service to the DER ecosystem’s entities. Selecting an appropriate RBAC technology is important for the RBAC administrator and users who request DER access authorization. The first architecture is centralized, based on the OpenLDAP, an open source implementation of the Lightweight Directory Access Protocol (LDAP). The second approach is decentralized, based on a private Ethereum blockchain test network, where the RBAC model is stored and efficiently retrieved via the utilization of a single Smart Contract. We have implemented two end-to-end Proofs-of-Concept (PoC), respectively, to offer the RBAC service to the DER entities as web applications. Finally, an evaluation of the two approaches is presented, highlighting the key speed, cost, usability, and security features.

42 ENGINEERING↗
DOE OSTI2022

A Methodology for Evaluating Operator Usage of Machine Learning Recommendations for Power Grid Contingency Analysis

This work presents the application of a methodology to measure domain expert trust and workload, elicit feedback, and understand the technological usability and impact when a machine learning assistant is introduced into contingency analysis for real-time power grid simulation. The goal of this framework is to rapidly collect and analyze a broad variety of human factors data in order to accelerate the development and evaluation loop for deploying machine learning applications. We describe our methodology and analysis, and we discuss insights gained from a pilot participant about the current usability state of an early technology readiness level (TRL) artificial neural network (ANN) recommender.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2024

Homomorphic Encryption for Electrical Metering Aggregation: Protecting the Privacy of Building Tenants

Electrical meters are devices that measure consumer electricity usage. The data collected by these meters is necessary for utility billing and electrical grid management but can also be used to assess the environmental impact of buildings. Prior research has found that unprotected metering data could potentially be used to infer some information about the behaviors of building tenants by detecting changes in electricity usage. For example, a period of low electricity usage could suggest that the tenants are not in the building. As smart metering becomes more common, there is a growing need for data privacy protections for metering data that do not negatively impact the quality and availability of data used for energy management and billing applications. To identify potential solutions, we developed a Python-based data aggregation platform to analyze the potential efficacy of privacy-enhancing technologies for energy metering applications. This platform aggregates groups of metering sites into virtual buildings, which could potentially detach changes in electrical activity from individual tenants, making it more difficult to track the activity of a specific tenant. To further protect data during analysis, this project utilizes homomorphic encryption as part of its initial approach. Homomorphic encryption offers a means of protecting energy consumption data while permitting mathematical operations to be performed without the need to know the data contents. This allows for data to be processed into usable statistics without revealing energy consumption information. A series of homomorphic encryption libraries were evaluated to determine their applicability and limitations in the context of metering data. The use of these techniques may help to reassure consumers and encourage further adoption of smart grid infrastructure.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

Reassessment of PHDS Fulcrum40h High Purity Germanium (HPGe) Detector System Performance

This report presents a comprehensive evaluation of the updated PHDS Fulcrum40h High Purity Germanium (HPGe) detector system, benchmarking its performance, usability, and software capabilities against the current National Nuclear Security Administration (NNSA) Nuclear Emergency Support Team (NEST) HPGe Detector System Requirements Document. Systematic measurements were conducted using a single Fulcrum40h detector to assess key parameters including gamma efficiency and resolution, neutron detection efficiency, gamma pulse-pileup response, and gamma-to-neutron crosstalk. The Fulcrum40h system, acquired in August 2022, has undergone recent updates by PHDS to address deficiencies identified following the initial NEST requirements release. The results provide critical insights into the detector’s operational capabilities and compliance with NNSA NEST standards.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗