Search NASA⌕ Search

SEARCH · Search NASA

Results for “trust”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Research Needs for Trusted Analytics in National Security Settings

As artificial intelligence, machine learning, and statistical modeling methods become commonplace in national security applications, the drive to create trusted analytics becomes increasingly important. The goal of this report is to identify areas of research that can provide the foundational understanding and technical prerequisites for the development and deployment of trusted analytics in national security settings. Our review of the literature covered several disjoint research communities, including computer science, statistics, human factors, and several branches of psychology and cognitive science, which tend not to interact with one another or cite each other's literatures. As a result, there exists no agreed-upon theoretical framework for understanding how various factors influence trust and no well-established empirical paradigm for studying these effects. This report therefore takes three steps. First, we define several key terms in an effort to provide a unifying language for trusted analytics and to manage the scope of the problem. Second, we outline an empirical perspective that identifies key independent, moderating, and dependent variables in assessing trusted analytics. Though not a substitute for a theoretical framework, the empirical perspective does support research and development of trusted analytics in the national security domain. Finally, we discuss several research gaps relevant to developing trusted analytics for the national security mission space.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2025

Zero Trust Strategies for Chemical, Biological, Radiological, and Nuclear Detection Systems: D.1 Cyber Scenarios

The evolving landscape of cybersecurity necessitates a paradigm shift to a Zero Trust (ZT) model, which assumes breaches and continuously verifies trust. This approach reshapes how trust boundaries are established, focusing on identities, devices, networks, applications, and data, rather than solely relying on perimeter defenses such as firewalls. Central to this transformation is the National Institute of Standards and Technology's (NIST) Special Publication 800-207, outlining the Zero Trust Architecture (ZTA), along with Executive Order 14028, which mandates federal agencies to adopt ZT principles. Complementary to these efforts, the Cybersecurity and Infrastructure Security Agency (CISA) developed the Zero Trust Maturity Model (ZTMM), providing a framework with five pillars and three cross-cutting capabilities to guide agencies toward enhanced cybersecurity maturity. In support of these initiatives, the DHS Countering Weapons of Mass Destruction Office (CWMD) is applying ZT principles to secure Chemical, Biological, Radiological, and Nuclear (CBRN) detection systems. Recognizing the diverse deployment models and network connectivity of these systems—from stationary, non-networked units to mobile, cloud-connected devices—the Pacific Northwest National Laboratory (PNNL) is developing cybersecurity scenarios specifically for CBRN environments. These scenarios examine various configurations and technological capabilities, offering insights into the application of ZTMM pillars in enhancing the security postures of CBRN devices. The cybersecurity scenarios presented by PNNL are hypothetical, crafted to explore theoretical situations and stimulate discussion on the potential use or compromise of CBRN detection systems in varied contexts. These narratives are illustrative and do not reference any real events or actual networks. Instead, they employ generalized reference models to highlight concepts and potential issues within CBRN security, focusing on how Zero Trust strategies can be adapted to address these challenges effectively.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
NASA NTRS2020

Designing and Training for Appropriate Trust in Increasingly Autonomous Advanced Air Mobility Operations: A Mental Model Approach: Version 1

To enable effective human-autonomy teaming (HAT) in Advanced Air Mobility (AAM) operations, the current paper presents a theoretical framework to design and train for appropriate trust in automation. The novel contribution of this work resides in connecting the construct of trust to mental models and showing how this method could be used to enable emerging HAT concepts such as Adaptive Trust Calibration. To contextualize this framework, in section 2 we discuss simplified vehicle operations (SVO) and remote vehicle operations (RVO), which are leading operational concepts within AAM. In section 3 we describe our perspective on automation and increasingly autonomous systems and present a brief discussion on human-automation interaction and human-autonomy teaming. In section 4 we provide a detailed discussion on the construct of trust in automation. In section 5 we present a framework that associates mental models with trust through principles of transparent design. Finally, in section 6 we present three descriptive models for designing and training for appropriate trust in increasingly autonomous systems.

Human-Autonomy Teaming↗
DOE OSTI2025

Trustworthiness and Trust: Identifying Factors that Drive Successful Human-AI Interaction in Nuclear Power Plant Applications

Emerging technologies such as artificial intelligence (AI) and machine learning (ML) are rapidly evolving and considered a promising tool for efficient and continued safe operations of the U.S. nuclear power plants (NPPs). Emerging AI techniques like large language models (LLMs) are one such technology that may support personnel at existing NPPs perform work more efficiently. For example, operators may query the current operational status of a power plant via a chat interface leveraging LLMs to access plant-related information in an interactive manner rather than manually collecting various sensor data for tasks such as surveillances or completing work orders. This is a fundamental shift in the way operators currently perform their tasks today. The literature of human-automation interaction indicates that trust is a crucial factor that drives successful interaction between a human operator and an automated system, like an AI-infused NPP application. This work presents the results of a literature review on key factors that relate to trust in AI/LLM technologies for NPP applications. The relevant literature of human factors and cognitive engineering has identified various factors related to trust including trustworthiness, performance characteristics, operator skill and perceived risk. This preliminary literature review will guide development and evaluation of models involving the identified factors influencing trust in AI and develop a framework for human-centered design for interface between humans and AI. By addressing trust, this work supports developing a technical basis for designing key characteristics of AI/LLM to support calibrated trust, which will ultimately support wide-scale adoption of AI/LLM technologies, as well as ensure safe, effective, and reliable use.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2021

Trust-but-Verify in Cyber-Physical Systems

Cyber-physical systems span a wide spectrum, from long-lived legacy systems to more modern installations. Trust is an issue that arises across the spectrum, albeit with different variants of goals and constraints. On the one end of the spectrum, legacy systems are characterized by function-based designs in which trust is an implicitly in-built concept -- the operation is historically designed, implemented, and optimized in a benign stance with respect to intended use. On the other end of the spectrum, modern systems are characterized by offerings from manufacturers, vendors, and system installers -- the devices and deployments use a variety of security features that offer promises of increased trust. All along this spectrum of cyber-physical systems, extending trust beyond the traditional cyber portions to the arteries that connect the physical portions to the cyber portions is a major challenge. Here, we identify a Trust-but-Verify approach that spans this spectrum in addressing trust.

Perumalla, Kalyan↗
DOE OSTI2024

TRUST Sensors in Environments: Fiber Optic Displacement (SE-FOD) Report, Release FY24

The objective of the Delivery Environments (DE) Testbeds to Reduce Uncertainty in Simulations and Tests (TRUST) project is to quantify and help increase confidence in specific areas of computational and experimental capabilities that are applicable to the development, on-target assessment, and qualification of current and future delivery environments. More complete quantification of confidence in experimental and computational capabilities and the sufficient increase of confidence in those capabilities is critical to improving weapons engineering design, qualification, and assessment efforts that are critical to the current and future stockpile. This work uses and provides feedback on analysis tools and experimental results databases for efficient and responsive engineering which are currently under development. Under the TRUST work package, there are five testbeds and their associated engineering analysis baseline models (EABMs): 1. Contact Thermal Conductivity (CTC), 2. Nonlinear Dynamics (ND), 3. Sensors in Environments: Accelerometers (SE-A), 4. Sensors in Environments: Fiber Optic Displacement Gauges (SE-FOD), and 5. Sensors in Environments: Thermocouples (SE-TC). The TRUST project uses single-feature testbeds to quantify uncertainties in specific models and experiments and to identify capability development needs that can help to reduce these uncertainties. Each testbed is designed, configured, and tested in collaboration with groups with design and experimental capability: E-14 and MPA-CINT. The complementary simulations are conducted using W-13 analysis tools and stored in model repositories with plans for incremental progress toward EABM requirements. W-13 extends and exercises the testbed simulations in collaboration with experimentalists for uncertainty quantification of current and future materials, geometries, and environments. Additionally, TRUST is intended to provide engineers in W-13 and E-14 with experience in both numerical simulations and experimental methods through cross-discipline collaborations. Following the introduction to the TRUST project, the remainder of this report focuses on experimental, modeling and simulation, and analysis efforts conducted in FY24 relevant to the TRUST Sensors in Environments: Fiber Optic Displacement (SE-FOD) testbed.

42 ENGINEERING↗
NASA NTRS2018

Serious Gaming for Building a Basis of Certification via Trust and Trustworthiness of Autonomous Systems

Autonomous systems governed by a variety of adaptive and nondeterministic algorithms are being planned for inclusion into safety-critical environments, such as unmanned aircraft and space systems in both civilian and military applications. However, until autonomous systems are proven and perceived to be capable and resilient in the face of unanticipated conditions, humans will be reluctant or unable to delegate authority, remaining in control aided by machine-based information and decision support. Proving capability, or trustworthiness, is a necessary component of certification. Perceived capability is a component of trust. Trustworthiness is an attribute of a cyber-physical system that requires context-driven metrics to prove and certify. Trust is an attribute of the agents participating in the system and is gained over time and multiple interactions through trustworthy behavior and transparency. Historically, artificial intelligence and machine learning systems provide answers without explanation - without a rationale or insight into the machine “thinking”. In order to function as trusted teammates, machines must be able to explain their decisions and actions. This transparency is a product of both content and communication. NASA’s Autonomy Teaming & TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR) project seeks to build a basis for certification of autonomous systems via establishing metrics for trustworthiness and trust in multi-agent team interactions, using AI (Artificial Intelligence) explainability and persistent modeling and simulation, in the context of mission planning and execution, with analyzable trajectories. Inspired by Massively Multiplayer Online Role Playing Games (MMORPG) and Serious Gaming, the proposed ATTRACTOR modeling and simulation environment is similar to online gaming environments in which player (aka agent) participants interact with each other, affect their environment, and expect the simulation to persist and change regardless of any individual agent’s active participation. This persistent simulation environment will accommodate individual agents, groups of self-organizing agents, and large-scale infrastructure behavior. The effects of the emerging adaptation and coevolution can be observed and measured to building a basis of measurable trustworthiness and trust, toward certification of safety-critical autonomous systems.

Allen, B. Danette↗
NASA NTRS2022

Adapting a Trusted AI Framework to Space Mission Autonomy

As artificial intelligence (AI) is increasingly pro- posed for new and future capabilities in space missions, the question of how to trust AI-enabled space autonomy has been explored. Recently, a collaboration between The Aerospace Corporation (Aerospace) and NASA’s Jet Propulsion Labora- tory (JPL) investigated how Aerospace’s Trusted AI Frame- work could be applied to two JPL projects that planned on lev- eraging AI for critical autonomous tasks. This combined effort led to many insights in the practical implementation of trusted AI along with considerable updates to the Trusted AI Frame- work that tailored its topic threads to space exploration. This document cohesively summarizes the enhanced framework as tailored to space missions as well as estimation of the level of trust required as a function of mission criticality and key stakeholders. The goal of this work is to provide a set of best practices to inform autonomy researchers, flight engineers, mission and proposal reviewers, and instrument and mission principal investigators (PI’s) to drive AI-based autonomy that maximizes trust and lowers the barriers to mission adoption for both science and engineering applications.

Amini, Rashied↗
DOE OSTI2022

Newton trust-region methods with primary variable switching for simulating high temperature multiphase porous media flow

Coupling multiphase flow with energy transport due to high temperature heat sources introduces significant new challenges since boiling and condensation processes can lead to dry-out conditions with subsequent re-wetting. The transition between two-phase and single-phase behavior can require changes to the primary dependent variables adding discontinuities as well as extending constitutive nonlinear relations to extreme physical conditions. Practical simulations of large-scale engineered domains lead to Jacobian systems with a very large number of unknowns that must be solved efficiently using iterative methods in parallel on high-performance computers. Performance assessment of potential nuclear repositories, carbon sequestration sites and geothermal reservoirs can require numerous Monte-Carlo simulations to explore uncertainty in material properties, boundary conditions, and failure scenarios. Due to the numerical challenges, standard NR iteration may not converge over the range of required simulations and require more sophisticated optimization method like trust-region. In this study, we use the open-source simulator PFLOTRAN for the important practical problem of the safety assessment of future nuclear waste repositories in the U.S. DOE geologic disposal safety assessment Framework. The simulator applies the PETSc parallel framework and a backward Euler, finite volume discretization. We demonstrate failure of the conventional NR method and the success of trust-region modifications to Newton’s method for a series of test problems of increasing complexity. Trust-region methods essentially modify the Newton step size and direction under some circumstances where the standard NR iteration can cause the solution to diverge or oscillate. Furthermore, we show how the Newton Trust-Region method can be adapted for Primary Variable Switching (PVS) when the multiphase state changes due to boiling or condensation. The simulations with high-temperature heat sources which led to extreme nonlinear processes with many state changes in the domain did not converge with NR, but they do complete successfully with the trust-region methods modified for PVS. This implementation effectively decreased weeks of simulation time needing manual adjustments to complete a simulation down to a day. Finally, we show the strong scalability of the methods on a single node and multiple nodes in an HPC cluster.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2016

Effects of Transparency on Pilot Trust and Agreement in the Autonomous Constrained Flight Planner

We performed a human-in-the-loop study to explore the role of transparency in engendering trust and reliance within highly automated systems. Specifically, we examined how transparency impacts trust in and reliance upon the Autonomous Constrained Flight Planner (ACFP), a critical automated system being developed as part of NASA's Reduced Crew Operations (RCO) Concept. The ACFP is designed to provide an enhanced ground operator, termed a super dispatcher, with recommended diversions for aircraft when their primary destinations are unavailable. In the current study, 12 commercial transport rated pilots who played the role of super dispatchers were given six time-pressured all land scenarios where they needed to use the ACFP to determine diversions for multiple aircraft. Two factors were manipulated. The primary factor was level of transparency. In low transparency scenarios the pilots were given a recommended airport and runway, plus basic information about the weather conditions, the aircraft types, and the airport and runway characteristics at that and other airports. In moderate transparency scenarios the pilots were also given a risk evaluation for the recommended airport, and for the other airports if they requested it. In the high transparency scenario additional information including the reasoning for the risk evaluations was made available to the pilots. The secondary factor was level of risk, either high or low. For high-risk aircraft, all potential diversions were rated as highly risky, with the ACFP giving the best option for a bad situation. For low-risk aircraft the ACFP found only low-risk options for the pilot. Both subjective and objective measures were collected, including rated trust, whether the pilots checked the validity of the automation recommendation, and whether the pilots eventually flew to the recommended diversion airport. Key results show that: 1) Pilots trust increased with higher levels of transparency, 2) Pilots were more likely to verify ACFPs recommendations with low levels of transparency and when risk was high, 3) Pilots were more likely to explore other options from the ACFP in low transparency conditions and when risk was high, and 4) Pilots decision to accept or reject ACFPs recommendations increased as a function of the transparency in the explanation. The finding that higher levels of transparency was coupled with higher levels of trust, a lower need to verify other options, and higher levels of agreement with ACFP recommendations, confirms the importance of transparency in aiding reliance on automated recommendations. Additional analyses of qualitative data gathered from subjects through surveys and during debriefing interviews also provided the basis for new design recommendations for the ACFP.

trust↗
NASA NTRSDate not supplied

The Role of Trust and Usability in Enabling Spaceflight Crew Autonomy

Future long duration exploration missions will require an increased use of onboard automated systems as spaceflight crews venture further than before and have longer communications delays with ground support. The design of these systems must support appropriate crew trust and have sufficient usability to enable spaceflight crew autonomy or risk being misused while crews wait to communicate with ground support. We evaluated trust & usability in our self-scheduling tool, Playbook, for crew mission timelines. Data was collected in a controlled lab experiment with 31 participants. Participants in the study conducted two tasks: scheduling, where participants were responsible for scheduling a majority of a day's operational tasks, and rescheduling, where participants were provided a schedule and asked to reschedule higher priority activities. We found a significant correlation between system trust and usability, irrespective of self-scheduling tasks. We conclude that system usability may play a bigger role in how trust is learned while conducting novel crew autonomy tasks such as self-scheduling. Future research should investigate the role of usability to encourage appropriate trust in onboard automated crew systems and enable crew autonomy.

crew autonomy↗
DOE OSTI2020

How Do Visual Explanations Foster End Users' Appropriate Trust in Machine Learning?

We investigated the effects of different visual explanations on users' trust in machine learning classification. We proposed three forms of visual explanations of a classification based on identifying relevant training instances. We conducted a user study to evaluate these visual explanations as well as a no explanation condition. We measured users' trust of a classifier, quantified the effects of these three forms of explanations, and assessed the changes in users' trust. We found that participants trust a classifier appropriately when an explanation is available. The combination of human, classification algorithm and understandable explanation makes better decisions than the classifier and human alone.This work advances the state-of-the-art closer to building trust-able machine learning models and informs the design and appropriate use of automated systems.

Yang, Fumeng↗
NASA NTRS2010

Effects of Secondary Task Modality and Processing Code on Automation Trust and Utilization During Simulated Airline Luggage Screening

The purpose of this research was to examine the impact of environmental distractions on human trust and utilization of automation during the process of visual search. Participants performed a computer-simulated airline luggage screening task with the assistance of a 70% reliable automated decision aid (called DETECTOR) both with and without environmental distractions. The distraction was implemented as a secondary task in either a competing modality (visual) or non-competing modality (auditory). The secondary task processing code either competed with the luggage screening task (spatial code) or with the automation's textual directives (verbal code). We measured participants' system trust, perceived reliability of the system (when a target weapon was present and absent), compliance, reliance, and confidence when agreeing and disagreeing with the system under both distracted and undistracted conditions. Results revealed that system trust was lower in the visual-spatial and auditory-verbal conditions than in the visual-verbal and auditory-spatial conditions. Perceived reliability of the system (when the target was present) was significantly higher when the secondary task was visual rather than auditory. Compliance with the aid increased in all conditions except for the auditory-verbal condition, where it decreased. Similar to the pattern for trust, reliance on the automation was lower in the visual-spatial and auditory-verbal conditions than in the visual-verbal and auditory-spatial conditions. Confidence when agreeing with the system decreased with the addition of any kind of distraction; however, confidence when disagreeing increased with the addition of an auditory secondary task but decreased with the addition of a visual task. A model was developed to represent the research findings and demonstrate the relationship between secondary task modality, processing code, and automation use. Results suggest that the nature of environmental distractions influence interaction with automation via significant effects on trust and system utilization. These findings have implications for both automation design and operator training.

Phillips, Rachel↗
NASA NTRS2021

Accessible Telemetry Streams using a Zero Trust Architecture for the Flight Operations Directorate

As a result of information technology based work becoming increasingly distributed, unique challenges have been presented within the realm of defined network perimeters, namely with respect to secure access to resources. Historically, and from a simplistic abstract perspective, the common approach has been to adopt the, so-called, moat model whereby a physical network perimeter (or interconnected perimeters) is defined to encapsulate resources behind a boundary protected by a firewall. Users are provisioned access through a virtual private network (VPN) and may be further constrained to resources through specific firewall allow and disallow rulesets. Virtual Private Networks and firewall rulesets lead to common problems, particularly at scale and, as a result, perimeter-less architectures provided over the public internet are increasingly becoming prevalent, particularly with its more popular implementation, the Zero Trust Architecture. We present a proposed implementation of the Zero Trust Architecture with a particular concrete example utilizing a de-perimeterized network that requires authentication and authorization for each action between nodes and does not operate within an implicit trust boundary. It should be noted that this paper is not an attempt at providing comprehensive resolutions for the specific problem space with respect to perimeter based security and is more directed at providing information with regard to our proposed implementation of a Zero Trust Architecture for the Flight Operations Directorate. We direct the reader to our Introduction and Background section for more details on specific documentation and where it can be located as it relates to de-perimeterization and Zero Trust.

Paul Shoemaker↗
NASA NTRS2023

Space Trusted Autonomy Readiness Levels

Technology Readiness Levels are a mainstay for organizations that fund, develop, test, acquire, or use technologies. Technology Readiness Levels provide a standardized assessment of a technology’s maturity and enable consistent comparison among technologies. They inform decisions throughout a technology’s development life cycle, from concept, through development, to use. A variety of alternative Readiness Levels have been developed, including Algorithm Readiness Levels, manufacturing Readiness Levels, Human Readiness Levels, Commercialization Readiness Levels, Machine Learning Readiness Levels, and Technology Commitment Levels. However, while Technology Readiness Levels have been increasingly applied to emerging disciplines, there are unique challenges to assessing the rapidly developing capabilities of autonomy. This paper adopts the moniker of Space Trusted Autonomy Readiness Levels to identify a two-dimensional scale of readiness and trust appropriate for the special challenges of assessing autonomy technologies that seek space use. It draws inspiration from other readiness levels’ definitions, and from the rich field of trust and trustworthiness. The Space Trusted Autonomy Readiness Levels were developed by a collaborative Space Trusted Autonomy subgroup, which was created from The Space Science and Technology Partnership Forum between the United States Space Force, the National Aeronautics and Space Administration, and the National Reconnaissance Office.

Kerianne L Hobbs↗
NASA NTRSDate not supplied

What is the Role of Usability and Trust in Autonomy?

Usability encompasses learnability, efficiency, memorability, effectiveness, and satisfaction. NASA’s standards for usability acceptance criteria focus on interfaces that help operators achieve their tasks efficiently, effectively, and with satisfaction. However, discussions on usability, especially regarding future highly automated and autonomous systems, rarely include trust. As NASA plans for long-duration exploration missions, it envisions astronauts operating more independently from Mission Control on Earth. This independence will drive the development of these highly automated and autonomous systems that astronauts will use daily. To prepare for this future, our team has developed a scheduling and execution software tool that facilitates self-scheduling, allowing astronauts to independently manage their own schedule without Mission Control’s involvement. Over many years, we have developed, matured, and evaluated our software tool in extreme environments, prioritizing user-centered design and high usability. These evaluations have included multiple campaigns in NASA analogs, including NEEMO, BASALT, and HERA, as well as technology demonstrations onboard the International Space Station. Our recent research on software interfaces for future astronaut autonomy revealed a strong correlation between usability and trust measures. In a controlled lab experiment, we asked novice users to perform a complex scheduling task, during which the software immediately validated the schedule’s constraints and checked for violations. We collected usability (User Experience Questionnaire, UEQ) and trust (Trust in Automated Systems scale, TAS) measures; significant, strong, and moderate correlations emerged between several of the UEQ metrics and TAS. These results support the argument for investing in usability early to enable and sustain trust in highly automated and autonomous systems.

usability↗
DOE OSTI2021

5G Zero Trust Architecture: A Testable and Phased Approach

With the emphasis being placed on Zero Trust Architecture in technology and by Executive Order (Executive Order on Improving the Nation’s Cybersecurity ), an investigation was performed as to the feasibility of implementing this approach into 5G. Reference documents include NIST Special Publication 800-207 and Department of Defense (DoD) Zero Trust Reference Architecture , and these provided guidance as to the overall approach. Progress is being made within 5G to address the lack of Zero Trust Architecture; a June 2021 DoD press release indicated that “The prototype 5G network is built on the next generation of Open Radio Network standards and designed to comply with DOD specifications for zero-trust architecture for native security and secure connectivity with other networks.”

5G↗
DOE OSTI2021

Adaptive Cognitive Mechanisms to Maintain Calibrated Trust and Reliance in Automation

Trust calibration for a human–machine team is the process by which a human adjusts their expectations of the automation’s reliability and trustworthiness; adaptive support for trust calibration is needed to engender appropriate reliance on automation. Herein, we leverage an instance-based learning ACT-R cognitive model of decisions to obtain and rely on an automated assistant for visual search in a UAV interface. This cognitive model matches well with the human predictive power statistics measuring reliance decisions; we obtain from the model an internal estimate of automation reliability that mirrors human subjective ratings. The model is able to predict the effect of various potential disruptions, such as environmental changes or particular classes of adversarial intrusions on human trust in automation. Finally, we consider the use of model predictions to improve automation transparency that account for human cognitive biases in order to optimize the bidirectional interaction between human and machine through supporting trust calibration. The implications of our findings for the design of reliable and trustworthy automation are discussed.

60 APPLIED LIFE SCIENCES↗