Search NASA⌕ Search

Engineering topics

Feather, Martin

Publications and source records attributed to Feather, Martin.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Hallmarks of an Autonomous Space System’s Development and V&V

NASA’s deep space missions are dependent on autonomy when conditions require a faster response than can be directed by communication to and from Earth. Verification and Validation (V&V) of the autonomy is an essential step for missions to be confident in its use. This paper provides an overview of the development and V&V of one such autonomous system, DIMES (Descent Image Motion Estimation System), used successfully to reduce a critical mission risk during the landings of NASA’s two Mars Exploration Rovers on Mars in 2004.

Feather, Martin↗
NASA NTRS2022

Space Applications of a Trusted AI Framework: Experiences and Lessons Learned

Artificial intelligence (AI), which encompasses machine learning (ML), has become a critical technology due to its well-established success in a wide array of applications. However, the proper application of AI remains a central topic of discussion in many safety-critical fields. This has limited its success in autonomous systems due to the difficulty of ensuring AI algorithms will perform as desired and that users will understand and trust how they operate. In response, there is growing demand for trustability in AI to address both the expectations and concerns regarding its use. The Aerospace Corporation (Aerospace) developed a Framework for Trusted AI (henceforth referred to as the framework) to encourage best practices for the implementation, assessment, and control of AI-based applications. It is generally applicable, being based on terms and definitions that cut across AI domains, and thus is a starting point for practitioners to tailor to their particular application. To help demonstrate how the framework can be tailored into mission assurance guidance for the space domain, Aerospace sought the involvement of the Jet Propulsion Laboratory (JPL) to engage with actual examples of AI-based space autonomy.

Kaufman, James↗
NASA NTRS2022

Adapting a Trusted AI Framework to Space Mission Autonomy

As artificial intelligence (AI) is increasingly pro- posed for new and future capabilities in space missions, the question of how to trust AI-enabled space autonomy has been explored. Recently, a collaboration between The Aerospace Corporation (Aerospace) and NASA’s Jet Propulsion Labora- tory (JPL) investigated how Aerospace’s Trusted AI Frame- work could be applied to two JPL projects that planned on lev- eraging AI for critical autonomous tasks. This combined effort led to many insights in the practical implementation of trusted AI along with considerable updates to the Trusted AI Frame- work that tailored its topic threads to space exploration. This document cohesively summarizes the enhanced framework as tailored to space missions as well as estimation of the level of trust required as a function of mission criticality and key stakeholders. The goal of this work is to provide a set of best practices to inform autonomy researchers, flight engineers, mission and proposal reviewers, and instrument and mission principal investigators (PI’s) to drive AI-based autonomy that maximizes trust and lowers the barriers to mission adoption for both science and engineering applications.

Amini, Rashied↗
NASA NTRS2022

Assurance Equations: A Cost and Criticality Model for Optimizing Quality Assurance Surveillance

The cost of quality vs cost of failure correction has been a long-running topic of discussion within the Aerospace community. It leads directly to concepts of “risk tolerance”, and risk-based decision-making. It would be valuable if there was a way to compute the optimal investment in customer-executed quality assurance activities using defect significance with respect to performance objectives, the activities’ defect detection effectiveness, and the cost-penalty for late discovery of impactful defects. This optimization is particularly of interest to projects whose budget constraints significantly limit their risk management options.The cost to fix defects (i.e., failure correction) escalates as the project matures. There have been studies attempting to determine the relative cost of fixing defects discovered during various phases of a project life cycle with important implications, all of which suggest growth factors are large. The commonly referred to 1:10:100 rule represents a cost multiplier for repair/rework across the Design to Fab to Test hardware development phases. Cost premiums for QA activities also accumulate when they are treated as mandatory (due to schedule drag) or are performed later than their assigned phase.This paper describes the modeling of development phase -dependencies in the conduct of typical customer-executed quality assurance activities. Our initial modeling encompasses:• Distinct phases of the production lifecycle• Multiple kinds of Defects, each with some a-priori likelihood of being present• Each defect’s impact on performance Objectives for a type of hardware• The cost and efficacy of assurance techniques at detecting such Defects• The costs of fixing those Defects detected in a given phase of the production lifecycleThe model captures assurance activities’ abilities to Detect defects. Upon detection it is assumed that the Defect is immediately fixed. Defects that “escape” detection by some activity may thereafter be detected by a later activity, but by then the cost of fixing the Defect may have escalated. Defects are related to the performance Objectives they would detract from, were those Defects to remain present in the operating system.We have constructed and are exploring, a model that relates the importance of hardware system elements to mission objectives, the impact of types of Defects on those hardware types, the cost of customer-executed assurance activities (i.e., supplier controls) and their effectiveness towards reducing an impactful quality escape, and the cost of Defect correction across production phase. We describe the approach taken to select the key model aspects, why they are relevant to our NASA mission, and our efforts to populate it with relevant and contemporary data. We use a notional example to illustrate model design and function.

Plante, Jeannette↗
NASA NTRS2021

Results from the ASTERIA CubeSat Extended Mission Experiments

Over the past two years, JPL has used the ASTERIA (Arcsecond Space Telescope Enabling Research In Astrophysics) CubeSat as an in-flight test platform during extended missions. ASTERIA successfully completed its prime mission in early 2018, and continued to operate in low Earth orbit (LEO) for an additional twenty months. This paper describes demonstrations that were performed on the spacecraft and on the ground-based testbed during the extended mission. These demonstrations fall into three categories: Autonomy technology maturation, hardware characterization, and science discovery. Autonomy technology maturation supported three development efforts. The first shifted the spacecraft commanding paradigm from time-based sequences to Task Networks (tasknets), which allow simpler commanding and more robust onboard execution. The second demonstrated onboard orbit determination in Low Earth Orbit (LEO) without GPS. This activity used a fully-independent means of spacecraft orbit determination for Earth orbiters using only passive imaging. The third technology provided in situ hardware health state estimation using a model-based reasoning technique. These three technologies were demonstrated either in flight or on the testbed individually, and then were combined to demonstrate the capability to perform autonomous navigation on board without ground intervention, even in the presence of anomalies. Hardware characterization involved both onboard and ground-based activities. On board, nonstandard attitude control modes were commanded to characterize the spacecraft pointing jitter as a function of target brightness, reaction wheel speed, controller gain, and the number of guide stars. The results provide insights into the contribution of jitter to the ASTERIA photometry and inform the feasibility of future astrophysics small satellite missions for which jitter control is an enabling technology. On the ground, the ASTERIA Operations Team coordinated with Amazon Web Services (AWS) to configure their new ground stations to communicate with ASTERIA to prove out their viability. ASTERIA used AWS ground stations for nominal operations for the last four months of the mission. Finally, ASTERIA continued to perform exoplanet science as the spacecraft was well-suited to execute long-term monitoring of stars such as alpha Centauri to search for small transiting planets. The science team also imaged a number of interesting objects including a comet, an asteroid, cities at night, and the moon, and coordinated with other projects on Targets of Opportunity for follow-up confirmations and co-observations. Throughout the prime and the extended missions, the ASTERIA spacecraft proved to be a mighty platform that “will go into history as an innovative milestone.”[1 - Zurbuchen]

Doran, Patrick↗
NASA NTRS2020

MEXEC: An Onboard Integrated Planning and Execution Approach for Spacecraft Commanding

The traditional form of spacecraft commanding is with sequences that specify when commands should execute based on a schedule generated on the ground. Some sequences have control logic and event driven responses to increase flexibility, but it is limited. An approach to increase autonomy is to use goal-based planning and commanding. Using this paradigm, intention and behavior is modeled on board the spacecraft. In this paper we describe MEXEC (Multi-mission EXECutive), a multi-mission, task-based, onboard planning and execution software designed specifically to be used as flight software. As a path to infusion for future flight projects, we describe two experiments performed on the ASTERIA CubeSat and testbed that demonstrate that MEXEC can be integrated and used for spacecraft operations and increase robustness and science return compared to the standard sequences that were being used.

Campuzano, Brian↗
NASA NTRS2020

Enabling Assurance in the MBSE Environment

A number of specific benefits that fit within the hallmarks of effective development are realized with implementation of model-based approaches to systems and assurance. Model Based Systems Engineering (MBSE) enabled by standardized modeling languages (e.g., SysML®) is at the core. These benefits in the context of spaceflight system challenges can include [1]: • Improved management of complex development • Reduced risk in the development process • Improved cost management • Improved design decisions With appropriate modeling techniques the assurance community also can improve early oversight and insight into project development. NASA has shown the basic constructs of SysML in an MBSE environment offer several key advantages, within a Model Based Mission Assurance (MBMA) initiative [2, 3]. These include the following: • Model viewpoints that promote rapid and systematic assessment of requirements coverage, hazard tagging and risk management • Embedded safety assessments for launch vehicles • Deployment of model assisted development of reliability products - Failure Modes and Effects Analyses (FMEAs) and Fault Trees • Test Planning • Validation and Verification of complex functions • Support of Assurance Case development for complex systems In addition, while there are benefits to be harvested, there is a realization that these do not come without effort and cost. Enabling model-based approaches requires structure, not only in an organizational context, but in a modeling context as well. There can be a steep learning curve and costs associated to train skilled modelers. But, on the other hand, not all of the assurance community need to be modelers. Models themselves must conform to ontologies that enable assurance. This places constraints upon the models and modelers. Optimums have yet to be developed where resources and constraints on modeling must be traded off in the organization and modeling efforts for projects. A number of barriers need to be overcome, as well, which pose challenges to the developers of the software that supports MBSE/MBMA. Information and data must be made to flow seamlessly through the life cycle. Because there is a wide variety of tools used in the community, to avoid the problems of the past of silos, delays, and diverging interests, information should flow among these tools to support the “single source of truth” paradigm of MBSE. This will greatly facilitate MBMA and advancement of assurance functions.

Evans, John W.↗
NASA NTRS2018

Assurance of Autonomy for Robotic Space Missions

While there have been meetings on assurance of autonomy in other application areas, this meeting seeks to develop a roadmap for assurance of autonomy specifically for robotic space missions. Key characteristics that distinguish this application area include: the lack of detailed prior knowledge of the environments in which those missions are to operate; the challenges of mimicking deep space operating conditions for purposes of testing; the need to be highly assured of failsafe operation; limited and delayed (due to speed of light over solar system distances) communication; and the one/few-of-a-kind, must-work-the-first-time nature of most space missions.

Feather, Martin↗
NASA NTRS2015

NASA System Safety Handbook. Volume 2: System Safety Concepts, Guidelines, and Implementation Examples

This is the second of two volumes that collectively comprise the NASA System Safety Handbook. Volume 1 (NASASP-210-580) was prepared for the purpose of presenting the overall framework for System Safety and for providing the general concepts needed to implement the framework. Volume 2 provides guidance for implementing these concepts as an integral part of systems engineering and risk management. This guidance addresses the following functional areas: 1.The development of objectives that collectively define adequate safety for a system, and the safety requirements derived from these objectives that are levied on the system. 2.The conduct of system safety activities, performed to meet the safety requirements, with specific emphasis on the conduct of integrated safety analysis (ISA) as a fundamental means by which systems engineering and risk management decisions are risk-informed. 3.The development of a risk-informed safety case (RISC) at major milestone reviews to argue that the systems safety objectives are satisfied (and therefore that the system is adequately safe). 4.The evaluation of the RISC (including supporting evidence) using a defined set of evaluation criteria, to assess the veracity of the claims made therein in order to support risk acceptance decisions.

Safety Case↗
NASA NTRS2008

Optimizing Requirements Decisions with KEYS

Recent work with NASA's Jet Propulsion Laboratory has allowed for external access to five of JPL's real-world requirements models, anonymized to conceal proprietary information, but retaining their computational nature. Experimentation with these models, reported herein, demonstrates a dramatic speedup in the computations performed on them. These models have a well defined goal: select mitigations that retire risks which, in turn, increases the number of attainable requirements. Such a non-linear optimization is a well-studied problem. However identification of not only (a) the optimal solution(s) but also (b) the key factors leading to them is less well studied. Our technique, called KEYS, shows a rapid way of simultaneously identifying the solutions and their key factors. KEYS improves on prior work by several orders of magnitude. Prior experiments with simulated annealing or treatment learning took tens of minutes to hours to terminate. KEYS runs much faster than that; e.g for one model, KEYS ran 13,000 times faster than treatment learning (40 minutes versus 0.18 seconds). Processing these JPL models is a non-linear optimization problem: the fewest mitigations must be selected while achieving the most requirements. Non-linear optimization is a well studied problem. With this paper, we challenge other members of the PROMISE community to improve on our results with other techniques.

requirements↗
NASA NTRS2008

Space Computing Systems Validation Challenges

To examine the challenges of spaceborne computing systems and the past, present and future approaches to verification and validation in hardware and software systems.

Commercial Orbital Transportation Services (COTS)↗
NASA NTRS2007

Robust Derivation of Risk Reduction Strategies

Effective risk reduction strategies can be derived mechanically given sufficient characterization of the risks present in the system and the effectiveness of available risk reduction techniques. In this paper, we address an important question: can we reliably expect mechanically derived risk reduction strategies to be better than fixed or hand-selected risk reduction strategies, given that the quantitative assessment of risks and risk reduction techniques upon which mechanical derivation is based is difficult and likely to be inaccurate? We consider this question relative to two methods for deriving effective risk reduction strategies: the strategic method defined by Kazman, Port et al [Port et al, 2005], and the Defect Detection and Prevention (DDP) tool [Feather & Cornford, 2003]. We performed a number of sensitivity experiments to evaluate how inaccurate knowledge of risk and risk reduction techniques affect the performance of the strategies computed by the Strategic Method compared to a variety of alternative strategies. The experimental results indicate that strategies computed by the Strategic Method were significantly more effective than the alternative risk reduction strategies, even when knowledge of risk and risk reduction techniques was very inaccurate. The robustness of the Strategic Method suggests that its use should be considered in a wide range of projects.

risk reduction↗
NASA NTRS2006

Using Dissimilarity Metrics to Identify Interesting Designs

A computer program helps to blend the power of automated-search software, which is able to generate large numbers of design solutions, with the insight of expert designers, who are able to identify preferred designs but do not have time to examine all the solutions. From among the many automated solutions to a given design problem, the program selects a smaller number of solutions that are worthy of scrutiny by the experts in the sense that they are sufficiently dissimilar from each other. The program makes the selection in an interactive process that involves a sequence of data-mining steps interspersed with visual displays of results of these steps to the experts. At crucial points between steps, the experts provide directives to guide the process. The program uses heuristic search techniques to identify nearly optimal design solutions and uses dissimilarity metrics defined by the experts to characterize the degree to which solutions are interestingly different. The search, data-mining, and visualization features of the program were derived from previously developed risk-management software used to support a risk-centric design methodology

Feather, Martin↗