SEARCH · Search NASA

Results for “uncertainty in complex systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

All sources NASA NTRS DOE OSTI

NASA NTRS2019

Recent Advancements in Modeling and Simulation of Entry Systems at NASA

This paper describes recent development of modeling and simulation technologies for entry systems in support of NASA's exploration missions. Mission-tailored research and development in modeling of entry systems occurs across the Agency (e.g., within the Orion and Mars 2020 Programs), however the aim of this paper is to discuss the broad, cross-mission research conducted by NASA's Entry Systems Modeling (ESM) Project, which serves as the Agency's only concerted effort toward advancing entry systems across a range of technical disciplines. Technology development in ESM is organized and prioritized from a system-level perspective, resulting in four broad technical areas of investment: (1) Predictive material modeling, (2) Shock layer kinetics and radiation, (3) Computational and experimental aerosciences, and (4) Guidance, navigation, and control. Investments in thermal protection material modeling are geared toward high-fidelity, predictive models capable of handling complex structures, with an eye toward optimizing design performance and quantifying thermal protection system reliability. New computational tools have been developed to characterize material properties and behavior at the microstructural level, and experimental techniques (molecular beam scattering, micro-computed tomography, among others) have been developed to measure material kinetics, morphology, and other parameters needed to inform and validate detailed simulations. Advancements have also been made in macrostructural simulation capability to enable 3-D system-scale calculations of material response with complex topological features, including differential recession of tile gaps. Research and development in the area of shock layer kinetics has focused on air and CO2-based atmospheres. Capacity and capability of the NASA Ames Electric Arc Shock Tube (EAST) have been expanded in recent years and analysis of resulting data has led to several improvements in kinetic models, while simultaneously reducing uncertainties associated with radiative heat transfer predictions. First-principles calculations of fundamental kinetic, thermodynamic, and transport data, along with state-specific models for non-equilibrium flow regimes, have also yielded new insights and have the potential to vastly improve model fidelity. Aerosciences is a very broad area of interest in entry systems, yet a number of important challenges are being addressed: Coupled fluid-structure simulations of parachute inflation and dynamics; Experimental and computational studies of vehicle dynamics; Multi-phase flow with dust particles to simulate entry environments at Mars during dust storms; Studies of roughness-induced heating augmentation relevant to tiled and woven thermal protection systems; and Advanced numerical methods to optimize computational analyses for desired accuracy versus cost. Guidance and control in the context of entry systems has focused on development of methods for multi-axis control (i.e. pitch and yaw, rather than bank angle alone) of spacecraft during entry and descent. With precision landing requirements driven by Mars human exploration goals, recent efforts have yielded 6-DOF models of multi-axis control with propulsive descent of both inflatable and rigid ellipsled-like architectures.

Barnhardt, Michael D.↗

NASA NTRS2022

A Multimodel Investigation of Asian Summer Monsoon UTLS Transport Over the Western Pacific

The Asian summer monsoon (ASM) as a chemical transport system is investigated using a suite of models in preparation for an airborne field campaign over the Western Pacific. Results show that the dynamical process of anticyclone eddy shedding in the upper troposphere rapidly transports convectively uplifted Asian boundary layer air masses to the upper troposphere and lower stratosphere over the Western Pacific. The models show that the transported air masses contain significantly enhanced aerosol loading and a complex chemical mixture of trace gases that are relevant to ozone chemistry. The chemical forecast models consistently predict the occurrence of the shedding events, but the predicted concentrations of transported trace gases and aerosols often differ between models. The airborne measurements to be obtained in the field campaign are expected to help reduce the model uncertainties. Furthermore, the large-scale seasonal chemical structure of the monsoon system is obtained from modeled carbon monoxide, a tracer of the convective transport of pollutants, which provides a new perspective of the ASM circulation, complementing the dynamical characterization of the monsoon.

Laura L. Pan↗

NASA NTRS2003

The Aeronautical Data Link: Decision Framework for Architecture Analysis

A decision analytic approach that develops optimal data link architecture configuration and behavior to meet multiple conflicting objectives of concurrent and different airspace operations functions has previously been developed. The approach, premised on a formal taxonomic classification that correlates data link performance with operations requirements, information requirements, and implementing technologies, provides a coherent methodology for data link architectural analysis from top-down and bottom-up perspectives. This paper follows the previous research by providing more specific approaches for mapping and transitioning between the lower levels of the decision framework. The goal of the architectural analysis methodology is to assess the impact of specific architecture configurations and behaviors on the efficiency, capacity, and safety of operations. This necessarily involves understanding the various capabilities, system level performance issues and performance and interface concepts related to the conceptual purpose of the architecture and to the underlying data link technologies. Efficient and goal-directed data link architectural network configuration is conditioned on quantifying the risks and uncertainties associated with complex structural interface decisions. Deterministic and stochastic optimal design approaches will be discussed that maximize the effectiveness of architectural designs.

Morris, A. Terry↗

NASA NTRSDate not supplied

Results of the NASA Prediction Uncertainty Reduction Tech Challenge

In January 2021 the Advanced Air Vehicles Program approved a new Tech Challenge (TC) to be run out of the Commercial Supersonic Technology (CST) Project. The objective of the TC was to bring uncertainties in the empirical noise modeling for civilian supersonic aircraft into the same range as that of conventional aircraft. The TC goal statement was to “Produce data and demonstrate tools that reduce the uncertainty in predicting Landing & Takeoff Noise levels of supersonic-relevant designs by 5 EPNdB and are ready for use in studies to inform FAA rule-making.” To make the required improvements in noise prediction methods for supersonic aircraft, NASA decided to validate and use high-fidelity numerical simulations to acquire the needed noise data on supersonic propulsion configurations. High-fidelity component designs were developed, such as a two-stage propulsor behind a supersonic inlet designed by General Electric Aviation and variable area exhaust systems using input from recent contracts with GE and Rolls Royce. A noise database was generated as a function of geometric and flow parameters, providing corrections to the empirical noise models and added new input variables to describe the complexities created by the supersonic-specific design features. Statistically, the new models were able to predict the acoustic impact of supersonic-specific features, the reduction in uncertainty being reduced from the baseline 7.8 EPNdB at the beginning of the TC to 2.0 EPNdB at the end.

noise prediction↗

NASA NTRS2000

Transport and Mixing in the Stratosphere and Troposphere

Long-term changes in the composition of the atmosphere are known to have significant effects on atmospheric chemistry and stratospheric ozone. Increasing levels of greenhouse gases have the potential to change the global climate in the middle and upper atmospheres, as well as in the troposphere. Volcanic eruptions, El Nino events, and other natural variations can also cause changes in atmospheric composition and climate. Whether the causes are natural or manmade, changes in the global climate system can have impacts on human society. In order to understand and predict the consequences of these changes, and of control measures such as the Montreal Protocol, it is necessary to understand the complex interactions between radiation, chemistry, and dynamics in the atmosphere. Much of the uncertainty in our understanding of atmospheric processes comes from an incomplete understanding of atmospheric transport. A complete and self-consistent model of transport requires not only an understanding of trace-species transport, but also the transport of dynamically active quantities such as heat and potential vorticity. Therefore, the goal of the proposed research is to better understand large-scale transport and mixing processes in the middle atmosphere and troposphere.

Bowman, Kenneth P.↗

NASA NTRS2013

Understanding Crew Decision-Making in the Presence of Complexity: A Flight Simulation Experiment

Crew decision making and response have long been leading causal and contributing factors associated with aircraft accidents. Further, it is anticipated that future aircraft and operational environments will increase exposure to risks related to these factors if proactive steps are not taken to account for ever-increasing complexity. A flight simulation study was designed to collect data to help in understanding how complexity can, or may, be manifest. More specifically, an experimental apparatus was constructed that allowed for manipulation of information complexity and uncertainty, while also manipulating operational complexity and uncertainty. Through these manipulations, and the aid of experienced airline pilots, several issues have been discovered, related most prominently to the influence of information content, quality, and management. Flight crews were immersed in an environment that included new operational complexities suggested for the future air transportation system as well as new technological complexities (e.g. electronic flight bags, expanded data link services, synthetic and enhanced vision systems, and interval management automation). In addition, a set of off-nominal situations were emulated. These included, for example, adverse weather conditions, traffic deviations, equipment failures, poor data quality, communication errors, and unexpected clearances, or changes to flight plans. Each situation was based on one or more reference events from past accidents or incidents, or on a similar case that had been used in previous developmental tests or studies. Over the course of the study, 10 twopilot airline crews participated, completing over 230 flights. Each flight consisted of an approach beginning at 10,000 ft. Based on the recorded data and pilot and research observations, preliminary results are presented regarding decision-making issues in the presence of the operational and technological complexities encountered during the flights.

Young, Steven D.↗

NASA NTRSDate not supplied

Model Assisted Probability of Detection for NASA Space Missions

Model assisted probability of detection (MAPOD) uses data from simulations to improve a traditional probability of detection (POD) study. This could include extending the parameter space to reduce uncertainty or substituting experimental data with simulated data to reduce the time and cost of a POD study. In the past MAPOD was difficult due to limited computational resources, but recent innovations in simulation tools and high-performance computing have made this type of high-degree-of-freedom modelling possible, and complex structures have made it necessary. This presentation will summarize the work done by the computational nondestructive evaluation (CNDE) group at NASA Langley Research Center (LaRC) to complete a MAPOD study for phased array ultrasound testing (PAUT) of a friction stir welding (FSW) method to be used on Space Launch System (SLS) structures. The three critical needs for a MAPOD study are a validated and verified model of the inspection technique for the structure being inspected, some experimental POD data, and an uncertainty model for both the model and the experimental data. PAUT was simulated using Extende CIVA’s UT module. The model was validated using laboratory inspection data from NASA Marshall Space Flight Center (MSFC) for a Hit/Miss POD for FSW in 2219-T87 aluminum panels representative of those used in the SLS. This model was then used to simulate flaw sizes that were originally omitted from the original POD study. The results of this new MAPOD study will be presented along with a discussion of the methods and processes used to analyze the original data, selected simulation parameters, and development of the uncertainty model used for the statistical analysis. The goal of this effort is not just to improve the POD study but to demonstrate the value of MAPOD and provide a roadmap for application of MAPOD on future projects.

Elizabeth Gregory↗

NASA NTRS2017

Supportability Challenges, Metrics, and Key Decisions for Future Human Spaceflight

Future crewed missions beyond Low Earth Orbit (LEO) represent a logistical challenge that is unprecedented in human space flight. Astronauts will travel farther and stay in space for longer than any previous mission, far from timely abort or resupply from Earth. Under these conditions, supportability { defined as the set of system characteristics that influence the logistics and support required to enable safe and effective operations of systems { will be a much more significant driver of space system lifecycle properties than it has been in the past. This paper presents an overview of supportability for future human space flight. The particular challenges of future missions are discussed, with the differences between past, present, and future missions highlighted. The relationship between supportability metrics and mission cost, performance, schedule, and risk is also discussed. A set of pro- posed strategies for managing supportability is presented (including reliability growth, uncertainty reduction, level of repair, commonality, redundancy, In-Space Manufacturing (ISM) (including the use of material recycling and In-Situ Resource Utilization (ISRU) for spares and maintenance items), reduced complexity, and spares inventory decisions such as the use of predeployed or cached spares - along with a discussion of the potential impacts of each of those strategies. References are provided to various sources that describe these supportability metrics and strategies, as well as associated modeling and optimization techniques, in greater detail. Overall, supportability is an emergent system characteristic and a holistic challenge for future system development. System designers and mission planners must carefully consider and balance the supportability metrics and decisions described in this paper in order to enable safe and effective beyond-LEO human space flight.

Owens, Andrew C.↗

NASA NTRS2016

The New Sun-Sky-Lunar Cimel CE318-T Multiband Photometer - A Comprehensive Performance Evaluation

This paper presents the new photometer CE318-T, able to perform daytime and night-time photometric measurements using the sun and the moon as light source. Therefore,this new device permits a complete cycle of diurnal aerosol and water vapour measurements valuable to enhance atmospheric monitoring to be extracted. In this study wehave found significantly higher precision of triplets when comparing the CE318-T master instrument and the Cimel AErosol RObotic NET work (AERONET) master (CE318-AERONET) triplets as a result of the new CE318-T tracking system. Regarding the instrument calibration, two new methodologies to transfer the calibration from a reference instrument using only daytime measurements (Sun Ratio and Sun-Moon gain factor techniques) are presented and discussed. These methods allow the reduction of the previous complexities inherent to nocturnal calibration. A quantitative estimation of CE318-T AOD uncertainty by means of error propagation theory during daytime revealed AOD uncertainties (u(sup D)(sub AOD)) for Langley-calibrated instruments similar to the expected values for other reference instruments (0.002-0.009). We have also found u(sup D)(sub AOD) values similar to the values reported in sun photometry for field instruments (approximately 0.015). In the case of the night-time period, the CE318-T-estimated standard combined uncertainty (u(sup N)(sub AOD)) is dependent not only on the calibration technique but also on illumination conditions and the instrumental noise. These values range from 0.011-0.018 for Lunar Langley-calibrated instruments to 0.012-0.021 for instruments calibrated using the Sun Ratio technique. In the case of moon-calibrated instruments using the Sun-Moon gain factor method and sun calibrated using the Langley technique, we found u(sup N)(sub AOD) ranging from 0.016 to 0.017 (up to 0.019 in 440 nm channel), not dependent on any lunar irradiance model. A subsequent performance evaluation including CE318-T and collocated measurements from independent reference instruments has served to assess the CE318-T performance as well as to confirm its estimated uncertainty. Daytime AOD evaluation, performed at Izana station from March to June 2014, encompassed measurements from a reference CE318-T, a CE318-AERONET master instrument, a Precision Filter Radiometer (PFR) and a Precision Spectroradiometer (PSR) prototype, reporting low AOD discrepancies between the four instruments (up to 0.006). The nocturnal AOD evaluation was performed using CE318-T- and starphotometer-collocated measurements and also by means of a day/night coherence transition test using the CE318-T master instrument and the CE318 daytime data from the CE318-AERONET master instrument. Results showed low discrepancies with the star photometer at 870 and 500 nm channels(less tna or equal to 0.013) and differences with AERONET daytime data (1 h after and before sunset and sunrise) in agreement with the estimated u(sup N)(sub AOD) values at all illumination conditions in the case of channels within the visible spectral range, and only for high moon's illumination conditions in the case of near infrared channels. Perceptible water vapour (PWV) validation showed a good agreement between CE318-T and Global Navigation Satellite System (GNSS) PWV values for all illumination conditions, within the expected precision for sun photometry. Finally, two case studies have been included to highlight the ability of the new CE318-T to capture the diurnal cycle of aerosols and water vapour as well as short-term atmospheric variations, critical for climate studies.

Barreto, Africa↗

NASA NTRS1994

A Supersonic Tunnel for Laser and Flow-Seeding Techniques

A supersonic wind tunnel with flow conditions of 3 lbm/s (1.5 kg/s) at a free-stream Mach number of 2.5 was designed and tested to provide an arena for future development work on laser measurement and flow-seeding techniques. The hybrid supersonic nozzle design that was used incorporated the rapid expansion method of propulsive nozzles while it maintained the uniform, disturbance-free flow required in supersonic wind tunnels. A viscous analysis was performed on the tunnel to determine the boundary layer growth characteristics along the flowpath. Appropriate corrections were then made to the contour of the nozzle. Axial pressure distributions were measured and Mach number distributions were calculated based on three independent data reduction methods. A complete uncertainty analysis was performed on the precision error of each method. Complex shock-wave patterns were generated in the flow field by wedges mounted near the roof and floor of the tunnel. The most stable shock structure was determined experimentally by the use of a focusing schlieren system and a novel, laser based dynamic shock position sensor. Three potential measurement regions for future laser and flow-seeding studies were created in the shock structure: deceleration through an oblique shock wave of 50 degrees, strong deceleration through a normal shock wave, and acceleration through a supersonic expansion fan containing 25 degrees of flow turning.

Bruckner, Robert J.↗

NASA NTRS1998

Neural Network Control of a Magnetically Suspended Rotor System

Magnetic bearings offer significant advantages because they do not come into contact with other parts during operation, which can reduce maintenance. Higher speeds, no friction, no lubrication, weight reduction, precise position control, and active damping make them far superior to conventional contact bearings. However, there are technical barriers that limit the application of this technology in industry. One of them is the need for a nonlinear controller that can overcome the system nonlinearity and uncertainty inherent in magnetic bearings. At the NASA Lewis Research Center, a neural network was selected as a nonlinear controller because it generates a neural model without any detailed information regarding the internal working of the magnetic bearing system. It can be used even for systems that are too complex for an accurate system model to be derived. A feed-forward architecture with a back-propagation learning algorithm was selected because of its proven performance, accuracy, and relatively easy implementation.

Choi, Benjamin B.↗

NASA NTRS2020

Improving Trust in Deep Neural Networks with Nearest Neighbors

Deep neural networks are used increasingly for perception and decision-making in UAVs. For example, they can be used to recognize objects from images and decide what actions the vehicle should take. While deep neural networks can perform very well at complex tasks, their decisions may be unintuitive to a human operator. When a human disagrees with a neural network prediction, due to the black box nature of deep neural networks, it can be unclear whether the system knows something the human does not or whether the system is malfunctioning. This uncertainty is problematic when it comes to ensuring safety. As a result, it is important to develop technologies for explaining neural network decisions for trust and safety. This paper explores a modification to the deep neural network classification layer to produce both a predicted label and an explanation to support its prediction. Specifically, at test time, we replace the final output layer of the neural network classifier by a k-nearest neighbor classifier. The nearest neighbor classifier produces 1) a predicted label through voting and 2) the nearest neighbors involved in the prediction, which represent the most similar examples from the training dataset. Because prediction and explanation are derived from the same underlying process, this approach guarantees that the explanations are always relevant to the predictions. We demonstrate the approach on a convolutional neural network for a UAV image classification task. We perform experiments using a forest trail image dataset and show empirically that the hybrid classifier can produce intuitive explanations without loss of predictive performance compared to the original neural network. We also show how the approach can be used to help identify potential issues in the network and training process.

Lee, Ritchie↗

NASA NTRSDate not supplied

Model Assisted Probability of Detection for NASA Space Missions

Model assisted probability of detection (MAPOD) uses data from simulations to improve a traditional probability of detection (POD) study. This could include extending the parameter space to reduce uncertainty or substituting experimental data with simulated data to reduce the time and cost of a POD study. In the past MAPOD was difficult due to limited computational resources, but recent innovations in simulation tools and high-performance computing have made this type of high-degree-of-freedom modeling possible, and complex structures have made it necessary. This presentation will summarize the work done by the computational nondestructive evaluation (CNDE) specialists within the Nondestructive Evaluation Sciences branch at NASA Langley Research Center (LaRC) to complete a MAPOD study for phased array ultrasound testing (PAUT) of a friction stir welding (FSW) method to be used on Space Launch System (SLS) structures. The three critical needs for a MAPOD study are a validated and verified model of the inspection technique for the structure being inspected, some experimental POD data, and an uncertainty model for both the model and the experimental data. PAUT was simulated using Extende CIVA’s UT module. The model was validated using laboratory inspection data from NASA Marshall Space Flight Center (MSFC) for a Hit/Miss POD for FSW in 2219-T87 aluminum panels representative of those used in the SLS. This model was then used to simulate flaw sizes that were originally omitted from the original POD study. The results of this new MAPOD study will be presented along with a discussion of the methods and processes used to analyze the original data, selected simulation parameters, and development of the uncertainty model used for the statistical analysis. The goal of this effort is not just to improve the POD study but to demonstrate the value of MAPOD and provide a roadmap for application of MAPOD on future projects.

Elizabeth Gregory↗

NASA NTRSDate not supplied

Variational Coupled Loads Analysis using the Hybrid Parametric Variation Method

Time-domain coupled loads analysis (CLA)is used to determine the response of a launch vehicle and payload system to transient forces, such as liftoff, engine ignitions and shutdowns, jettison events, and atmospheric flight loads, such as buffet. CLA, using Hurty/Craig-Bampton (HCB)component models, is the accepted method for the establishment of design-level loads for launch systems. However, uncertainty in the component models flows into uncertainty in predicted system results. Uncertainty in the structural responses during launch is a significant concern because small variations in launch vehicle and payload mode shapes and their interactions can result in significant variations in system loads. Uncertainty quantification (UQ)is used to determine statistical bounds on prediction accuracy based on model uncertainty. In this paper uncertainty is treated at the HCB component-model level. In an effort to account for model uncertainties and statistically bound their effect on CLA predictions, this work combines CLA with UQ in a process termed variational coupled loads analysis (VCLA). The modeling of uncertainty using a parametric approach, in which input parameters are represented by random variables, is common, but its major drawback is the resulting uncertainty is limited to the form of the nominal model. Uncertainty in model form is one of the biggest contributors to uncertainty in complex built-up structures. Model-form uncertainty can be represented using a nonparametric approach based on random matrix theory (RMT). In this work, UQ is performed using the hybrid parametric variation (HPV)method, which combines parametric with nonparametric uncertainty at the HCB component model level. The HPV method requires the selection of dispersion values for the HCB fixed-interface (FI)eigenvalues, and the HCB mass and stiffness matrices. The dispersions are based upon component test-analysis modal correlation results. During VCLA, random component models are assembled into an ensemble of random systems using a Monte Carlo (MC)approach. CLA is applied to each of the ensemble members to produce an ensemble of system-level responses for statistical analysis. The proposed methodology is demonstrated through its application to a buffet loads analysis of NASA’s Space Launch System (SLS)during the transonic regime fifty seconds after liftoff. Core stage (CS)section shears and moments are recovered, and statistics are computed.

Uncertainty Quantification↗

NASA NTRS1992

A feasibility assessment of magnetic bearings for free-piston Stirling space power converters

This report describes a design and analysis study performed by Mechanical Technology Incorporated (MTI) under NASA Contract NAS3-26061. The objective of the study was to assess the feasibility and efficacy of applying magnetic bearings to free-piston Stirling-cycle power conversion machinery of the type currently being evaluated for possible use in long-term space missions. The study was performed for a 50-kWe Reference Stirling Space Power Converter (RSSPC) system consisting of two 25-kWe free-piston Stirling engine modules. Two different versions of the RSSPC engine modules have been defined under NASA Contract NAS3-25463. These modules currently use hydrostatic gas bearings to support the reciprocating displacer and power piston assemblies. Results of this study show that active magnetic bearings of the attractive electromagnetic type are technically feasible for RSSPC application provided that wire insulation with 60,000-hr life capability at 300 C can be developed for the bearing coils. From a design integration standpoint, both versions of the RSSPC were found to be conceptually amenable to magnetic support of the power piston assembly. However, only one version of the RSSPC was found to be amendable to magnetic support of the displacer assembly. Unacceptable changes to the basic engine design would be required to incorporate magnetic displacer bearings into the second version. Complete magnetic suspension of the RSSPC can potentially increase overall efficiency of the Stirling cycle power converter by 0.53 to 1.4 percent (0.15 to 0.4 efficiency points). Magnetic bearings will also overcome several operational concerns associated with hydrostatic gas bearing systems. However, these advantages are accompanied by a 5 to 8 percent increase in specific mass of the RSSPC, depending on the RSSPC version employed. Additionally, magnetic bearings are much more complex, both mechanically and particularly electronically, than hydrostatic bearings. Accordingly, long-term stability and reliability represent areas of uncertainty for magnetic bearings. Considerable development effort will be required to establish the long-term suitability of these bearings for Stirling space power applications.

Curwen, Peter W.↗

NASA NTRS2015

A Framework for the Cross-Sectoral Integration of Multi-Model Impact Projections: Land Use Decisions Under Climate Impacts Uncertainties

Climate change and its impacts already pose considerable challenges for societies that will further increase with global warming (IPCC, 2014a, b). Uncertainties of the climatic response to greenhouse gas emissions include the potential passing of large-scale tipping points (e.g. Lenton et al., 2008; Levermann et al., 2012; Schellnhuber, 2010) and changes in extreme meteorological events (Field et al., 2012) with complex impacts on societies (Hallegatte et al., 2013). Thus climate change mitigation is considered a necessary societal response for avoiding uncontrollable impacts (Conference of the Parties, 2010). On the other hand, large-scale climate change mitigation itself implies fundamental changes in, for example, the global energy system. The associated challenges come on top of others that derive from equally important ethical imperatives like the fulfilment of increasing food demand that may draw on the same resources. For example, ensuring food security for a growing population may require an expansion of cropland, thereby reducing natural carbon sinks or the area available for bio-energy production. So far, available studies addressing this problem have relied on individual impact models, ignoring uncertainty in crop model and biome model projections. Here, we propose a probabilistic decision framework that allows for an evaluation of agricultural management and mitigation options in a multi-impactmodel setting. Based on simulations generated within the Inter-Sectoral Impact Model Intercomparison Project (ISI-MIP), we outline how cross-sectorally consistent multi-model impact simulations could be used to generate the information required for robust decision making. Using an illustrative future land use pattern, we discuss the trade-off between potential gains in crop production and associated losses in natural carbon sinks in the new multiple crop- and biome-model setting. In addition, crop and water model simulations are combined to explore irrigation increases as one possible measure of agricultural intensification that could limit the expansion of cropland required in response to climate change and growing food demand. This example shows that current impact model uncertainties pose an important challenge to long-term mitigation planning and must not be ignored in long-term strategic decision making

farmlands↗

NASA NTRS2017

How Engineers Negotiate Domain Boundaries in a Complex, Interdisciplinary Engineering Project

Engineering educators have an essential role in preparing engineers to work in a complex, interdisciplinary workforce. While much engineering education focuses on teaching students to develop disciplinary expertise in specific engineering domains, there is a strong need to teach engineers about the knowledge that they develop or use in their work (Bucciarelli 1994, Allenby Sarewitz, 2011; Frodeman, 2013). The purpose of this research is to gain a better understanding of the knowledge systems of practicing engineers through observations of their practices such that the insights learned can guide future education efforts. Using an example from a complex and interdisciplinary engineering project, this paper presents a case study overviewing the types of epistemological (or knowledge-acquiring or using) complexities that engineers navigate. Specifically, we looked at a discussion of the thermal design of a CubeSat that occurred during an engineering review at NASA. We analyzed the review using a framework that we call 'peak events', or pointed discussions between reviewers, project engineers, and managers. We examined the dialog within peak events to identify the ways that knowledge was brought to bear, highlighting discussions of uncertainty and the boundaries of knowledge claims. We focus on one example discussion surrounding the thermal design of the CubeSat, which provides a particularly thorough example of a knowledge system since the engineers present explained, justified, negotiated, and defended knowledge within a social setting. Engineering students do not get much practice or instruction in explicitly negotiating knowledge systems and epistemic standards in this way. We highlight issues that should matter to engineering educators, such as the need to discuss what level of uncertainty is sufficient and the need to negotiate boundaries of system responsibility. Although this analysis is limited to a single discussion or 'peak event', our case shows that this type of discussion can occur in engineering and suggests that it could be important for future engineering education research.

ethnography↗

NASA NTRS2022

Reliability-Aware Requirements Development for Autonomy Software

Autonomy is the ability of a system to make decisions and take actions in the presence of uncertainty. Validating autonomy, therefore, is a matter of validating that the system makes intelligent decisions even when there might be discrepancies between the observed and the expected state of the world as well as when the possible outcome of each of the possible courses of action is uncertain. During the requirements engineering phase, the functions, including those that are required to be autonomous and their expected behavior are specified. However, in the case of autonomous software system, it is challenging to specify all possible scenarios that the software may encounter. Additionally, since autonomous behaviors cannot always be fully predetermined, it is difficult to reason about their completeness and correctness. While these challenges are also faced with non-autonomous system, they are more prevalent for autonomous system due to their complexity and emergent behavior. Our efforts have been three-fold. On one hand, we have developed a goal-based method for requirements decomposition. We have embedded this approach into a higher-level systems engineering framework and then developed methods for decision and reliability analysis within this framework. And finally, we use simulation analysis to validate the more theoretical methods and provide a feedback mechanism. The decision analysis approach (DA) helps in identifying and mitigating the run-time risks, by bringing to the forefront the uncertainties, decisions, interactions, and other factors that may cause autonomous software to make erroneous decisions. The DA framework is used to quantify, for each decision that the software can make, failure risk due to uncertainties. The information about the key decisions and circumstances that can cause autonomous system to make incorrect decisions are used to generate test scenarios to be run in a simulator to ensure that the system can handle error-prone circumstances. The results or outcomes from decisions, based on the simulated scenarios, can then be leveraged to further gain insight of the assured system. We combine a goal-based approach with simulation analysis to facilitate requirements development for autonomous systems and further provide a method for run-time systems level reliability considerations during the autonomous software development process. The DA and the reliability analysis methods provide a formalism for the consideration of uncertainty in the run time decision-making process for autonomous software. This formalism considers the uncertainty involved in the outcome of different courses of action, in terms of performance and cost, as well as a method to consider the system level reliability implications. System level reliability considerations for autonomous software decision making contribute to the satisfaction of the system level goals over the lifecycle of a given mission. We demonstrate our approach using a hypothetical rover path planning example. The requirements for our rover simulator are based on existing literature about Mars rovers and lessons learned from real world observations at JPL. The specifics of the autonomy design for the Mars rovers, that have not been previously cleared for external release, are not used for our demonstrations.

Lindvall, Mikael↗