SEARCH · Search NASA

Results for “metric learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

NASA NTRS2003

Managing a project's legacy: implications for organizations and project management

Organizations that rely on projects to implement their products must find effective mechanisms for propagating lessons learned on one project throughout the organization. A broad view of what constitutes a project's 'legacy' is presented that includes not just the design products and leftover parts, but new processes, relationships, technology, skills, planning data, and performance metrics. Based on research evaluating knowledge reuse in innovative contexts, this paper presents an approach to project legacy management that focuses on collecting and using legacy knowledge to promote organizational learning and effective reuse, while addressing factors of post-project responsibility, information obsolescence, and the importance of ancillary contextual information. .

risk management↗

NASA NTRS2024

An Extensible Perturbed Parameter Ensemble for the Community Atmosphere Model Version 6

This paper documents the methodology and preliminary results from a Perturbed Parameter Ensemble (PPE) technique, where multiple parameters are varied simultaneously and the parameter values are determined with Latin hypercube sampling. This is done with the Community Atmosphere Model version 6 (CAM6), the atmospheric component of the Community Earth System Model version 2 (CESM2). We apply the PPE method to CESM2-CAM6 to understand climate sensitivity to atmospheric physics parameters. The initial simulations vary 45 parameters in the microphysics, convection, turbulence and aerosol schemes with 263 ensemble members. These atmospheric parameters are typically the most uncertain in many climate models. Control simulations are analyzed and targeted simulations to understand climate forcing due to aerosols and fast climate feedbacks. The use of various emulators is explored in the multi- dimensional space mapping input parameters to output metrics. Parameter impacts on various model outputs, such as radiation, cloud and aerosol properties are evaluated. Machine learning is also used to probe optimal parameter values against observations. Our findings show that using PPE is a valuable tool for climate uncertainty analysis. Furthermore, by varying many parameters simultaneously, we find that many different combinations of parameter values can produce results consistent with observations, and thus careful analysis of tuning is important. The CESM2-CAM6 PPE is publicly available, and extensible to other configurations to address questions of other model processes in the atmosphere and other model components (e.g. coupling to the land surface).

Machine learning↗

NASA NTRSDate not supplied

Open-Source Data Engineering at NASA: CCMC's Approach to Managing Petabyte-Scale Heliophysics Data

The Community Coordinated Modeling Center (CCMC) at NASA Goddard Space Flight Center (GSFC) leads heliophysics research by providing open access to numerous models and their outputs. Our resources are available on-demand and continuously updated with real-time data, covering sun-earth interactions across multiple domains. These domains include coronal, heliosphere, inner and global magnetosphere, ionosphere, thermosphere, and lower atmosphere interactions. Operating in a hybrid environment, CCMC utilizes both self-owned hardware and Amazon Web Services (AWS) cloud infrastructure. Managing petabytes of data across multiple locations necessitates robust data engineering solutions. To address this challenge, CCMC has adopted industry-standard and open-source tools. We use Apache Airflow as our primary data engineering platform, Python for scripting and data processing, and GitLab for version control and CI/CD. Additionally, we employ Kubernetes for containerized services, Grafana and Prometheus for metrics and monitoring, and Terraform and Puppet for reproducible infrastructure as code. This presentation will discuss lessons learned from our data engineering experiences, platforms evaluated but found unsuitable for our scientific data requirements, and specific techniques developed to enhance data transfer speed and reliability. By using these technologies effectively, CCMC continues to advance heliophysics research through efficient data management and open-access modeling.

space weather↗

NASA NTRSDate not supplied

Analyzing Natural Language Context in Human-Machine Teaming using Supervised Machine Learning

Building a foundation for trustworthiness and trust verification in multi-asset teaming is the research challenge of Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR). The Design Reference Mission (DRM) for ATTRACTOR is a search and rescue mission objective governed by a multi-member team consisting of human and machine operators. A crucial component to the effort is the communication between humans and autonomous agents throughout both planning and execution stages of the mission. Intuitive communication methods and modalities are posited as critical enablers for certifying trust and trustworthiness. This paper reports on the data collection and analysis conducted in support of the Human Informed Natural-language GANs Evaluation (HINGE)project to attain explainable and trusted communication between human-machine assets. Two identically curated image description datasets were acquired for HINGE, both consisting of two unique input modalities (typed vs. verbal) and retrieved in two distinct contexts (general vs. specific). The gathered datasets were assessed and compared using Parts-of-Speech (POS)features, sentence similarity metrics, and linguistic analysis. Then, the datasets were modeled and tested separately and in combination with one another using machine learning algorithms. The comparison and testing results reveal a superior dataset, by which a preferred context and input is understood, for generating image representations of missing persons using a Generative Adversarial Network (GAN).

Bryan A Barrows↗

NASA NTRSDate not supplied

Lessons Learned From Smallsat Microwave Sounders Data Assimilation in the NASA Goddard Earth Observing System (GEOS)

The NASA Time-Resolved Observations of Precipitation structure and storm Intensity with a Constellation of Smallsats (TROPICS) mission is a constellation of small satellites carrying state-of-art microwave temperature and humidity sounders with 12 channels between 91 GHz and 205 GHz frequency. Including TROPICS-pathfinder, launched on 30 June 2021, five TROPICS satellites operate and provide temperature and humidity data to NWP and atmospheric retrieval communities. This study evaluates the quality of radiance data from these five satellites. It assesses data impacts on NASA Goddard Earth Observing System (GEOS) global NWP analysis and forecasts while seeking answers to the following key questions: 1. What information do TROPICS satellites bring to NWP analysis? 2. How will TROPICS data assimilation affect NWP forecasts, including tropical cyclone analysis and forecasts? 3. What are the benefits of having multiple TROPICS satellites in NWP forecasts? 4. What are the weaknesses regarding instrument stability, consistency, and data quality of these CubeSats? 5. Can multiple TROPICS make similar impacts as conventional MW sensors such as ATMS or GPM Microwave Imager (GMI)? The sensitivities of each TROPICS channel to temperature, water vapor, cloud, and precipitation are examined in various weather conditions. Results from observing system experiments assimilating TROPICS radiance are used for evaluation metrics, including forecast skills and fit to other observations such as radiosondes and microwave and infrared sounders. Finally, the lessons learned from using these cube satellite radiance data in the GEOS atmospheric data assimilation system are shared.

Min-Jeong Kim↗

NASA NTRS2012

Estimation and Bias Correction of Aerosol Abundance using Data-driven Machine Learning and Remote Sensing

Air quality information is increasingly becoming a public health concern, since some of the aerosol particles pose harmful effects to peoples health. One widely available metric of aerosol abundance is the aerosol optical depth (AOD). The AOD is the integrated light extinction coefficient over a vertical atmospheric column of unit cross section, which represents the extent to which the aerosols in that vertical profile prevent the transmission of light by absorption or scattering. The comparison between the AOD measured from the ground-based Aerosol Robotic Network (AERONET) system and the satellite MODIS instruments at 550 nm shows that there is a bias between the two data products. We performed a comprehensive analysis exploring possible factors which may be contributing to the inter-instrumental bias between MODIS and AERONET. The analysis used several measured variables, including the MODIS AOD, as input in order to train a neural network in regression mode to predict the AERONET AOD values. This not only allowed us to obtain an estimate, but also allowed us to infer the optimal sets of variables that played an important role in the prediction. In addition, we applied machine learning to infer the global abundance of ground level PM2.5 from the AOD data and other ancillary satellite and meteorology products. This research is part of our goal to provide air quality information, which can also be useful for global epidemiology studies.

Malakar, Nabin K.↗

NASA NTRSDate not supplied

Applying Machine Learning Tools for Runway Configuration Decision Support

Determining optimal runway configurations at airports, a responsibility assigned to air traffic controllers, is a challenging task. The decision-making process is intricate and involves consideration of many factors such as prevailing wind condition, convective weather, visibility, cloud ceilings, departure and arrival demand, traffic flow, equipment status, and other airport constraints. In a previous work, we developed a Runway Configuration Assistance tool using an offline reinforcement learning method called conservative Q-learning. In this paper, we evaluate and validate our Runway Configuration Assistance tool as a decision support for air traffic controllers. We validated our tool using three airports with differing levels of complexity: Charlotte Douglas International Airport, Denver International Airport, and Dallas Fort Worth International Airport. We quantified the performance of the Runway Configuration Assistance tool based on (1) agreement with historical air traffic controller decisions and (2) violation of decisions that would be obvious to subject-matter experts. Our tool showed promising results in both performance metrics for the three airports, despite the complexities in the runway configuration decision-making process. We also discuss challenges in using machine learning in general to aid air traffic management and identify deployment considerations for the Runway Configuration Assistance tool.

Runway Configuration Management↗

NASA NTRSDate not supplied

Machine Learning Tools for Runway Configuration Decision Support

Runway Configuration Management↗

NASA NTRS2014

Customizing Countermeasure Prescriptions using Predictive Measures of Sensorimotor Adaptability

Astronauts experience sensorimotor disturbances during the initial exposure to microgravity and during the readapation phase following a return to a gravitational environment. These alterations may lead to disruption in the ability to perform mission critical functional tasks during and after these gravitational transitions. Astronauts show significant inter-subject variation in adaptive capability following gravitational transitions. The ability to predict the manner and degree to which each individual astronaut will be affected would improve the effectiveness of a countermeasure comprised of a training program designed to enhance sensorimotor adaptability. Due to this inherent individual variability we need to develop predictive measures of sensorimotor adaptability that will allow us to predict, before actual space flight, which crewmember will experience challenges in adaptive capacity. Thus, obtaining this information will allow us to design and implement better sensorimotor adaptability training countermeasures that will be customized for each crewmember's unique adaptive capabilities. Therefore the goals of this project are to: 1) develop a set of predictive measures capable of identifying individual differences in sensorimotor adaptability, and 2) use this information to design sensorimotor adaptability training countermeasures that are customized for each crewmember's individual sensorimotor adaptive characteristics. To achieve these goals we are currently pursuing the following specific aims: Aim 1: Determine whether behavioral metrics of individual sensory bias predict sensorimotor adaptability. For this aim, subjects perform tests that delineate individual sensory biases in tests of visual, vestibular, and proprioceptive function. Aim 2: Determine if individual capability for strategic and plastic-adaptive responses predicts sensorimotor adaptability. For this aim, each subject's strategic and plastic-adaptive motor learning abilities are assessed using a test of locomotor function designed specifically to delineate both mechanisms. Aim 3: Develop predictors of sensorimotor adaptability using brain structural and functional metrics. We will measure individual differences in regional brain volumes (structural MRI), white matter integrity (diffusion tensor imaging, or DTI), functional network integrity (resting state functional connectivity MRI), and sensorimotor adaptation task-related functional brain activation (functional MRI). We decided to complete the data collection for Specific Aims 1, 2 and 3 simultaneously on the same subjects to increase data capture. By having the same subjects perform all three specific aims we can enhance our ability to detect how a wider range of factors can predict adaptability in a specific individual. This provides a much richer database and potentially a better understanding of the predictive power of the selected factors. In this presentation I will discuss preliminary data obtained to date.

Bloomberg, J. J.↗

NASA NTRS2008

Closing the Certification Gaps in Adaptive Flight Control Software

Over the last five decades, extensive research has been performed to design and develop adaptive control systems for aerospace systems and other applications where the capability to change controller behavior at different operating conditions is highly desirable. Although adaptive flight control has been partially implemented through the use of gain-scheduled control, truly adaptive control systems using learning algorithms and on-line system identification methods have not seen commercial deployment. The reason is that the certification process for adaptive flight control software for use in national air space has not yet been decided. The purpose of this paper is to examine the gaps between the state-of-the-art methodologies used to certify conventional (i.e., non-adaptive) flight control system software and what will likely to be needed to satisfy FAA airworthiness requirements. These gaps include the lack of a certification plan or process guide, the need to develop verification and validation tools and methodologies to analyze adaptive controller stability and convergence, as well as the development of metrics to evaluate adaptive controller performance at off-nominal flight conditions. This paper presents the major certification gap areas, a description of the current state of the verification methodologies, and what further research efforts will likely be needed to close the gaps remaining in current certification practices. It is envisioned that closing the gap will require certain advances in simulation methods, comprehensive methods to determine learning algorithm stability and convergence rates, the development of performance metrics for adaptive controllers, the application of formal software assurance methods, the application of on-line software monitoring tools for adaptive controller health assessment, and the development of a certification case for adaptive system safety of flight.

Jacklin, Stephen A.↗

NASA NTRSDate not supplied

The Lunar Lab Initiative

Analogous to terrestrial Antarctic basecamps at our south pole, space exploration outposts will be a combination of habitation and science-focused assets. The Artemis Exploration Roadmap endeavors to establish a sustained human presence on the lunar south pole starting in 2028. Within the planned Artemis Base Camp, most efforts to date have focused on the habitation assets or a mixture of science and habitation assets . With the recent advent of commercial lunar landing capabilities, the trade space can be further expanded to include dedicated science focused assets that ensure adequate science capability. To that end, the Forge, an innovation team initiative created at the Johnson Space Center (JSC), explored alternative ways to increase the science capabilities within the basecamp. The Forge uses structured brainstorming and facilitation to increase innovation within the assigned team, while executing its process in a rapid turnaround fashion for ideation and design study. As a pilot study for the Forge, a multi-disciplinary team investigated a lunar lab module to augment the lunar surface plans for Artemis. This formulation study evaluated the scientific and operational considerations for this dedicated crew space that would increase the capabilities of lunar surface science operations. The trade space evaluated single lunar lander delivery versus multiple lunar lander delivery for module assembly, alternative structural designs, the science instrument outfitting with mapping to scientific goals to maximize lunar science objectives, and how the lab would be operated. The lab operations options explored a stand-alone facility, as well as a facility integrated into a larger lunar surface complex, while evaluating crew usage and habitability considerations. The resulting alternatives were evaluated by the Forge team with a set of metrics to determine their originality, feasibility, and science performance, and thereby provide a recommendation to the lunar architecture planning team. Lessons learned from the lunar lab study were compiled to improve the process and tools for future Forge studies. Forward work, open issues, and challenges to further refine the design of the various science-focused asset options were also documented. By providing a robust science capability at humanity’s furthest outpost, we can plan for a sustained human lunar presence once the initial lunar surface access capability is well established.

Lunar outpost↗

NASA NTRS2013

NASA Software Engineering Benchmarking Study

To identify best practices for the improvement of software engineering on projects, NASA's Offices of Chief Engineer (OCE) and Safety and Mission Assurance (OSMA) formed a team led by Heather Rarick and Sally Godfrey to conduct this benchmarking study. The primary goals of the study are to identify best practices that: Improve the management and technical development of software intensive systems; Have a track record of successful deployment by aerospace industries, universities [including research and development (R&D) laboratories], and defense services, as well as NASA's own component Centers; and Identify candidate solutions for NASA's software issues. Beginning in the late fall of 2010, focus topics were chosen and interview questions were developed, based on the NASA top software challenges. Between February 2011 and November 2011, the Benchmark Team interviewed a total of 18 organizations, consisting of five NASA Centers, five industry organizations, four defense services organizations, and four university or university R and D laboratory organizations. A software assurance representative also participated in each of the interviews to focus on assurance and software safety best practices. Interviewees provided a wealth of information on each topic area that included: software policy, software acquisition, software assurance, testing, training, maintaining rigor in small projects, metrics, and use of the Capability Maturity Model Integration (CMMI) framework, as well as a number of special topics that came up in the discussions. NASA's software engineering practices compared favorably with the external organizations in most benchmark areas, but in every topic, there were ways in which NASA could improve its practices. Compared to defense services organizations and some of the industry organizations, one of NASA's notable weaknesses involved communication with contractors regarding its policies and requirements for acquired software. One of NASA's strengths was its software assurance practices, which seemed to rate well in comparison to the other organizational groups and also seemed to include a larger scope of activities. An unexpected benefit of the software benchmarking study was the identification of many opportunities for collaboration in areas including metrics, training, sharing of CMMI experiences and resources such as instructors and CMMI Lead Appraisers, and even sharing of assets such as documented processes. A further unexpected benefit of the study was the feedback on NASA practices that was received from some of the organizations interviewed. From that feedback, other potential areas where NASA could improve were highlighted, such as accuracy of software cost estimation and budgetary practices. The detailed report contains discussion of the practices noted in each of the topic areas, as well as a summary of observations and recommendations from each of the topic areas. The resulting 24 recommendations from the topic areas were then consolidated to eliminate duplication and culled into a set of 14 suggested actionable recommendations. This final set of actionable recommendations, listed below, are items that can be implemented to improve NASA's software engineering practices and to help address many of the items that were listed in the NASA top software engineering issues. 1. Develop and implement standard contract language for software procurements. 2. Advance accurate and trusted software cost estimates for both procured and in-house software and improve the capture of actual cost data to facilitate further improvements. 3. Establish a consistent set of objectives and expectations, specifically types of metrics at the Agency level, so key trends and models can be identified and used to continuously improve software processes and each software development effort. 4. Maintain the CMMI Maturity Level requirement for critical NASA projects and use CMMI to measure organizations developing software for NASA. 5.onsolidate, collect and, if needed, develop common processes principles and other assets across the Agency in order to provide more consistency in software development and acquisition practices and to reduce the overall cost of maintaining or increasing current NASA CMMI maturity levels. 6. Provide additional support for small projects that includes: (a) guidance for appropriate tailoring of requirements for small projects, (b) availability of suitable tools, including support tool set-up and training, and (c) training for small project personnel, assurance personnel and technical authorities on the acceptable options for tailoring requirements and performing assurance on small projects. 7. Develop software training classes for the more experienced software engineers using on-line training, videos, or small separate modules of training that can be accommodated as needed throughout a project. 8. Create guidelines to structure non-classroom training opportunities such as mentoring, peer reviews, lessons learned sessions, and on-the-job training. 9. Develop a set of predictive software defect data and a process for assessing software testing metric data against it. 10. Assess Agency-wide licenses for commonly used software tools. 11. Fill the knowledge gap in common software engineering practices for new hires and co-ops.12. Work through the Science, Technology, Engineering and Mathematics (STEM) program with universities in strengthening education in the use of common software engineering practices and standards. 13. Follow up this benchmark study with a deeper look into what both internal and external organizations perceive as the scope of software assurance, the value they expect to obtain from it, and the shortcomings they experience in the current practice. 14. Continue interactions with external software engineering environment through collaborations, knowledge sharing, and benchmarking.

Rarick, Heather L.↗

NASA NTRS1997

IV and V Issues in Achieving High Reliability and Safety in Critical Control System Software

Risk analysis and integrated verification and validation are two important elements in a plan for ensuring the safety of critical software systems. We describe an approach we are currently developing for integrating risk analysis, and metrics analysis, and propose a fault predictor that would integrate the results of these activities. Practical difficulties associated with our approach are also discussed, as are limitations of the proposed predictor. We conclude with a discussion of what as been learned to date, and with suggestions for future work.

validation software reliability software risk asse↗

NASA NTRS2018

Serious Gaming for Building a Basis of Certification via Trust and Trustworthiness of Autonomous Systems

Autonomous systems governed by a variety of adaptive and nondeterministic algorithms are being planned for inclusion into safety-critical environments, such as unmanned aircraft and space systems in both civilian and military applications. However, until autonomous systems are proven and perceived to be capable and resilient in the face of unanticipated conditions, humans will be reluctant or unable to delegate authority, remaining in control aided by machine-based information and decision support. Proving capability, or trustworthiness, is a necessary component of certification. Perceived capability is a component of trust. Trustworthiness is an attribute of a cyber-physical system that requires context-driven metrics to prove and certify. Trust is an attribute of the agents participating in the system and is gained over time and multiple interactions through trustworthy behavior and transparency. Historically, artificial intelligence and machine learning systems provide answers without explanation - without a rationale or insight into the machine “thinking”. In order to function as trusted teammates, machines must be able to explain their decisions and actions. This transparency is a product of both content and communication. NASA’s Autonomy Teaming & TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR) project seeks to build a basis for certification of autonomous systems via establishing metrics for trustworthiness and trust in multi-agent team interactions, using AI (Artificial Intelligence) explainability and persistent modeling and simulation, in the context of mission planning and execution, with analyzable trajectories. Inspired by Massively Multiplayer Online Role Playing Games (MMORPG) and Serious Gaming, the proposed ATTRACTOR modeling and simulation environment is similar to online gaming environments in which player (aka agent) participants interact with each other, affect their environment, and expect the simulation to persist and change regardless of any individual agent’s active participation. This persistent simulation environment will accommodate individual agents, groups of self-organizing agents, and large-scale infrastructure behavior. The effects of the emerging adaptation and coevolution can be observed and measured to building a basis of measurable trustworthiness and trust, toward certification of safety-critical autonomous systems.

Allen, B. Danette↗

NASA NTRS1985

A software technology evaluation program

A set of quantitative approaches is presented for evaluating software development methods and tools. The basic idea is to generate a set of goals which are refined into quantifiable questions which specify metrics to be collected on the software development and maintenance process and product. These metrics can be used to characterize, evaluate, predict, and motivate. They can be used in an active as well as passive way by learning form analyzing the data and improving the methods and tools based upon what is learned from that analysis. Several examples were given representing each of the different approaches to evaluation. The cost of the approaches varied inversely with the level of confidence in the interpretation of the results.

Novaes-Card, David N.↗

NASA NTRS2024

Flight Mechanics Modeling and Simulation of the Earth Entry System

Introduction: The Mars Sample Return (MSR) Campaign being planned by NASA and ESA has the ambitious goal to return Mars samples back to Earth. This international collaboration had developed a concept of operations that included a ESA-designed Earth Return Orbiter (ERO) and NASA-designed Capture, Containment, and Return System (CCRS). The Earth Entry System (EES), consisting of a protective aeroshell that houses the samples as well as sample containment vessels, would conduct entry, descent, and landing (EDL) on a direct Earth trajectory. The EES would enter on a spin-stabilized ballistic trajectory with the goal to passively achieve aerodynamic stability throughout all regions of flight. The EDL sequence would end with the EES impacting the soft playa soil of the Utah Test and Training Range (UTTR). As of the submission of this abstract, the MSR campaign is undergoing a re-architecture leading to a pause in EES development. However, the novel approaches developed in flight mechanics modeling and simulation can significantly benefit the greater IPPW community in the development of Earth return vehicles. This paper will present the latest state of EES flight mechanics modeling and simulation. The paper will highlight the simulation architecture developed and key lessons learned from understanding of EDL trajectory sensitivities. Modeling and Simulation: Figure 1 provides a high-level concept of operations for the approach, entry, descent, and landing (AEDL) phase of the CCRS-portion of MSR. The objective of EES flight mechanics is to model and simulate the EES trajectory from ERO separation to ground impact at UTTR. A variety of flight mechanics simulation models were utilized to model both exo-atmopsheric and atmospheric portions of flight. 42, a 6-DOF simulation developed at Goddard Space Flight Center, is utilized for propagating the attitude of EES during exo-atmospheric flight. 42 allows for a variety of spin eject mechanism scenarios to be simulated for analysis. 10 minutes prior to entry, the 42 states are handed off to the EDL sims. The prime EDL sim utilized by EES is the Program to Optimize Simulated Trajectories II (POST2), a 6-DOF sim developed at Langley Research Center, and the independent verification and validation EDL sim utilized is DSENDS, a 6-DOF sim developed at Jet Propulsion Laboratory. Figure 2 provides a visualization of the flight mechanics simulation model flow through various points in the AEDL phase. Due to the existence of a variety of sim models, the EES flight mechanics team developed processes for data hand-off. These processes included the development of a centralized coordinate frame document, utilization of a single, centralized simulation input document for all sims to reference, and hand-off files containing both the technical data to be ingested by other flight mechanics sims as well as annotations of modeling assumptions utilized to generate the data. Figure~\ref{fig:post2simarchitecture} provides an overview of the POST2 sim architecture wherein POST2 ingests numerous subsystem models and input files. The dispersed state file generated by MONTE provides the position/velocity state of the trajectory while the 42 Handoff file provides the attitude. The aerodynamics database, delivered by the EES aeroscience team, is utilized to simulate the aerodynamic forces and moments experienced during EDL. A custom atmosphere model, developed by EES atmosphere team, is utilized to simulate the anticipated atmosphere environment around the region of Earth through which the EES trajectory flys. These inputs and subsystem models can be varied depending on the AEDL flight mechanics scenario being simulated. Monte Carlo simulations are utilized to generate statistical AEDL performance metrics in the form of scorecards and violin plots. Furthermore, outputs from the POST2 simulation are utilized for follow-on analyses including aerothermal and landing performance. \section{Flight Mechanics Lessons Learned} Though the EES flight mechanics team uncovered a variety of lessons learned through the analysis conducted to support CCRS through preliminary design review, this paper will highlight the most important lessons. A key AEDL performance goal is to ensure the landing footprint of EES remains on the UTTR south range. A common modeling strategy used in EDL analysis is One-Variable-At-a-Time (OVAT). OVAT analysis provides insight into the key drivers that affect AEDL performance metrics. Figure 3 shows the landing ellipses for single dispersion sources as compared to the baseline aggregate of all dispersions. The figure shows that atmosphere winds alone dominate the size of the footprint ellipse (note: EES does not use a parachute unlike previous Earth-return missions and is in wind-driven free fall for ~5min). The significance of the wind led the EES flight mechanics team to pursue the development of a Custom Atmosphere Model [4], in lieu of EarthGRAM [1], built on actual radiosonde wind measurements around the UTTR-region. This decision was driven by the realism in the generated footprint ellipses and lessons-learned from Stardust [5]. These findings will be invaluable for future Earth-return missions in providing an early understanding of the key drivers affecting footprint size and modeling considerations for which to account. Another lesson learned is tied to the AEDL performance goal of achieving passive stability throughout all regions of flight. It is well understood that blunt-body aeroshells are less stable as they transition from supersonic to subsonic. Eliminating a backshell does help improvestability; however, other phenomena such as roll-induced instability during terminal descent can still arise. The EES flight mechanics team developed stability metrics as tools to better understand the causes of and better predict the onset of dynamic instability. These tools were built upon analytical models developed by Jaffe [3] and Murphy [2]. The tools were shown to both be very accurate in correlation with actual unstable cases and useful in developing stability margin policies based on the vehicle design and simulation considerations (e.g. sphere-cone angle change, mass change, wind turbulence). These tools allowed for the current EES design to demonstrate the ability to achieve passive stability and can be an invaluable tool for consideration in the design of parachute-less Earth-return vehicles.

Rohan Deshmukh↗

NASA NTRSDate not supplied

Benchmark Problem Development for Testing Maturity of Intelligent Contingency Management Tools

This paper presents a process used to develop appropriate scenarios and metrics for evaluating the maturity of intelligent contingency management algorithms. A benchmark scenario is a reference point against which something can be measured, compared, or assessed. Creating an accurate benchmark requires considerable research and expertise. The scenario itself is an artificial representation of a real-world event, designed to achieve a set of learning objectives through experiential learning. Designing an effective benchmark simulation scenario requires careful planning, including identification of clear objectives; capability assessment of the algorithm/tool being evaluated; assessment of necessary levels of fidelity; development of a process flow map of events and event interactions; and identification of metrics that map back to objectives. Thus, a benchmark scenarios for contingency management might consist of one or several commonly used functions taken from real world applications, used for evaluation, characterization and performance measurement of a contingency management algorithm. Behavior of the contingency management algorithm under different environmental conditions should then be able to be predicted using a set of benchmark functions. The paper describes the resulting benchmark problem as an illustration of the application of this process.

Jon Holbrook↗

NASA NTRSDate not supplied

A Novel Machine Learning Method for Surface PM2.5 Estimations from Geostationary Satellites

Particulate matter (PM) with a diameter of less or equal to 2.5 μm, known as PM , affects human health as it penetrates the respiratory system. The Environmental Protection Agency (EPA) measures the atmospheric concentration of PM using air quality monitors stationed throughout the Continental United States (CONUS). Such measurements are points on a spatial domain and therefore, might not be representative of the air quality at nearby areas considering that the composition of the atmosphere is highly variable from place to place. Satellite based AOD permits a spatially uniform means of estimating PM and new geostationary satellites provide high temporal and spatial resolution estimation of AOD. However, the concentration of PM is non-linearly dependent on other atmospheric parameters that include relative humidity, temperature, and height of the planetary boundary layer. This information may be estimated at similar spatial and temporal resolutions as AOD from numerical modeling such as from the National Oceanic and Atmospheric Administration’s (NOAA) High Resolution Rapid Refresh (HRRR) model which resolves near real-time atmospheric conditions over the CONUS. The estimation of PM concentration is a multi-parametric problem that considers the effect of temporal dependencies among the different parameters. Deep learning approaches are appropriate for such complex estimation problems as they intrinsically capture relations among multiple non-linear parameters. This study compares deep-learning methods to traditional regression analysis to demonstrate the capabilities of these methods in predicting PM2.5 concentrations. Additionally, a novel ensemble learning approach is employed to identify scientific processes that could further improve the estimation of PM concentration. Utilizing Long Short-Term Memory (LSTM) neural networks, which are suitable for multivariate time series estimation problems as they are capable of learning long-term dependencies, individual models are created for each EPA station and trained on the aforementioned dataset collocated over each station. Individual station models are merged if the model's performance is improved by reducing the root mean squared error (RMSE) metric. This ensemble training method ultimately reduces the RMSE value. Evaluation of these results provide insights into physical processes and related observable parameters that may contribute to PM concentrations. Identified parameters evaluated to be statistically different between the merged and unmerged models are expected to improve overall performance. These new parameters are then utilized for reevaluation of the deep learning methods with an extreme gradient boosting model with an RMSE of 5.5 providing the best results.

George Priftis↗