Search NASA⌕ Search

SEARCH · Search NASA

Results for “Critical Benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

187 records · Page 11

Unsupervised Anomaly Detection in High-Dimensional Flight Data Using Convolutional Variational Auto-Encoder

The modern National Airspace System (NAS) is an extremely safe system and the aviation industry has experienced a steady decrease in fatalities over the years. This can be attributed to both improved flight critical systems with redundant hardware and software protections, as well as an increased focus on active monitoring and response to real time and historically identified vulnerabilities by implementing more resilient procedures and protocols. The main approach for identifying vulnerabilities in operations leverages domain expertise using knowledge about how the system should behave within the expected tolerances to known safety margins. This approach works well when the system has a well-defined operating condition. However, the operations in the NAS can be highly complex with various nuances that render it difficult to clearly pre-define all known safety vulnerabilities. With the advancement of data science and machine learning techniques, the potential to automatically identify emerging vulnerabilities in the observed operations has become more practical in recent years. The state-of-the-art anomaly detection approaches in aerospace data usually rely on supervised or semi-supervised learning. However, in many real-world problems such as flight safety, creating labels for the data requires huge amount of effort and is largely impractical. To address this challenge, we developed a Convolutional Variational Auto-Encoder (CVAE), which is an unsupervised learning approach for anomaly detection in high-dimensional heterogeneous time-series data. We validate performance of CVAE compared to the state-of-the-art supervised learning approach as well as unsupervised clustering-based approach using KMeans++ and kernel-based approach using One-Class Support Vector Machine (OC-SVM) on Yahoo!'s benchmark time series anomaly detection data. Finally, we showcase performance of CVAE on a case study of identifying anomalies in the first 60 seconds of commercial flights' take-offs using Flight Operational Quality Assurance (FOQA) data.

Memarzadeh, Milad↗

Objective Structured Clinical Evaluation (OSCE) of an Artificial Intelligence (AI) Clinical Decision Support System (CDSS) Tool

BACKGROUND Objective Structured Clinical Evaluations (OSCEs) have long been established as a robust methodology for summative assessment of clinical skills and decision-making during medical education. The recent integration of Artificial Intelligence (AI) into clinical decision-making processes has prompted the need for novel evaluation frameworks to assess the efficacy and reliability of AI clinical decision support system (CDSS) tools. This abstract outlines the process of quantitatively evaluating a novel CDSS (“Doc in a Box” Google 2024) trained on curated medical spaceflight data in the psychomotor domain as it interfaces with a human volunteer acting as the crew medical officer (CMO). PURPOSE The AI CDSS under review was developed as part of the Lunar Command and Control Interoperability (LuCCI) project, which is intended to address a gap in how Lunar Surface Systems (LSS) would interoperate across multiple programs, commercial partners, and international partners. The project objective is to define, prototype, integrate, and evaluate an interoperable lunar command, control, data, and software reference architecture to enable autonomy and informatics capability through common standards across LSS. A multi-modal AI-based CDSS compatible with Federated LSS will assist clinicians in diagnosing and managing complex medical conditions by providing evidence-based recommendations through predictive analytics. Given the critical role of decision-support as NASA continues to evolve its Earth-independent medical operations (EIMO), it is imperative to ensure that such AI tools perform reliably and align with clinical standards during progressive lunar and Martian exploration class missions. METHODS The OSCE framework, traditionally used for evaluating human clinicians, was adapted to assess the AI tool's decision-making capabilities in simulated clinical scenarios. In this adapted OSCE, the AI CDSS was tested across a series of structured clinical scenarios designed to mimic real-life spaceflight patient cases. These scenarios included a range of conditions and complexities, allowing for comprehensive assessment of the tool's performance. Key evaluation metrics included accuracy of diagnosis, timeliness of decision-making, and appropriate recommendations for therapies. The OSCE was scored by human physician evaluators who assessed the AI's recommendations in comparison with expert clinicians' medical decision making to ensure alignment with best practices and the standard of care. RESULTS Preliminary results indicate that the AI CDSS demonstrated high accuracy in diagnostic recommendations and decision support across various scenarios. However, certain limitations were noted, such as occasional discrepancies in handling complex or nuanced cases that required a more contextual understanding. Additionally, the tool scored higher on the diagnostic portion of the rubric, with lower scores in the therapeutic recommendations. These findings highlight the importance of continuous refinement and validation of AI tools through rigorous evaluation frameworks like the OSCE. The adaptation of OSCEs for AI tools presents several advantages, including a structured and reproducible approach to evaluation, the ability to test AI systems in diverse clinical scenarios, and the opportunity to benchmark AI performance against established clinical standards to permit charting of future progress as aerospace medicine evolves as a discipline. Remaining challenges include ensuring that these evaluations capture the full spectrum of clinical decision-making scenarios that will be confronted by CMOs during missions and adequately reflecting real-world variability of the austere spaceflight environment. CONCLUSION Employing OSCEs to evaluate AI clinical decision support tools offers a promising approach to validating their clinical utility and efficacy. This methodology not only provides insights into the tool's performance but also fosters ongoing improvement and alignment with standard of care practices. Future research should focus on refining these evaluation processes and addressing limitations to enhance the integration of AI tools in clinical spaceflight settings. REFERENCES Scott S, Hearns V, Barker MA. Testing Clinical Skills: A Look at the OSCE and USMLE Clinical Skills Exams. S D Med. 2019 Oct;72(10):451-453. Majumder MAA, Kumar A, Krishnamurthy K, Ojeh N, Adams OP, Sa B. An evaluative study of objective structured clinical examination (OSCE): students and examiners perspectives. Adv Med Educ Pract. 2019 Jun 5;10:387-397. Karam VY, Park YS, Tekian A, Youssef N. Evaluating the validity evidence of an OSCE: results from a new medical school. BMC Med Educ. 2018 Dec 20;18(1):313.

Ariana M Nelson↗

Risky Business

During my internship I worked on two major projects, recommending improvements for the Center's Risk Management Workshop and helping with the strategic planning efforts for Safety and Mission Assurance (S&MA). The risk management improvements is the key project I worked on this semester through my internship, while the strategic planning is the secondary assignment. S&MA Business Office covers both aspects in its delegation, getting both spans some of the work done in the office. A risk is a future event with a negative consequence that has some probability of occurring. Safety and Mission Assurance identifies, analyzes, plans, and tracks risk. The directorate offers the Center a Risk Management Workshop, and part of the ongoing efforts of S&MA is to make continuous improvements to the RM Workshop. By using the Project Management Institute's (PMI) Standard for Risk Management, I performed a gap analysis to make improvements for our materials. I benchmarked the PMI's Risk Management Standard, compared our Risk Management Workshop materials to PMI's standard, and identified any gaps in our material. My major findings were presented to the Business Office of S&MA for a decision on whether or not to incorporate the improvements. These suggestions were made by attending JSC working group meetings, Health, Safety and Environment (HSE) panel reviews and various risk review meetings. The improvements provide better understanding of risk management processes and enhanced risk tracking knowledge and skills. Risk management is an integral part of any engineering discipline, getting exposed to this section of engineering will greatly help shape my career in the future. Johnson Space Center is a world leader in risk management processes; learning risk management here gives me a huge advantage over my peers, as well as understanding decision making in the context of risk management will help me to be a well-rounded engineer. Strategic planning is an area I had not previously studied. Helping with the strategic planning efforts in S&MA has taught me how organizations think and function as a whole. S&MA is adopting a balanced scorecard approach to strategic planning. As part of this planning method strategic themes, objectives, and initiatives are formed. I attended strategic theme team workshops that formed the strategy map for the directorate and gave shape to the plan. Also during these workshops the objectives were discussed and built. Learning the process for strategic planning has helped me better understand how organizations and businesses function, which also helps me to be a more effective employee. Other assignments I had during my internship included completing the Safety and Mission Assurance Technical Excellent Program (STEP) Level 1, as well as doing a two week rotation through the Space Exploration division in S&MA, specifically working with a thermal protection systems (TPS) engineer. While working there, I learned about the Orion capsule and the SpaceX Dragon cargo capsule. I attended meetings to prepare the engineers for the upcoming Critical Design Reviews for both capsules and reviewed test data. Learning risk management, strategic planning, and working in the Space Exploration division has taught me about many aspects of S&MA. My internship at NASA has given me new experiences and taught me numerous subjects that I would have otherwise not learned. This opportunity has expanded my educational horizons and is helping me to become a more useful engineer and employee.

Yarbrough, Katherine↗

The Determination of Forces and Moments on a Gimballed SRM Nozzle Using a Cold Flow Model

The Solid Rocket Motor Air Flow Facility (SAF) at NASA Marshall Space Flight Center was used to characterize the flow in the critical aft end and nozzle of a solid propellant rocket motor (SRM) as part of the design phase of development. The SAF is a high pressure, blowdown facility which supplies a controlled flow of air to a subscale model of the internal port and nozzle of a SRM to enable measurement and evaluation of the flow field and surface pressure distributions. The ASRM Aft Section/Nozzle Model is an 8 percent scale model of the 19 second burn time aft port geometry and nozzle of the Advanced Solid Rocket Motor, the now canceled new generation space Shuttle Booster. It has the capability to simulate fixed nozzle gimbal angles of 0, 4, and 8 degrees. The model was tested at full scale motor Reynolds Numbers with extensive surface pressure instrumentation to enable detailed mapping of the surface pressure distributions over the nozzle interior surface, the exterior surface of the nozzle nose and the surface of the simulated propellant grain in the aft motor port. A mathematical analysis and associated numerical procedure were developed to integrate the measured surface pressure distributions to determine the lateral and axial forces on the moveable section of the nozzle, the effective model thrust and the effective aerodynamic thrust vector (as opposed to the geometric nozzle gimbal angle). The nozzle lateral and axial aerodynamic loads and moments about the pivot point are required for design purposes and require complex, three dimensional flow analyses. The alignment of the thrust vector with the nozzle geometric centerline is also a design requirement requiring three dimensional analyses which were supported by this experimental program. The model was tested with all three gimbal angles at three pressure levels to determine Reynolds number effects and reproducibility. This program was successful in demonstrating that a measured surface pressure distribution could be integrated to determine the lateral and axial loads, moments and thrust vector alignment for the scaled model of a large space booster nozzle. Numerical results were provided which are scaleable to the full scale rocket motor and can be used as benchmark data for 3-D CFD analyses.

Whitesides, R. Harold↗

3D Material Response of the MSL Heatshield Using NuSil-Coated PICA

The Mars Science Laboratory (MSL) was protected during its atmospheric entry by an instrumented heatshield that used NASA's Phenolic Impregnated Carbon Ablator (PICA) material [1]. PICA is a lightweight carbon fiber/polymeric resin material that offers outstanding performance for protecting probes during planetary entry. Data from the Mars Entry Descent and Landing Instrument (MEDLI) suite on MSL offers unique in-flight validation data for models of material response and atmospheric entry. MEDLI recorded, among other things, time-resolved in-depth temperature data of PICA using thermocouple sensors assembled in the MEDLI Integrated Sensor Plugs (MISP) [2]. A space-grade silicone-based coating commercially known as NuSil CV-1144-0 [3] was applied to the entire MSL heatshield, including the MEDLI plugs, to mitigate the spread of dust from PICA. Modeling the thermal response of PICA-NuSil (PICA-N) system is still an open challenge. Ground testing of PICA-N models exhibited surface temperature jumps of the order of 150 K due to oxide scale formation and sub-sequent NuSil burn-off. It is therefore critical to include a validated model for the material response of the coating in engineering codes. A test campaign has been conducted at the NASA’s Langley HyMETS [4] facility to screen the response of PICA-N and gather detailed data on its behavior [5]. A first model of PICA-N thermal response has been developed using the Hy-METS experiments [6]. The objective of this work is to analyze the material response of the latest PICA-N model compared to the engineering model used to simulate the entry of MSL. The environment and material response around the MSL aeroshell during Mars atmospheric entry is simulated using a collection of tools. The Direct Simulation Monte Carlo SPARTA code [7] is used in the rarefied regime, the Data Parallel Line Relaxation (DPLR) code [8] is used in the continuum regime and radiative heating conditions are provided by the Nonequilibrium air radiation (NEQAIR) code [9] to estimate the environmental conditions. The thermal response inside the material is computed using the Porous material Analysis Toolbox based on Open-FOAM (PATO) [10,11,12]. Thermodynamic and chemistry properties are estimated using the Mutation++ library [13]. The approach implemented in PATO as a first cut PICA-N thermal response model is outlined in Figure 1. While the recession is less than the coating thickness, the Surface mass and energy balance Boundary Condition (SBC) uses the NuSil B’ tables. Once the recession removes the coating, the usual PICA B’ tables are used for the SBC. The B’ tables are computed using an equilibrium solver implemented in Mutation++, given the temperature, pressure, blowing rate, composition of the pyrolysis and environment gases, and the condensed species at the surface. Preliminary results of the 3D material response of the MSL heat-shield at the peak heating (80 sec after Entry Interface) are shown in Figure 2. Current NASA’s mission to Mars, Mars 2020, used the spare heatshield of MSL for thermal protection during entry, descent, and landing. In preparation for Mars 2020 post-flight analysis, the PATO high-fidelity material response capability was benchmarked against flight data from MEDLI. This effort represents an important milestone toward the development of validated predictive capabilities for designing thermal protection systems for planetary probes. This bench-marking is awaiting the final release of the MEDLI-2 data.

Aerospace↗

Feeding Ten Billion People Is Possible Within Four Terrestrial Planetary Boundaries

Global agriculture puts heavy pressure on planetary boundaries, posing the challenge to achieve future food security without compromising Earth system resilience. On the basis of process-detailed, spatially explicit representation of four interlinked planetary boundaries (biosphere integrity, land-system change, freshwater use, nitrogen flows) and agricultural systems in an internally consistent model framework, we here show that almost half of current global food production depends on planetary boundary transgressions. Hotspot regions, mainly in Asia, even face simultaneous transgression of multiple underlying local boundaries. If these boundaries were strictly respected, the present food system could provide a balanced diet (2,355 kcal per capita per day) for 3.4 billion people only. However, as we also demonstrate, transformation towards more sustainable production and consumption patterns could support 10.2 billion people within the planetary boundaries analysed. Key prerequisites are spatially redistributed cropland, improved water–nutrient management, food waste reduction and dietary changes. Adoption of the Sustainable Development Goals by all nations in 2015 is the first ever commitment to a world development path that safeguards the stability of the Earth system as a prerequisite for meeting universal human standards1. The longstanding challenge of achieving food security through sustainable agriculture is particularly acute in this context as world agriculture is a leading cause for the current transgressions of multiple planetary boundaries (PBs) globally and regionally2–5. The PB framework is a comprehensive scientific attempt to synoptically define our planet’s biogeophysical limits to anthropogenic interference. It suggests bounds to nine interacting processes that together delineate a Holocene-like Earth system state. The Holocene is chosen as the reference state as it is the only period known to provide a safe operating space for a world population of several billion people, and according to a precautionary principle, the PBs are set in sufficient distance from processes that may critically undermine Earth system resilience and global sustainability. A challenging question, thus, is whether human development goals such as food security can be met while maintaining multiple PBs along with their subglobal manifestations. Further PB transgressions could jeopardize the chances of providing sufficient food for a world population projected to be wealthier and reach >9 billion by 2050. This conundrum portrays a tradeoff between Earth’s biophysical carrying capacity and humankind’s rising food demand, calling in response for radical rethinking of food production and consumption patterns6–9. Yield gap closures, avoidance of excessive input use, shifts towards less resource-demanding diets, food waste reductions and efficient international trade are crucial options for sustainably increasing the food supply10–15. For example, enhancing water-use efficiency on irrigated and rain-fed farms can triple or quadruple crop yields in low-performing systems, suggesting possible global gains of >20% (ref. 16). Even higher gains appear feasible through globally optimized configurations of the land-use pattern17, and cutting food losses by half could generate food for another billion people18. Thus, collective large-scale implementation of such options could sustain food for a further growing world population19. Yet achieving this within a safe operating space as defined by PBs requires not only a halt to but actually a reversal of existing PB transgressions. Previous studies suggest that such a reconciliation might be possible, but these were based on aggregate representations of PBs (not accounting for the spatial patterns of limits, transgressions and interactions) or considered only one boundary in isolation17,20–23. Here, we systematically quantify to what extent current food production depends on local to global transgressions of the PBs for biosphere integrity, land-system change, freshwater use and nitrogen (N) flows, along with the potential of a range of solutions to avoid these transgressions and still increase food supply (Table 1). To this end, we configured an internally consistent process-based model of the terrestrial biosphere including agriculture (LPJmL) with multiple spatially distributed PBs and their interactions. LPJmL is among the longest-established and best-evaluated biosphere models, showing robust performance regarding simulation of, for example, carbon, water and crop yield dynamics (Supplementary Figs. 1 and 2 and Supplementary Table 1; see ref. 24 for a comprehensive benchmarking and Supplementary Methods for more detail on model evaluations). In principle following established definitions4, we refine the computation of some PBs with respect to their regional patterns and interactions (Methods), providing globally gridded precautionary limits to human interference with the Earth system at a level of great detail. In particular, we account for the evidence that many PBs need to be represented spatially explicitly4 to cover their

Gerten, Dieter↗

Wildfire Segmentation From Remotely Sensed Data Using Quantum-Compatible Conditional Vector Quantized-Variational Autoencoders

Wildfires represent a critical environmental hazard with multifaceted implications for ecosystems, communities, and public health [1]. The escalating frequency and intensity of wildfires globally have intensified the urgency for robust segmentation methodologies to facilitate effective mitigation, response, and recovery strategies [2]. Accurate wildfire segmentation is pivotal for delineating fire boundaries, assessing progression patterns, and prioritizing resource allocation during emergency scenarios. Furthermore, precise segmentation enables stakeholders, including policymakers, environmental scientists, and emergency responders, to formulate evidence-based strategies, thereby minimizing socio-economic disruptions and ecological degradation. Consequently, advancing wildfire segmentation techniques through innovative technological interventions remains a paramount research imperative. Although foundational in wildfire segmentation, traditional deterministic models exhibit inherent limitations that compromise their efficacy in dynamic and uncertain environments. These models often operate on rigid algorithms prioritizing deterministic classifications, thereby overlooking the inherent complexities and uncertainties associated with wildfire behavior and satellite data variability. Such deterministic frameworks tend to produce oversimplified representations that fail to capture the intricate nuances of evolving fire dynamics, spatial heterogeneity, and environmental interactions [1]. Consequently, the deterministic approach’s propensity for uncertainty collapsing [1, 3] hampers the accuracy, reliability, and applicability of segmentation outcomes in real-world scenarios. Contrastingly, stochastic models offer a more nuanced and adaptable framework for wildfire segmentation. By integrating probabilistic elements into the modeling paradigm, stochastic approaches, particularly probabilistic approaches such as variational auto encoders (VAEs) [4], facilitate comprehensive uncertainty assessment, enabling researchers to quantify and incorporate uncertainties into segmentation outcomes effectively. This probabilistic nature empowers stochastic models to encapsulate variability, account for data inconsistencies, and adapt to evolving environmental conditions, enhancing segmentation accuracy, reliability, and robustness. Embracing stochastic methodologies thus catalyzes advancements in wildfire science by fostering a more holistic, adaptive, and resilient segmentation framework. Despite VAEs demonstrating significant promise in various applications, they come with inherent limitations that have garnered attention within the machine learning community. One of the primary drawbacks lies in their reliance on static priors, which essentially assume a fixed distribution for latent variables, thereby limiting the model’s flexibility to capture complex data structures effectively [5]. This static nature leads to suboptimal representations, especially when dealing with complex and high-dimensional data. Additionally, VAEs often struggle with generating sharp and realistic samples, a phenomenon commonly referred to as mode collapse [5, 7, 6]. Furthermore, the optimization process in VAEs, which involves balancing the reconstruction loss and the regularization term, can sometimes be challenging to fine-tune [7]. In recent efforts to address these shortcomings, alternative approaches like Vector Quantized Variational Auto encoders(VQ-VAEs) [7], address the challenges by incorporating discrete latent variables and leveraging techniques that enhance the quality and diversity of generated samples while maintaining efficient training dynamics. VQ-VAEs propose a dynamic prior distribution generation mechanism that diverges from the static priors commonly associated with traditional VAEs. This dynamic approach allows for more adaptive and context-aware latent variable representations, thereby potentially capturing complex data structures more effectively. Unlike autoregressive prior models such as PixelCNN, which, despite their ability to model dependencies across data dimensions, suffer from significant computational inefficiencies and lack flexibility in handling diverse datasets. In our work, we propose to use a generative quantum-compatible approach to help alleviate the shortcomings of autoregressive prior model in VQ-VAEs. Restricted Boltzmann Machines (RBMs) are a viable alternative prior model that can learn prior distributions in a faster and more flexible manner. In this research endeavor, we meticulously curate a state-of-the-art dataset leveraging satellite MODIS data in conjunction with VIIRS fire masks, derived from Fire Radiative Power (FRP), thereby encapsulating diverse wildfire scenarios and environmental contexts. We developed a conditional VQ-VAE architecture with the RBM prior model that is trained in a supervised manner for segmenting wildfire masks. This innovative approach synergistically harnesses deep learning capabilities, enabling the generation of segmentation maps characterized by heightened precision, granularity, and contextual relevance. Furthermore, replacing the autoregressive prior learning method proposed by the original VQ-VAE with a prior density approximation via quantum-compatible RBM facilitates expedited inference processes, augments flexibility in prior sampling, optimizes computational efficiency and establishes a groundbreaking benchmark in wildfire segmentation methodologies.

quantum machine learning↗