Search NASA⌕ Search

SEARCH · Search NASA

Results for “AI Evaluations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS1985

Evaluation of AIS Data for Agronomic and Rangeland Vegetation: Preliminary Results for August 1984 Flight over Nebraska Sandhills Agricultural Laboratory

Since 1978 scientists from the Center for Agricultural Meteorology and Climatology at the University of Nebraska have been conducting research at the Sandhills Agricultural Laboratory on the effects of water stress on crop growth, development and yield using remote sensing techniques. We have been working to develop techniques, both remote and ground-based, to monitor water stress, phenological development, leaf area, phytomass production and grain yields of corn, soybeans and sorghum. Because of the sandy soils and relatively low rainfall at the site it is an excellent location to study water stress without the necessity of installing expensive rainout shelters. The primary objectives of research with the airborne imaging spectrometer (AIS) data collected during an August 1984 flight over the Sandhills Agricultural Laboratory are to evaluate the potential of using AIS to: (1) discriminate crop type; (2) to detect subtle architectural differences that exist among different cultivars or hybrids of agronomic crops; (3) to detect and quantify, if possible, the level of water stress imposed on the crops; and (4) to evaluate leaf area and biomass differences for different crops.

Blad, B. L.↗
NASA NTRS1985

Preliminary Evaluation of AIS Spectra Along a Topographic/moisture Gradient in the Nebraska Sandhills

Six spectral plots, each summarizing single-pixel reflectance for 128 channels of Airborne Imaging Spectrometer (AIS) data, were examined. The six sample pixels were located along a topographic/moisture gradient from lake surface to dune top in the Nebraska Sandhills. AIS spectra for various moisture regimes/vegetative zones appear quite logical, with a general positive relationship between increasing elevation (i.e., decreasing access of plant roots to water) and increasing reflectance in the spectral regions diagnostic of leaf-water content (i.e., bands centered on 1.65 and 2.20 microns).

Runquist, D. C.↗
NASA NTRS1987

Evaluation of AIS-2 (1986) data over hydrothermally altered granitoid rocks of the Singatse Range (Yerington) Nevada and comparison with 1985 AIS-1 data

The Airborne Imaging Spectrometer-2 (AIS-2) flights along 2 subparallel lines (bearing 013) were designed to traverse 3 major rock assemblages - the Triassic sedimentary sequence; the granitoid rocks of the Yerington batholith and the Tertiary ignimbritic ash flow and ash fall tuffs. The first 2 sites are hydrothermally altered to a quartz-sericite-tourmaline mineralogy. The first AIS-2 data set showed numerous line dropouts and a considerable number of randomly distributed dark pixels. A second decommutation reduced the dropout essentially to near zero and the dark pixels by about 75 percent. Vertical striping was removed by histogram matching, column by column. A log residual spectrum was calculated which showed the departure of a 2 x 2 pixel area from the spatially and spectrally averaged scene. A 1:1 correlation was found with the log residual AIS-2 data and a large open pit area of gypsum. An area with known sericite agreed with the overflight data, and an area known to be free of any significant amount of O-H bearing materials showed no evidence of any in the AIS-2 log residuals.

Lyon, R. J. P.↗
NASA NTRS2004

Approach for Autonomous Control of Unmanned Aerial Vehicle Using Intelligent Agents for Knowledge Creation

This paper describes the development of a planned approach for Autonomous operation of an Unmanned Aerial Vehicle (UAV). A Hybrid approach will seek to provide Knowledge Generation thru the application of Artificial Intelligence (AI) and Intelligent Agents (IA) for UAV control. The application of many different types of AI techniques for flight will be explored during this research effort. The research concentration will be directed to the application of different AI methods within the UAV arena. By evaluating AI approaches, which will include Expert Systems, Neural Networks, Intelligent Agents, Fuzzy Logic, and Complex Adaptive Systems, a new insight may be gained into the benefits of AI techniques applied to achieving true autonomous operation of these systems thus providing new intellectual merit to this research field. The major area of discussion will be limited to the UAV. The systems of interest include small aircraft, insects, and miniature aircraft. Although flight systems will be explored, the benefits should apply to many Unmanned Vehicles such as: Rovers, Ocean Explorers, Robots, and autonomous operation systems. The flight system will be broken down into control agents that will represent the intelligent agent approach used in AI. After the completion of a successful approach, a framework of applying a Security Overseer will be added in an attempt to address errors, emergencies, failures, damage, or over dynamic environment. The chosen control problem was the landing phase of UAV operation. The initial results from simulation in FlightGear are presented.

Dufrene, Warren R., Jr.↗
NASA NTRS2004

Approach for Autonomous Control of Unmanned Aerial Vehicle Using Intelligent Agents for Knowledge Creation

This paper describes the development of a planned approach for Autonomous operation of an Unmanned Aerial Vehicle (UAV). A Hybrid approach will seek to provide Knowledge Generation through the application of Artificial Intelligence (AI) and Intelligent Agents (IA) for UAV control. The applications of several different types of AI techniques for flight are explored during this research effort. The research concentration is directed to the application of different AI methods within the UAV arena. By evaluating AI and biological system approaches. which include Expert Systems, Neural Networks. Intelligent Agents, Fuzzy Logic, and Complex Adaptive Systems, a new insight may be gained into the benefits of AI and CAS techniques applied to achieving true autonomous operation of these systems. Although flight systems were explored, the benefits should apply to many Unmanned Vehicles such as: Rovers. Ocean Explorers, Robots, and autonomous operation systems. A portion of the flight system is broken down into control agents that represent the intelligent agent approach used in AI. After the completion of a successful approach, a framework for applying an intelligent agent is presented. The initial results from simulation of a security agent for communication are presented.

Dufrene, Warren R., Jr.↗
NASA NTRSDate not supplied

A Systems Approach to AI Model Integration and Performance Evaluation for the Generic UAM Simulation Framework

This paper introduces py-guam, an open-source experimentation framework developed for the NASA Generic Urban Air Mobility simulation (GUAM) environment, facilitating the integration and evaluation of advanced artificial intelligence (AI) algorithms. We present a systems approach which enables the seamless incorporation of data-driven models, including off-nominal and failure state detection, into the GUAM’s Cognitive Architecture (CA). The framework supports customizable experimentation parameters, derives Safety Performance Indicators (SPIs) from UL 4600 safety case analyses, and employs rapid UAM simulations to assess AI impacts on flight performance across diverse scenarios. Through comprehensive testing and validation experiments, we demonstrate GUAM’s capability to enhance safety and efficiency in urban air mobility operations. Additionally, the open-source nature of py-guam fosters community collaboration, ensuring continuous improvement and adaptability to evolving technological advancements. This work establishes a robust tool for developing and testing AI-driven urban air mobility (UAM) systems, advancing the safety and reliability of autonomous urban air vehicles.

Artificial Intelligence↗
NASA NTRSDate not supplied

Objective Structured Clinical Evaluation (OSCE) of an Artificial Intelligence (AI) Clinical Decision Support System (CDSS) Tool

BACKGROUND Objective Structured Clinical Evaluations (OSCEs) have long been established as a robust methodology for summative assessment of clinical skills and decision-making during medical education. The recent integration of Artificial Intelligence (AI) into clinical decision-making processes has prompted the need for novel evaluation frameworks to assess the efficacy and reliability of AI clinical decision support system (CDSS) tools. This abstract outlines the process of quantitatively evaluating a novel CDSS (“Doc in a Box” Google 2024) trained on curated medical spaceflight data in the psychomotor domain as it interfaces with a human volunteer acting as the crew medical officer (CMO). PURPOSE The AI CDSS under review was developed as part of the Lunar Command and Control Interoperability (LuCCI) project, which is intended to address a gap in how Lunar Surface Systems (LSS) would interoperate across multiple programs, commercial partners, and international partners. The project objective is to define, prototype, integrate, and evaluate an interoperable lunar command, control, data, and software reference architecture to enable autonomy and informatics capability through common standards across LSS. A multi-modal AI-based CDSS compatible with Federated LSS will assist clinicians in diagnosing and managing complex medical conditions by providing evidence-based recommendations through predictive analytics. Given the critical role of decision-support as NASA continues to evolve its Earth-independent medical operations (EIMO), it is imperative to ensure that such AI tools perform reliably and align with clinical standards during progressive lunar and Martian exploration class missions. METHODS The OSCE framework, traditionally used for evaluating human clinicians, was adapted to assess the AI tool's decision-making capabilities in simulated clinical scenarios. In this adapted OSCE, the AI CDSS was tested across a series of structured clinical scenarios designed to mimic real-life spaceflight patient cases. These scenarios included a range of conditions and complexities, allowing for comprehensive assessment of the tool's performance. Key evaluation metrics included accuracy of diagnosis, timeliness of decision-making, and appropriate recommendations for therapies. The OSCE was scored by human physician evaluators who assessed the AI's recommendations in comparison with expert clinicians' medical decision making to ensure alignment with best practices and the standard of care. RESULTS Preliminary results indicate that the AI CDSS demonstrated high accuracy in diagnostic recommendations and decision support across various scenarios. However, certain limitations were noted, such as occasional discrepancies in handling complex or nuanced cases that required a more contextual understanding. Additionally, the tool scored higher on the diagnostic portion of the rubric, with lower scores in the therapeutic recommendations. These findings highlight the importance of continuous refinement and validation of AI tools through rigorous evaluation frameworks like the OSCE. The adaptation of OSCEs for AI tools presents several advantages, including a structured and reproducible approach to evaluation, the ability to test AI systems in diverse clinical scenarios, and the opportunity to benchmark AI performance against established clinical standards to permit charting of future progress as aerospace medicine evolves as a discipline. Remaining challenges include ensuring that these evaluations capture the full spectrum of clinical decision-making scenarios that will be confronted by CMOs during missions and adequately reflecting real-world variability of the austere spaceflight environment. CONCLUSION Employing OSCEs to evaluate AI clinical decision support tools offers a promising approach to validating their clinical utility and efficacy. This methodology not only provides insights into the tool's performance but also fosters ongoing improvement and alignment with standard of care practices. Future research should focus on refining these evaluation processes and addressing limitations to enhance the integration of AI tools in clinical spaceflight settings. REFERENCES Scott S, Hearns V, Barker MA. Testing Clinical Skills: A Look at the OSCE and USMLE Clinical Skills Exams. S D Med. 2019 Oct;72(10):451-453. Majumder MAA, Kumar A, Krishnamurthy K, Ojeh N, Adams OP, Sa B. An evaluative study of objective structured clinical examination (OSCE): students and examiners perspectives. Adv Med Educ Pract. 2019 Jun 5;10:387-397. Karam VY, Park YS, Tekian A, Youssef N. Evaluating the validity evidence of an OSCE: results from a new medical school. BMC Med Educ. 2018 Dec 20;18(1):313.

Ariana M Nelson↗
NASA NTRS1991

AI aerospace components

An evaluation is made of the application of novel, AI-capabilities-related technologies to aerospace systems. Attention is given to expert-system shells for Space Shuttle Orbiter mission control, manpower and processing cost reductions at the NASA Kennedy Space Center's 'firing rooms' for liftoff monitoring, the automation of planetary exploration systems such as semiautonomous mobile robots, and AI for battlefield staff-related functions.

Heindel, Troy A.↗
NASA NTRS1988

Quality measures and assurance for AI (Artificial Intelligence) software

This report is concerned with the application of software quality and evaluation measures to AI software and, more broadly, with the question of quality assurance for AI software. Considered are not only the metrics that attempt to measure some aspect of software quality, but also the methodologies and techniques (such as systematic testing) that attempt to improve some dimension of quality, without necessarily quantifying the extent of the improvement. The report is divided into three parts Part 1 reviews existing software quality measures, i.e., those that have been developed for, and applied to, conventional software. Part 2 considers the characteristics of AI software, the applicability and potential utility of measures and techniques identified in the first part, and reviews those few methods developed specifically for AI software. Part 3 presents an assessment and recommendations for the further exploration of this important area.

Rushby, John↗
NASA NTRS1987

Measuring near infrared spectral reflectance changes from water stressed conifer stands with AIS-2

Airborne Imaging Spectrometer-2 (AIS-2) data was acquired over two paired conifer stands for the purpose of detecting differences in spectral reflectance between stressed and natural canopies. Water stress was induced in a stand of Norway spruce and white pine by severing the sapwood near the ground. Water stress during the AIS flights was evaluated through shoot water potential and relative water content measurements. Preliminary analysis with raw AIS-2 data using SPAM indicates that there were small, inconsistent differences in absolute spectral reflectance in the near infrared 0.97 to 1.3 micron between the stressed and natural canopies.

Riggs, George↗
NASA NTRS1993

Automated inspection of solder joints for surface mount technology

Researchers at NASA/GSFC evaluated various automated inspection systems (AIS) technologies using test boards with known defects in surface mount solder joints. These boards were complex and included almost every type of surface mount device typical of critical assemblies used for space flight applications: X-ray radiography; X-ray laminography; Ultrasonic Imaging; Optical Imaging; Laser Imaging; and Infrared Inspection. Vendors, representative of the different technologies, inspected the test boards with their particular machine. The results of the evaluation showed limitations of AIS. Furthermore, none of the AIS technologies evaluated proved to meet all of the inspection criteria for use in high-reliability applications. It was found that certain inspection systems could supplement but not replace manual inspection for low-volume, high-reliability, surface mount solder joints.

Savage, Robert M.↗
NASA NTRSDate not supplied

Designing and Evaluating NASA SPoRT Center’s DustTracker-AI Model for Detecting Dust in NASA/NOAA Geostationary Satellite Imagery

- Near real-time identification of airborne dust in satellite imagery is important for mitigating the adverse effects of dust storms on human activities. - False color Red-Green-Blue (RGB) imagery has been used for dust detection, but it has limitations and can be difficult to interpret. - The NASA Short-term Research and Transition (SPoRT) center has developed a night-time dust detection random forest (NT-DustTracker-AI, Berndt et al. 2021) model using NASA/NOAA Geostationary Operational Environmental Satellite-16 (GOES-16) Advanced Baseline Imager (ABI) infrared imagery as inputs. - The SPoRT center has partnered with the NOAA National Weather Service to evaluate the model for use in weather forecasting operations, and preliminary results have been positive. - The SPoRT center has expanded the model to cover both day and night, continuing to use infrared imagery as inputs. This new model, known as DustTracker-AI, has shown good agreement with available dust observations

Robert A. Junod↗
NASA NTRS2016

Evaluation of the MICAST #2-12 AI-7wt%Si Sample Directionally Solidified Aboard the International Space Station

The US team of the European led "MIcrostructure Formation in CASTing of Technical Alloys under Diffusive and Magnetically Controlled Convective Conditions" (MICAST) program recently received a third Aluminum - 7wt% silicon alloy that was processed in the microgravity environment aboard the International Space Station. The sample, designated MICAST#2-12, was directionally solidified in the Solidification with Quench Furnace (SQF) at a constant rate of 40micometers/s through an imposed temperature gradient of 31K/cm. Procedures taken to evaluate the state of the sample prior to sectioning for metallographic analysis are reviewed and rational for measuring the microstructural constituents, in particular the primary dendrite arm spacing (Lambda (sub1)), is given. The data are presented, put in context with the earlier samples, and evaluated in view of a relevant theoretical model.

Tewari, Surendra N.↗
NASA NTRSDate not supplied

Multiscale Modeling of Reconstructed Tricalcium Silicate using NASA Multiscale Analysis Tool

To study microstructure characteristics of cementitious materials hydrated in space; previously, cement binder formations were processed under microgravity conditions and was further compared against ground-based experiments. For accurate estimation of process-structure-property linkage, particularly on samples hydrated in the microgravity environment, it is desired to have a high-fidelity volumetric representation of the microstructure. However, owing to small sample size and high porosity of the space-returned samples, conventional experimental characterization techniques are not viable. Hence, a deep learning-based reconstruction algorithm was employed to obtain high fidelity 3D volumes from sparse high resolution 2D Scanning Electron Microscopy (SEM) images, as inputs to micromechanics-based modeling. This machine learning-based reconstruction methodology validated against low-order statistical descriptors, captured the microstructural topology of both sample types (ground, 1g and microgravity, μg). Due to the lack of gravity, hydration products of the samples processed in space differed from those processed-on ground. Such AI-generated virtual samples were analyzed in a multiscale recursive micromechanics approach using the NASA Multiscale Analysis Tool (NASMAT). Here, we present a methodology to rapidly integrate and evaluate these AI-generated volumes in NASMAT. The synthesized microstructural volumes are directly employed as Representative Volume Elements (RVEs) to preserve the fidelity (1 pixel = 0.54 m). Invariably, analysis of such largescale problems (5123 voxels) requires huge amount of computational resources. By taking advantage of the NASMAT architecture, we also focused on systematic multiscale integration of these AI-reconstructed virtual volumes to reduce the computational demands. In this work, this methodology is demonstrated on the ground-based, 1g samples. The estimated stiffness value of 15.90 GPa is comparable to experimentally obtained modulus of hydrated tricalcium silicate sample. The workflow presented here paves the way for utilizing the NASMAT tool to perform multiscale analyses of other multi-phase material systems using either 3D virtual datasets synthesized using AI or obtained via micro-CT.

Machine Learning↗
NASA NTRSDate not supplied

Transforming Science Prioritization Processes Using Artificial Intelligence

Artificial Intelligence (AI) and Machine Learning (ML) have potential to augment significantly the current labor-intensive processes of science prioritization, specifically by the National Academies’ Decadal Survey on behalf of NASA and NSF. Here we summarize what we believe to be the first exploratory demonstration-of-concept results from an application of AI/ML to Survey science prioritization. Specifically, we applied Latent Dirichlet Allocation (LDA) and Natural Language Processing (NLP) to reveal trends in published astrophysics research that may indicate science priorities and which could be applied to strategic planning. For the purpose of the work that we summarize here, AI/ML is able to analyze – that is, to “understand,” in a manner of speaking – a vast amount of text to reveal complex relationships among research topics, including the growth or decline of science community activities in those topics over time. We trained ourselves and AI/ML algorithms by using ~400,000 abstracts in the period 1998 to 2010 to “forecast” the Academies’ Astro2010 recommendations and compare with the solicited white papers. Comparing our results with actual Astro2010 recommendations allowed us to identify candidate metrics that better predicted the actual results of the Survey. We found, for example, that Compound Annual Growth Rate (CAGR) of papers published in a topic area is a good proxy measure for importance of this topic area of research. With this training complete, we identified candidate astrophysics astrophysics science priorities for the 2021+ period using the research during 2007 - 2019 . We conclude that appropriate application of AI can potentially significantly reduce the current workload of the Decadal Survey processes and reveal otherwise unrecognized characteristics in the body of astronomical research. We emphasize throughout the exploratory nature of our work, encouraging colleagues to pursue promising results further. Our most critical governing assumption was that increased (or decreased) research activity can be used to identify scientific or technology topic areas worthy of increased (or decreased) future emphasis. We discuss advantages, limitations, and recognize the “black box” nature of our technique. We note ethics issues associated, for example, with using AI/ML to reveal “hidden” meanings and biases in published work. Furthermore, inevitable improvements in AI may soon enable widespread and welcome identification of and advocacy for science and technology priorities by disparate and diverse groups and organizations. Consequently, we continue to urge a near-term, in-depth evaluation of appropriate applications of AI, including implications and consequences, as well as support for multiple follow-on assessments, of which ours is only a beginning.

Artificial Intelligence↗
NASA NTRS1987

Calibrating AIS images using the surface as a reference

A method of evaluating the initial assumptions and uncertainties of the physical connection between Airborne Imaging Spectrometer (AIS) image data and laboratory/field spectrometer data was tested. The Tuscon AIS-2 image connects to lab reference spectra by an alignment to the image spectral endmembers through a system gain and offset for each band. Images were calibrated to reflectance so as to transform the image into a measure that is independent of the solar radiant flux. This transformation also makes the image spectra directly comparable to data from lab and field spectrometers. A method was tested for calibrating AIS images using the surface as a reference. The surface heterogeneity is defined by lab/field spectral measurements. It was found that the Tuscon AIS-2 image is consistent with each of the initial hypotheses: (1) that the AIS-2 instrument calibration is nearly linear; (2) the spectral variance is caused by sub-pixel mixtures of spectrally distinct materials and shade, and (3) that sub-pixel mixtures can be treated as linear mixtures of pure endmembers. It was also found that the image can be characterized by relatively few endmembers using the AIS-2 spectra.

Smith, M. O.↗