Search NASA⌕ Search

SEARCH · Search NASA

Results for “dissimilarity metric”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2006

Using Dissimilarity Metrics to Identify Interesting Designs

A computer program helps to blend the power of automated-search software, which is able to generate large numbers of design solutions, with the insight of expert designers, who are able to identify preferred designs but do not have time to examine all the solutions. From among the many automated solutions to a given design problem, the program selects a smaller number of solutions that are worthy of scrutiny by the experts in the sense that they are sufficiently dissimilar from each other. The program makes the selection in an interactive process that involves a sequence of data-mining steps interspersed with visual displays of results of these steps to the experts. At crucial points between steps, the experts provide directives to guide the process. The program uses heuristic search techniques to identify nearly optimal design solutions and uses dissimilarity metrics defined by the experts to characterize the degree to which solutions are interestingly different. The search, data-mining, and visualization features of the program were derived from previously developed risk-management software used to support a risk-centric design methodology

Feather, Martin↗
DOE OSTI2025

Unsupervised Image-Based Classification of Corrosion Severity in Automobile Engine Connecting Rods

Corrosion in engine connecting rods is a critical issue in the automotive industry, potentially leading to catastrophic engine failure, monetary losses, and safety hazards. The labor shortage in the industry further emphasizes the need for fast, accurate, and automated corrosion detection methods to ensure appropriate surface treatments can be applied to restore component integrity. We present an unsupervised image-based framework for classifying corrosion severity in automobile engine connecting rods using short-wave infrared (SWIR) and telecentric grayscale imaging. We employ the structural similarity index measure (SSIM) as a dissimilarity metric and the k-medians clustering algorithm for classification. Our algorithm achieves an overall accuracy of 80.64% for SWIR images, with 100% accuracy in classifying highly corroded samples. For grayscale images, the method attains an overall accuracy of 77.42%, with 90.91% accuracy for highly corroded samples. The method’s ability to work with different imaging modalities and its high accuracy in identifying severe corrosion cases make it a promising tool for automated corrosion assessment in the automotive industry, potentially improving efficiency and safety in engine component maintenance.

42 ENGINEERING↗
DOE OSTI2022

Rigid registration algorithm based on the minimization of the total variation of the difference map

Image registration is broadly used in various scenarios in which similar scenes in different images are to be aligned. However, image registration becomes challenging when the contrasts and backgrounds in the images are vastly different. This work proposes using the total variation of the difference map between two images (TVDM) as a dissimilarity metric in rigid registration. A method based on TVDM minimization is implemented for image rigid registration. The method is tested with both synthesized and real experimental data that have various noise and background conditions. The performance of the proposed method is compared with the results of other rigid registration methods. It is demonstrated that the proposed method is highly accurate and robust and outperforms other methods in all of the tests. The new algorithm provides a robust option for image registrations that are critical to many nano-scale X-ray imaging and microscopy applications.

36 MATERIALS SCIENCE↗
NASA NTRS2005

Automated Image Registration Using Morphological Region of Interest Feature Extraction

With the recent explosion in the amount of remotely sensed imagery and the corresponding interest in temporal change detection and modeling, image registration has become increasingly important as a necessary first step in the integration of multi-temporal and multi-sensor data for applications such as the analysis of seasonal and annual global climate changes, as well as land use/cover changes. The task of image registration can be divided into two major components: (1) the extraction of control points or features from images; and (2) the search among the extracted features for the matching pairs that represent the same feature in the images to be matched. Manual control feature extraction can be subjective and extremely time consuming, and often results in few usable points. Automated feature extraction is a solution to this problem, where desired target features are invariant, and represent evenly distributed landmarks such as edges, corners and line intersections. In this paper, we develop a novel automated registration approach based on the following steps. First, a mathematical morphology (MM)-based method is used to obtain a scale-orientation morphological profile at each image pixel. Next, a spectral dissimilarity metric such as the spectral information divergence is applied for automated extraction of landmark chips, followed by an initial approximate matching. This initial condition is then refined using a hierarchical robust feature matching (RFM) procedure. Experimental results reveal that the proposed registration technique offers a robust solution in the presence of seasonal changes and other interfering factors. Keywords-Automated image registration, multi-temporal imagery, mathematical morphology, robust feature matching.

Plaza, Antonio↗
NASA NTRS2006

NASA Tech Briefs, July 2006

Topics covered include: Airport Remote Tower Sensor Systems; Implantable Wireless MEMS Sensors for Medical Uses; Embedded Sensors for Measuring Surface Regression; Coordinating an Autonomous Earth-Observing Sensorweb; Range-Measuring Video Sensors; Stability Enhancement of Polymeric Sensing Films Using Fillers; Sensors for Using Times of Flight to Measure Flow Velocities; Receiver Would Control Phasing of a Phased-Array Antenna; Modern Design of Resonant Edge-Slot Array Antennas; Carbon-Nanotube Schottky Diodes; Simplified Optics and Controls for Laser Communications; Coherent Detection of High-Rate Optical PPM Signals; Multichannel Phase and Power Detector; Using Satellite Data in Weather Forecasting: I; Using Dissimilarity Metrics to Identify Interesting Designs; X-Windows PVT Widget Class; Shuttle Data Center File-Processing Tool in Java; Statistical Evaluation of Utilization of the ISS; Nanotube Dispersions Made With Charged Surfactant; Aerogels for Thermal Insulation of Thermoelectric Devices; Low-Density, Creep-Resistant Single-Crystal Superalloys; Excitations for Rapidly Estimating Flight-Control Parameters; Estimation of Stability and Control Derivatives of an F-15; Tool for Coupling a Torque Wrench to a Round Cable Connector; Ultrasonically Actuated Tools for Abrading Rock Surfaces; Active Struts With Variable Spring Stiffness and Damping; Multiaxis, Lightweight, Computer-Controlled Exercise System; Dehydrating and Sterilizing Wastes Using Supercritical CO2; Alpha-Voltaic Sources Using Liquid Ga as Conversion Medium; Ice-Borehole Probe; Alpha-Voltaic Sources Using Diamond as Conversion Medium; White-Light Whispering-Gallery-Mode Optical Resonators; Controlling Attitude of a Solar-Sail Spacecraft Using Vanes; and Wire-Mesh-Based Sorber for Removing Contaminants from Air.

Source record↗
DOE OSTI2023

EI_MS_ML

The unambiguous identification of compounds from their electron ionization mass (EI-MS) spectra remains a significant unsolved problem in the field of metabolomics and analytical chemistry as a whole. Typically EI-MS spectra are compared using various mathematical operations that convert the spectral similarity or differences into a distance-like metric that roughly approximates the similarity of any two spectra. A commonly used metric for this is the cosine similarity metric which has values close to one for very similar spectra and a value of zero for very dissimilar spectra; however, no metric is perfect. Due to the prevalence of structurally-similar compounds such as isomers and the prevalence of certain fragmentation patterns across structurally-dissimilar compounds, the unambiguous assignment of EI-MS spectra compounds remains difficult. Frequently, querying an observed EI-MS spectrum against a large database such as the NIST17 library yields multiple possible assignments requiring the end user to distinguish between multiple high scoring hits, or multiple low scoring hits while keeping in mind that the correct hit may not be in the database at all. Although techniques such as orthogonal information from techniques such as chromatography can greatly aid in unambiguous assignment, this also requires more complicated experimental designs and access to more complicated analytical instrumentation. Substructures can be trivially detected and represented as strings using a previously published technique called node coloring from a known chemical structure. However, for experimentally-derived EI-MS spectra this information must be derived from the spectra itself (i.e., because we do not know what compound it represents). To achieve this, the software uses techniques from the field of machine learning and a large training dataset of EI-MS spectra corresponding to known structures annotated with substructure strings, to build models that can predict the presence of a given chemical substructure from an EI-MS spectrum directly.If these predictions are of high-quality (i.e., are unlikely to be false positives), the presence of one or more predicted substructures can be used to constrain the number of possible hits for a query spectrum. Mathematically, this restriction could be expressed in many forms, but the most straight-forward implementation is to weight the cosine similarity of a query spectrum and a plausible database match with a Tanimoto-like coefficient based on the ratio of the number of substructures predicted to the number of substructures present in the potential database hit. Determining which combination of models best reduces assignment ambiguity will be achieved using a combination of manual curation and optimization techniques such as genetic algorithms. This software will perform all the steps necessary to construct said models from a training dataset and evaluate them using a holdout dataset. Various statistical analyses can be performed to determine if this approach does decrease assignment ambiguity. For example, if this approach works, on average, the rank-order of the correct assignment for the holdout set of EI-MS spectra should decrease and the weighted cosine similarities for most of the possible matches in the database should be better than the unweighted cosine similarities. Furthermore, this same pipeline can be used on real experimental data to generate less ambiguous assignments.

Mitchell, Joshua↗
DOE OSTI2024

Automating the Analysis of Large Language Models Responses through Zero-Shot Question Answering

Recent advancements in Large Language Models (LLMs) have shown significant potential in various applications, yet their evaluation, particularly in zero-shot question answering scenarios, remains a challenging task. In this study, our objective was to explore precision metrics for Large Language Models (LLM) and design and implement a software pipeline to automatically evaluate LLMs' outputs under zero-shot question answering. Zero-shot question answering involves a model providing answers to questions about topics it hasn't seen during training. It leverages the principles of zero-shot learning by relying on semantic understanding and generalization from related knowledge. The data used was metadata from medical databases on congenital heart disease. We explored eleven LLM metrics and selected three for our evaluation: BLEU, BERTScore, and MoverScore. BLEU calculates a score based on the overlap of n-grams (contiguous sequences of n items, typically words) between the machine-generated translation and the reference translations. Higher BLEU scores indicate better correspondence between the machine-generated and human-generated translations. BERTScore is a metric used to evaluate the quality of machine-generated text by measuring the similarity of token embeddings produced by BERT (Bidirectional Encoder Representations from Transformers) between the generated text and reference text. MoverScore is a metric that quantifies the dissimilarity between the distributions of word embeddings from machine-generated text and reference text, emphasizing semantic similarity over exact token overlap. We also introduced HBKI, a composite metric summarizing these approaches. We tested five models —GPT-3, Llama-2, Gemini 1.5 Pro, Solar 10.7B, and Mixtral-8x7b. Our software pipeline, designed and implemented using Object-Oriented Programming principles, allows users to customize the selection and extraction of features for topics of interest in their own research. Our results show that MoverScore delivered the most precise evaluation of the LLM's outputs, while Mixtral-8x7b achieved the best overall performance in extracting metadata from the databases.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

Genetic Determinants of Microbial Survival in Space

Space flight agencies envision a future for humankind beyond Earth, including missions back to the Moon and to Mars in the coming decades. Sending humans into space inevitably includes their microbiomes as well, leading to trillions of bacteria being shed in their living areas. These bacteria shape the lives of their hosts as well as their environment; thus, it is crucial to understand the adaptations of these microbial spacefarers in spaceflight conditions. We aimed to elucidate the genetic determinants of microbial survival in space using a pan-genome analysis of 12 genera cultured from the International Space Station (ISS) from 2017 to 2018. Analysis was performed on each of the genera individually with terrestrial analogs to identify the core and accessory genomes of the spaceflight and terrestrial strains. We then compared the flight and terrestrial core and accessory genomes for each genera using a Bray-Curtis index and visualized the resulting dissimilarity using an Non-Metric Dimensional Scaling plot. The core proteins available in only the spaceflight organisms were then manually characterized for function and genomic location. In every core genome comparison in each genus, there was significant dissimilarity in the core of the spaceflight organisms when compared to the terrestrial organisms. This trend was present in some of the accessory genomes, but was not ubiquitous. Functional analysis of the core content of the ISS genomes showed the majority of genes unique to the core were clustered by location. These gene clusters suggested a set of genetic determinants confer survival in spacecraft-built environments, notably through the uptake of extracellular DNA such as bacteriophage and plasmids. The clear difference between spaceflight and terrestrial microorganisms shows that spaceflight conditions are selective, which has long term implications for their human hosts and environments.

MoBE↗
DOE OSTI2023

Reference data, predictors, and probability grids for forest degradation classes in three sites in the Brazilian Amazon

Forest degradation by fires and selective logging is widespread in the Amazon region. We implemented a gradient boosted classification modeling framework to classify intact, logged, and burned forests at three Amazonian sites: Feliz Natal Municipality and Xingu Indigenous Territory in Mato Grosso State, and Saracá-Taquera National Forest in Pará State. We used forest degradation history from Landsat time-series as reference data and textural metrics derived from PlanetScope images as predictors. Textural metrics were computed using the Gray-Level Co-Occurrence Matrix (GLCM) textural technique. Included in the attached zip file are ten files: - a shapefile containing the reference data (fire and selective logging polygons and year of event) for each site; - a multiband tif file containing the 8 GLCM metrics used as predictors (Mean, Variance, Homogeneity, Contrast, Dissimilarity, Entropy, Angular Second Moment, Correlation) at the original PlanetScope resolution (3.125m) for each site; - a multiband tif file containing the 72 aggregated GLCM metrics used as predictors (Mean, Variance, Homogeneity, Contrast, Dissimilarity, Entropy, Angular Second Moment, and Correlation aggregated using the mean, first quartile, third quartile, maximum, median, minimum, root mean square, standard deviation, and skewness statistics) at 562m resolution for each site; - a multiband tif file containing the 3 probability grids for either intact, logged, or burned forests at the aggregation resolution (562m) for each site.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Evaluating a Laboratory Flume Microbiome as a Window Into Natural Riverbed Biogeochemistry

Riverbeds are hotspots for microbially-mediated reactions that exhibit pronounced variability in space and time. It is challenging to resolve biogeochemical mechanisms in natural riverbeds, as uncontrolled settings complicate data collection and interpretation. To overcome these challenges, laboratory flumes are often used as proxies for natural riverbed systems. Flumes capture spatiotemporal variability and thus allow for controlled investigations of riverbed biogeochemistry. These investigations implicitly rely on the assumption that the flume microbiome is similar to the microbiome of natural riverbeds. However, this assumption has not been tested and it is unknown how the microbiome of a flume compares to natural aquatic settings, including riverbeds. To evaluate the fundamental assumption that a flume hosts a microbiome similar to natural riverbed systems, we used 16s rRNA gene sequencing and publicly available data to compare the sediment microbiome of a single large laboratory flume to a wide variety of natural ecosystems including lake and marine sediments, river, lake, hyporheic, soil, and marine water, and bank and wetland soils. Richness and Shannon diversity metrics, analyses of variance, Bray-Curtis dissimilarity, and analysis of the common microbiomes between flume and river sediment all indicated that the flume microbiome more closely resembled natural riverbed sediments than other ecosystems, supporting the use of flume experiments for investigating natural microbially-mediated biogeochemical processes in riverbeds.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2013

Software Tools to Support the Assessment of System Health

This presentation provides an overview of three software tools that were developed by the NASA Glenn Research Center to support the assessment of system health: the Propulsion Diagnostic Method Evaluation Strategy (ProDIMES), the Systematic Sensor Selection Strategy (S4), and the Extended Testability Analysis (ETA) tool. Originally developed to support specific NASA projects in aeronautics and space, these software tools are currently available to U.S. citizens through the NASA Glenn Software Catalog. The ProDiMES software tool was developed to support a uniform comparison of propulsion gas path diagnostic methods. Methods published in the open literature are typically applied to dissimilar platforms with different levels of complexity. They often address different diagnostic problems and use inconsistent metrics for evaluating performance. As a result, it is difficult to perform a one ]to ]one comparison of the various diagnostic methods. ProDIMES solves this problem by serving as a theme problem to aid in propulsion gas path diagnostic technology development and evaluation. The overall goal is to provide a tool that will serve as an industry standard, and will truly facilitate the development and evaluation of significant Engine Health Management (EHM) capabilities. ProDiMES has been developed under a collaborative project of The Technical Cooperation Program (TTCP) based on feedback provided by individuals within the aircraft engine health management community. The S4 software tool provides a framework that supports the optimal selection of sensors for health management assessments. S4 is structured to accommodate user ]defined applications, diagnostic systems, search techniques, and system requirements/constraints. One or more sensor suites that maximize this performance while meeting other user ]defined system requirements that are presumed to exist. S4 provides a systematic approach for evaluating combinations of sensors to determine the set or sets of sensors that optimally meet the performance goals and the constraints. It identifies optimal sensor suite solutions by utilizing a merit (i.e., cost) function with one of several available optimization approaches. As part of its analysis, S4 can expose fault conditions that are difficult to diagnose due to an incomplete diagnostic philosophy and/or a lack of sensors. S4 was originally developed and applied to liquid rocket engines. It was subsequently used to study the optimized selection of sensors for a simulation ]based aircraft engine diagnostic system. The ETA Tool is a software ]based analysis tool that augments the testability analysis and reporting capabilities of a commercial ]off ]the ]shelf (COTS) package. An initial diagnostic assessment is performed by the COTS software using a user ]developed, qualitative, directed ]graph model of the system being analyzed. The ETA Tool accesses system design information captured within the model and the associated testability analysis output to create a series of six reports for various system engineering needs. These reports are highlighted in the presentation. The ETA Tool was developed by NASA to support the verification of fault management requirements early in the Launch Vehicle process. Due to their early development during the design process, the TEAMS ]based diagnostic model and the ETA Tool were able to positively influence the system design by highlighting gaps in failure detection, fault isolation, and failure recovery.

Melcher, Kevin J.↗
DOE OSTI2020

Predicting Device Parameters for Dye-Sensitized Solar Cells from Electronic Structure Calculations to Reproduce Experiment.

Given that improvements to the power-conversion efficiency (PCE) of dye-sensitized solar cells (DSSCs) have slowed in recent years, a means to accurately predict the device parameters yielded by trial dyes in silico, without having to synthesize them, would be extremely valuable to speed up the design process. Currently, the best-performing methods of calculating device parameters rely on a set of experimentally determined kinetic coefficients. In practice, it is very difficult to measure these kinetic parameters accurately, limiting the overall accuracy of such predictive methods. This work proposes a model to obtain key parameters such as J(SC), V-OC, and PCE using only the results from density functional theory (DFT) and time-dependent DFT calculations, noting that rates of electron-transfer steps are ultimately linked to the electronic structure of the dye center dot center dot center dot TiO2 working electrode. Six organic DSSC dyes from dissimilar chemical classes (L0, L1, L2, WS-2, WS-92, and C281) were chosen to demonstrate the power of this approach. Their a priori known experimentally determined device performance metrics served to validate our predictions. The greatest absolute error in our predicted PCE values was 0.36% relative to the experiment, while the greatest fractional error was 0.042. This indicates that the proposed model offers a dramatic improvement on previous predictive methods for DSSC device parameters, both in accuracy and in consistency. Moreover, such a predictive model has great potential to be applied to other photovoltaic applications, further enabling the design of novel, highly efficient photoactive materials.

density functional theory↗
DOE OSTI2021

Welding of Haynes 282 to Steels to Enable Modular Rotors for Advanced Ultra Super-Critical Steam Turbines

Steam turbines for an Advanced Ultra Super Critical (AUSC) fossil fired power plant will operate at temperatures well above those of current commercial steam cycles with inlet temperatures more than 760 C , which is beyond the capabilities of alloy steels presently used for steam turbine applications and requires advanced materials. Large components such as steam turbine rotors may be made of nickel based super alloys such as Haynes 282 (H282). However, monolithic forgings of superalloys in the sizes required for large steam turbines rotors can be prohibitively expensive besides many technical challenges. To minimize cost and alleviate the related technical challenges, superalloy use needs to be limited to locations on the steam turbine rotor where strength and temperature requirements cannot be met by conventional steels. This is possible if nickel-based superalloys can be successfully welded to steels and the related technical challenges - machining parts made of dissimilar welded materials, non-destructive examination of such welds for flaw detection; and, material properties of such hybrid components – are sufficiently addressed. In this technology development project, we successfully welded H282 to plates up to ~ 75 mm (~ 3 inches) to a 3.5NiCrMoV steel of similar thickness. Advanced ultrasonic inspection technique called Phased Array Ultrasonic Testing (PAUT) was employed to examine the dissimilar H282-Steel welds, into which flat bottomed side drilled holes (SDH) of various diameters were introduced, to determine the minimum detectable feature sizes; it was shown that with PAUT SDH of dia. down to 0.5 mm could be detected in the base alloys and SDH with dia. down to 2.4 mm could be detected in the weld metal under multiple orientations successfully. An autonomous machining process monitoring system was developed and demonstrated whereby the forces acting on the cutting tool could be actively monitored as the cutting tool transitioned from H282 to Steel across the weld using which the machining parameters can be potentially altered without interruption to extend tool life. This project successfully achieved its objectives of - i. Developing a welding methodology and viable welding geometries, to successfully join H282 to steel 3.5CrMoNiV steel to enable manufacture of modular steam turbine rotors for AUSC applications (conditions of at least 760 °Celsius and 3,100 psia (pounds per square inch absolute pressure) and evaluate the material properties of the welded specimen. ii. Employ the advanced ultrasonic inspection technique to the dissimilar weld metal joint and determine the minimum detectable feature sizes. iii. Develop effective machining techniques to machine such hybrid structures with online tool force monitoring and effect machine state metrics for optimal results.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2025

Implementation of disruptive designs for gas turbine components using direct energy deposition additive manufacturing

This research aims to develop a framework for establishing the correlation between in-situ monitoring data, process parameters, and microstructure evolution in blown-powder laser-directed energy deposition (DED) additive manufacturing (AM). To achieve this, a comprehensive manufacturing framework has been developed, spanning from in-situ data acquisition, melt-pool simulation, microstructure modeling, and statistical microstructure quantification. A machine learning-based surrogate model is constructed to predict melt pool geometry directly from in-situ coaxial camera data. The surrogate model is trained using outputs from a high-fidelity melt pool simulation, which provides accurate melt pool dimension data under varying process conditions. The predicted melt pool geometry is then used as input to a microstructure model to predict microstructural features. To rigorously compare and analyze microstructures, the project introduces statistical metrics that quantify differences based on key features such as morphology and texture. Microstructures are represented using advanced statistical descriptors including angular chord length distribution, two-point spatial statistics, orientation distribution function, and global spherical harmonic. These representations are used to compute four distinct “dissimilarity scores” that quantitatively capture differences in texture and morphology. This framework is demonstrated to enable automated calibration of simulation parameters by minimizing discrepancies between simulated and target microstructures. The technology developed in this project enables direct correlation between in-situ monitoring data and resulting microstructure, paving the way for adaptive microstructure control in metal AM. This capability strengthens the connection between process parameters and final material properties, facilitating more precise and reliable material design.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Comparative Assessment of Data-driven Process Models in Health Information Technology

Process mining for conformance analysis consists of comparing a reference process model against a data-driven process model generated via log files from information technology systems. However, in the absence of a complete reference process model, we found no suggested approaches in the literature to address the need for evaluating process conformance among different healthcare facilities to assess standardization of care. Our goal is to find similarities and dissimilarities in data-driven process models among US Veterans Health Administration (VHA) facilities that can be indicative of patient safety issues. Our hypothesis was that the analysis would not produce statistically significant differences in outcome. We present a unique implementation of conformance analysis in process mining that consists of combining process mining, process mapping and statistical metrics. We illustrate our approach by applying it to the analysis of two clinical radiology order process models generated from healthcare data provided by two similar facilities in the VHA. The comparative assessment showed that about 70% of the orders completed successfully and 30% were not completed due to policy and duplications. Our analysis found a good statistical correlation between both facilities, as the Spearman’s correlation coefficient between facilities for the frequency of cases per total hours was 0.87879, for the frequency of cases by state transition was 0.79702 and for the throughput time per state transition was 0.63582. Additional statistical analyses using the Mann-Whitney U test and the root mean square error both produced values that were not significant. The foregoing approach validated our hypothesis by demonstrating a good statistical correlation of data describing the flow of clinical radiology orders absent a credible reference model. Finding good agreement between both facilities was important in confirming that the clinical orders flow in a similar manner, suggesting standardization of care.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2017

Similarity Assessment of Land Surface Model Outputs in the North American Land Data Assimilation System

Multimodel ensembles are often used to produce ensemble mean estimates that tend to have increased simulation skill over any individual model output. If multimodel outputs are too similar, an individual LSM would add little additional information to the multimodel ensemble, whereas if the models are too dissimilar, it may be indicative of systematic errors in their formulations or configurations. The article presents a formal similarity assessment of the North American Land Data Assimilation System (NLDAS) multimodel ensemble outputs to assess their utility to the ensemble, using a confirmatory factor analysis. Outputs from four NLDAS Phase 2 models currently running in operations at NOAA/NCEP and four new/ upgraded models that are under consideration for the next phase of NLDAS are employed in this study. The results show that the runoff estimates from the LSMs were most dissimilar whereas the models showed greater similarity for root zone soil moisture, snow water equivalent, and terrestrial water storage. Generally, the NLDAS operational models showed weaker association with the common factor of the ensemble and the newer versions of the LSMs showed stronger association with the common factor, with the model similarity increasing at longer time scales. Trade-offs between the similarity metrics and accuracy measures indicated that the NLDAS operational models demonstrate a larger span in the similarity-accuracy space compared to the new LSMs. The results of the article indicate that simultaneous consideration of model similarity and accuracy at the relevant time scales is necessary in the development of multimodel ensemble.

land surface models↗
NASA NTRSDate not supplied

Runway Sign Classifier: A DAL C Certifiable Machine Learning System

In recent years, the remarkable progress of Machine Learning (ML) technologies within the domain of Artificial Intelligence (AI) systems has presented unprecedented opportunities for the aviation industry, paving the way for further advancements in automation, including the potential for single pilot or fully autonomous operation of large commercial airplanes. However, ML technology faces major incompatibilities with existing airborne certification standards, such as ML model traceability and explainability issues or the inadequacy of traditional coverage metrics. Certification of ML-based airborne systems using current standards is problematic due to these challenges. This paper presents a case study of an airborne system utilizing a Deep Neural Network (DNN) for airport sign detection and classification. Building upon our previous work, which demonstrates compliance with Design Assurance Level (DAL) ”D”, we upgrade the system to meet the more stringent requirements of Design Assurance Level ”C”. To achieve DAL C, we employ an established architectural mitigation technique involving two redundant and dissimilar Deep Neural Networks. The application of novel ML-specific data management techniques further enhances this approach. This work is intended to illustrate how the certification challenges of ML-based systems can be addressed for medium criticality airborne applications.

Flight Software↗