Search NASA⌕ Search

SEARCH · Search NASA

Results for “probabilistic machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Probabilistic Machine Learning Estimation of Ocean Mixed Layer Depth from Dense Satellite and Sparse In-Situ Observations

The ocean mixed layer plays an important role in the coupling between the upper ocean and atmosphere across a wide range of time scales. Estimation of the variability of the ocean mixed layer is therefore important for atmosphere-ocean prediction and analysis. The increasing coverage of in situ Argo profile data allows for an increasingly accurate analysis of the mixed layer depth (MLD) variability associated with deviations from the seasonal climatology. However, sampling rates are not sufficient to fully resolve subseasonal (<90 day) MLD variability. Yet, many multivariate observations-based analyses include implicit modeled subseasonal MLD variability. One analysis method is optimal interpolation of in situ data, but the interior analysis can be improved by leveraging surface data with regression or variational approaches. Here, we demonstrate how machine learning methods and satellite sea surface temperature, salinity, and height facilitate MLD estimation in a pilot study of two regions: the mid-latitude southern Indian and the eastern equatorial Pacific Oceans. We construct multiple machine learning architectures to produce weekly 1/2° gridded MLD anomaly fields (relative to a monthly climatology) with uncertainty estimates. We test multiple traditional and probabilistic machine learning techniques to compare both accuracy and probabilistic calibration. We validate our methodology by applying it to ocean model simulations. We find that incorporating sea surface data through a machine learning model improves the performance of spatiotemporal MLD variability estimation compared to optimal interpolation of Argo observations alone. These preliminary results are a promising first step for the application of machine learning to MLD prediction.

Machine Learning↗

Probabilistic Modeling of Heavy Machinery Using Machine Learning

NASA Glenn Research Center’s facility operations seeks to leverage its extensive instrumentation and historical data with machine learning to increase system efficiencies. The ultimate goal of this effort is to probabilistically model the behavior of Glenn’s central air service compressors for optimal decision making and planning. This project is a first step in that direction. We propose a multimodel approach that uses high-dimensional models to ask simple questions about complex dependent structures, and low-dimensional models to ask complex questions about simple dependent structures. While the low-dimensional models make strong assumptions, they can be visualized and they can be insightful. We show good fits for univariate models of compressor sensors, and preliminary work on high-dimensional multivariate models.

Machine learning↗

Image-to-Image Wildfire Detection via Quantum-Compatible Variational Segmentation from Remotely-sensed Data

Over the last decade, the incidence of wildfires has surged, causing widespread destruction globally. To better comprehend and manage these incidents, remote sensing and aerial missions have been implemented in recent efforts. However, this has resulted in an exponential rise in the amount of remote sensing data utilization, leading to a need for intelligent automation of data extraction in wildfire studies. Machine learning provides an accurate automated approach for detecting these natural anomalies and facilitates decision-makers to take prompt actions. To make insightful decisions in wildfire management, it is imperative to move beyond simple detection and explore the potential of probabilistic generative machine learning for creating "what-if" scenarios for various wildfire conditions. Such models offer improved representation of the stochastic nature of wildfire events. However, the optimization of these models can be computationally expensive, especially when using classical computers. Quantum computers have recently emerged as a promising solution to reduce the computational cost of training such models and improve their performance. In this study, we aim to utilize quantum-compatible machine learning techniques to implement our probabilistic generative approach. To that end, we propose a supervised probabilistic variational model consisting of a U-NET-based image-to-image component along with encoder and decoder networks which work as a variational autoencoder (VAE) component. Additionally, we explore the type of latent distribution type in the VAE component and implement different means for modeling the prior distribution. We further investigate the quantum-compatible versions of the model compared to the classical counterpart and benchmark potential benefits of quantum compatibility over the classical model.

quantum machine learning↗

Automated Probabilistic Finite Element Model Calibration Tool Based on Uncertainty Quantification and Machine Learning

Qualification and certification of safety critical parts is a hurdle to the adoption of metallic additively manufactured components for aerospace vehicle applications. Challenges include variability in part properties due to inconsistent defect distribution and microstructure. Understanding of the process through finite element modeling (FEM), and process control through in-situ monitoring, may result in significant improvements; however, solutions useful to manufacturers will require large volumes of data and automated data utilization. Toward this end, a generalizable automated FEM calibration paradigm is developed. This paradigm leverages existing and novel tools from machine learning and uncertainty quantification to enable the automatic calibration of FEMs without requiring prior knowledge of the model performance across input parameter space, including meshing and solver settings, which can require time consuming manual model probing or cause noisy and inconsistent predictions. The result is a probabilistic distribution of calibrated and validated FEM input parameters targeting measured data.

Additive manufacturing model calibration finite el↗

Quantum-Compatible Variational Segmentation for Image-to-Image Wildfire Detection Using Satellite Data

Wildfire occurrences have been increasing for the past decade, leaving devastating traces across the world. In the recent efforts, remote sensing and airborne missions have been utilized to better understand and manage wildfires. This has resulted in an exponential increase in volume of remote sensing data, which has pushed the need for intelligent automation of data extraction for wildfire studies. Machine learning offers accurate automation in detecting such natural anomalies and enable decision-makers to take actions in a timely manner. Recent advances in machine learning algorithms, namely probabilistic generative methods, allow researchers and decisionmakers to step beyond detection and study “what-if” scenarios for wildfire occurrences. Additionally, they offer better imitations to the stochastic behavior of nature, and wildfire events. However, optimizing the performance of these probabilistic generative models is a computationally expensive process, specially using digital computers. On the other hand, quantum computers have recently shown a promise to reduce computationally costly training of such models and provide performance improvements. There is a body of research investigating the potential for improved machine learning methods in which key operations are performed on a quantum computer. In this study, we propose a probabilistic image-toimage segmentation approach combining a very well-known segmentation method, U-NET, with a Conditional Variational Auto-Encoder (CVAE) to not only detect wildfires but also describe the stochasticity of the phenomenon and be capable of running “what-if” scenarios. Our proposed model is compatible with training on quantum computers, which results in a quantum-assisted image-to-image segmentation approach and can be used to benchmark the potential benefit of quantum computing over the classical one.

quantum↗

Use of Machine Learning Techniques for Iidentification of Robust Teleconnections to East African Rainfall Variability in Observations and Models

Providing advance warning of East African rainfall variations is a particular focus of several groups including those participating in the Famine Early Warming Systems Network. Both seasonal and long-term model projections of climate variability are being used to examine the societal impacts of hydrometeorological variability on seasonal to interannual and longer time scales. The NASA / USAID SERVIR project, which leverages satellite and modeling-based resources for environmental decision making in developing nations, is focusing on the evaluation of both seasonal and climate model projections to develop downscaled scenarios for using in impact modeling. The utility of these projections is reliant on the ability of current models to capture the embedded relationships between East African rainfall and evolving forcing within the coupled ocean-atmosphere-land climate system. Previous studies have posited relationships between variations in El Niño, the Walker circulation, Pacific decadal variability (PDV), and anthropogenic forcing. This study applies machine learning methods (e.g. clustering, probabilistic graphical model, nonlinear PCA) to observational datasets in an attempt to expose the importance of local and remote forcing mechanisms of East African rainfall variability. The ability of the NASA Goddard Earth Observing System (GEOS5) coupled model to capture the associated relationships will be evaluated using Coupled Model Intercomparison Project Phase 5 (CMIP5) simulations.

Roberts, J. Brent↗

Machine-Learning-Based Adaptive Thinning of CrIS Radiances to Improve Global Tropical Cyclone Analysis and Forecasts

This work is focused on optimizing the assimilation of hyperspectral infrared (IR) radiances from the Cross-track Infrared Sounder (CrIS) with the goal of improving the representation of tropical cyclones (TCs) in global analyses and forecasts. Current operational assimilation systems rely on subsampling IR radiances on a regular thinning grid. A new and improved adaptive methodology based on machine learning (ML) recognizes TCs from geostationary satellite imagery and is implemented in the Goddard Earth Observing System (GEOS) model and data assimilation framework. The ML methodology is extensively trained on existing TC data sets and creates for each TC a dynamic mask, based on the evolving shape and life cycle of that specific event. Once a TC mask is created, a switch is then activated in the data assimilation system to alter the thinning, ingesting more CrIS radiances within the moving mask, thus increasing the TC sampling. After the TC dissipates, the assimilation of CrIS radiances reverts to normal data density. Results of TC segmentation provided by a state-of-the-art generative machine learning model known as the Denoising Diffusion Probabilistic Model (DDPM) are compared to the previously used U-Net model. The new approach surpasses the performance of the previously developed one. The methodology is applied to both clear-sky and cloud-cleared radiances. Benefits from the latter methodology, particularly in improving the structure of TCs and the intensity forecasts, are presented.

Oreste Reale↗

VFR Trajectory Forecasting using Deep Generative Model for Autonomous Airspace Operations

To enable the airspace integration of autonomous operations, such as uncrewed aircraft conducting cargo deliveries, there is a need to forecast the positions of the surrounding traffic with which they may interact. This paper focuses on forecasting Visual Flight Rules traffic, a significant source of uncertainty and risk in the airspace, especially around small regional airports, due to the unplanned and often untracked nature of such flights. A deep generative model is developed, trained on historical traffic data at example towered and non-towered airports, and used to predict flight trajectories. Experimental results are presented comparing the performance of variational autoencoder and classical machine learning forecasting when applied to both the towered and non-towered airports over varying time horizons. The results show the advantages of the variational autoencoder in producing accurate probabilistic forecasts over varying time horizons.

uncrewed aircraft↗

VFR Trajectory Forecasting using Deep Generative Model for Autonomous Airspace Operations

To enable the airspace integration of autonomous operations, such as uncrewed aircraft conducting cargo deliveries, there is a need to forecast the positions of the surrounding traffic with which they may interact. This paper focuses on forecasting Visual Flight Rules traffic, a significant source of uncertainty and risk in the airspace, especially around small regional airports, due to the unplanned and often untracked nature of such flights. A deep generative model is developed, trained on historical traffic data at example towered and non-towered airports, and used to predict flight trajectories. Experimental results are presented comparing the performance of variational autoencoder and classical machine learning forecasting when applied to both the towered and non-towered airports over varying time horizons. The results show the advantages of the variational autoencoder in producing accurate probabilistic forecasts over varying time horizons.

uncrewed aircraft↗

Probabilistic Design Methodology and its Application to the Design of an Umbilical Retract Mechanism

A lot has been learned from past experience with structural and machine element failures. The understanding of failure modes and the application of an appropriate design analysis method can lead to improved structural and machine element safety as well as serviceability. To apply Probabilistic Design Methodology (PDM), all uncertainties are modeled as random variables with selected distribution types, means, and standard deviations. It is quite difficult to achieve a robust design without considering the randomness of the design parameters which is the case in the use of the Deterministic Design Approach. The US Navy has a fleet of submarine-launched ballistic missiles. An umbilical plug joins the missile to the submarine in order to provide electrical and cooling water connections. As the missile leaves the submarine, an umbilical retract mechanism retracts the umbilical plug clear of the advancing missile after disengagement during launch and retrains the plug in the retracted position. The design of the current retract mechanism in use was based on the deterministic approach which puts emphasis on factor of safety. A new umbilical retract mechanism that is simpler in design, lighter in weight, more reliable, easier to adjust, and more cost effective has become desirable since this will increase the performance and efficiency of the system. This paper reports on a recent project performed at Tennessee State University for the US Navy that involved the application of PDM to the design of an umbilical retract mechanism. This paper demonstrates how the use of PDM lead to the minimization of weight and cost, and the maximization of reliability and performance.

Onyebueke, Landon↗

Using Satellite Soil Moisture and Rainfall in the Landslide Hazard Assessment for Situational Awareness System

The Landslide Hazard Assessment for Situational Awareness system(LHASA)gives a global view of landslide hazard in nearly real time. Currently, it is being upgraded from version 1 to version 2, which entails improvements along several dimensions. These include the incorporation of new predictors, machine learning, and new event-based landslide inventories. As a result, LHASA version 2 substantially improves on the prior performanceand introduces a probabilistic element to the global landslide nowcast. Data from the soil moisture active-passive (SMAP) satellite has been assimilated into a globally consistent data product with a latency less than 3 days, known as SMAP Level 4. In LHASA, thesedata representthe antecedent conditions prior to landslide-triggering rainfall. In some cases, soil moisture may have accumulated over aperiod of many months. The model behind SMAP Level 4 also estimates the amount of snow on the ground, which is an important factor in some landslide events. LHASA also incorporates this information as an antecedent condition that modulates the response torainfall. Slope, lithology, and active faults were also used as predictor variables. These factors can have a strong influence on where landslides initiate.LHASA relies on precipitation estimates from the Global Precipitation Measurement mission to identify the locations where landslides are most probable. The low latency and consistent global coverage of these data make them ideal for real-time applications at continental to global scales. LHASA relies primarily on rainfall from the last 24 hours to spothazardous sites, which is rescaled by the local 99thpercentile rainfall.However, the multi-day latency of SMAP requires the use of a 2-day antecedent rainfall variable to represent the accumulation of rain between the antecedent soil moisture and current rainfall. LHASA merges these predictors with XGBoost, a commonly used machine-learning tool, relying on historical landslide inventories to develop the relationship between landslide occurrence and various risk factors. The resulting model relies heavily on current daily rainfall, but other factors also play an important role. LHASA outputsthe probability oflandslide occurrence ona grid of roughly one kilometer over all continents from 60 North to 60 South latitude. Evaluation over the period 2019-2020 showsthat LHASA version 2 doubles the accuracy of the global landslide nowcast without increasing the global false alarm rate. LHASA also identifies the areas where the human exposure to landslide hazard is most intense. Landslide hazard is divided into 4 levels: minimal, low, moderate, and high. Next, the number of persons and the length of major roads (primary and secondary roads)within each of these areas is calculated for every second-level administrative district (county). These results can be viewedthrough a web portal hosted at the Goddard Space Flight Center. In addition, users can download daily hazard and exposure data.LHASAversion 2uses machine learning and satellite data to identify areas of probable landslide hazard within hours of heavy rainfall. Itsglobal maps are significantly more accurate, and it now includes rapid estimates of exposed populations and infrastructure. In addition, a forecast mode will be implemented soon.

Thomas Stanley↗

Chance-Constrained System Identification of Nonlinear Discrete Systems with Safety and Stability Guarantees

This paper presents a discrete-time nonlinear system identification method while satisfying the stability and safety properties of the system with high probability. An Extreme Learning Machine (ELM) is used with a Gaussian assumption on the function reconstruction error. A quadratically constrained quadratic program (QCQP) is developed with probabilistic safety and stability constraints that are only required to be satisfied at sampled points inside the invariant region. The proposed method is validated using two simulation examples: a two degrees-of-freedom (DoF) robot manipulator with constraints on joint angles whose trajectories are guaranteed to remain inside a safe set and on motion trajectories data of a hand-drawn shape.

Iman Salehi↗

An Unobtrusive System to Measure, Assess, and Predict Cognitive Workload in Real-World Environments

Across many careers, individuals face alternating periods of high and low attention and cognitive workload, which can result in impaired cognitive functioning and can be detrimental to job performance. For example, some professions (e.g., fire fighters, emergency medical personnel, doctors and nurses working in an emergency room, pilots) require long periods of low workload (boredom), followed by sudden, high-tempo operations during which they may be required to respond to an emergency and perform at peak cognitive levels. Conversely, other professions (e.g., air traffic controllers, market investors in financial industries, analysts) require long periods of high workload and multitasking during which the addition of just one more task results in cognitive overload resulting in mistakes. An unobtrusive system to measure, assess, and predict cognitive workload could warn individuals, their teammates, or their supervisors when steps should be taken to augment cognitive readiness. In this talk I will describe an approach to this problem that we have found to be successful across work domains including: (1) a suite of unobtrusive, field-ready neurophysiological, physiological, and behavioral sensors that are chosen to best suit the target environment; (2) custom algorithms and statistical techniques to process and time-align raw data originating from the sensor suite; (3) probabilistic and statistical models designed to interpret the data into the human state of interest (e.g., cognitive workload, attention, fatigue); (4) and machine-learning techniques to predict upcoming performance based on the current pattern of events, and (5) display of each piece of information depending on the needs of the target user who may or may not want to drill down into the functioning of the system to determine how conclusions about human state and performance are determined. I will then focus in on our experimental results from our custom functional near-infrared spectroscopy sensor, designed to operate in real-world environments to be worn comfortably (e.g., positioned into a baseball cap or a surgeons cap) to measure changes in brain blood oxygenation without adding burden to the individual being assessed.

workload collection↗

Sampling Functions from Gaussian Processes and Structured Covariance Gaussian Networks

When learning aerodynamic models from data, it is critical to incorporate estimates of model uncertainty. This motivates the design of probabilistic aerodynamic databases which can be sampled to generate physically and statistically plausible aerodynamic models. In this talk we discuss how to sample deterministic functions from two different kinds of probabilistic models and demonstrate their use. First, Gaussian Process Regressors (GPRs) are a widely used probabilistic kernel-based model which can be thought of as Gaussian distributions over functions. GPRs are generally trained by maximizing the marginal likelihood of seeing the training data over the kernel parameter space. Sample functions are easily generated by drawing points from the Gaussian distribution at desired input points. However, when the points are not known ahead of time, the classical sampling approach is not possible since successive function samples will generate different function realizations. We present an approach for sampling consistent function evaluations from a GPR over multiple samples. Second, we describe a neural network architecture which learns a conditional Gaussian distribution by maximizing the marginal likelihood at each point in the input space. We then discuss and compare several options for generating sample functions which match this distribution. Finally, we demonstrate the use of these probabilistic aerodynamic models in an atmospheric reentry simulation.

Gaussian process regression↗

Probabilistic Prognosis of Non-Planar Fatigue Crack Growth

Quantifying the uncertainty in model parameters for the purpose of damage prognosis can be accomplished utilizing Bayesian inference and damage diagnosis data from sources such as non-destructive evaluation or structural health monitoring. The number of samples required to solve the Bayesian inverse problem through common sampling techniques (e.g., Markov chain Monte Carlo) renders high-fidelity finite element-based damage growth models unusable due to prohibitive computation times. However, these types of models are often the only option when attempting to model complex damage growth in real-world structures. Here, a recently developed high-fidelity crack growth model is used which, when compared to finite element-based modeling, has demonstrated reductions in computation times of three orders of magnitude through the use of surrogate models and machine learning. The model is flexible in that only the expensive computation of the crack driving forces is replaced by the surrogate models, leaving the remaining parameters accessible for uncertainty quantification. A probabilistic prognosis framework incorporating this model is developed and demonstrated for non-planar crack growth in a modified, edge-notched, aluminum tensile specimen. Predictions of remaining useful life are made over time for five updates of the damage diagnosis data, and prognostic metrics are utilized to evaluate the performance of the prognostic framework. Challenges specific to the probabilistic prognosis of non-planar fatigue crack growth are highlighted and discussed in the context of the experimental results.

Leser, Patrick E.↗

Identification of Robust Terminal-Area Routes in Convective Weather

Convective weather is responsible for large delays and widespread disruptions in the U.S. National Airspace System, especially during summer. Traffic flow management algorithms require reliable forecasts of route blockage to schedule and route traffic. This paper demonstrates how raw convective weather forecasts, which provide deterministic predictions of the vertically integrated liquid (the precipitation content in a column of airspace) can be translated into probabilistic forecasts of whether or not a terminal area route will be blocked. Given a flight route through the terminal area, we apply techniques from machine learning to determine the likelihood that the route will be open in actual weather. The likelihood is then used to optimize terminalarea operations by dynamically moving arrival and departure routes to maximize the expected capacity of the terminal area. Experiments using real weather scenarios on stormy days show that our algorithms recommend that a terminal-area route be modified 30% of the time, opening up 13% more available routes that were forecast to be blocked during these scenarios. The error rate is low, with only 5% of cases corresponding to a modified route being blocked in reality, whereas the original route is in fact open. In addition, for routes predicted to be open with probability 0.95 or greater by our method, 96% of these routes (on average over time horizon) are indeed open in the weather that materializes

Pfeil, Diana Michalek↗

Quantum-Assisted Variational Segmentation for Image-to-Image Wildfire Detection Using Satellite Data

The quantum computing community has been searching for suitable applications to demonstrate the potential of near-term quantum devices. Quantum machine learning is a potential candidate, particularly using models that cannot be efficiently simulated with classical computers [1, 2]. This work focuses on a transition phase of quantum computers where the quantum machine learning model is still simulable classically but projected not to be simulable as the size of the model grows. Ultimately quantum computers may have advantages for high-dimensional real-world problems. Due to the limited number of qubits in current noisy intermediate-scale quantum (NISQ) devices, the direct application of quantum computers in high dimensional data is not feasible. To remedy this problem, an encoder-decoder architecture can be utilized. The encoder model would transform the high-dimensional data into a compact representation, to a level that small quantum computers can be used today (or in the near future), and the decoder would take the quantum processed outputs back to the high-dimensional space. Addressing the two challenges of quantum machine learning, this work investigates a hybrid supervised generative model with a quantum Ising Born machine embedded as the latent distribution. The model contains four main parts (Figure 1.a.): (1) a U-NET architecture responsible for learning segmentation flow, (2) a Prior network responsible for learning an encoded latent distribution of the input data, (3) a Born machine which represents the latent distribution, and (4) a Posterior network in charge of learning the joint encoded latent distribution of inputs and target data. The initial model, proposed by [3], is optimized by (1) maximizing the overlap of the prior and posterior latent distributions, and (2) minimizing the segmentation loss. The proposed model is designed to be investigated in a simulation environment applied to the real-world application of wildfire segmentation. Specifically, the model is designed to solve the patchy wildfire segmentations of Moderate Resolution Imaging Spectroradiometer (MODIS) by taking the MODIS observations and using Visible Infrared Imaging Radiometer Suite’s (VIIRS) consistent wildfire product as the target. The model solves patchy wildfire segmentations and provides insight into the epistemic errors sourced from model variation. The model utilizes the Born machine as a QUBO solver to represent the latent space as a Bernoulli distribution. The proposed configuration allows the variational segmentation model to leverage the true quantum probabilistic nature and derive a more expressive latent configuration, increasing the model performance in describing wildfire segmentations. The quantum probabilistic information of the Born machine is directly incorporated in the Kullback-Leibler divergence loss in the prior and posterior distributions, forcing the Bernoulli latent distribution to maximize the overlap of input and joint input-target distributions. The proposed model is then trained and compared with a baseline only consisting of direct Bernoulli latent distribution with no Born machine representing the latent space. The models are evaluated based on the segmentation metrics, such as precision, recall, intersect of union, with uncertainty boundaries accounting for the stochastic nature of the model. Our findings show that even in low latent-dimensional space (due to the limit in computational power of the classical quantum simulator), we are able to effectively capture the latent representation and hence the model performs better than the baseline. The findings are a projection for scaling the model into higher dimensional latent space with the Born machine surpassing the baseline performance. Figure 1. Sub-figure (a) demonstrates the architecture for the training phase. The model consists of a Prior and Posterior network that encode inputs and joint input-target data into compact representations, respectively. The Born machine represents the latent distribution, and the U-NET branch learns the segmentation patterns of the data. The stochasticity is introduced to the U-NET through its last layer to create meaningful but stochastic segmentations. Sub-figure (b) represents the inference phase where the model takes the stochastic behavior from the prior network and injects that into the U-NET. Each attempt of inference will generate different but similar segmentations from the same distribution of the wildfire event. REFERENCES [1] Coyle, B., Mills, D., Danos, V., & Kashefi, E. (2020). The Born supremacy: quantum advantage and training of an Ising Born machine. npj Quantum Information, 6(1), 1-11. [2] Liu, J. G., & Wang, L. (2018). Differentiable learning of quantum circuit born machines. Physical Review A, 98(6), 062324. [3] Kohl, S., Romera-Paredes, B., Meyer, C., De Fauw, J., Ledsam, J. R., Maier-Hein, K., ... & Ronneberger, O. (2018). A probabilistic u-net for segmentation of ambiguous images. Advances in neural information processing systems, 31.

quantum machine learning↗

Representing Learning With Graphical Models

Probabilistic graphical models are being used widely in artificial intelligence, for instance, in diagnosis and expert systems, as a unified qualitative and quantitative framework for representing and reasoning with probabilities and independencies. Their development and use spans several fields including artificial intelligence, decision theory and statistics, and provides an important bridge between these communities. This paper shows by way of example that these models can be extended to machine learning, neural networks and knowledge discovery by representing the notion of a sample on the graphical model. Not only does this allow a flexible variety of learning problems to be represented, it also provides the means for representing the goal of learning and opens the way for the automatic development of learning algorithms from specifications.

Buntine, Wray L.↗