Search NASA⌕ Search

SEARCH · Search NASA

Results for “Contrastive learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS1991

Identification of linear multivariable systems from a single set of data by identification of observers with assigned real eigenvalues

This paper presents a formulation for identification of linear multivariable systems from a single set of input-output data. The identification method is formulated with the mathematical framework of learning identification, by extension of the repetition domain concept to include shifting time intervals. This contrasts existing learning approaches that require data from multiple experiments. In this method, the system input-output relationship is expressed in terms of an observer, which is made asymptotically stable by an embedded real eigenvalue assignment procedure. Through this relationship, the Markov parameters of the observer are identified. The Markov parameters of the actual system are recovered from those of the observer, and then used to obtain a state space model of the system by standard realization techniques. The basic mathematical formulation is derived, and numerical examples presented to illustrate the proposed method.

Phan, Minh↗
DOE OSTI2025

Microscopic Imprints of Learned Solutions in Tunable Networks

In physical networks trained using supervised learning, physical parameters are adjusted to produce desired responses to inputs. An example is an electrical contrastive local learning network of nodes connected by edges that adjust their conductances during training. When an edge conductance changes, it upsets the current balance of every node. In response, physics adjusts the node voltages to minimize the dissipated power. Learning in these systems is therefore a coupled double-optimization process, in which the network descends both a cost landscape in the high-dimensional space of edge conductances and a physical landscape—the power dissipation—in the high-dimensional space of node voltages. Because of this coupling, the physical landscape of a trained network contains information about the learned task. Here, we derive a structure-function relation for trained tunable networks and demonstrate that all the physical information relevant to the trained input-output relation can be captured by a tuning susceptibility, an experimentally measurable quantity. We supplement our theoretical results with simulations to show that the tuning susceptibility is correlated with functional importance and that we can extract physical insight into how the system performs the task from the conductances of highly susceptible edges. Our analysis is general and can be applied directly to mechanical networks, such as networks trained for protein-inspired function such as allostery.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Machine learning without a processor: Emergent learning in a nonlinear analog network

Standard deep learning algorithms require differentiating large nonlinear networks, a process that is slow and power-hungry. Electronic contrastive local learning networks (CLLNs) offer potentially fast, efficient, and fault-tolerant hardware for analog machine learning, but existing implementations are linear, severely limiting their capabilities. These systems differ significantly from artificial neural networks as well as the brain, so the feasibility and utility of incorporating nonlinear elements have not been explored. Here, we introduce a nonlinear CLLN—an analog electronic network made of self-adjusting nonlinear resistive elements based on transistors. We demonstrate that the system learns tasks unachievable in linear systems, including XOR (exclusive or) and nonlinear regression, without a computer. We find our decentralized system reduces modes of training error in order (mean, slope, curvature), similar to spectral bias in artificial neural networks. The circuitry is robust to damage, retrainable in seconds, and performs learned tasks in microseconds while dissipating only picojoules of energy across each transistor. This suggests enormous potential for fast, low-power computing in edge systems like sensors, robotic controllers, and medical devices, as well as manufacturability at scale for performing and studying emergent learning.

Science & Technology - Other Topics↗
NASA NTRS1999

Establishing a Distance Learning Plan for International Space Station (ISS) Interactive Video Education Events (IVEE)

Educational outreach is an integral part of the International Space Station (ISS) mandate. In a few scant years, the International Space Station has already established a tradition of successful, general outreach activities. However, as the number of outreach events increased and began to reach school classrooms, those events came under greater scrutiny by the education community. Some of the ISS electronic field trips, while informative and helpful, did not meet the generally accepted criteria for education events, especially within the context of the classroom. To make classroom outreach events more acceptable to educators, the ISS outreach program must differentiate between communication events (meant to disseminate information to the general public) and education events (designed to facilitate student learning). In contrast to communication events, education events: are directed toward a relatively homogeneous audience who are gathered together for the purpose of learning, have specific performance objectives which the students are expected to master, include a method of assessing student performance, and include a series of structured activities that will help the students to master the desired skill(s). The core of the ISS education events is an interactive videoconference between students and ISS representatives. This interactive videoconference is to be preceded by and followed by classroom activities which help the students aftain the specified learning objectives. Using the interactive videoconference as the centerpiece of the education event lends a special excitement and allows students to ask questions about what they are learning and about the International Space Station and NASA. Whenever possible, the ISS outreach education events should be congruent with national guidelines for student achievement. ISS outreach staff should recognize that there are a number of different groups that will review the events, and that each group has different criteria for acceptance. For example, school administrators are more likely to be concerned about an event meeting national standards and the cost of the event. In contrast, a teacher's acceptance of an education event may be directly related to the amount of extra work the event imposes upon that teacher. ISS education events must be marketed differently to the different groups of educators, and must never increase the workload of the average teacher.

Wallington, Clint↗
DOE OSTI2021

Binary Complex Neural Network Acceleration on FPGA

Being able to learn from complex data with phase information is imperative for many signal processing applications. Today’s real-valued deep neural networks (DNNs) have shown efficiency in latent information analysis but fall short when applied to the complex domain. Deep complex networks (DCN) , in contrast, can learn from complex data, but have high computational costs; therefore, they cannot satisfy the instant decision making requirements of many deployable systems dealing with short observations or short signal bursts. Recent, Binarized Complex Neural Network (BCNN), which integrates DCNs with binarized neural networks (BNN), shows great potential in classifying complex data in real-time. In this paper, we propose a structural pruning based accelerator of BCNN, which is able to provide more than 5000 frames/s inference throughput on edge devices. The high performance comes from both the algorithm and hardware sides. On the algorithm side, we conduct structural pruning to the original BCNN models and obtain 20 × pruning rates with negligible accuracy loss; on the hardware side, we propose a novel 2D convolution operation accelerator for the binary complex neural network. Experimental results show that the proposed design works with over 90% utilization and is able to achieve the inference throughput of 5882 frames/s and 4938 frames/s for complex NIN-Net and ResNet-18 using CIFAR-10 dataset and Alveo U280 Board.

Peng, Hongwu↗
DOE OSTI2023

TransformerG2G: Adaptive time-stepping for learning temporal graph embeddings using transformers

Dynamic graph embedding has emerged as a very effective technique for addressing diverse temporal graph analytic tasks (i.e., link prediction, node classification, recommender systems, anomaly detection, and graph generation) in various applications. Such temporal graphs exhibit heterogeneous transient dynamics, varying time intervals, and highly evolving node features throughout their evolution. Hence, incorporating long-range dependencies from the historical graph context plays a crucial role in accurately learning their temporal dynamics. In this paper, we develop a graph embedding model with uncertainty quantification, TransformerG2G, by exploiting the advanced transformer encoder to first learn intermediate node representations from its current state (t) and previous context (over timestamps [t–1,t–l], l is the length of context). Moreover, we employ two projection layers to generate lower-dimensional multivariate Gaussian distributions as each node's latent embedding at timestamp t. We consider diverse benchmarks with varying levels of "novelty" as measured by the TEA (Temporal Edge Appearance) plots. Here, our experiments demonstrate that the proposed TransformerG2G model outperforms conventional multi-step methods and our prior work (DynG2G) in terms of both link prediction accuracy and computational efficiency, especially for high degree of novelty. Furthermore, the learned time-dependent attention weights across multiple graph snapshots reveal the development of an automatic adaptive time stepping enabled by the transformer. Importantly, by examining the attention weights, we can uncover temporal dependencies, identify influential elements, and gain insights into the complex interactions within the graph structure. For example, we identified a strong correlation between attention weights and node degree at the various stages of the graph topology evolution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Explicit physics-informed neural networks for nonlinear closure: The case of transport in tissues

In upscaling methods, closures for nonlinear problems present a well-known challenge. While a number of theoretical methods have been proposed for handling such closures, nonlinearities still remain a significant obstacle for many problems. In this work, we use a combination of formal upscaling and data-driven machine learning for explicitly closing a nonlinear transport and reaction process in multiscale tissues. The classical effectiveness factor model is used to formulate the macroscale reaction kinetics. We train a multilayer perceptron network using training data generated by direct numerical simulations over microscale examples. Once trained, the network is used in an algorithm for numerically solving the upscaled (coarse-grained) differential equation describing mass transport and reaction in two example tissues. The network is described as being explicit in the sense that the network is trained using macroscale concentrations and gradients of concentration as components of the feature space rather than incorporating them as part of a constraint in the optimization process. Network training and solutions to the macroscale transport equations were computed for two different tissues. The two tissue types (brain and liver) exhibit markedly different geometrical complexity and spatial scale (cell size and sample size). The upscaled solutions for the average concentration are compared with numerical solutions derived from the microscale concentration fields by a posteriori averaging. There are three outcomes of this work of particular note. 1) Our overall approach results in an upscaled nonlinear PDE. The PDE is closed using a neural network, and our approach results in the definition of the classical effectiveness factor for effecting closure. 2) We identify particular source terms for the closure problem that are important for representing the structure of the closure. These source terms involve macroscale concentrations and their gradients. We adopt these source terms to use as explicit features in the learning algorithm. We find the trained networks that include the macroscale source terms generate models that are able to predict the correction factor with increased fidelity over those that do not. 3) We find that the trained network exhibits good generalizability, and it is able to predict the effectiveness factor with high fidelity for realistically-structured tissues despite the significantly different scale and geometrical complexity of the two example tissue types. This latter result emphasizes our purposeful connection between conventional averaging methods with the use of machine learning for closure; this contrasts with some machine learning methods for upscaling where the exact form of the macroscale equation remains unknown.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Information-entropy-driven generation of material-agnostic datasets for machine-learning interatomic potentials

In contrast to their empirical counterparts, machine-learning interatomic potentials (MLIAPs) promise to deliver near-quantum accuracy over broad regions of configuration space. However, due to their generic functional forms and extreme flexibility, they can catastrophically fail to capture the properties of novel, out-of-sample configurations, making the quality of the training set a determining factor, especially when investigating materials under extreme conditions. We propose a novel automated dataset generation method based on the maximization of the information entropy of the feature distribution, aiming at an extremely broad coverage of the configuration space in a way that is agnostic to the properties of specific target materials. The ability of the dataset to capture unique material properties is demonstrated on a range of unary materials, including elements with the FCC (Al), BCC (W), HCP (Be, Re and Os), graphite (C), and trigonal (Sb, Te) ground states. MLIAPs trained to this dataset are shown to be accurate over a range of application-relevant metrics, as well as extremely robust over very broad swaths of configurations space, even without dataset fine-tuning or hyper-parameter optimization, making the approach extremely attractive to rapidly and autonomously develop general-purpose MLIAPs suitable for simulations in extreme conditions.

36 MATERIALS SCIENCE↗
NASA NTRS1997

Uncertain responses by humans and rhesus monkeys (Macaca mulatta) in a psychophysical same-different task

The authors asked whether animals, like humans, use an uncertain response adaptively to escape indeterminate stimulus relations. Humans and monkeys were placed in a same-different task, known to be challenging for animals. Its difficulty was increased further by reducing the size of the stimulus differences, thereby making many same and different trials difficult to tell apart. Monkeys do escape selectively from these threshold trials, even while coping with 7 absolute stimulus levels concurrently. Monkeys even adjust their response strategies on short time scales according to the local task conditions. Signal-detection and optimality analyses confirm the similarity of humans' and animals' performances. Whereas associative interpretations account poorly for these results, an intuitive uncertainty construct does so easily. The authors discuss the cognitive processes that allow uncertainty's adaptive use and recommend further comparative studies of metacognition.

NASA Discipline Space Human Factors↗
NASA NTRSDate not supplied

An LES Survey of Cold-Air Outbreaks Spanning ACTIVATE and COMBLE

Two recent field campaigns offer unique new opportunities to survey coupled aerosol and cloud processes operating within cold-air outbreaks (CAOs). NASA's 2019-2023 Aerosol Cloud meTeorology Interactions oVer the western Atlantic Experiment (ACTIVATE) project provided airborne in situ and remote sensing observations in multiple CAOs off the central US Eastern Seaboard. Contemporaneously, DOE's 2019-2020 Cold-air Outbreaks in the Marine Boundary Layer Experiment (COMBLE) provided continuous ground-based in situ and remote sensing measurements during hundreds of hours of CAO conditions at two sites flanking the Norwegian Sea. Here we highlight lessons learned from contrasting sets of aerosol-aware Lagrangian large-eddy simulation case studies derived from each campaign, as well as outstanding uncertainties. Cases simulated include all of the most extensively sampled CAO flights during ACTIVATE, and a wide range of CAO strengths and mesoscale structures during COMBLE. Each case study is also suitable for simulation with large-scale models in single-column model mode or with a limited-domain approach using periodic boundary conditions. We seek to address the central question: what microphysical process pathways control the Lagrangian evolution of surface and top-of-atmosphere radiative fluxes under ACTIVATE and COMBLE conditions? Many science team members, co-investigators, and collaborators will be gratefully acknowledged in association with their contributions to making this work possible.

cold-air outbreaks↗
DOE OSTI2021

Operating a commercial building HVAC load as a virtual battery through airflow control

Virtual battery (VB) is an innovative method to model flexibility of building loads and effectively coordinate them with other resources at a system level. Unlike a real battery with a dedicated power conversion system for charging control, methods are required for operating building loads to deviate from the baseline to respond to grid signals. This paper presents a VB control for a commercial heating, ventilation, and air conditioning (HVAC) system to follow the desired power consumption in real-time by adjusting zonal airflow rates. The proposed method consists of two parts. At the system level, a mixed feedforward and feedback control is used to estimate the desired total airflow rate. At the zone level, two priority-based algorithms are then proposed to distribute the total airflow rate to individual zones. In particular, a zonal airflow limit estimation method is proposed using machine-learning techniques, in contrast to physics-based thermal models in existing studies, to more accurately capture zonal thermal dynamics and improve temperature control performance. An office building on the Pacific Northwest National Laboratory campus is implemented in EnergyPlus, and used to illustrate and validate the proposed control.

Wang, Jiyu↗
NASA NTRS1991

Identification of linear multivariable systems from a single set of data by identification of observers with assigned real eigenvalues

A formulation is presented for identification of linear multivariable from a single set of input-output data. The identification method is formulated with the mathematical framework of learning identifications, by extension of the repetition domain concept to include shifting time intervals. This method contrasts with existing learning approaches that require data from multiple experiments. In this method, the system input-output relationship is expressed in terms of an observer, which is made asymptotically stable by an embedded real eigenvalue assignment procedure. Through this relationship, the Markov parameters of the observer are identified. The Markov parameters of the actual system are recovered from those of the observer, and then used to obtain a state space model of the system by standard realization techniques. The basic mathematical formulation is derived, and numerical examples presented to illustrate.

Phan, Minh↗
DOE OSTI2022

Feedback Control Approaches for Restoration of Power Grids from Blackouts

The automated restoration of power systems with variable energy resources is a timely problem to tackle. Automated restoration advice can support operators in deciding on strategic actions to restore power grids from a blackout with a mix of conventional and renewable generation resources. To this end, this paper frames the restoration process of power grids with solar resources as a nonlinear dynamic model with algebraic constraints in discrete time which is steered by feedback control loops. We discuss two feedback-control strategies based on greedy and reinforcement learning algorithms, and contrast their performance with restoration plans generated by a mixed-integer linear program. We found that the reinforcement learning algorithm infers restoration actions faster than the greedy one. However, the tuning process of the reinforcement learning parameters is slower than for the greedy one.

machine learning↗
DOE OSTI2025

A Study on Contrastive Graph Neural Network Pretraining for Predicting Transcriptome Profiles

We study graph neural network learning for transcriptomics with limited amount of labeled data. Our study reveals that simple GNN architectures perform well and do not suffer from over-fitting as the more sophisticated ones. Our study shows that although contrastive learning as a pretraining strategy has been successful in predicting properties such as formation and binding energy, it is not effective for transcriptomics.

Ma, Jiaji [University of Virginia]↗