Search NASA⌕ Search

SEARCH · Search NASA

Results for “scientific training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

One Million Open-source Cislunar Orbits

Cislunar space, encompassing the region from geosynchronous orbit to beyond the Moon, is poised to become a cornerstone for future exploration, scientific discovery, and national security. Missions in this region, spanning durations from weeks to decades, require robust infrastructure and reliable transit capabilities. The complex gravitational influences of the Moon, Sun, and planets, along with thermal radiation from Earth and the Sun, lead to significant trajectory deviations, resulting in kilometer-scale errors within days. Leveraging the high-performance computing resources at Lawrence Livermore National Laboratory (LLNL), we have simulated one million high-fidelity cislunar trajectories, now publicly available via LLNL’s Green Data Oasis and the Unified Data Library. Generated using the open-source Space Situational Awareness Python package, these trajectories match the precision of commercial tools such as AGI’s Systems Tool Kit and NASA’s General Mission Analysis Tool. This data set is a valuable resource for reference, statistical analysis of cislunar orbit populations, and training machine learning models for rapid orbit classification with minimal observational input. Preliminary analysis reveals stable bands in Keplerian element space, particularly around five geosynchronous radii across a range of inclinations and eccentricities. Beyond this threshold, the Moon’s influence disrupts most unassisted orbits, though co-orbiting L4/L5 Lunar Trojans persist throughout the six-year simulation.

Astronomy and AstroPhysics↗
DOE OSTI2023

Development of Scalable Reactive Transport Framework for PFAS

Research on PFAS chemicals is extensive and covers all aspects, including analytical quantification, determination of properties, toxicology, ex situ treatment, and in situ remediation. PFAS chemicals are very stable and thus persistent/recalcitrant in the environment. Although there are many unknowns about PFAS chemicals, degradation pathways, reaction rates, etc., many different sorption, oxidation, reduction, biological, and innovative treatment approaches are being developed. Sorption with activated carbon is currently the only fully available in situ treatment technology for PFAS-impacted groundwater. Given the wide array of PFAS chemicals and transformation products, remediation may need multi-step treatment trains to fully address the PFAS contamination. The work here provides kinetic reaction modules that represent an initial cut at functionality representing PFAS migration and reaction in groundwater aquifer flow and transport models. One reaction kinetics module provides a method to model kinetically limited adsorption using a mass transfer model. The second reaction module represents biological transformation of 8:2 FTOH and daughter species, illustrating how a complex reaction pathway network can be represented. Both reaction modules allow for spatially variable parameter values so that a variety of remediation approaches (e.g., a PRB or volumetric treatment or variations in geochemical conditions) can be simulated. The intent with these PFAS reaction modules is to provide tools for practitioners to aid in the selection, design, and assessment of potential in situ PFAS remediation strategies. It is anticipated that, as PFAS remediation technologies and scientific understanding advances, these modules would be refined or replaced to match new knowledge.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Machine Learning for DUNE Supernova Trigger

One of the major scientific goals of the Deep Underground Neutrino Experiment (DUNE) is to detect and measure the neutrino flux originating from galactic core-collapse supernovae. These neutrinos provide an opportunity to study the end of life evolution of massive stars, and reveal information about the structure of core-collapse that is not visible in observations of the electromagnetic spectrum. Because of the rarity of these events, it is crucial that DUNE is able to detect supernova neutrino interactions when they occur. However, this will require sifting through a large quantity of data, motivating the development of a trigger algorithm to identify significant events and discard irrelevant data. Machine learning provides a potential approach to building this trigger. This project generates ADC and ground truth images of simulated neutrino interactions in a LArTPC detector to be used for machine learning, and uses them to train a sparse Convolutional Neural Network (C NN). The performance of this model when applied to the task of pixel classification based on interaction type is examined. This project found that the sparse CNN approach has the potential to have high accuracy in pixel classification, meaning it may be highly relevant to the development of a supernova neutrino trigger for the DUNE far detector.

Damish, S.↗
DOE OSTI2021

PPPL Laboratory Directed Research and Development (Project Final Reports, FY2018 - FY2020)

The U.S. Department of Energy’s (DOE) Princeton Plasma Physics Laboratory (PPPL) is a collaborative national center for fusion energy science, basic sciences, and advanced technology. The Laboratory has three major missions: (1) to develop the scientific knowledge and advanced engineering to enable fusion to power the U.S. and the world; (2) to advance the science of nanoscale fabrication for technologies of tomorrow; and (3) to further the development of the scientific understanding of the plasma universe from laboratory to astrophysical scales. PPPL’s Laboratory Directed Research and Development (LDRD) program supports and encourages creativity and innovation and contributes to its long-term viability. New scientific and technical research areas emerge and are nurtured through the program. Furthermore, new capabilities are developed to enable the Laboratory to meet its and DOE’s missions. The program is used to systematically diversify the Laboratory’s programs and mission. In the last few years, the program has started projects in nanomaterial synthesis, microelectronics, advanced x-ray spectroscopy, high-energy-density physics, superconducting magnet technology, machine learning and artificial intelligence, 3D magnetic fields to optimize fusion plasmas, integration of permanent magnets with simple high-field magnets to reduce the cost of producing complex 3D magnetic fields, advanced computational methods for predictive understanding and control of fusion plasma, development of quantum computing algorithms for plasma physics, liquid metal plasma-facing components for fusion reactors, virtual engineering, and plasma-based space propulsion. The program is also the vehicle to recruit and train talented scientists and engineers with the new skills needed to perform the Laboratory’s mission. Many of the new hires through the program go on to become world-class scientists and engineers in their fields. This report provides descriptions and accomplishments of those LDRD projects that were completed during fiscal years 2018 through 2020.

36 MATERIALS SCIENCE↗
DOE OSTI2024

A Comprehensive Review of Latent Space Dynamics Identification Algorithms for Intrusive and Non-Intrusive Reduced-Order-Modeling

Numerical solvers of partial differential equations (PDEs) have been widely employed for simulating physical systems. However, the computational cost remains a major bottleneck in various scientific and engineering applications, which has motivated the development of reduced-order models (ROMs). Recently, machine-learning-based ROMs have gained significant popularity and are promising for addressing some limitations of traditional ROM methods, especially for advection dominated systems. In this chapter, we focus on a particular framework known as Latent Space Dynamics Identification (LaSDI), which transforms the high-fidelity data, governed by a PDE, to simpler and low-dimensional latent-space data, governed by ordinary differential equations (ODEs). These ODEs can be learned and subsequently interpolated to make ROM predictions. Each building block of LaSDI can be easily modulated depending on the application, which makes the LaSDI framework highly flexible. In particular, we present strategies to enforce the laws of thermodynamics into LaSDI models (tLaSDI), enhance robustness in the presence of noise through the weak form (WLaSDI), select high-fidelity training data efficiently through active learning (gLaSDI, GPLaSDI), and quantify the ROM prediction uncertainty through Gaussian processes (GPLaSDI). We demonstrate the performance of different LaSDI approaches on Burgers equation, a non-linear heat conduction problem, and a plasma physics problem, showing that LaSDI algorithms can achieve relative errors of less than a few percent and up to thousands of times speed-ups.

Computational Engineering, Finance, and Science (c↗
DOE OSTI2025

Nature of innovations affecting photovoltaic system costs

Innovations improve technology costs through various kinds of engineering advancements, including changes to materials choices and device or process designs. Understanding how these innovations relate to cost change can reveal aspects of the process of technology evolution, yet developing such understanding is often not possible with a strictly quantitative approach due to data limitations. In this paper we develop a hybrid quantitative-qualitative framework for relating specific innovations to cost change by using the variables in a quantitative technology cost change model as an organizing principle. We demonstrate this framework by applying it to the cost decline in photovoltaic (PV) systems over the last five decades. This framework generates new understanding of a set of innovations that contributed to PV modules’ sustained cost decline and the more modest trends observed in balance-of-system (BOS) costs. The results show the great diversity of innovations that affected PV costs, drawing on wide-ranging fields of expertise within scientific research and practice. We find that there are differences in the characteristics of innovations that reduced the cost of PV modules compared to innovations influencing BOS costs. Numerous module innovations reduced costs by advancing manufacturing tools and processes that improved material quality. Many BOS innovations reduced costs through a combination of component design changes, integration, automation, digitalization, and standardization. Overall, most innovations in our sample affected PV hardware. However, some also target ‘soft technologies’ such as task durations through innovations like fast-track permitting, which require improved collaboration and process streamlining. This framework also provides insight into the nature of knowledge spillovers between technologies. Both module and BOS hardware innovations show the benefits of PV’s position within an ‘ecosystem’ of continuously advancing technologies in many industries, in particular semiconductors and electronics, and also point to the importance of public institutions for accelerating testing, permitting, and training.

14 SOLAR ENERGY↗
DOE OSTI2022

Model-Form Epistemic Uncertainty Quantification for Modeling with Differential Equations: Application to Epidemiology

Modeling real-world phenomena to any degree of accuracy is a challenge that the scientific research community has navigated since its foundation. Lack of information and limited computational and observational resources necessitate modeling assumptions which, when invalid, lead to model-form error (MFE). The work reported herein explored a novel method to represent model-form uncertainty (MFU) that combines Bayesian statistics with the emerging field of universal differential equations (UDEs). The fundamental principle behind UDEs is simple: use known equational forms that govern a dynamical system when you have them; then incorporate data-driven approaches – in this case neural networks (NNs) – embedded within the governing equations to learn the interacting terms that were underrepresented. Utilizing epidemiology as our motivating exemplar, this report will highlight the challenges of modeling novel infectious diseases while introducing ways to incorporate NN approximations to MFE. Prior to embarking on a Bayesian calibration, we first explored methods to augment the standard (non-Bayesian) UDE training procedure to account for uncertainty and increase robustness of training. In addition, it is often the case that uncertainty in observations is significant; this may be due to randomness or lack of precision in the measurement process. This uncertainty typically manifests as “noisy” observations which deviate from a true underlying signal. To account for such variability, the NN approximation to MFE is endowed with a probabilistic representation and is updated using available observational data in a Bayesian framework. By representing the MFU explicitly and deploying an embedded, data-driven model, this approach enables an agile, expressive, and interpretable method for representing MFU. In this report we will provide evidence that Bayesian UDEs show promise as a novel framework for any science-based, data-driven MFU representation; while emphasizing that significant advances must be made in the calibration of Bayesian NNs to ensure a robust calibration procedure.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

N 2 Onet: a global collaborative network facilitating advances in measurement, modeling, and mitigation of agricultural soil nitrous oxide emissions

Nitrogen (N) fertilizer supports global food production, but its use and overuse drive emissions of nitrous oxide (N 2 O), a potent and long-lived greenhouse gas. Understanding the drivers of N 2 O fluxes remains elusive, making it difficult to predict emissions in time and space and to develop and evaluate ways to lower emissions through management. Major scientific uncertainties underlying the understanding of the drivers of N 2 O fluxes identified in a workshop of N 2 O emissions experts include poor process-based understanding of controls on soil N 2 O emissions in the field; insufficient data to reduce uncertainty in N 2 O budgets from the field to regional scales, including N 2 O emission measurements and importantly, field-scale N balances; and high uncertainty in model predictions of soil N 2 O emissions across environmental and management conditions. To reduce these uncertainties, we present the concept of N 2 Onet, a global collaborative initiative to accelerate advances in N 2 O measurement, analyses, and mitigation. N 2 Onet will serve as an observational network of supersites with multi-scale measurements; a database hub for N 2 O flux and ancillary data; and a catalyst for community building, information sharing, and training. By coalescing and coordinating the global community of researchers, N 2 Onet will provide a roadmap for reducing N 2 O emissions from agriculture worldwide.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

MOFSimplify, machine learning models with extracted stability data of three thousand metal–organic frameworks

Abstract We report a workflow and the output of a natural language processing (NLP)-based procedure to mine the extant metal–organic framework (MOF) literature describing structurally characterized MOFs and their solvent removal and thermal stabilities. We obtain over 2,000 solvent removal stability measures from text mining and 3,000 thermal decomposition temperatures from thermogravimetric analysis data. We assess the validity of our NLP methods and the accuracy of our extracted data by comparing to a hand-labeled subset. Machine learning (ML, i.e. artificial neural network) models trained on this data using graph- and pore-geometry-based representations enable prediction of stability on new MOFs with quantified uncertainty. Our web interface, MOFSimplify, provides users access to our curated data and enables them to harness that data for predictions on new MOFs. MOFSimplify also encourages community feedback on existing data and on ML model predictions for community-based active learning for improved MOF stability models.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Plant Cell Biology International Meeting 2022

There is a strong need to develop the knowledge base that will enable the rational design of improved crop species for sustainable agriculture and biomass production. To attain this goal, researchers must learn how plant cells integrate metabolism and cellular dynamics during growth and in response to environmental challenges. Meeting these goals require data sharing and the development and training of faculty, post-doctoral fellows, graduate and undergraduate students in state-of-the-art quantitative integrative cell biology methods. The inaugural Plant Cell Biology International (PCBI) meeting capitalized on the interaction between two well-established plant cell biology communities across the Atlantic, the Midwest Plant Cell Dynamics (PCD) and the European Network for Plant Endomembrane Research (ENPER) and provided a framework to meet these goals. The conference took place in person successfully during August 1-5th 2022 in Crete, Greece. The conference offered opportunities for undergraduate and graduate students, postdocs, young investigators across the globe to present their results and interact across different disciplines. The conference provided an open and inclusive forum for oral presentations that allowed all labs to share research results. It was attended over 80 % by students, postdocs and young investigators. Workshops on best practices for image acquisition and processing pipelines, publication guidelines by scientific journal editors, professional development and well-being benefited the participants at large. Overall, this was the inaugural joint Plant Cell Biology International conference that for the first time officially brought together plant cell biologists to discuss opportunities and challenges in the field to gain a mechanistic understanding of complex biological processes and advance sustainable agriculture. Due to its success, it was rescheduled for 2025 in Crete, to continue strengthening the plant cell biology community and exchanging knowledge and expertise across the globe.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Method to simultaneously facilitate all jet physics tasks

Machine learning has become an essential tool in jet physics. Due to their complex, high-dimensional nature, jets can be explored holistically by neural networks in ways that are not possible manually. However, innovations in all areas of jet physics are proceeding in parallel. We show that specially constructed machine learning models trained for a specific jet classification task can improve the accuracy, precision, or speed of all other jet physics tasks. This is demonstrated by training on a particular multiclass generation and classification task and then using the learned representation for different generation and classification tasks, for datasets with a different (full) detector simulation, for jets from a different collision system ($pp$ versus $ep$), for generative models, for likelihood ratio estimation, and for anomaly detection. We consider our omnilearn approach thus as a jet-physics foundation model. It is made publicly available for use in any area where state-of-the-art precision is required for analyses involving jets and their substructure.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

ThunderSecure: deploying real-time intrusion detection for 100G research networks by leveraging stream-based features and one-class classification network

Nowadays, data generated by large-scale scientific experiments are on the scale of petabytes per month. These data are transferred through dedicated high-bandwidth networks (40/100G) across distributed sites for processing, storage, and analysis. Like general purpose networks, research networks experience intrusions. However, monitoring anomalies in such high-speed network traffics is challenging given current cyber-infrastructure. Moreover, traditional network intrusion detection systems (NIDS) are signature based. However, anomaly patterns are difficult to define and that rulesets are often not updated frequently enough to reflect the changes of attack behaviors. We present ThunderSecure, a high-throughput, unsupervised learning-based intrusions detection system for 100G research networks. ThunderSecure implements an efficient packet processing and detection pipeline using multi-cores and GPUs. It extracts statistical and temporal features from real-time network data streams and feeds them to a one-class anomaly detection network. A baseline of normal distribution will be created based on the training observation. Testing traffic deviated from the learned profile will be marked as anomalies. We trained ThunderSecure on hundreds of billions of science data packets mirrored from two 100G network connections at Fermi National Accelerator Laboratory. The detection performance was evaluated on traffic captured from the same research network days and weeks after the training with different types of attack flows injected. Results show that ThunderSecure can recognize science data traffic captured long after the training and made nearly certain detection on the segment of the streams where anomalous flows were injected.

100G research network↗
DOE OSTI2021

Randomized Algorithms for Scientific Computing (RASC)

Randomized algorithms have propelled advances in artificial intelligence (AI) and represent a foundational research area in advancing AI for Science. Future advancements in DOE Office of Science priority areas such as climate science, astrophysics, fusion, advanced materials, combustion, and quantum computing all require randomized algorithms for surmounting challenges of complexity, robustness, and scalability. Advances in data collection and numerical simulation have changed the dynamics of scientific research and motivate the need for randomized algorithms. For instance, advances in imaging technologies such as X-ray ptychography, electron microscopy, electron energy loss spectroscopy, or adaptive optics lattice light-sheet microscopy collect hyperspectral imaging and scattering data in terabytes, at breakneck speed enabled by state-of-the-art detectors. The data collection is exceptionally fast compared with its analysis. Likewise, advances in high-performance architectures have made exascale computing a reality and changed the economies of scientific computing in the process. Floating-point operations that create data are essentially free in comparison with data movement. Thus far, most approaches have focused on creating faster hardware. Ironically, this faster hardware has exacerbated the problem by making data still easier to create. Under such an onslaught, scientists often resort to heuristic deterministic sampling schemes (e.g., low-precision arithmetic, sampling every nth element) and sacrifice potentially valuable accuracy. Dramatically better results can be achieved via randomized algorithms, reducing the data size as much as or more than naive deterministic subsampling can achieve, while retaining the high accuracy of computing on the full data set. By randomized algorithms we mean those algorithms that employ some form of randomness in internal algorithmic decisions to accelerate time to solution, increase scalability, or improve reliability. Examples include matrix sketching for solving large-scale least-squares problems (see Figure 1) and stochastic gradient descent for training machine learning models. We are not recommending heuristic methods but rather randomized algorithms that have certificates of correctness and probabilistic guarantees of optimality and near-optimality. Such approaches can be useful beyond acceleration, for example, in understanding how to avoid measure zero worst-case scenarios that plague methods such as QR matrix factorization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Microsecond-latency feedback at a particle accelerator by online reinforcement learning on hardware

The commissioning and operation of future large-scale scientific experiments will challenge current tuning and control methods. Reinforcement learning (RL) algorithms are a promising solution due to their ability to dynamically adapt to changing environments and consider delayed consequences. In many real-world applications, RL policies must produce actions in real time, often within microseconds to milliseconds, imposing significant constraints on system latency and computational overhead that conventional machine learning libraries are not designed to handle. To control phenomena in real time at these timescales, RL needs to be deployed on-the-edge, namely on dedicated hardware located near the system it controls, without relying on a host CPU or cloud-based inference. In this work we present the design and deployment of an experience accumulator system in a particle accelerator. In this system, deep-RL algorithms run using hardware acceleration and act within a few microseconds, enabling the use of RL for control of phenomena like beam instabilities. The training uses the collected data offline to reduce the number of operations carried out on the acceleration hardware. The proposed architecture was tested in real experimental conditions at the Karlsruhe research accelerator, a synchrotron light source, where the system was used to control artificially induced horizontal betatron oscillations in real-time, with a control loop period of just 2.7 μs. The results showed a performance comparable to the commercial feedback system available at the accelerator, demonstrating the viability and potential of this approach. Due to the self-learning and reconfiguration capability of this implementation, a seamless application to other control problems is possible. Applications range from particle accelerators to large-scale research and industrial facilities.

FPGA↗
DOE OSTI2020

Clustering-Based Predictive Analytics to Improve Scientific Data Discovery

Given the sheer volume of scientific data archived within the data-intensive projects at the US Department of Energy's Oak Ridge National Laboratory, finding precisely what data we are looking for may not be a trivial task; conversely, we may also miss a more prominent data product. To address such issues, we propose improving the data discovery system and using data analytics methods to comprehend what specific users might be interested in based on their physiological state, search patterns, and past data usage history. This work's primary goal is to prune the complexity, increase the visibility of popular data products, and direct users toward the data that best meet their needs. The proposed algorithm constructs a user profile based on the user's explicit or implicit interactions with the system, such as items they are currently looking at on-site and the key metadata mappings related to the data set. The pattern is then used to build a training data set, which will help find relevant data to recommend to the user.

Devarakonda, Ranjeet↗
DOE OSTI2020

Developing a simulator-based satellite dataset for using machine learning techniques to derive aerosol-cloud-precipitation interactions in models and observations in a consistent framework

Aerosol-cloud-precipitation interactions (ACPI) remain a major uncertainty in understanding the Earth’s radiation budget and water cycle (including extremes). After decades of active research, various observationally based metrics have been developed to constrain ACPI in Earth System Models (ESMs), but direct comparison of model and data estimates can confound scientific understanding because limitations and uncertainties in sampling and retrieval procedures may combine with model deficiencies in process representations of ACPI to obstruct understanding. Furthermore, conventional ACPI metrics often vary from one regime to another, and the ACPI process representation in ESMs is also typically derived based on only a limited area/regime (even though the parameterization applies globally). To bridge the gap between models and data and to correctly describe ACPI in all regimes, we propose to construct a new CALIPSO-CloudSat merged dataset that is produced by the same algorithms used in satellite simulators in ESMs, and to use machine learning techniques to derive new ACPI metrics that can be accurately estimated by satellites and can provide meaningful constraints on cloud microphysical process representations in ESMs. The dataset will include measured and retrieved variables for aerosol, cloud, and precipitation from CALIPSO and CloudSat, and environmental variables from meteorological reanalysis. The data will be used to train a neural network to construct the ACPI metrics as a function of environmental conditions. The new ACPI formula will be used to constrain the ACPI in the Energy Exascale Earth System Model (E3SM), and to augment/reformulate the ACPI process representation in the E3SM to improve the simulation of the evolution of the atmosphere under different environmental conditions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

PlasmoData.jl — A Julia framework for modeling and analyzing complex data as graphs

Datasets encountered in scientific and engineering applications appear in complex formats (e.g., images, multivariate time series, molecules, video, text strings, networks). Graph theory provides a unifying framework to model such datasets and enables the use of powerful tools that can help analyze, visualize, and extract value from data. In this work, we present PlasmoData.jl, an open-source, Julia framework that uses concepts of graph theory to facilitate the modeling and analysis of complex datasets. The core of our framework is a general data modeling abstraction, which we call a DataGraph. We show how the abstraction and software implementation can be used to represent diverse data objects as graphs and to enable the use of tools from topology, graph theory, and machine learning (e.g., graph neural networks) to conduct a variety of tasks. We illustrate the versatility of the framework by using real datasets: (i) an image classification problem using topological data analysis to extract features from the graph model to train machine learning models; (ii) a disease outbreak problem where we model multivariate time series as graphs to detect abnormal events; and (iii) a technology pathway analysis problem where we highlight how we can use graphs to navigate connectivity. Further, our discussion also highlights how PlasmoData.jl leverages native Julia capabilities to enable compact syntax, scalable computations, and interfaces with diverse packages. Overall, we show that the DataGraph abstraction and PlasmoData.jl Julia package are able to model data within graphs and enable useful analysis.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2023

KEBLM: Knowledge-Enhanced Biomedical Language Models

Pretrained language models (PLMs) have demonstrated strong performance on many natural language processing (NLP) tasks. Despite their great success, these PLMs are typically pretrained only on unstructured free texts without leveraging existing structured knowledge bases that are readily available for many domains, especially scientific domains. As a result, these PLMs may not achieve satisfactory performance on knowledge-intensive tasks such as biomedical NLP. Comprehending a complex biomedical document without domain-specific knowledge is challenging, even for humans. Inspired by this observation, we propose a general framework for incorporating various types of domain knowledge from multiple sources into biomedical PLMs. We encode domain knowledge using lightweight adapter modules, bottleneck feed-forward networks that are inserted into different locations of a backbone PLM. For each knowledge source of interest, we pretrain an adapter module to capture the knowledge in a self-supervised way. We design a wide range of self-supervised objectives to accommodate diverse types of knowledge, ranging from entity relations to description sentences. Once a set of pretrained adapters is available, we employ fusion layers to combine the knowledge encoded within these adapters for downstream tasks. Each fusion layer is a parameterized mixer of the available trained adapters that can identify and activate the most useful adapters for a given input. Our method diverges from prior work by including a knowledge consolidation phase, during which we teach the fusion layers to effectively combine knowledge from both the original PLM and newly-acquired external knowledge using a large collection of unannotated texts. After the consolidation phase, the complete knowledge-enhanced model can be fine-tuned for any downstream task of interest to achieve optimal performance. Extensive experiments on many biomedical NLP datasets show that our proposed framework consistently improves the performance of the underlying PLMs on various downstream tasks such as natural language inference, question answering, and entity linking. These results demonstrate the benefits of using multiple sources of external knowledge to enhance PLMs and the effectiveness of the framework for incorporating knowledge into PLMs. Finally, while primarily focused on the biomedical domain in this work, our framework is highly adaptable and can be easily applied to other domains, such as the bioenergy sector.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗