SEARCH · Search NASA

Results for “ensemble distillation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

10 records

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Using ensembles and distillation to optimize the deployment of deep learning models for the classification of electronic cancer pathology reports

One of the goals of the Surveillance, Epidemiology, and End Results (SEER) program is to estimate incidence, prevalence, and mortality of all cancers. To that end, cancer registries across the country maintain a massive database of cancer pathology reports which contain rich information to understand cancer trends. However, these reports are stored in the form of unstructured text, and human annotators are required to read and extract relevant information. In this article, we show that existing deep learning models for automating information extraction from cancer pathology reports can be significantly improved by using ensemble model distillation. We found that by training multiple predictive models and transferring their knowledge to a single, low-resource model, we can reduce the number of highly confident wrong predictions. Our results show that our implemented methods could save 1000s of manual annotation hours.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2022

Distilling Knowledge from Ensembles of Cluster-Constrained-Attention Multiple-Instance Learners for Whole Slide Image Classification

The peculiar nature of whole slide imaging (WSI), digitizing conventional glass slides to obtain multiple high resolution images which capture microscopic details of a patient’s histopathological features, has garnered increased interest from the computer vision research community over the last two decades. Given the unique computational space and time complexity inherent to gigapixel-size whole slide image data, researchers have proposed novel machine learning algorithms to aid in the performance of diagnostic tasks in clinical pathology. One effective algorithm represents a Whole slide image as a bag of smaller image patches, which can be represented as low-dimension image patch embeddings. Weakly supervised deep-learning methods, such as cluster-constrained-attention multiple instance learning (CLAM), have shown promising results when combined with image patch embeddings. While traditional ensemble classifiers yield improved task performance, such methods come with a steep cost in model complexity. Through knowledge distillation, it is possible to retain some performance improvements from an ensemble, while minimizing costs to model complexity. In this work, we implement a weakly supervised ensemble using clustering-constrained-attention multiple-instance learners (CLAM), which uses attention and instance-level clustering to identify task salient regions and feature extraction in whole slides. By applying logit-based and attention-based knowledge distillation, we show it is possible to retain some performance improvements resulting from the ensemble at zero cost to model complexity.

Alamudun, Folami↗

DOE OSTI2022

Distilling physical origins of hardness in multi-principal element alloys directly from ensemble neural network models

Abstract Despite a plethora of data being generated on the mechanical behavior of multi-principal element alloys, a systematic assessment remains inaccessible via Edisonian approaches. We approach this challenge by considering the specific case of alloy hardness, and present a machine-learning framework that captures the essential physical features contributing to hardness and allows high-throughput exploration of multi-dimensional compositional space. The model, tested on diverse datasets, was used to explore and successfully predict hardness in Al x Ti y (CrFeNi) 1- x - y , Hf x Co y (CrFeNi) 1- x - y and Al x (TiZrHf) 1- x systems supported by data from density-functional theory predicted phase stability and ordering behavior. The experimental validation of hardness was done on TiZrHfAl x . The selected systems pose diverse challenges due to the presence of ordering and clustering pairs, as well as vacancy-stabilized novel structures. We also present a detailed model analysis that integrates local partial-dependencies with a compositional-stimulus and model-response study to derive material-specific insights from the decision-making process.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2022

Shallow Cumulus Entrainment Dynamics in a Sheared Environment

Abstract Vertical wind shear has long been known to tilt convective towers and reduce thermal ascent rates. The purpose of this study is to better understand the physical mechanisms responsible for reduced ascent rates in shallow convection. In particular, the study focuses on cloud-edge mass flux to assess how shear impacts mass-flux profiles of both the ensemble and individual clouds of various depths. A compositing algorithm is used to distill large-eddy simulation (LES) output to focus on up- and down-shear cloud edges that are not influenced by complex cloud geometry or nearby clouds. A direct entrainment algorithm is used to estimate the mass flux through the cloud surface. We find that the dynamics on the up- and down-shear sides are fundamentally different, with the entrainment of environmental momentum and dilution of buoyancy being primarily responsible for the reduced down-shear ascent rates. Direct estimates of fluid flow through the cloud interface indicate a counter-shear organized flow pattern that entrains on the down-shear side and detrains on the up-shear side, resulting from the subcloud shear being lifted into the cloud layer by the updraft. In spite of organized regions of entrainment and detrainment, the overall net lateral mass flux remains unchanged with respect to the no shear run, with weak detrainment present throughout cloud depth.

Meteorology & Atmospheric Sciences↗

DOE OSTI2026

Accessing the gluon momentum fraction of nucleons through the gradient flow

We calculate the gluon momentum fraction of the nucleon using lattice QCD, with a nonperturbative renormalization technique based on the gradient flow. The gluon momentum fraction is determined on a single Wilson-clover ensemble using 𝑁 𝑓 =2 +1 flavors with pion mass 358 MeV and lattice spacing 0.094 fm. We employ the variational method to reduce excited-state contamination and apply the distillation framework to ensure a large operator basis. To reduce systematic uncertainties, we apply Bayesian model averaging to all fit procedures. We apply matching coefficients to the flow-time dependent lattice results to recover the gluon momentum fraction in the $\overline{MS}$-scheme at 2 GeV. Our final result is ⟨𝑥⟩ 𝑔 ⁢(𝜇 =2 GeV) =0.482⁢(35), where we quote only statistical uncertainties.

Lattice QCD↗

DOE OSTI2021

Towards high-precision parton distributions from lattice QCD via distillation

We apply the Distillation spatial smearing program to the extraction of the unpolarized isovector valence PDF of the nucleon. The improved volume sampling and control of excited-states afforded by distillation leads to a dramatically improved determination of the requisite Ioffe-time Pseudo-distribution (pITD). The impact of higher-twist effects is subsequently explored by extending the Wilson line length present in our non-local operators to one half the spatial extent of the lattice ensemble considered. The valence PDF is extracted by analyzing both the matched Ioffe-time Distribution (ITD), as well as a direct matching of the pITD to the PDF. Through development of a novel prescription to obtain the PDF from the pITD, we establish a concerning deviation of the pITD from the expected DGLAP evolution of the pseudo-PDF. The presence of DGLAP evolution is observed once more following introduction of a discretization term into the PDF extractions. Observance and correction of this discrepancy further highlights the utility of distillation in such structure studies.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NASA NTRS1986

Interstellar fossil Mg-26 and its possible relationship to excess meteoritic Mg-26

A plausible scenario is advanced for explainig a linear correlation found in some solar system solids between their Mg-26/Mg-24 isotopic ratios and their Al/Mg elemental abundance ratios. This scenario involves three stages: (1) the mechanical aggregation of an average ensemble of Al-bearing dust particles that is postulated to be modestly enriched in the Al/Mg abundance ratio because the aggregated particles themselves are; (2) the extraction, perhaps but not necessarily by hot distillation, of almost all Mg, leaving an aggregate with a large Al/Mg ratio and a large Mg-26 excess; and (3) the uptake of normal ambient Mg by the resulting hot Al-rich solid as it cools in Mg-rich vapor. A linear correlation in solids between their Mg-26/Mg-24 isotopic ratio and their aluminum enrichment may be a fossil correlation inherited from interstellar dust.

Clayton, Donald D.↗

DOE OSTI2024

Distilling the Evolving Contributions of Anthropogenic Aerosols and Greenhouse Gases to Large‐Scale Low‐Frequency Surface Ocean Changes Over the Past Century

Abstract Anthropogenic aerosols (AER) and greenhouse gases (GHG)—the leading drivers of the forced historical change—produce different large‐scale climate response patterns, with correlations trending from negative to positive over the past century. To understand what caused the time‐evolving comparison between GHG and AER response patterns, we apply a low‐frequency component analysis to historical surface ocean changes from CESM1 single‐forcing large‐ensemble simulations. While GHG response is characterized by its first leading mode, AER response consists of two distinct modes. The first one, featuring long‐term global AER increase and global cooling, opposes GHG response patterns up to the mid‐twentieth century. The second one, featuring multidecadal variations in AER distributions and interhemispheric asymmetric surface ocean changes, appears to reinforce the GHG warming effect over recent decades. AER thus can have both competing and synergistic effects with GHG as their emissions change temporally and spatially.

Dong, Yue↗

DOE OSTI2020

Mechanistic understanding of catalytic conversion of ethanol to 1-butene over 2D-pillared MFI zeolite

Ethanol is an important C2 platform molecule for producing value-added chemicals and distillate hydrocarbon fuels (e.g., jet and diesel). Among these, catalytic upgrading of ethanol to butenes can generate valuable commodity chemicals (e.g., 1-butene) and provide C4 olefin intermediates that can be further upgraded to jet/diesel fuels. 2D zeolites offer hierarchical mesoporous structures, leading to improved mass transport and reduced diffusion length, which can help to address the coking challenges faced by ethanol conversion to hydrocarbons over 3D zeolites. In this study, we investigate the acid-catalyzed conversion of ethanol to 1-butene over the Brønsted acid sites (BAS) in 2D-pillared MFI zeolite (2D-PMFI) using ab initio molecular dynamics (AIMD) simulations, in situ diffuse reflectance infrared Fourier transform spectroscopy (DRIFTS), and calorimetric measurements. A detailed thermodynamic analysis, using quasi-harmonic approximation (QHA), on the Gibbs free energy pathway of ethanol conversion shows that the consideration of entropy is critical to accurately capture the detailed thermodynamic profiles. The formation of bulky ethoxy-ethene complex is found to be a potential rate-determining step (RDS), proceeding via stepwise mechanism. The reactivity of 2D-PMFI can be further tuned by manipulating RDS through a careful control of number of BAS and operating temperatures. The accuracy of computational model is confirmed through the comparison of calculated vibrational density of states (VDOS) against the experimental DRIFTS measurements. Overall, our study provides mechanistic insights into ethanol upgrading over the 2D-PMFI and shows the importance of evaluating entropic effects in such confined system.

Ethanol conversion, 2D-pillared zeolite, MFI, Anha↗

DOE OSTI2023

Adaptive Pore Opening to Form Tailored Adsorption Sites in a Cooperatively Flexible Framework Enables Record Inverse Propane/Propylene Separation

A proposed low-energy alternative to the separation of alkanes from alkenes by energy-intensive cryogenic distillation is separation by porous adsorbents. Unfortunately, most adsorbents preferentially take up the desired, high-value major component alkene, requiring frequent regeneration. Adsorbents with inverse selectivity for the minor component alkane would enable the direct production of purified, reagent-grade alkene, greatly reducing global energy consumption. However, such materials are exceedingly rare, especially for propane/propylene separation. Here, we report that through adaptive and spontaneous pore size and shape adaptation to optimize an ensemble of weak noncovalent interactions, the structurally responsive metal-organic framework CdIF-13 (sod-Cd(benzimidazolate) 2 ) exhibits inverse selectivity for propane over propylene with record-setting separation performance under industrially relevant temperature, pressure, and mixture conditions. Powder synchrotron X-ray diffraction measurements combined with first-principles calculations yield atomic-scale insight and reveal the induced fit mechanism of adsorbate-specific pore adaptation and ensemble interactions between ligands and adsorbates. Dynamic column breakthrough measurements confirm that CdIF-13 displays selectivity under mixed-component conditions of varying ratios, with a record measured selectivity factor of α ≈ 3 at 95:5 propylene:propane at 298 K and 1 bar. When sequenced with a low-cost rigid adsorbent, we demonstrated the direct purification of propylene under ambient conditions. In conclusion, this combined atomic-level structural characterization and performance testing firmly establishes how cooperatively flexible materials can be capable of unprecedented separation factors.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗