Search NASA⌕ Search

SEARCH · Search NASA

Results for “Gradient boosting trees”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Large Destabilization of (TiVNb)-Based Hydrides via (Al, Mo) Addition: Insights from Experiments and Data-Driven Models

High-entropy alloys (HEAs) represent an interesting alloying strategy that can yield exceptional performance properties needed across a variety of technology applications, including hydrogen storage. Examples include ultrahigh volumetric capacity materials (BCC alloys → FCC dihydrides) with improved thermodynamics relative to conventional high-capacity metal hydrides (like MgH 2 ), but still further destabilization is needed to reduce operating temperature and increase system-level capacity. Here, in this work, we demonstrate efficient hydride destabilization strategies by synthesizing two new Al 0.05 (TiVNb) 0.95–x Mo x (x = 0.05, 0.10) compositions. We specifically evaluate the effect of molybdenum (Mo) addition on the phase structure, microstructure, hydrogen absorption, and desorption properties. Both alloys crystallize in a bcc structure with decreasing lattice parameters as the Mo content increases. The alloys can rapidly absorb hydrogen at 25 °C with capacities of 1.78 H/M (2.79 wt %) and 1.79 H/M (2.75 wt %) with increasing Mo content. Pressure-composition isotherms suggest a two-step reaction for hydrogen absorption to a final fcc dihydride phase. The experiments demonstrate that increasing Mo content results in a significant hydride destabilization, which is consistent with predictions from a gradient boosting tree data-driven model for metal hydride thermodynamics. Furthermore, improved desorption properties with increasing Mo content and reversibility were observed by in situ synchrotron X-ray diffraction, in situ neutron diffraction, and thermal desorption spectroscopy.

36 MATERIALS SCIENCE↗

Learning the Low Frequency Earthquake Activity on the Central San Andreas Fault

Abstract Low frequency earthquakes (LFEs) originating below the central San Andreas Fault are associated with slow‐slip beneath the seismogenic zone within the more ductile portion of the crust. Monitoring efforts over 15 years detected >1 million LFEs. We train a gradient boosted tree model using statistical features describing the seismic waveforms to estimate the hourly LFE event count. The burst‐like LFE behavior is reproduced, while lower amplitudes are predicted during the most active periods. The hourly event counts are up to 18% greater than the catalog. The ability to continuously monitor LFE activity provides insight to when geodetic measurements of slow slip are possible, without the need for developing a computational‐intensive template‐matching catalog. Similar waveform statistical features are found between detecting LFEs and tremors, which provides additional evidence tremors are composed of LFEs. The approach extracts information contained in continuous seismic waveforms that might benefit detecting precursory signals.

58 GEOSCIENCES↗

Seismic Features Predict Ground Motions During Repeating Caldera Collapse Sequence

Abstract Applying machine learning to continuous acoustic emissions, signals previously deemed noise, from laboratory faults and slowly slipping subduction‐zone faults, demonstrates hidden signatures are emitted that describe physical details, including fault displacement and friction. However, no evidence currently exists to demonstrate that similar hidden signals occur during seismogenic stick‐slip on earthquake faults—the damaging earthquakes of most societal interest. We show that continuous seismic emissions emitted during the 2018 multi‐month caldera collapse sequence at the Kı̄lauea volcano in Hawai'i contain hidden signatures characterizing the earthquake cycle. Multi‐spectral data features extracted from 30 s intervals of the continuous seismic emission are used to train a gradient boosted tree regression model to predict the GNSS‐derived contemporaneous surface displacement and time‐to‐failure of the upcoming collapse event. This striking result suggests that at least some faults emit such signals and provide a potential path to characterizing the instantaneous and future behavior of earthquake faults.

58 GEOSCIENCES↗

Gray matter volume drives the brain age gap in schizophrenia: a SHAP study

Neuroimaging-based brain age is a biomarker that is generated by machine learning (ML) predictions. The brain age gap (BAG) is typically defined as the difference between the predicted brain age and chronological age. Studies have consistently reported a positive BAG in individuals with schizophrenia (SCZ). However, there is little understanding of which specific factors drive the ML-based brain age predictions, leading to limited biological interpretations of the BAG. We gathered data from three publicly available databases - COBRE, MCIC, and UCLA - and an additional dataset (TOPSY) of early-stage schizophrenia (82.5% untreated first-episode sample) and calculated brain age with pre-trained gradient-boosted trees. Then, we applied SHapley Additive Explanations (SHAP) to identify which brain features influence brain age predictions. We investigated the interaction between the SHAP score for each feature and group as a function of the BAG. These analyses identified total gray matter volume (group × SHAP interaction term β = 1.71 [0.53; 3.23]; p corr < 0.03) as the feature that influences the BAG observed in SCZ among the brain features that are most predictive of brain age. Other brain features also presented differences in SHAP values between SCZ and HC, but they were not significantly associated with the BAG. We compared the findings with a non-psychotic depression dataset (CAN-BIND), where the interaction was not significant. This study has important implications for the understanding of brain age prediction models and the BAG in SCZ and, potentially, in other psychiatric disorders.

60 APPLIED LIFE SCIENCES↗

Prediction of carbon nanostructure mechanical properties and the role of defects using machine learning

Graphene-based nanostructures hold immense potential as strong and lightweight materials, however, their mechanical properties such as modulus and strength are difficult to fully exploit due to challenges in atomic-scale engineering. This study presents a database of over 2,000 pristine and defective nanoscale CNT bundles and other graphitic assemblies, inspired by microscopy, with associated stress–strain curves from reactive molecular dynamics (MD) simulations using the reactive INTERFACE force field (IFF-R). These 3D structures, containing up to 80,000 atoms, enable detailed analyses of structure-stiffness-failure relationships. By leveraging the database and physics- and chemistry-informed machine learning (ML), accurate predictions of elastic moduli and tensile strength are demonstrated at speeds 1,000 to 10,000 times faster than efficient MD simulations. Hierarchical Graph Neural Networks with Spatial Information (HS-GNNs) are introduced, which integrate chemistry knowledge. HS-GNNs as well as extreme gradient boosted trees (XGBoost) achieve forecasts of mechanical properties of arbitrary carbon nanostructures with only 3 to 6% mean relative error. The reliability equals experimental accuracy and is up to 20 times higher than other ML methods. Predictions maintain 8 to 18% accuracy for large CNT bundles, CNT junctions, and carbon fiber cross-sections outside the training distribution. The physics- and chemistry-informed HS-GNN works remarkably well for data outside the training range while XGBoost works well with limited training data inside the training range. The carbon nanostructure database is designed for integration with multimodal experimental and simulation data, scalable beyond 100 nm size, and extendable to chemically similar compounds and broader property ranges. The ML approaches have potential for applications in structural materials, nanoelectronics, and carbon-based catalysts.

Winetrout, Jordan J.↗

The ASHRAE Great Energy Predictor III competition: Overview and results

In late 2019, ASHRAE hosted the Great Energy Predictor III (GEPIII) machine learning competition on the Kaggle platform. This launch marked the third energy prediction competition from ASHRAE and the first since the mid-1990s. In this updated version, the competitors were provided with over 20 million points of training data from 2,380 energy meters collected for 1,448 buildings from 16 sources. This competition’s overall objective was to find the most accurate modeling solutions for the prediction of over 41 million private and public test data points. Furthermore, the competition had 4,370 participants, split across 3,614 teams from 94 countries who submitted 39,403 predictions. In addition to the top five winning workflows, the competitors publicly shared 415 reproducible online machine learning workflow examples (notebooks), including over 40 additional, full solutions. This paper gives a high-level overview of the competition preparation and dataset, competitors and their discussions, machine learning workflows and models generated, winners and their submissions, discussion of lessons learned, and competition outputs and next steps. The most popular and accurate machine learning workflows used large ensembles of mostly gradient boosting tree models, such as LightGBM. Similar to the first predictor competition, preprocessing of the data sets emerged as a key differentiator.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Insights into the origin of halo mass profiles from machine learning

ABSTRACT The mass distribution of dark matter haloes is the result of the hierarchical growth of initial density perturbations through mass accretion and mergers. We use an interpretable machine-learning framework to provide physical insights into the origin of the spherically-averaged mass profile of dark matter haloes. We train a gradient-boosted-trees algorithm to predict the final mass profiles of cluster-sized haloes, and measure the importance of the different inputs provided to the algorithm. We find two primary scales in the initial conditions (ICs) that impact the final mass profile: the density at approximately the scale of the haloes’ Lagrangian patch RL ($R\sim 0.7\, R_L$) and that in the large-scale environment (R ∼ 1.7 RL). The model also identifies three primary time-scales in the halo assembly history that affect the final profile: (i) the formation time of the virialized, collapsed material inside the halo, (ii) the dynamical time, which captures the dynamically unrelaxed, infalling component of the halo over its first orbit, (iii) a third, most recent time-scale, which captures the impact on the outer profile of recent massive merger events. While the inner profile retains memory of the ICs, this information alone is insufficient to yield accurate predictions for the outer profile. As we add information about the haloes’ mass accretion history, we find a significant improvement in the predicted profiles at all radii. Our machine-learning framework provides novel insights into the role of the ICs and the mass assembly history in determining the final mass profile of cluster-sized haloes.

79 ASTRONOMY AND ASTROPHYSICS↗

Detection of topological materials with machine learning

Databases compiled using ab initio and symmetry-based calculations now contain tens of thousands of topological insulators and topological semimetals. This makes the application of modern machine learning methods to topological materials possible. Using gradient boosted trees, we show how to construct a machine learning model which can predict the topology of a given existent material with an accuracy of 90%. Such predictions are orders of magnitude faster than actual ab initio calculations. In this work, we use machine learning models to probe how different material properties affect topological features. Notably, we observe that topology is mostly determined by the “coarse-grained” chemical composition and crystal symmetry and depends little on the particular positions of atoms in the crystal lattice. We identify the sources of our model's errors and we discuss approaches to overcome them.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Composite Qdrift-product formulas for quantum and classical simulations in real and imaginary time

Recent study has shown that it can be advantageous to implement a composite channel that partitions the Hamiltonian H for a given simulation problem into subsets A and B such that H = A + B , where the terms in A are simulated with a Trotter-Suzuki channel and the B terms are randomly sampled via the Qdrift algorithm. Here we extend Qdrift and composite product formulas to imaginary time, formulating candidate classical algorithms for quantum Monte Carlo calculations. We upper bound the induced Schatten- 1 → 1 norm on both imaginary-time Qdrift and composite channels. Another recent result demonstrated that simulations of lattice Hamiltonians containing geometrically local interactions can be improved using a Lieb-Robinson argument to decompose H into subsets that contain only terms supported on that subset of the lattice. Here, we provide a quantum algorithm by unifying this result with the composite approach into “local composite channels” and we upper bound the diamond distance. We provide exact numerical simulations of algorithmic cost by counting the number of gates of the form e − i H j t and e − H j β to meet a certain error tolerance ε . In doing so, we optimize the partitioning into sets A and B using gradient boosted tree models from machine learning. These numerical studies are important given that product formulas have been historically known to outperform analytic upper bounds. We show constant factor advantages for a variety of interesting Hamiltonians, the maximum of which is a ≈ 20 -fold speedup that occurs in the simulation of Jellium. Published by the American Physical Society 2024

Pocrnic, Matthew (ORCID:0000000203089376)↗

Classification of AB O 3 perovskite solids: a machine learning study

Here we explored the use of machine learning methods for classifying whether a particular AB O 3 chemistry forms a perovskite or non-perovskite structured solid. Starting with three sets of feature pairs (the tolerance and octahedral factors, the A and B ionic radii relative to the radius of O, and the bond valence distances between the A and B ions from the O atoms), we used machine learning to create a hyper-dimensional partial dependency structure plot using all three feature pairs or any two of them. Doing so increased the accuracy of our predictions by 2–3 percentage points over using any one pair. We also included the Mendeleev numbers of the A and B atoms to this set of feature pairs. Moreover, doing this and using the capabilities of our machine learning algorithm, the gradient tree boosting classifier, enabled us to generate a new type of structure plot that has the simplicity of one based on using just the Mendeleev numbers, but with the added advantages of having a higher accuracy and providing a measure of likelihood of the predicted structure.

36 MATERIALS SCIENCE↗

Machine Learning Benchmarks for the Classification of Equivalent Circuit Models from Electrochemical Impedance Spectra

Analysis of Electrochemical Impedance Spectroscopy (EIS) data for electrochemical systems often consists of defining an Equivalent Circuit Model (ECM) using expert knowledge and then optimizing the model parameters to deconvolute various resistance, capacitive, inductive, or diffusion responses. For small data sets, this procedure can be conducted manually; however, it is not feasible to manually define a proper ECM for extensive data sets with a wide range of EIS responses. Automatic identification of an ECM would substantially accelerate the analysis of large sets of EIS data. We showcase machine learning methods to classify the ECMs of 9,300 impedance spectra provided by QuantumScape for the BatteryDEV hackathon. The best-performing approach is a gradient-boosted tree model utilizing a library to automatically generate features, followed by a random forest model using the raw spectral data. A convolutional neural network using boolean images of Nyquist representations is presented as an alternative, although it achieves a lower accuracy. We publish the data and open source the associated code. The approaches described in this article can serve as benchmarks for further studies. A key remaining challenge is the identifiability of the labels, underlined by the model performances and the comparison of misclassified spectra.

25 ENERGY STORAGE↗

fSCAml

A repository for predicting fractional snow covered area using a gradient boosted trees machine learning.

Crumley, Ryan L [Self-Employed]↗

Machine learning framework for assessment of microbial factory performance

Metabolic models can estimate intrinsic product yields for microbial factories, but such frameworks struggle to predict cell performance (including product titer or rate) under suboptimal metabolism and complex bioprocess conditions. On the other hand, machine learning, complementary to metabolic modeling necessitates large amounts of data. Building such a database for metabolic engineering designs requires significant manpower and is prone to human errors and bias. We propose an approach to integrate data-driven methods with genome scale metabolic model for assessment of microbial bio-production (yield, titer and rate). Using engineered E. coli as an example, we manually extracted and curated a data set comprising about 1200 experimentally realized cell factories from ~100 papers. We furthermore augmented the key design features (e.g., genetic modifications and bioprocess variables) extracted from literature with additional features derived from running the genome-scale metabolic model iML1515 simulations with constraints that match the experimental data. Then, data augmentation and ensemble learning (e.g., support vector machines, gradient boosted trees, and neural networks in a stacked regressor model) are employed to alleviate the challenges of sparse, non-standardized, and incomplete data sets, while multiple correspondence analysis/principal component analysis are used to rank influential factors on bio-production. The hybrid framework demonstrates a reasonably high cross-validation accuracy for prediction of E.coli factory performance metrics under presumed bioprocess and pathway conditions (Pearson correlation coefficients between 0.8 and 0.93 on new data not seen by the model).

59 BASIC BIOLOGICAL SCIENCES↗

Benchmarking Parametric and Machine Learning Models for Genomic Prediction of Complex Traits

The usefulness of genomic prediction in crop and livestock breeding programs has prompted efforts to develop new and improved genomic prediction algorithms, such as artificial neural networks and gradient tree boosting. However, the performance of these algorithms has not been compared in a systematic manner using a wide range of datasets and models. Using data of 18 traits across six plant species with different marker densities and training population sizes, we compared the performance of six linear and six non-linear algorithms. First, we found that hyperparameter selection was necessary for all non-linear algorithms and that feature selection prior to model training was critical for artificial neural networks when the markers greatly outnumbered the number of training lines. Across all species and trait combinations, no one algorithm performed best, however predictions based on a combination of results from multiple algorithms (i.e., ensemble predictions) performed consistently well. While linear and non-linear algorithms performed best for a similar number of traits, the performance of non-linear algorithms vary more between traits. Although artificial neural networks did not perform best for any trait, we identified strategies (i.e., feature selection, seeded starting weights) that boosted their performance to near the level of other algorithms. Our results highlight the importance of algorithm selection for the prediction of trait values.

59 BASIC BIOLOGICAL SCIENCES↗

Dataset for 'Stream Temperature Predictions for River Basin Management in the Pacific Northwest and Mid-Atlantic Regions Using Machine Learning', Water 2022

This data package presents forcing data, model code, and model output for classical machine learning models that predict monthly stream water temperature as presented in the manuscript ‘Stream Temperature Predictions for River Basin Management in the Pacific Northwest and Mid-Atlantic Regions Using Machine Learning’, Water (Weierbach et al., 2022). Specifically, for input forcing datasets we include two files each generated using the BASIN-3D data integration tool (Varadharajan et al., 2022) for stations in the Pacific Northwest and Mid Atlantic Hydrologic regions. Model code (written in python with the use of jupyter notebooks) includes codes for data preprocessing, training Multiple Linear Regression, Support Vector Regression, and Extreme Gradient Boosted Tree models, and additional notebooks for analysis of model output. We include specific model output files which represent modeling configurations presented in the manuscript also presented in an hdf5 format. Together, these data make up the workflow for predictions across three scenarios (single station, regional, and predictions in unmonitored basins) presented in the manuscript and allow for reproducibility of modeling procedures.

54 ENVIRONMENTAL SCIENCES↗

Stream Temperature Predictions for River Basin Management in the Pacific Northwest and Mid-Atlantic Regions Using Machine Learning

Stream temperature (Ts) is an important water quality parameter that affects ecosystem health and human water use for beneficial purposes. Accurate Ts predictions at different spatial and temporal scales can inform water management decisions that account for the effects of changing climate and extreme events. In particular, widespread predictions of Ts in unmonitored stream reaches can enable decision makers to be responsive to changes caused by unforeseen disturbances. In this study, we demonstrate the use of classical machine learning (ML) models, support vector regression and gradient boosted trees (XGBoost), for monthly Ts predictions in 78 pristine and human-impacted catchments of the Mid-Atlantic and Pacific Northwest hydrologic regions spanning different geologies, climate, and land use. The ML models were trained using long-term monitoring data from 1980–2020 for three scenarios: (1) temporal predictions at a single site, (2) temporal predictions for multiple sites within a region, and (3) spatiotemporal predictions in unmonitored basins (PUB). In the first two scenarios, the ML models predicted Ts with median root mean squared errors (RMSE) of 0.69–0.84 °C and 0.92–1.02 °C across different model types for the temporal predictions at single and multiple sites respectively. For the PUB scenario, we used a bootstrap aggregation approach using models trained with different subsets of data, for which an ensemble XGBoost implementation outperformed all other modeling configurations (median RMSE 0.62 °C).The ML models improved median monthly Ts estimates compared to baseline statistical multi-linear regression models by 15–48% depending on the site and scenario. Air temperature was found to be the primary driver of monthly Ts for all sites, with secondary influence of month of the year (seasonality) and solar radiation, while discharge was a significant predictor at only 10 sites. The predictive performance of the ML models was robust to configuration changes in model setup and inputs, but was influenced by the distance to the nearest dam with RMSE <1 °C at sites situated greater than 16 and 44 km from a dam for the temporal single site and regional scenarios, and over 1.4 km from a dam for the PUB scenario. Our results show that classical ML models with solely meteorological inputs can be used for spatial and temporal predictions of monthly Ts in pristine and managed basins with reasonable (<1 °C) accuracy for most locations.

54 ENVIRONMENTAL SCIENCES↗

CoSHA: Code for Stellar Properties Heuristic Assignment—for the MaStar Stellar Library

We introduce CoSHA: a Code for Stellar properties Heuristic Assignment. In order to estimate the stellar properties, CoSHA implements a Gradient Tree Boosting algorithm to label each star across the parameter space (T eff , $\mathrm{log}g$, [Fe/H], and [α/Fe]). We use CoSHA to estimate the stellar atmospheric parameters of 22,000 unique stars in the MaNGA Stellar Library (MaStar). To quantify the reliability of our approach, we run internal tests, using both the Göttingen Stellar Library (a theoretical library) and the first data release of MaStar, and external tests, by comparing the resulting distributions in the parameter space with the APOGEE estimates of the same properties. In summary, our parameter estimates span the ranges T eff = [2900, 12,000] K, $\mathrm{log}g=[-0.5,5.6]$, [Fe/H] = [-3.74, 0.81], and αM = [-0.22, 1.17]. We report internal (external) uncertainties of the properties of ${\sigma }_{{T}_{\mathrm{eff}}}\sim 43(240)$ K, ${\sigma }_{\mathrm{log}g}\sim 0.2(0.4)$, σ [Fe/H] ~ 0.16(0.24), and σ [α/Fe] ~ 0.09(0.08). These uncertainties are comparable to those of other methods with similar objectives. Despite the fact that CoSHA is not aware of the spatial distributions of these physical properties in the Milky Way, we are able to recover the main trends known in the literature. The catalog of physical properties for MaStar can be accessed online.

79 ASTRONOMY AND ASTROPHYSICS↗

Benchmarking parametric and machine learning models for genomic prediction of complex traits

The usefulness of genomic prediction in crop and livestock breeding programs has prompted efforts to develop new and improved genomic prediction algorithms, such as artificial neural networks and gradient tree boosting. However, the performance of these algorithms has not been compared in a systematic manner using a wide range of datasets and models. Using data of 18 traits across six plant species with different marker densities and training population sizes, we compared the performance of six linear and six non-linear algorithms. First, we found that hyperparameter selection was necessary for all non-linear algorithms and that feature selection prior to model training was critical for artificial neural networks when the markers greatly outnumbered the number of training lines. Across all species and trait combinations, no one algorithm performed best, however predictions based on a combination of results from multiple algorithms (i.e. ensemble predictions) performed consistently well. While linear and non-linear algorithms performed best for a similar number of traits, the performance of non-linear algorithms vary more between traits. Although artificial neural networks did not perform best for any trait, we identified strategies (i.e. feature selection, seeded starting weights) that boosted their performance to near the level of other algorithms. Our results highlight the importance of algorithm selection for the prediction of trait values.

59 BASIC BIOLOGICAL SCIENCES↗