Search NASA⌕ Search

SEARCH · Search NASA

Results for “operator regression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Learning the structure of wind: A data-driven nonlocal turbulence model for the atmospheric boundary layer

In this work, we develop a novel data-driven approach to modeling the atmospheric boundary layer. This approach leads to a nonlocal, anisotropic synthetic turbulence model which we refer to as the deep rapid distortion (DRD) model. Our approach relies on an operator regression problem that characterizes the best fitting candidate in a general family of nonlocal covariance kernels parameterized in part by a neural network. This family of covariance kernels is expressed in Fourier space and is obtained from approximate solutions to the Navier–Stokes equations at very high Reynolds numbers. Each member of the family incorporates important physical properties such as mass conservation and a realistic energy cascade. The DRD model can be calibrated with noisy data from field experiments. After calibration, the model can be used to generate synthetic turbulent velocity fields. To this end, we provide a new numerical method based on domain decomposition which delivers scalable, memory-efficient turbulence generation with the DRD model as well as others. We demonstrate the robustness of our approach with both filtered and noisy data coming from the 1968 Air Force Cambridge Research Laboratory Kansas experiments. Using these data, we witness exceptional accuracy with the DRD model, especially when compared to the International Electrotechnical Commission standard.

17 WIND ENERGY↗

Mechanistic studies of small molecule ligands selective to RNA single G bulges

Abstract Small-molecule RNA binders have emerged as an important pharmacological modality. A profound understanding of the ligand selectivity, binding mode, and influential factors governing ligand engagement with RNA targets is the foundation for rational ligand design. Here, we report a novel class of coumarin derivatives exhibiting selective binding affinity towards single G RNA bulges. Harnessing the computational power of all-atom Gaussian accelerated molecular dynamics simulations, we unveiled a rare minor groove binding mode of the ligand with a key interaction between the coumarin moiety and the G bulge. This predicted binding mode is consistent with results obtained from structure-activity relationship studies and transverse relaxation measurements by nuclear magnetic resonance spectroscopy. We further generated 444 molecular descriptors from 69 coumarin derivatives and identified key contributors to the binding events, such as charge state and planarity, by lasso (least absolute shrinkage and selection operator) regression. Our work deepened the understanding of RNA-small molecule interactions and integrated a new framework for the rational design of selective small-molecule RNA binders.

Biochemistry & Molecular Biology↗

Nonlocal Operator Learning with Uncertainty Quantification

The goal of this work is to develop a Bayesian framework to characterize the uncertainty of material response when using a nonlocal, homogenized model to describe wave propagation through heterogeneous, disordered materials. Our approach is based on an operator regression technique combined with Bayesian optimization, through which the nonlocal kernel for a specific disordered microstructure is investigated.

36 MATERIALS SCIENCE↗

Machine-Learning of Nonlocal Kernels for Anomalous Subsurface Transport from Breakthrough Curves

Anomalous behavior is ubiquitous in subsurface solute transport due to the presence of high degrees of heterogeneity at different scales in the media. Although fractional models have been extensively used to describe the anomalous transport in various subsurface applications, their application is hindered by computational challenges. Simpler nonlocal models characterized by integrable kernels and finite interaction length represent a computationally feasible alternative to fractional models; yet, the informed choice of their kernel functions still remains an open problem. We propose a general data-driven framework for the discovery of optimal kernels on the basis of very small and sparse data sets in the context of anomalous subsurface transport. Using spatially sparse breakthrough curves recovered from fine-scale particle-density simulations, we learn the best coarse-scale nonlocal model using a nonlocal operator regression technique. Predictions of the breakthrough curves obtained using the optimal nonlocal model show good agreement with fine-scale simulation results even at locations and time intervals different from the ones used to train the kernel, confirming the excellent generalization properties of the proposed algorithm. A comparison with trained classical models and with black-box deep neural networks confirms the superiority of the predictive capability of the proposed model.

97 MATHEMATICS AND COMPUTING↗

Optimization and Multimachine Learning Algorithms to Predict Nanometal Surface Area Transfer Parameters for Gold and Silver Nanoparticles

Interactions between gold metallic nanoparticles and molecular dyes have been well described by the nanometal surface energy transfer (NSET) mechanism. However, the expansion and testing of this model for nanoparticles of different metal composition is needed to develop a greater variety of nanosensors for medical and commercial applications. In this study, the NSET formula was slightly modified in the size-dependent dampening constant and skin depth terms to allow for modeling of different metals as well as testing the quenching effects created by variously sized gold, silver, copper, and platinum nanoparticles. Overall, the metal nanoparticles followed more closely the NSET prediction than for Förster resonance energy transfer, though scattering effects began to occur at 20 nm in the nanoparticle diameter. To further improve the NSET theoretical equation, an attempt was made to set a best-fit line of the NSET theoretical equation curve onto the Au and Ag data points. An exhaustive grid search optimizer was applied in the ranges for two variables, 0.1≤C≤2.0 and 0≤α≤4, representing the metal dampening constant and the orientation of donor to the metal surface, respectively. Three different grid searches, starting from coarse (entire range) to finer (narrower range), resulted in more than one million total calculations with values C=2.0 and α=0.0736. The results improved the calculation, but further analysis needed to be conducted in order to find any additional missing physics. With that motivation, two artificial intelligence/machine learning (AI/ML) algorithms, multilayer perception and least absolute shrinkage and selection operator regression, gave a correlation coefficient, R2, greater than 0.97, indicating that the small dataset was not overfitting and was method-independent. This analysis indicates that an investigation is warranted to focus on deeper physics informed machine learning for the NSET equations.

Demers, Steven M. E. (ORCID:0000000192213246)↗

Global Aerodynamic Modeling Using Automated Local Model Networks in Real Time

A novel method is presented for automated real-time global aerodynamic modeling using local model networks, known as Smoothed Partitioning with Localized Trees in Real Time (SPLITR), as part of NASA’s Learn-to-Fly technology development initiative. The global nonlinear aerodynamics are partitioned into several local regions known as cells, with the dimension, location, and timing of each partition automatically selected based on a residual characterization procedure, under the constraints of real-time operation. Regression trees represent the successive partitioning of the global flight envelope and describe the evolution of the cell structure. Recursive equation-error least-squares parameter estimation in the time domain is used to estimate a model that represents the local aerodynamics in each region, so that it can be updated independently with non-contiguous data in the range of each cell over time. A weighted superposition of these piecewise local models across the flight envelope forms a global nonlinear model that also accurately captures the local aerodynamics. The SPLITR approach is demonstrated using both simulation and flight data, and the results are analyzed in terms of model predictive capabilities as well as interpretability. The results show that SPLITR can be used to automatically partition complex nonlinear aerodynamic behavior, produce an accurate model, and provide valuable physical insight into the local and global aerodynamics.

Weinstein, Rose↗

Differential Equation Approximation Using Gradient-Boosted Quantile Regression

The operation of cyber-physical-human (CPH) systems is subject to various epistemic and aleatory uncertainties. Overall trustworthiness of CPH systems relies on the trustworthiness of its components and their interactions. It is important that computational models comprising the cyber component of CPH provide predictions accompanied by a measure of confidence in model outcomes. Uncertainty quantification (UQ) and propagation are especially important in safety critical CPH systems. Gradient-boosted trees is a modeling approach capable both of learning the dynamics of a system and performing UQ. In this paper, we devise a method for using gradient boosting to learn the dynamics of a second order differential equation and estimate uncertainty at the same time. We do this by creating a custom loss function that trains the model to approximate the second derivative of a noisy time series, and to penalize based on a parameter that corresponds to the desired quantile. The resulting gradient boosting model can simulate stochastic trajectories of the system given a single starting point, that is, it can estimate both the expected trajectory and its uncertainty. We show that the uncertainty estimation is well calibrated and that the model can learn the dynamics even in the presence of noise. We demonstrate the approach on a simple cartpole system.

Autonomous systems↗

Lowering post‐construction yield assessment uncertainty through better wind plant power curves

Abstract Many operational analyses of wind power plants require a statistical relationship, which can be called the wind plant power curve, to be developed between wind plant energy production and concurrent atmospheric variables. Currently, a univariate linear regression at monthly resolution is the industry standard for post‐construction yield assessments. Here, we evaluate the benefits in augmenting this conventional approach by testing alternative regressions performed with multiple inputs, at a finer time resolution, and using nonlinear machine‐learning algorithms. We utilize the National Renewable Energy Laboratory's open‐source software package OpenOA to assess wind plant power curves for 10 wind plants. When a univariate generalized additive model at daily or hourly resolution is used, regression uncertainty is reduced, in absolute terms, by up to 1.0 % and 1.2 % (corresponding to a −59 % and −80 % relative change), respectively, compared to a univariate linear regression at monthly resolution; also, a more accurate assessment of the mean long‐term wind plant production is achieved. Additional input variables also reduce the regression uncertainty: when temperature is added as an input to the conventional monthly linear regression, the operational analysis uncertainty connected to regression is reduced, in absolute terms, by up to 0.5 % (−43 % relative change) for wind power plants with strong seasonal variability. Adding input variables to the machine‐learning model at daily resolution can further reduce regression uncertainty, with up to a −10 % relative change. Based on these results, we conclude that a multivariate nonlinear regression at daily or hourly resolution should be recommended for assessing wind plant power curves.

17 WIND ENERGY↗

Empirical radius formulas for canonical neutron stars from bidirectionally selecting features of equations of state in extended Bayesian analyses of observational data

Significant advancement in Bayesian inference of nuclear equation of state (EOS) from gravitational wave and x-ray observations of neutron stars (NSs) has been made by the nuclear astrophysics community especially since GW170817. By extending the traditional Bayesian analysis which normally ends at presenting the marginalized posterior probability distribution functions (PDFs) of individual EOS parameters and their correlations (or sometimes only the Pearson correlation coefficients which are only reliably useful when the variables are linearly correlated while they are actually often not), we search for a data-driven and robust empirical formula for the radius 𝑅 1.4 of canonical NSs in terms of the characteristic EOS parameters (features). We also identify the single most important but currently poorly known EOS parameter for determining the 𝑅 1.4 . Using three regression-model-building methodologies: bidirectional stepwise feature selection, least absolute shrinkage selection operator (LASSO) regression, and neural network regression on a large set of posterior EOSs and the corresponding 𝑅 1.4 values inferred from earlier comprehensive Bayesian analyses of NS observational data, we systematically and rigorously develop the most probable 𝑅 1.4 formulas with varying statistical accuracy and technical complexity. Here, the most important EOS parameters for determining 𝑅 1.4 are found consistently in each of the feature selection processes to be (in order of decreasing importance): curvature 𝐾 sym , slope 𝐿, skewness 𝐽 sym of nuclear symmetry energy, skewness 𝐽 0 , incompressibility 𝐾 0 of symmetric nuclear matter, and the magnitude 𝐸 sym ⁡(𝜌 0 ) of symmetry energy at the saturation density 𝜌 0 of nuclear matter.

Bayesian methods↗

NOAA/NASA Joint Simulation of AMTS and HIRS

Sounding simulation test was designed to compare the relative accuracies of atmospheric temperature profiles retrieved from HIRS2, the current operational infrared temperature sounder, and AMTS, a proposed advanced high spectral resolution infrared sounder. Retrievals generated by GLAS, using their physical retrieval algorithm, and NESDIS, using their operational statistical regression algorithm, for both instruments under clear and cloudy conditions were compared. In the cloudy portion of the test, MSU data, corresponding to the microwave component of the current operational sounding system, was used in conjunction with both instruments to aid in cloud seeding.

Susskind, J.↗

Two dimensional convolute integers for machine vision and image recognition

Machine vision and image recognition require sophisticated image processing prior to the application of Artificial Intelligence. Two Dimensional Convolute Integer Technology is an innovative mathematical approach for addressing machine vision and image recognition. This new technology generates a family of digital operators for addressing optical images and related two dimensional data sets. The operators are regression generated, integer valued, zero phase shifting, convoluting, frequency sensitive, two dimensional low pass, high pass and band pass filters that are mathematically equivalent to surface fitted partial derivatives. These operators are applied non-recursively either as classical convolutions (replacement point values), interstitial point generators (bandwidth broadening or resolution enhancement), or as missing value calculators (compensation for dead array element values). These operators show frequency sensitive feature selection scale invariant properties. Such tasks as boundary/edge enhancement and noise or small size pixel disturbance removal can readily be accomplished. For feature selection tight band pass operators are essential. Results from test cases are given.

Edwards, Thomas R.↗

Pitfalls in Prediction Modeling for Normal Tissue Toxicity in Radiation Therapy: An Illustration With the Individual Radiation Sensitivity and Mammary Carcinoma Risk Factor Investigation Cohorts

Purpose: To identify the main causes underlying the failure of prediction models for radiation therapy toxicity to replicate. Methods and Materials: Data were used from two German cohorts, Individual Radiation Sensitivity (ISE) (n=418) and Mammary Carcinoma Risk Factor Investigation (MARIE) (n=409), of breast cancer patients with similar characteristics and radiation therapy treatments. The toxicity endpoint chosen was telangiectasia. The LASSO (least absolute shrinkage and selection operator) logistic regression method was used to build a predictive model for a dichotomized endpoint (Radiation Therapy Oncology Group/European Organization for the Research and Treatment of Cancer score 0, 1, or ≥2). Internal areas under the receiver operating characteristic curve (inAUCs) were calculated by a naïve approach whereby the training data (ISE) were also used for calculating the AUC. Cross-validation was also applied to calculate the AUC within the same cohort, a second type of inAUC. Internal AUCs from cross-validation were calculated within ISE and MARIE separately. Models trained on one dataset (ISE) were applied to a test dataset (MARIE) and AUCs calculated (exAUCs). Results: Internal AUCs from the naïve approach were generally larger than inAUCs from cross-validation owing to overfitting the training data. Internal AUCs from cross-validation were also generally larger than the exAUCs, reflecting heterogeneity in the predictors between cohorts. The best models with largest inAUCs from cross-validation within both cohorts had a number of common predictors: hypertension, normalized total boost, and presence of estrogen receptors. Surprisingly, the effect (coefficient in the prediction model) of hypertension on telangiectasia incidence was positive in ISE and negative in MARIE. Other predictors were also not common between the 2 cohorts, illustrating that overcoming overfitting does not solve the problem of replication failure of prediction models completely. Conclusions: Overfitting and cohort heterogeneity are the 2 main causes of replication failure of prediction models across cohorts. Cross-validation and similar techniques (eg, bootstrapping) cope with overfitting, but the development of validated predictive models for radiation therapy toxicity requires strategies that deal with cohort heterogeneity.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Multiple-Instance Regression with Structured Data

We present a multiple-instance regression algorithm that models internal bag structure to identify the items most relevant to the bag labels. Multiple-instance regression (MIR) operates on a set of bags with real-valued labels, each containing a set of unlabeled items, in which the relevance of each item to its bag label is unknown. The goal is to predict the labels of new bags from their contents. Unlike previous MIR methods, MI-ClusterRegress can operate on bags that are structured in that they contain items drawn from a number of distinct (but unknown) distributions. MI-ClusterRegress simultaneously learns a model of the bag's internal structure, the relevance of each item, and a regression model that accurately predicts labels for new bags. We evaluated this approach on the challenging MIR problem of crop yield prediction from remote sensing data. MI-ClusterRegress provided predictions that were more accurate than those obtained with non-multiple-instance approaches or MIR methods that do not model the bag structure.

learning↗

Results of the joint NASA/NOAA AMTS/HIRS2 sounding simulation test

The relative accuracies of atmospheric temperature profiles retrieved from HIRS2, the current operational infra-red temperature sounder, and AMTS, a proposed advanced high-spectral resolution infra-red sounder were compared. The sounding simulation test compared retrievals generated by GLAS, using their physical retrieval algorithm, and NESDIS, using their operational statistical regression algorithm, for both instruments under clear and cloudy conditions. Test results for both conditions are summarized in tables showing RMS temperature errors for 18 tropospheric layers, 4 stratospheric layers, and surface skin temperature. The improvement of the AMTS instrument over HIRS2, particularly for cloudy conditions is demonstrated. The GLAS retrievals are more accurate than the operational NESDIS retrieval, especially with increasing cloud cover.

Susskind, J.↗

Data-driven learning of nonlocal physics from high-fidelity synthetic data

A key challenge to nonlocal models is the analytical complexity of deriving them from first principles, and frequently their use is justified a posteriori. Here, we extract nonlocal models from data, circumventing these challenges and providing data-driven justification for the resulting model form. Extracting data-driven surrogates is a major challenge for machine learning (ML) approaches, due to nonlinearities and lack of convexity — it is particularly challenging to extract surrogates which are provably well-posed and numerically stable. Our scheme not only yields a convex optimization problem, but also allows extraction of nonlocal models whose kernels may be partially negative while maintaining well-posedness even in small-data regimes. To achieve this, based on established nonlocal theory, we embed in our algorithm sufficient conditions on the non-positive part of the kernel that guarantee well-posedness of the learnt operator. These conditions are imposed as inequality constraints to meet the requisite conditions of the nonlocal theory. We demonstrate this workflow for a range of applications, including reproduction of manufactured nonlocal kernels; numerical homogenization of Darcy flow associated with a heterogeneous periodic microstructure; nonlocal approximation to high-order local transport phenomena; and approximation of globally supported fractional diffusion operators by truncated kernels.

42 ENGINEERING↗

A comprehensive and fair comparison of two neural operators (with practical extensions) based on $\mathrm{FAIR}$ data

Neural operators can learn nonlinear mappings between function spaces and offer a new simulation paradigm for real-time prediction of complex dynamics for realistic diverse applications as well as for system identification in science and engineering. Herein, we investigate the performance of two neural operators, which have shown promising results so far, and we develop new practical extensions that will make them more accurate and robust and importantly more suitable for industrial-complexity applications. The first neural operator, DeepONet, was published in 2019 (Lu et al., 2019), and its original architecture was based on the universal approximation theorem of Chen & Chen (1995). The second one, named Fourier Neural Operator or FNO, was published in 2020, and it is based on parameterizing the integral kernel in the Fourier space. DeepONet is represented by a summation of products of neural networks (NNs), corresponding to the branch NN for the input function and the trunk NN for the output function; both NNs are general architectures, e.g., the branch NN can be replaced with a CNN or a ResNet. According to Kovachki et al. (2021), FNO in its continuous form can be viewed conceptually as a DeepONet with a specific architecture of the branch NN and a trunk NN represented by a trigonometric basis. In order to compare FNO with DeepONet computationally for realistic setups, we develop several extensions of FNO that can deal with complex geometric domains as well as mappings where the input and output function spaces are of different dimensions. We also develop an extended DeepONet with special features that provide inductive bias and accelerate training, and we present a faster implementation of DeepONet with cost comparable to the computational cost of FNO, which is based on the Fast Fourier Transform. Here we consider 16 different benchmarks to demonstrate the relative performance of the two neural operators, including instability wave analysis in hypersonic boundary layers, prediction of the vorticity field of a flapping airfoil, porous media simulations in complex-geometry domains, etc. We follow the guiding principles of FAIR (Findability, Accessibility, Interoperability, and Reusability) for scientific data management and stewardship. The performance of DeepONet and FNO is comparable for relatively simple settings, but for complex geometries the performance of FNO deteriorates greatly. We also compare theoretically the two neural operators and obtain similar error estimates for DeepONet and FNO under the same regularity assumptions.

42 ENGINEERING↗

Forecasting Solar-Thermal Systems Performance under Transient Operation Using a Data-Driven Machine Learning Approach Based on the Deep Operator Network Architecture

Modeling and prediction of the dynamic behavior of thermal systems operating under intermittent energy input and variable load requirements represent one of the greatest challenges in the development of efficient and reliable renewable-based power generation technologies. In this work, a data-driven machine learning modeling framework was developed based on a modified version of the Deep Operator Network architecture where the time coordinate in the trunk net is replaced with historical data of the predicting quantity. The modeling framework can be used to accurately predict the performance of renewable-based energy conversion technologies including wind- and solar-based power plants. This novel framework was applied on a solar-thermal system that consists of a solar collection loop using a flat plate collector, a power generation loop comprising an Organic Rankine Cycle, and a thermal energy storage tank connecting both loops. Variable solar irradiance, air temperature, and power load profiles were used by the Deep Operator Network to predict the State-of-Charge and the efficiency of the thermal system for several days. The results were compared with the State-of-Charge and efficiency functions calculated using a physics-based model. For a simple operation scenario, characterized by a clear sky solar irradiance profile and constant load, the standard deviation in the State-of-Charge prediction by Deep Operator Network is below 0.9% during a seven-day prediction time horizon. For the most realistic operation scenario that considers real solar irradiance and a rough load profile, the maximum standard deviation in the predictions for the State-of-Charge and efficiency are below 6.8% and 2.5%, respectively. A comparison between Deep Operator Network and Long Short Term Memory network was also performed. In general, both networks predict very well the State-of-Charge for different data density conditions; however, a higher accuracy, with a standard deviation below 2.0%, is obtained by the Deep Operator Network during three and half days using sparser training data of 20-minute points. The same accuracy for the State-of-Charge prediction with the Long Short Term Memory network is achieved only for 14 h. Average standard deviations for the State-of-Charge prediction of 1.1% with the Deep Operator Network and 1.5% with the Long Short Term Memory network are obtained for a four-day prediction time using a denser training data of 5-minute points.

DeepONet↗