Search NASA⌕ Search

SEARCH · Search NASA

Results for “Integrated mean squared error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Nonparametric probability density estimation by optimization theoretic techniques

Two nonparametric probability density estimators are considered. The first is the kernel estimator. The problem of choosing the kernel scaling factor based solely on a random sample is addressed. An interactive mode is discussed and an algorithm proposed to choose the scaling factor automatically. The second nonparametric probability estimate uses penalty function techniques with the maximum likelihood criterion. A discrete maximum penalized likelihood estimator is proposed and is shown to be consistent in the mean square error. A numerical implementation technique for the discrete solution is discussed and examples displayed. An extensive simulation study compares the integrated mean square error of the discrete and kernel estimators. The robustness of the discrete estimator is demonstrated graphically.

Scott, D. W.↗

Assessment of Crustal Leg Uncertainties for RSTT Path-dependent Uncertainty

The Regional Seismic Travel Time (RSTT) model and software were recently updated to allow for path-dependent uncertainty (PDU) estimates for travel-time (Begnaud et al. 2021; Begnaud et al. 2020). Begnaud et al. (2021) describes the process of improving the current distance-dependent uncertainty parameterization for RSTT using a random effects model (REM) to estimate slowness (inverse velocity) uncertainty as a mean squared error for each model parameter. The REM separates the error between observed slowness and model predicted slowness into bias and random components. The PDU is calculated by integrating these mean squared errors along a seismic-phase ray path.

58 GEOSCIENCES↗

Updates to the Regional Seismic Travel Time (RSTT) Model: 2. Path-dependent Travel-time Uncertainty

Abstract The regional seismic travel time (RSTT) model and software were developed to improve travel-time prediction accuracy by accounting for three-dimensional crust and upper mantle structure. Travel-time uncertainty estimates are used in the process of associating seismic phases to events and to accurately calculate location uncertainty bounds (i.e. event location error ellipses). We improve on the current distance-dependent uncertainty parameterization for RSTT using a random effects model to estimate slowness (inverse velocity) uncertainty as a mean squared error for each model parameter. The random effects model separates the error between observed slowness and model predicted slowness into bias and random components. The path-specific travel-time uncertainty is calculated by integrating these mean squared errors along a seismic-phase ray path. We demonstrate that event location error ellipses computed for a 90% coverage ellipse metric (used by the Comprehensive Nuclear-Test-Ban Treaty Organization International Data Centre (IDC)), and using the path-specific travel-time uncertainty approach, are more representative (median 82.5% ellipse percentage) of true location error than error ellipses computed using distance-dependent travel-time uncertainties (median 70.1%). We also demonstrate measurable improvement in location uncertainties using the RSTT method compared to the current station correction approach used at the IDC (median 74.3% coverage ellipse).

58 GEOSCIENCES↗

StressNet - Deep learning to predict stress with fracture propagation in brittle materials

Abstract Catastrophic failure in brittle materials is often due to the rapid growth and coalescence of cracks aided by high internal stresses. Hence, accurate prediction of maximum internal stress is critical to predicting time to failure and improving the fracture resistance and reliability of materials. Existing high-fidelity methods, such as the Finite-Discrete Element Model (FDEM), are limited by their high computational cost. Therefore, to reduce computational cost while preserving accuracy, a deep learning model, StressNet, is proposed to predict the entire sequence of maximum internal stress based on fracture propagation and the initial stress data. More specifically, the Temporal Independent Convolutional Neural Network (TI-CNN) is designed to capture the spatial features of fractures like fracture path and spall regions, and the Bidirectional Long Short-term Memory (Bi-LSTM) Network is adapted to capture the temporal features. By fusing these features, the evolution in time of the maximum internal stress can be accurately predicted. Moreover, an adaptive loss function is designed by dynamically integrating the Mean Squared Error (MSE) and the Mean Absolute Percentage Error (MAPE), to reflect the fluctuations in maximum internal stress. After training, the proposed model is able to compute accurate multi-step predictions of maximum internal stress in approximately 20 seconds, as compared to the FDEM run time of 4 h, with an average MAPE of 2% relative to test data.

36 MATERIALS SCIENCE↗

Augmenting a Simulation Campaign for Hybrid Computer Model and Field Data Experiments

The Kennedy and O’Hagan (KOH) calibration framework uses coupled Gaussian processes (GPs) to meta-model an expensive simulator (first GP), tune its “knobs” (calibration inputs) to best match observations from a real physical/field experiment and correct for any modeling bias (second GP) when predicting under new field conditions (design inputs). There are well-established methods for placement of design inputs for data-efficient planning of a simulation campaign in isolation, that is, without field data: space-filling, or via criterion like minimum integrated mean-squared prediction error (IMSPE). Analogues within the coupled GP KOH framework are mostly absent from the literature. Here, in this study, we derive a closed form IMSPE criterion for sequentially acquiring new simulator data for KOH. We illustrate how acquisitions space-fill in design space, but concentrate in calibration space. Closed form IMSPE precipitates a closed-form gradient for efficient numerical optimization. We demonstrate that our KOH-IMSPE strategy leads to a more efficient simulation campaign on benchmark problems, and conclude with a showcase on an application to equilibrium concentrations of rare earth elements for a liquid–liquid extraction reaction.

97 MATHEMATICS AND COMPUTING↗

A critical analysis of the accuracy of several numerical techniques for combustion kinetic rate equations

A detailed analysis of the accuracy of several techniques recently developed for integrating stiff ordinary differential equations is presented. The techniques include two general-purpose codes EPISODE and LSODE developed for an arbitrary system of ordinary differential equations, and three specialized codes CHEMEQ, CREK1D, and GCKP4 developed specifically to solve chemical kinetic rate equations. The accuracy study is made by application of these codes to two practical combustion kinetics problems. Both problems describe adiabatic, homogeneous, gas-phase chemical reactions at constant pressure, and include all three combustion regimes: induction, heat release, and equilibration. To illustrate the error variation in the different combustion regimes the species are divided into three types (reactants, intermediates, and products), and error versus time plots are presented for each species type and the temperature. These plots show that CHEMEQ is the most accurate code during induction and early heat release. During late heat release and equilibration, however, the other codes are more accurate. A single global quantity, a mean integrated root-mean-square error, that measures the average error incurred in solving the complete problem is used to compare the accuracy of the codes. Among the codes examined, LSODE is the most accurate for solving chemical kinetics problems. It is also the most efficient code, in the sense that it requires the least computational work to attain a specified accuracy level. An important finding is that use of the algebraic enthalpy conservation equation to compute the temperature can be more accurate and efficient than integrating the temperature differential equation.

Radhadrishnan, Krishnan↗

LSTM-Based Data Integration to Improve Snow Water Equivalent Prediction and Diagnose Error Sources

Accurate prediction of snow water equivalent (SWE) can be valuable for water resource managers. Recently, deep learning methods such as long short-term memory (LSTM) have exhibited high accuracy in simulating hydrologic variables and can integrate lagged observations to improve prediction, but their benefits were not clear for SWE simulations. Here we tested an LSTM network with data integration (DI) for SWE in the western United States to integrate 30-day-lagged or 7-day-lagged observations of either SWE or satellite-observed snow cover fraction (SCF) to improve future predictions. SCF proved beneficial only for shallow-snow sites during snowmelt, while lagged SWE integration significantly improved prediction accuracy for both shallow- and deep-snow sites. The median Nash–Sutcliffe model efficiency coefficient (NSE) in temporal testing improved from 0.92 to 0.97 with 30-day-lagged SWE integration, and root-mean-square error (RMSE) and the difference between estimated and observed peak SWE values d max were reduced by 41% and 57%, respectively. DI effectively mitigated accumulated model and forcing errors that would otherwise be persistent. Moreover, by applying DI to different observations (30-day-lagged, 7-day-lagged), we revealed the spatial distribution of errors with different persistent lengths. For example, integrating 30-day-lagged SWE was ineffective for ephemeral snow sites in the southwestern United States, but significantly reduced monthly-scale biases for regions with stable seasonal snowpack such as high-elevation sites in California. These biases are likely attributable to large interannual variability in snowfall or site-specific snow redistribution patterns that can accumulate to impactful levels over time for nonephemeral sites. These results set up benchmark levels and provide guidance for future model improvement strategies.

54 ENVIRONMENTAL SCIENCES↗

Synthesis of hover autopilots for rotary-wing VTOL aircraft

The practical situation is considered where imperfect information on only a few rotor and fuselage state variables is available. Filters are designed to estimate all the state variables from noisy measurements of fuselage pitch/roll angles and from noisy measurements of both fuselage and rotor pitch/roll angles. The mean square response of the vehicle to a very gusty, random wind is computed using various filter/controllers and is found to be quite satisfactory although, of course, not so good as when one has perfect information (idealized case). The second part of the report considers precision hover over a point on the ground. A vehicle model without rotor dynamics is used and feedback signals in position and integral of position error are added. The mean square response of the vehicle to a very gusty, random wind is computed, assuming perfect information feedback, and is found to be excellent. The integral error feedback gives zero position error for a steady wind, and smaller position error for a random wind.

Hall, W. E.↗

Use of multiple gauges and microwave attenuation of precipitation for satellite verification

In this paper both a microwave attenuation measurement along a horizontal line and multiple point gauge measurements are analyzed as possible ground-truth designs to validate satellite precipitation retrieval algorithms at the field of view spatial level (typically about 20 km). The design consists of comparing a sequence of pairs of contemporaneous measurements taken from the ground and from space. The authors examine theoretically the variance of expected differences between the two systems. The line average measurement leads to a smaller mean-square error compared to the case of a single point gauge, since some of the small-scale variability of the rain field is smoothed away by the line integration. The multiple point gauge measurements also give smaller mean-square error than that of a single point gauge. The centroid of the line and point gauge configurations are considered to be located randomly inside the field of view for different overpasses. A space-time spectral formalism is used with a noise-forced diffusive rain field to find the mean-square error. By considering instantaneous ground and satellite measurement pairs over about 50 visits when raining, we can reduce the expected error to approximately 10% of the standard deviation of climatological variability. This is considered to be a useful level of tolerance for identifying biases in the retrieval algorithms. It is found that the multiple point gauges (especially two gauges) are the economical ground-truth design compared to the microwave attenuation based on the mean-square error comparison. The major finding of this study is that a significant improvement over the point gauge is obtained by adding a single additional piece of information; adding more gauges or extending the line of attenuation is not an important improvement.

Ha, Eunho↗

Theoretical Control-Centric Modeling for Precision Model-Based Sliding Mode Control of a Hydraulic Artificial Muscle Actuator

Artificial muscles (AMs) typically rely on pneumatic sources of fluid power. The use of hydraulics can increase the power and force to weight and volume ratios of AM actuators. This paper develops a control-centric third-order single-input single-output (SISO) lumped-parameter dynamic model and sliding mode position controller based on Filippov's principle of equivalent dynamics for a braided hydraulic artificial muscle (HAM) actuator. The model predicts the nonlinear behavior of the HAM free contraction and captures the fluid and actuator nonlinear dynamic interactions in addition to the braid deformation. Model simulations are compared to experimental results for quasi-static pressurization, isometric pressurization, and open-loop square wave commands at 0.25, 0.5, and 1 Hz. Experiments of sine wave tracking at 0.25, 0.5, and 1 Hz and continuous square wave tracking at 0.067 Hz are conducted using a sliding mode controller (SMC) derived from the model. The SMC achieves a steady-state error of 6 μm at multiple setpoints within the actuator's 17 mm stroke. Compared to a proportional-integral-derivative (PID) controller, the SMC root-mean-square (RMS) error, mean error, and absolute maximum error are reduced on average by 53%, 61%, and 44%, respectively, demonstrating the benefit of model-based approaches for controlling HAMs.

42 ENGINEERING↗

Test functions for the moment method which yield the minimum mean square error

The method of moments has found extensive applications in the solution of a wide class of electromagnetic problems. Additionally, there are a number of basis functions which have found favor and with them different types of testing methods. This work examines the selection of test functions, given a selected set of basis functions, which serve to minimize the mean square error. These test functions will guarantee equal or lesser error compared with any other test function. Additionally, these test functions provide a monotonic improvement of the approximation as the increase in the order of the system. Hence while these test functions require the application of the differential or integral operator one additional time, they provide an absolute lower bound to the mean square error in the approximation and a means to systematically improve accuracy with each increase in the order of the unknowns.

Shamansky, H.↗

Detection of signals by weighted integrate-and-dump filter

A Weighted Integrate and Dump Filter (WIDF) is presented that results in reducing those losses in telemetry symbol signal to noise ratio (SNR) which occur in digital Integrate and Dump Filters (IDFs) when the samples are not phase locked to the input data symbol clock. The Minimum Mean Square Error (MMSE) criterion is used to derive a set of weights for approximating the analog integrate and dump filter, which is the matched filter for detection of signals in additive white Gaussian noise. This new digital matched filter results in considerable performance improvement compared to unweighted digital matched filters. An example is presented for a sampling rate of four times the symbol rate. As the sampling offset (or phase) varies with respect to the data symbol boundaries, the output SNR varies 1 dB for an unweighted IDF, but only 0.3 dB for the optimum WIDF, averaged over random data patterns. This improvement in performance relative to unweighted IDF means that significantly lower sampling and processing rates can be used for given telemetry symbol rates, resulting in reduced system cost.

Sadr, R.↗

Integration of GOES Data for Solar Resource Assessment of the Contiguous United States

The National Solar Radiation Database (NSRDB), produced by the National Laboratory of the Rockies (NLR), provides high-resolution solar resource data for the contiguous United States (CONUS) using Geostationary Operational Environmental Satellite (GOES) East and West observations. This study evaluates the integration of multi-satellite data within the GOES-East/West overlap regions, where conventional longitude-based selection methods often produce an artificial boundary seam. Our results demonstrate that an advanced blending algorithm, which incorporates sun-satellite scattering angles and satellite viewing zenith angles, improves NSRDB accuracy and creates a spatially continuous dataset. Validation against ground-based irradiance measurements reveals reductions in both percentage error (PE) and normalized Root Mean Square Error (nRMSE), particularly in the central United States. The dynamical integration of multi-satellite data provides a robust foundation for more precise modeling of solar resource and improved spatiotemporal analysis of solar ramp across the CONUS.

14 SOLAR ENERGY↗

Analyzing and Exploring Training Recipes for Large-Scale Transformer-Based Weather Prediction

Abstract The rapid rise of deep learning (DL) in numerical weather prediction (NWP) has led to a proliferation of models which forecast atmospheric variables with comparable or superior skill than traditional physics-based NWP. However, among these leading DL models, there is a wide variance in both the training settings and architecture used. Further, the lack of thorough ablation studies makes it hard to discern which components are most critical to success. In this work, we show that it is possible to attain high forecast skill even with relatively off-the-shelf architectures, simple training procedures, and moderate compute budgets. Specifically, we train a minimally modified Swin Transformer V2 (SwinV2) on ERA5 data and find that it attains superior skill in terms of mean-square errors of deterministic forecasts when compared against the European Centre for Medium-Range Weather Forecasts’ Integrated Forecasting System (IFS). Almost all DL–NWP systems share a core set of hyperparameters and design decisions. To aid and expedite future DL–NWP research, we present an in-depth, systematic exploration of different loss functions, model sizes and depths, patch sizes, and multistep training objectives. We also examine the model performance with metrics beyond the typical accuracy (ACC) and RMSE and investigate how the performance scales with model size. Through our open-source code, scoring pipelines, and models, we share our findings on key aspects of the training pipeline. These ablations reduce the necessity for expensive hyperparameter tuning and lower the barrier to entry for future DL–NWP research. Significance Statement This study investigates the potential of using large-scale transformer-based models for weather prediction, showing that it is possible to achieve high forecast accuracy with simpler, off-the-shelf architectures. By training a minimally modified SwinV2 transformer on ERA5 data, we show that the model achieves competitive forecast skill in terms of mean-square error for key variables, outperforming the European Centre for Medium-Range Weather Forecasts’ Integrated Forecasting System (IFS) at all lead times. Our findings suggest that effective training strategies, such as multistep fine-tuning and channel-weighted losses, significantly enhance the model’s performance. However, we also highlight that these improvements come with trade-offs in other areas, such as ensemble spread and high-frequency spatial detail. This work highlights the promise of deep learning in improving weather forecasts, which could lead to better preparedness and response to weather events, ultimately benefiting society by providing more reliable weather predictions.

Willard, Jared D. [Lawrence Berkeley National Labo↗

A note on obtaining an approximate value for the period of a periodic solution of x-double prime = -V prime /x/

The first integral of x-double prime = - V prime (x) yields an integral for the period of a periodic solution, if such exists. In general, this integral cannot be evaluated. By means of an approximate solution along with the minimization of a mean-square error, one can obtain an approximate value for the period in terms of the amplitude of the motion. The calculated period agrees very well with the period obtained by means of numerical integration for the case of orbit-orbit resonance involving the motion of two satellites of a planet. The same method is applied to obtain an alternative derivation of the first Krylov-Bogoliuboff averaging method in nonlinear mechanics.

Lass, H.↗

Resilience Measurement Framework For Post-deployment Artificial Intelligence (ai) Integrated Systems

Resilience is largely defined as the ability to adapt or recover from adverse conditions, stresses, attacks, or compromises on systems that use or are enabled by digital resources. In Artificial Intelligence Management and Research for Advanced Networked Testbed Hub (AMARANTH), resilience is measured in the amount of time it took from the beginning of a testing period for the model to reach predictions outside of the original 95% confidence interval or using the Kullback-Leibler (KL) divergence theorem, the Population Stability Index (PSI), and traditional methods such as root mean squared error (RMSE) threshold. Artificial Intelligence (AI) model drift is of significant concern when deploying AI-integrated systems into critical and/or secure environments. Drift can impact resilience of the AI-integrated system post-deployment and requires consistent maintenance and upkeep to ensure the model is accurate and precise. To quantify model drift and predict the point when a model's drift becomes unacceptable, we describe using Kullback-Leibler (KL) divergence, Population Stability Index (PSI) and/or confidence interval width estimations to determine the point of failure and time to failure of a model post-deployment. Through simple code functions, the KL-divergence, PSI, confidence interval, and root mean squared (RMSE) point of failures can be used to derive when a model needs to be maintained as well as the impact of adversarial action through statistical means.

Yockey, Patience [Idaho National Laboratory (INL),↗