Search NASA⌕ Search

SEARCH · Search NASA

Results for “Model error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32

SU-E-T-377: Inaccurate Positioning Might Introduce Significant MapCheck Calibration Error in Flatten Filter Free Beams

Purpose: This study investigates the calibration error of detector sensitivity for MapCheck due to inaccurate positioning of the device, which is not taken into account by the current commercial iterative calibration algorithm. We hypothesize the calibration is more vulnerable to the positioning error for the flatten filter free (FFF) beams than the conventional flatten filter flattened beams. Methods: MapCheck2 was calibrated with 10MV conventional and FFF beams, with careful alignment and with 1cm positioning error during calibration, respectively. Open fields of 37cmx37cm were delivered to gauge the impact of resultant calibration errors. The local calibration error was modeled as a detector independent multiplication factor, with which propagation error was estimated with positioning error from 1mm to 1cm. The calibrated sensitivities, without positioning error, were compared between the conventional and FFF beams to evaluate the dependence on the beam type. Results: The 1cm positioning error leads to 0.39% and 5.24% local calibration error in the conventional and FFF beams respectively. After propagating to the edges of MapCheck, the calibration errors become 6.5% and 57.7%, respectively. The propagation error increases almost linearly with respect to the positioning error. The difference of sensitivities between the conventional and FFF beams was small (0.11 ± 0.49%). Conclusion: The results demonstrate that the positioning error is not handled by the current commercial calibration algorithm of MapCheck. Particularly, the calibration errors for the FFF beams are ~9 times greater than those for the conventional beams with identical positioning error, and a small 1mm positioning error might lead to up to 8% calibration error. Since the sensitivities are only slightly dependent of the beam type and the conventional beam is less affected by the positioning error, it is advisable to cross-check the sensitivities between the conventional and FFF beams to detect potential calibration errors due to inaccurate positioning. This work was partially supported by a DOD Grant No.; DOD W81XWH1010862.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Streamlining Ocean Dynamics Modeling with Fourier Neural Operators: A Multiobjective Hyperparameter and Architecture Optimization Approach

Training an effective deep learning model to learn ocean processes involves careful choices of various hyperparameters. We leverage DeepHyper’s advanced search algorithms for multiobjective optimization, streamlining the development of neural networks tailored for ocean modeling. The focus is on optimizing Fourier neural operators (FNOs), a data-driven model capable of simulating complex ocean behaviors. Selecting the correct model and tuning the hyperparameters are challenging tasks, requiring much effort to ensure model accuracy. DeepHyper allows efficient exploration of hyperparameters associated with data preprocessing, FNO architecture-related hyperparameters, and various model training strategies. We aim to obtain an optimal set of hyperparameters leading to the most performant model. Moreover, on top of the commonly used mean squared error for model training, we propose adopting the negative anomaly correlation coefficient as the additional loss term to improve model performance and investigate the potential trade-off between the two terms. The numerical experiments show that the optimal set of hyperparameters enhanced model performance in single timestepping forecasting and greatly exceeded the baseline configuration in the autoregressive rollout for long-horizon forecasting up to 30 days. Utilizing DeepHyper, we demonstrate an approach to enhance the use of FNO in ocean dynamics forecasting, offering a scalable solution with improved precision.

97 MATHEMATICS AND COMPUTING↗

Case study: Accounting for response measurement error in fitting a regression model

This article presents a case study motivated by a plot of data that suggested an emerging trend that the authors were faced with explaining. When the measurement error of the data is accounted for, it turns out there was no real trend. Furthermore, this article shows how to use a Bayesian modeling approach to account for the measurement error.

42 ENGINEERING↗

Integrated Sensing and Controls for Coal Gasification - Development of Model-Based Controls for GE's Gasifier and Syngas Cooler (Final Technical Report)

This report summarizes the achievements and final results of this program. The objective of this program is to develop a comprehensive systems approach to integrated design of sensing and control systems for an Integrated Gasification Combined Cycle (IGCC) plant, using advanced model-based techniques. In particular, this program is focused on the model-based sensing and control system design for the core gasification section of an IGCC plant. The overall approach consists of (i) developing a first-principles physics-based dynamic model of the gasification section, (ii) performing model-reduction where needed to derive low-order models suitable for controls analysis and design, (iii) developing a sensing system solution combining online sensors with model-based estimation for important process variables not measured directly, and (iv) optimizing the steady-state and transient operation of the plant for normal operation as well as for startup using model predictive controls (MPC). Initially, available process unit models were implemented in a common platform using Matlab/Simulink{reg_sign}, and appropriate model reduction and model updates were performed to obtain the overall gasification section dynamic model. Also, a set of sensor packages were developed through extensive lab testing and implemented in the Tampa Electric Company IGCC plant at Polk power station in 2009, to measure temperature and strain in the radiant syngas cooler (RSC). Plant operation data was also used to validate the overall gasification section model. The overall dynamic model was then used to develop a sensing solution including a set of online sensors coupled with model-based estimation using nonlinear extended Kalman filter (EKF). Its performance in terms of estimating key unmeasured variables like gasifier temperature, carbon conversion, etc., was studied through extensive simulations in the presence sensing errors (noise and bias) and modeling errors (e.g. unknown gasifier kinetics, RSC fouling). In parallel, an MPC solution was initially developed using ideal sensing to optimize the plant operation during startup pre-heating as well as steady state and transient operation under normal high-pressure conditions, e.g. part-load, base-load, load transition and fuel changes. The MPC simulation studies showed significant improvements both for startup pre-heating and for normal operation. Finally, the EKF and MPC solutions were coupled to achieve the integrated sensing and control solution and its performance was studied through extensive steady state and transient simulations in the presence of sensor and modeling errors. The results of each task in the program and overall conclusions are summarized in this final report.

01 COAL, LIGNITE, AND PEAT↗

Explaining Multimodal Deceptive News Prediction Models

Social media plays a valuable role in rapid news dissemination, but it also serves as a vehicle to propagate unverified information. For example, news shared on Facebook or Twitter may actually contain disinformation, propaganda, hoaxes, conspiracies, clickbait or satire. This paper presents an in-depth analysis of the behavior of suspicious news classification models including error analysis and prediction confidence. We consider five deep learning architectures that leverage combinations of text, linguistic and image input signals from tweets. The behavior of these models is analyzed across four suspicious news prediction tasks. Our findings include that models leveraging only the text of tweets outperform those leveraging only the image (by 3-13% absolute in F-measure), and that models that combine image and text signals with linguistic cues e.g., biased and subjective language markers can, but do not always, perform even better. Finally, our main contribution is a series of analyses, in which we characterize text and image traits of our classes of suspicious news and analyze patterns of errors made by the various models to inform the design of future deceptive news prediction models.

deep learning, social media, deception↗

Model Test Bed for Evaluating Unstructured-Grid Wave Models for Resource Assessment and Characterization

Recent development of unstructured-grid wave models demonstrates the advantage of using an unstructured-grid modeling framework in wave modeling, especially in coastal areas that have complex coastlines, islands, shallow-water bays, and estuaries. This study evaluated the performance of unstructured-grid models in comparison to structured-grid models for predicting the IEC wave energy resource parameters at a common test bed site. Model skill was assessed using a set of model performace error statistics. Sensitivity of open boundary conditions and ]wind forcing on model results, computational efficiency, and the advantage and limitation of the unstructured-grid model are summarized in the report.

16 TIDAL AND WAVE POWER↗

On representation of temporal variability in electricity capacity planning models

This study systematically investigates how to represent intra-annual temporal variability in models of optimum electricity capacity investment. Inappropriate aggregation of temporal resolution can introduce substantial error into model outputs and associated economic insight. The mechanisms underlying the introduction of this error are shown. How many representative periods are needed to fully capture the variability is then investigated. For a sample dataset, a scenario-robust aggregation of hourly (8760) resolution is possible in the order of 10 representative hours when electricity demand is the only source of variability. The inclusion of wind and solar supply variability increases the resolution of the robust aggregation to the order of 1000. A similar scale of expansion is shown for representative days and weeks. These concepts can be applied to any such temporal dataset, providing, at the least, a benchmark that any other aggregation method can aim to emulate. Finally, how prior information about peak pricing hours can potentially reduce resolution further is also discussed.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Contaminant point source localization error estimates as functions of data quantity and model quality

We develop empirically-grounded error envelopes for localization of a point contamination release event in the saturated zone of a previously uncharacterized heterogeneous aquifer into which a number of plume-intercepting wells have been drilled. We assume that flow direction in the aquifer is known exactly and velocity is known to within a factor of two of our best guess from well observations prior to source identification. Other aquifer and source parameters must be estimated by interpretation of well breakthrough data via the advection-dispersion equation. We employ high performance computing to generate numerous random realizations of aquifer parameters and well locations, simulate well breakthrough data, and then employ unsupervised machine optimization techniques to estimate the most likely spatial (or space-time) location of the source. Tabulating the accuracy of these estimates from the multiple realizations, we relate the size of 90% and 95% confidence envelopes to the data quantity (number of wells) and model quality (fidelity of ADE interpretation model to actual concentrations in a heterogeneous aquifer with channelized flow). We find that for purely spatial localization of the contaminant source, increased data quantities can make up for reduced model quality. For space-time localization, we find similar qualitative behavior, but significantly degraded spatial localization reliability and less improvement from extra data collection. Since the space-time source localization problem is much more challenging, we also tried a multiple-initial-guess optimization strategy. Furthermore, this greatly enhanced performance, but gains from additional data collection remained limited.

54 ENVIRONMENTAL SCIENCES↗

Can Simple Machine Learning Tools Extend and Improve Temperature-Based Methods to Infer Streambed Flux?

Temperature-based methods have been developed to infer 1D vertical exchange flux between a stream and the subsurface. Current analyses rely on fitting physically based analytical and numerical models to temperature time series measured at multiple depths to infer daily average flux. These methods have seen wide use in hydrologic science despite strong simplifying assumptions including a lack of consideration of model structural error or the impacts of multidimensional flow or the impacts of transient streambed hydraulic properties. We performed a “perfect-model experiment” investigation to examine whether regression trees, with and without gradient boosting, can extract sufficient information from model-generated subsurface temperature time series, with and without added measurement error, to infer the corresponding exchange flux time series at the streambed surface. Using model-generated, synthetic data allowed us to assess the basic limitations to the use of machine learning; further examination of real data is only warranted if the method can be shown to perform well under these ideal conditions. We also examined whether the inherent feature importance analyses of tree-based machine learning methods can be used to optimize monitoring networks for exchange flux inference.

54 ENVIRONMENTAL SCIENCES↗

Energy considerations in the Community Atmosphere Model (CAM)

An error in the energy formulation in the Community Atmosphere Model (CAM) is identified and corrected. Ten year AMIP simulations are compared using the correct and incorrect energy formulations. Statistics of selected primary variables all indicate physically insignificant differences between the simulations, comparable to differences with simulations initialized with rounding sized perturbations. The two simulations are so similar mainly because of an inconsistency in the application of the incorrect energy formulation in the original CAM. CAM used the erroneous energy form to determine the states passed between the parameterizations, but used a form related to the correct formulation for the state passed from the parameterizations to the dynamical core. If the incorrect form is also used to determine the state passed to the dynamical core the simulations are significantly different. In addition, CAM uses the incorrect form for the global energy fixer, but that seems to be less important. The difference of the magnitude of the fixers using the correct and incorrect energy definitions is very small.

54 ENVIRONMENTAL SCIENCES↗

Utilizing physics-based input features within a machine learning model to predict wind speed forecasting error

Machine learning is quickly becoming a commonly used technique for wind speed and power forecasting. Many machine learning methods utilize exogenous variables as input features, but there remains the question of which atmospheric variables are most beneficial for forecasting, especially in handling non-linearities that lead to forecasting error. This question is addressed via creation of a hybrid model that utilizes an autoregressive integrated moving-average (ARIMA) model to make an initial wind speed forecast followed by a random forest model that attempts to predict the ARIMA forecasting error using knowledge of exogenous atmospheric variables. Variables conveying information about atmospheric stability and turbulence as well as inertial forcing are found to be useful in dealing with non-linear error prediction. Streamwise wind speed, time of day, turbulence intensity, turbulent heat flux, vertical velocity, and wind direction are found to be particularly useful when used in unison for hourly and 3 h timescales. The prediction accuracy of the developed ARIMA–random forest hybrid model is compared to that of the persistence and bias-corrected ARIMA models. The ARIMA–random forest model is shown to improve upon the latter commonly employed modeling methods, reducing hourly forecasting error by up to 5 % below that of the bias-corrected ARIMA model and achieving an R 2 value of 0.84 with true wind speed.

17 WIND ENERGY↗

A data-driven framework for error estimation and mesh-model optimization in system-level thermal-hydraulic simulation

We report over the past decades, several computer codes have been developed for simulation and analysis of thermal-hydraulics and system response in nuclear reactors under operating, abnormal transient, and accident conditions. However, simulation errors and uncertainties still inevitably exist even while these codes have been extensively assessed and used. In this work, a data-driven framework (Optimal Mesh/Model Information System, OMIS) is formulated and demonstrated to estimate simulation error and suggest optimal selection of computational mesh size (i.e., nodalization) and constitutive correlations (e.g., wall functions and turbulence models) for low-fidelity, coarse-mesh thermal-hydraulic simulation, in order to achieve accuracy comparable to that of high-fidelity simulation. Using results from high-fidelity simulations and experimental data with many fast-running low-fidelity simulations, an error database is built and used to train a machine learning model that can determine the relationship between local simulation error and local physical features. This machine learning model is then used to generate insight and help correct low-fidelity simulations for similar physical conditions. The OMIS framework is designed as a modularized six-step procedure and accomplished with state-of-the-art methods and algorithms. A mixed-convection case study was performed to illustrate the entire framework.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

A high-performance workflow system for subsurface simulation

Subsurface modeling applications typically neglect uncertainty in the conceptual models, past or future scenarios, and attribute most or all uncertainty to errors in model parameters. In this contribution, uncertainty in technetium-99 transport in a heterogeneous, deep vadose zone is explored with respect to the conceptual model using a next generation user environment called Akuna. Akuna provides a range of tools to manage environmental modeling projects, from managing simulation data to visualizing results from high-performance computational simulators. Core toolsets accessible through the user interface include model setup, grid generation, parameter estimation, and uncertainty quantification. The BC Cribs site at Hanford in southeastern Washington State is used to demonstrate Akuna capabilities. At the BC Cribs site, conceptualization of the system is highly uncertain because only sparse information is available for the geologic conceptual model, the physical and chemical properties of the sediments, and the history of waste disposal operations. Using the Akuna toolset to perform an analysis of conservative solute transport, significant prediction uncertainty in simulated concentrations is demonstrated by conceptual model variation. This demonstrates that conceptual model uncertainty is an important consideration in sparse data environments such as BC Cribs. It is also demonstrated that Akuna and the underlying toolset provides an integrated modeling environment that streamlines model setup, parameter optimization, and uncertainty analyses for high-performance computing applications.

Akuna↗

Scalar excursions in large-eddy simulations

Here, the range of values of scalar fields in turbulent flows is bounded by their boundary values, for passive scalars, and by a combination of boundary values, reaction rates, phase changes, etc., for active scalars. The current investigation focuses on the local conservation of passive scalar concentration fields and the ability of the large-eddy simulation (LES) method to observe the boundedness of passive scalar concentrations. In practice, as a result of numerical artifacts, this fundamental constraint is often violated with scalars exhibiting unphysical excursions. The present study characterizes passive-scalar excursions in LES of a shear flow and examines methods for diagnosis and assesment of the problem. The analysis of scalar-excursion statistics provides support of the main hypothesis of the current study that unphysical scalar excursions in LES result from dispersive errors of the convection-term discretization where the subgrid-scale model (SGS) provides insufficient dissipation to produce a sufficiently smooth scalar field. In the LES runs three parameters are varied: the discretization of the convection terms, the SGS model, and grid resolution. Unphysical scalar excursions decrease as the order of accuracy of non-dissipative schemes is increased, but the improvement rate decreases with increasing order of accuracy. Two SGS models are examined, the stretched-vortex and a constant-coefficient Smagorinsky. Scalar excursions strongly depend on the SGS model. The excursions are significantly reduced when the characteristic SGS scale is set to double the grid spacing in runs with the stretched-vortex model. The maximum excursion and volume fraction of excursions outside boundary values show opposite trends with respect to resolution. The maximum unphysical excursions increase as resolution increases, whereas the volume fraction decreases. The reason for the increase in the maximum excursion is statistical and traceable to the number of grid points (sample size) which increases with resolution. In contrast, the volume fraction of unphysical excursions decreases with resolution because the SGS models explored perform better at higher grid resolution.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Short–Term Forecasting of Wind Gusts at Airports Across CONUS Using Machine Learning

Short–term forecasting of wind gusts, particularly those of higher intensity, is of great societal importance but is challenging due to the presence of multiple gust generation mechanisms. Wind gust observations from eight high–passenger–volume airports across the continental United States (CONUS) are summarized and used to develop predictive models of wind gust occurrence and magnitude. These short–term (same hour) forecast models are built using multiple logistic and linear regression, as well as artificial neural networks (ANNs) of varying complexity. A suite of 19 upper–air predictors drawn from the ERA5 reanalysis and an autoregressive (AR) term are used. Stepwise procedures instruct predictor selection, and resampling is used to quantify model stability. All models are developed separately for the warm (April–September) and cold (October–March) seasons. Results show that ANNs of 3–5 hidden layers (HLs) generally exhibit higher hit rates than logistic regression models and also improve skill with respect to wind gust magnitudes. However, deeper networks with more HLs increase false alarm rates in occurrence models and mean absolute error in magnitude models due to model overfitting. For model skill, inclusion of the AR term is critical while the majority of the remaining skill derives from wind speeds and lapse rates. A predictive ceiling is also clearly demonstrated, particularly for the strong and damaging gust magnitudes, which appears to be partially due to ERA5 predictor characteristics and the presence of mixed wind climates.

54 ENVIRONMENTAL SCIENCES↗

The Schottky barrier modulation at PtSi/Si interface by strain and structural deformation

We show, using density functional theory (DFT) calculations, that the Schottky barrier height (SBH) at the PtSi/Si interface can be lowered by uniaxial strain applied not only on Si but also on PtSi. The strain was applied to the (001) direction of Si and PtSi, which is normal for the interface. The SBH of the hole is lowered by 0.08 eV under 2% of tensile strain on Si and by 0.09 eV under 4 % of compressive strain on PtSi. Because the SBH at PtSi/Si contact is approximately 0.2 eV, this amount of reduction can significantly lower the resistance of the PtSi/Si contact; thus applying uniaxial strain on both PtSi and Si possibly enhances the performance of Schottky barrier field effect transistors. Theoretical models of SB formation and conventional structure model are evaluated. It is found that Pt penetration into Si stabilizes the interface and lowers the SBH by approximately 0.1 eV from the bulk-terminated interface model, which implies that conventionally used bulk-terminated interface models have significant errors. Among the theoretical models of SB formation, the model of strong Fermi level pining adequately explains the electron transfer phenomena and SBH, but it has limited ability to explain SBH changes induced by changes of interface structure.

36 MATERIALS SCIENCE↗

Machine learning predictions of high-Curie-temperature materials

Technologies that function at room temperature often require magnets with a high Curie temperature, $T$ C , and can be improved with better materials. Discovering magnetic materials with a substantial $T$ C is challenging because of the large number of candidates and the cost of fabricating and testing them. Using the two largest known datasets of experimental Curie temperatures, we develop machine-learning models to make rapid $T$ C predictions solely based on the chemical composition of a material. We train a random-forest model and a k -NN one and predict on an initial dataset of over 2500 materials and then validate the model on a new dataset containing over 3000 entries. The accuracy is compared for multiple compounds' representations (“descriptors”) and regression approaches. A random-forest model provides the most accurate predictions and is not improved by dimensionality reduction or by using more complex descriptors based on atomic properties. Further, a random-forest model trained on a combination of both datasets shows that cobalt-rich and iron-rich materials have the highest Curie temperatures for all binary and ternary compounds. An analysis of the model reveals systematic error that causes the model to over-predict low-$T$ C materials and under-predict high-$T$ C materials. For exhaustive searches to find new high-$T$ C materials, analysis of the learning rate suggests either that much more data is needed or that more efficient descriptors are necessary.

36 MATERIALS SCIENCE↗

Many-body expansion based machine learning models for octahedral transition metal complexes

Abstract Graph-based machine learning (ML) models for material properties show great potential to accelerate virtual high-throughput screening of large chemical spaces. However, in their simplest forms, graph-based models do not include any 3D information and are unable to distinguish stereoisomers such as those arising from different orderings of ligands around a metal center in coordination complexes. In this work we present a modification to revised autocorrelation descriptors, a molecular graph featurization method, for predicting spin state dependent properties of octahedral transition metal complexes (TMCs). Inspired by analytical semi-empirical models for TMCs, the new modeling strategy is based on the many-body expansion (MBE) and allows one to tune the captured stereoisomer information by changing the truncation order of the MBE. We present the necessary modifications to include this approach in two commonly used ML methods, kernel ridge regression and feed-forward neural networks. On a test set composed of all possible isomers of binary TMCs, the best MBE models achieve mean absolute errors (MAEs) of 2.75 kcal mol −1 on spin-splitting energies and 0.26 eV on frontier orbital energy gaps, a 30%–40% reduction in error compared to models based on our previous approach. We also observe improved generalization to previously unseen ligands where the best-performing models exhibit MAEs of 4.00 kcal mol −1 (i.e. a 0.73 kcal mol −1 reduction) on the spin-splitting energies and 0.53 eV (i.e. a 0.10 eV reduction) on the frontier orbital energy gaps. Because the new approach incorporates insights from electronic structure theory, such as ligand additivity relationships, these models exhibit systematic generalization from homoleptic to heteroleptic complexes, allowing for efficient screening of TMC search spaces.

Meyer, Ralf (ORCID:0000000322360261)↗