Search NASA⌕ Search

SEARCH · Search NASA

Results for “Optimal estimation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Optimal Estimation Retrievals and Their Uncertainties: What Every Atmospheric Scientist Should Know

Remote sensing instruments are heavily used to provide observations for both the operational and research communities. These sensors do not provide direct observations of the desired atmospheric variables, but instead, retrieval algorithms are necessary to convert the indirect observations into the variable of interest. It is critical to be aware of the underlying assumptions made by many retrieval algorithms, including that the retrieval problem is often ill posed and that there are various sources of uncertainty that need to be treated properly. In short, the retrieval challenge is to invert a set of noisy observations to obtain estimates of atmospheric quantities. The problem is often complicated by imperfect forward models, by imperfect prior knowledge, and by the existence of nonunique solutions. Optimal estimation (OE) is a widely used physical retrieval method that combines measurements, prior information, and the corresponding uncertainties based on Bayes’s theorem to find an optimal solution for the atmospheric state. Furthermore, OE also allows the relative contributions of the different sources of error to the uncertainty in the final retrieved atmospheric state to be understood. Here, we provide a novel Python library to illustrate the use of OE for inverse problems in the atmospheric sciences. We introduce two example problems: how to retrieve drop size distribution parameters from radar observations and how to retrieve the temperature profile from ground-based microwave sensors. Using these examples, we discuss common pitfalls, how the various error sources impact the retrieval, and how the quality of the retrieval results can be quantified.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

An Online Joint Optimization–Estimation Architecture for Distribution Networks

Here in this article, we propose an optimal joint optimization-estimation architecture for distribution networks, which jointly solves the optimal power flow (OPF) problem and static state estimation (SE) problem through an online gradient-based feedback algorithm. The main objective is to enable a fast and timely interaction between the OPF decisions and state estimators with limited sensor measurements. First, convergence and optimality of the proposed algorithm are analytically established. Then, the proposed gradient-based algorithm is modified by introducing statistical information of the inherent estimation and linearization errors for an improved and robust performance of the online OPF decisions. Overall, the proposed method eliminates the traditional separation of operation and monitoring, where optimization and estimation usually operate at distinct layers and different time scales. Hence, it enables a computationally affordable, efficient, and robust online operational framework for distribution networks under time-varying settings.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Light in the dark forest. Part I. An efficient optimal estimator for 3D Lyman-alpha forest power spectrum

The highly anisotropic nature of the Lyman-alpha (Lyα) forest data introduces a complex survey window function that complicates the measurement of the three-dimensional power spectrum ( P 3D ). In this paper, we present the first fully optimal estimator for P 3D , which exactly deconvolves the survey window function and marginalizes contaminated modes that distort the power spectrum. Our approach adapts optimal estimator techniques developed for the 2D cosmic microwave background data to the 3D case. To achieve computational feasibility, we employ the conjugate gradient method and implement the P 3 M formalism to handle large-scale and small-scale operations separately and efficiently. We validate our estimator using Monte Carlo mocks and Gaussian simulations, demonstrating its accuracy and computational efficiency. We confirm that mode marginalization eliminates distortions arising from quasar continuum errors and delivers robust power spectrum estimation, though it also inflates errors at large scales. This first implementation works in the flat-sky case; we discuss the remaining steps needed to generalize it to the curved-sky case. This formalism offers a foundation for the Lyα forest P 3D measurements and a new path toward cosmological constraints from the Lyα forest data.

Lyman alpha forest↗
DOE OSTI2022

Synergistic Retrieval of Temperature and Humidity Profiles from Space-Based and Ground-Based Infrared Sounders Using an Optimal Estimation Method

The atmospheric temperature and humidity profiles of the troposphere are generally measured by radiosondes and satellites, which are essential for analyzing and predicting weather. Nevertheless, the insufficient observation frequencies and low detection accuracy of the boundary layer restricts the description of atmospheric state changes by the temperature and humidity profiles. Therefore, this work focus on retrieving the temperature and humidity profiles using observations of the FengYun-4 (FY-4) Geostationary Interferometric Infrared Sounder (GIIRS) combined with ground-based infrared spectral observations from the Atmospheric Emitted Radiance Interferometer (AERI), which are more accurate than space-based individual retrieval results and have a wider effective retrieval range than ground-based individual retrieval results. Based on the synergistic observations, which are made by matching the space-based and ground-based data with those of different spatial and temporal resolutions, a synergistic retrieval process is proposed to obtain the temperature and humidity profiles at a high frequency under clear-sky conditions based on the optimal estimation method. In this research, using the line-by-line radiative transfer model (LBLRTM) as the forward model for observing simulations, a retrieval experiment was carried out in Qingdao, China, where an AERI is situated. Taking radiosonde data as a reference for comparing the retrieval results of the temperature and humidity profiles of the troposphere, the root-mean-square error (RMSE) of the synergistic retrieval algorithm below 400 hPa is within 2 K for temperature and within 12% for relative humidity. Compared with the GIIRS individual retrieval, the RMSE of temperature and relative humidity for the synergistic method is reduced by 0.13–1.5 K and 2.7–4.4% at 500 hPa, and 0.13–2.1 K and 2.5–7.2% at 900 hPa. Moreover, the forecast index (FI) calculated from the retrieval results shows reasonable consistency with the FIs calculated from the ERA5 reanalysis and from radiosonde data. The synergistic retrieval results have higher temporal resolution than space-based retrieval results and can reflect the changes in the atmospheric state more accurately. Overall, the results demonstrated the promising potential of the synergistic retrieval of temperature and humidity profiles at high accuracy and high temporal resolution under clear-sky conditions from FY-4/GIIRS and AERI.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Using optimal estimation to retrieve winds from velocity-azimuth display (VAD) scans by a Doppler lidar

Abstract. Low-powered commercially available coherent Doppler lidar (CDL) wind profilers provide continuous measurement of vertical profiles of wind in the lower troposphere, usually close to or up to the top of the planetary boundary layer. The vertical extent of these wind profiles is limited by the availability of scatterers and thus varies substantially throughout the day and from one day to the next. This makes it challenging to develop continuous products that rely on CDL-observed wind profiles. In order to overcome this problem, we have developed a new method for wind profile retrievals from CDL that combines the traditional velocity-azimuth display (VAD) technique with optimal estimation (OE) to provide continuous wind profiles up to 3 km. The new method exploits the level-to-level covariance present in the wind profile to fill in the gaps where the signal-to-noise ratio of the CDL return is too low to provide reliable results using the traditional VAD method. Another advantage of the new method is that it provides the full error covariance matrix of the solution and profiles of information content, which more easily facilitates the assimilation of the observed wind profiles into numerical weather prediction models. This method was tested using yearlong CDL measurements at the Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) Central Facility in 2019. Comparison with the ARM operational CDL wind profile product and collocated radiosonde wind measurements shows excellent agreement (R2>0.99) with no degradation in results where the traditional VAD provided a valid solution. In the region where traditional VAD does not provide results, the OE wind speed and wind vector have uncertainties of 3.44 and 4.33 m s−1, respectively. As a result, the new method provides additional information over the standard technique and increases the effective range of existing CDL systems without the need for additional hardware.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

DESI DR1 Lyα 1D power spectrum: the optimal estimator measurement

The one-dimensional power spectrum P 1D of Lyα forest offers rich insights into cosmological and astrophysical parameters, including constraints on the sum of neutrino masses, warm dark matter models, and the thermal state of the intergalactic medium. We present the measurement of P 1D using the optimal quadratic maximum likelihood estimator applied to over 300,000 Lyα quasars from Data Release 1 (DR1) of the Dark Energy Spectroscopic Instrument (DESI) survey. This sample represents the largest to date for P 1D measurements and is larger than the Extended Baryon Oscillation Spectroscopic Survey (eBOSS) by a factor of 1.7. We conduct a meticulous investigation of instrumental and analysis systematics and quantify their impact on P 1D . This includes the development of a cross-exposure estimator that eliminates the need to model the pipeline noise and has strong potential for future P 1D measurements. We also present new insights into metal contamination through the 1D correlation function. Using a fitting function we measure the evolution of the Lyα forest bias with high precision: b F (z) = (-0.218 ± 0.002) × ((1 + z)/4) 2.96±0.06 . In a companion validation paper, we substantially extend our previous suite of CCD image simulations to quantify the pipeline's exquisite performance accurately. In another companion paper, we present DR1 P 1D measurements using the Fast Fourier Transform (FFT) approach to power spectrum estimation. These two measurements produce a forest bias parameter that differs by 2.2 sigma. However, our model is simplistic, so this disagreement will be investigated in future work.

Lyman alpha forest↗
DOE OSTI2024

Optimal function estimation with photonic quantum sensor networks

The problem of optimally measuring an analytic function of unknown local parameters each linearly coupled to a qubit sensor is well understood, with applications ranging from field interpolation to noise characterization. Here we resolve a number of open questions that arise when extending this framework to Mach-Zehnder interferometers and quadrature displacement sensing. In particular, we derive lower bounds on the achievable mean square error in estimating a linear function of either local phase shifts or quadrature displacements. In the case of local phase shifts, these results prove, and somewhat generalize, a conjecture by Proctor []. For quadrature displacements, we extend proofs of lower bounds to the case of arbitrary linear functions. We provide optimal protocols achieving these bounds up to small (multiplicative) constants and describe an algebraic approach to deriving new optimal protocols, possibly subject to additional constraints. Using this approach, we prove necessary conditions for the amount of entanglement needed for any optimal protocol for both local phase and displacement sensing. Published by the American Physical Society 2024

Bringewatt, Jacob↗
DOE OSTI2020

An a priori evaluation of a principal component and artificial neural network based combustion model in diesel engine conditions

A principal component analysis (PCA) and artificial neural network (ANN) based chemistry tabulation approach is presented. ANNs are used to map the thermochemical state onto a low-dimensional manifold consisting of five control variables that have been identified using PCA. Three canonical configurations are considered to train the PCA-ANN model: a series of homogeneous reactors, a nonpremixed flamelet, and a two-dimensional lifted flame. The performance of the model in predicting the thermochemical manifold of a spatially-developing turbulent jet flame in diesel engine thermochemical conditions is a priori evaluated using direct numerical simulation (DNS) data. The PCA-ANN approach is compared with a conventional tabulation approach (tabulation using ad hoc defined control variables and linear interpolation). The PCA-ANN model provides higher accuracy and requires several orders of magnitude less memory. Here, these observations indicate that the PCA-ANN model is superior for chemistry tabulation, especially for modelling complex chemistries that present multiple combustion modes as observed in diesel combustion. The performance of the PCA-ANN model is then compared to the optimal estimator, i.e. the conditional mean from the DNS. The results indicate that the PCA-ANN model gives high prediction accuracy, comparable to the optimal estimator, especially for major species and the thermophysical properties. Higher errors are observed for the minor species and reaction rate predictions when compared to the optimal estimator. It is shown that the prediction of minor species and reaction rates can be improved by using training data that exhibits a variation of parameters as observed in the turbulent flame. The output of the ANN is analysed to assess mass conservation. It is observed that the ANN incurs a mean absolute error of 0.05% in mass conservation. Furthermore, it is demonstrated that this error can be reduced by modifying the cost function of the ANN to penalise for deviation from mass conservation.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Impact of Seasonal Snow-Cover Change on the Observed and Simulated State of the Atmospheric Boundary Layer in a High-Altitude Mountain Valley

The structure and evolution of the atmospheric boundary layer (ABL) under clear-sky fair weather conditions over mountainous terrain is dominated by the diurnal cycle of the surface energy balance and thus strongly depends on surface snow cover. We use data from three passive ground-based infrared spectrometers deployed in the East River Valley in Colorado's Rocky Mountains to investigate the response of the thermal ABL structure to changes in surface energy balance during the seasonal transition from low to high snow cover. Temperature profiles were retrieved from the infrared radiances using the optimal estimation physical retrieval Tropospheric Remotely Observed Profiling via Optimal Estimation. A nocturnal surface inversion formed in the valley during clear-sky days, which was subsequently mixed out during daytime with the development of a convective boundary layer when snow cover was low. Over high snow cover, a very shallow convective boundary layer formed, above which the inversion persisted through the daytime hours. We compare these observations to NOAA's operational High-Resolution-Rapid-Refresh model and find large warm biases on clear-sky days resulting from the model's inability to form strong nocturnal inversions and to maintain the stable stratification in the valley during daytime when there was snow on the ground. We suggest several factors contributing to the large model errors. These are (a) the inability of the model to represent well-developed thermally driven flows likely due to the too coarse horizontal grid spacing (3 km), (b) too much convective mixing during daytime, and (c) too strong vertical coupling between the valley atmosphere and the free troposphere.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Predictive modeling of a subcritical pulverized-coal power plant for optimization: Parameter estimation, validation, and application

As renewable power generation deployment increases, fossil fuel plants are increasingly required to operate more flexibly. Many coal-fired power plants were originally designed to operate at base load and do not operate optimally at partial load. Predictive first-principles plant-wide models can be employed to identify opportunities for flexibility improvements and diagnose low-load operating issues. This paper describes the application of the Institute for the Design of Advanced Energy Systems Integrated Platform (IDAES) to model and optimize flexible power plant operations. The key benefits of using IDAES are that it provides an open-source, fully equation-oriented modeling framework for efficient modular model construction, reuse, and customization, together with a mathematical optimization framework leveraging powerful, state-of-the-art solvers. The process systems engineering workflow from predictive process simulation to parameter estimation, model validation, and plant optimization is applicable to a variety of existing and next-generation energy systems as well as other chemical and environmental processes. Here, to demonstrate this capability, a physics-based, steady-state model was developed to improve full- and part-load performance of the Escalante Generating Station, a 245 MWe (net) subcritical pulverized coal-fired power plant owned and operated by Tri-State Generation and Transmission Association. Specifically, sixty-nine model parameters were simultaneously estimated from several months of operating data enabling prediction of flow rates, temperatures, pressures, and steam quality throughout the plant. The validated model was leveraged by Escalante to reduce the minimum operating load from 90 MW to 50 MW by diagnosing a low-load water-hammer issue, enabling coal usage and emissions reductions during periods of low power demand. Additionally, opportunities for heat rate reduction (i.e., efficiency improvement) through a steeper sliding-pressure approach to load-following and optimization of other boiler operating variables were also identified and quantified. For example, a potential efficiency improvement of 0.7 percentage points was observed at half-load operation.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2021

Dakota A Multilevel Parallel Object-Oriented Framework for Design Optimization Parameter Estimation Uncertainty Quantification and Sensitivity Analysis (V.6.14) (Theory Manual)

The Dakota toolkit provides a flexible and extensible interface between simulation codes and iterative analysis methods. Dakota contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quantification with sampling, reliability, and stochastic expansion methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the Dakota toolkit provides a flexible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a theoretical manual for selected algorithms implemented within the Dakota software. It is not intended as a comprehensive theoretical treatment, since a number of existing texts cover general optimization theory, statistical analysis, and other introductory topics. Rather, this manual is intended to summarize a set of Dakota-related research publications in the areas of surrogate-based optimization, uncertainty quantification, and optimization under uncertainty that provide the foundation for many of Dakota's iterative analysis capabilities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Dakota, A Multilevel Parallel Object-Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis: Theory Manual (V.6.15)

The Dakota toolkit provides a flexible and extensible interface between simulation codes and iterative analysis methods. Dakota contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quantification with sampling, reliability, and stochastic expansion methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the Dakota toolkit provides a flexible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a theoretical manual for selected algorithms implemented within the Dakota software. It is not intended as a comprehensive theoretical treatment, since a number of existing texts cover general optimization theory, statistical analysis, and other introductory topics. Rather, this manual is intended to summarize a set of Dakota-related research publications in the areas of surrogate-based optimization, uncertainty quantification, and optimization under uncertainty that provide the foundation for many of Dakota's iterative analysis capabilities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Dakota A Multilevel Parallel Object-Oriented Framework for Design Optimization Parameter Estimation Uncertainty Quantification and Sensitivity Analysis: Version 6.12 Theory Manual

The Dakota toolkit provides a flexible and extensible interface between simulation codes and iterative analysis methods. Dakota contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quantification with sampling, reliability, and stochastic expansion methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the Dakota toolkit provides a flexible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a theoretical manual for selected algorithms implemented within the Dakota software. It is not intended as a comprehensive theoretical treatment, since a number of existing texts cover general optimization theory, statistical analysis, and other introductory topics. Rather, this manual is intended to summarize a set of Dakota-related research publications in the areas of surrogate-based optimization, uncertainty quantification, and optimization under uncertainty that provide the foundation for many of Dakota's iterative analysis capabilities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Dakota A Multilevel Parallel Object-Oriented Framework for Design Optimization Parameter Estimation Uncertainty Quantification and Sensitivity Analysis (V.6.14) (User's Manual)

The Dakota toolkit provides a flexible and extensible interface between simulation codes and iterative analysis methods. Dakota contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quantification with sampling, reliability, and stochastic expansion methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the Dakota toolkit provides a flexible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a users manual for the Dakota software and provides capability overviews and procedures for software execution, as well as a variety of example studies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Dakota, A Multilevel Parallel Object-Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis: Version 6.13 Theory Manual

The Dakota toolkit provides a flexible and extensible interface between simulation codes and iterative analysis methods. Dakota contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quantification with sampling, reliability, and stochastic expansion methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the Dakota toolkit provides a flexible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a theoretical manual for selected algorithms implemented within the Dakota software. It is not intended as a comprehensive theoretical treatment, since a number of existing texts cover general optimization theory, statistical analysis, and other introductory topics. Rather, this manual is intended to summarize a set of Dakota-related research publications in the areas of surrogate-based optimization, uncertainty quantification, and optimization under uncertainty that provide the foundation for many of Dakota's iterative analysis capabilities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Dakota, A Multilevel Parallel Object-Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis (V.6.16 User's Manual)

The Dakota toolkit provides a flexible and extensible interface between simulation codes and iterative analysis methods. Dakota contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quantification with sampling, reliability, and stochastic expansion methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the Dakota toolkit provides a flexible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a user's manual for the Dakota software and provides capability overviews and procedures for software execution, as well as a variety of example studies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Dakota, A Multilevel Parallel Object-Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis: Version 6.16 Theory Manual

The Dakota toolkit provides a flexible and extensible interface between simulation codes and iterative analysis methods. Dakota contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quantification with sampling, reliability, and stochastic expansion methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the Dakota toolkit provides a flexible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a theoretical manual for selected algorithms implemented within the Dakota software. It is not intended as a comprehensive theoretical treatment, since a number of existing texts cover general optimization theory, statistical analysis, and other introductory topics. Rather, this manual is intended to summarize a set of Dakota-related research publications in the areas of surrogate-based optimization, uncertainty quantification, and optimization under uncertainty that provide the foundation for many of Dakota’s iterative analysis capabilities.

97 MATHEMATICS AND COMPUTING↗