SEARCH · Search NASA

Results for “Value function”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

All sources NASA NTRS DOE OSTI

NASA NTRS2004

Dynamic Programming for Structured Continuous Markov Decision Problems

We describe an approach for exploiting structure in Markov Decision Processes with continuous state variables. At each step of the dynamic programming, the state space is dynamically partitioned into regions where the value function is the same throughout the region. We first describe the algorithm for piecewise constant representations. We then extend it to piecewise linear representations, using techniques from POMDPs to represent and reason about linear surfaces efficiently. We show that for complex, structured problems, our approach exploits the natural structure so that optimal solutions can be computed efficiently.

Dearden, Richard↗

NASA NTRS2005

Efficient Construction of Discrete Adjoint Operators on Unstructured Grids Using Complex Variables

A methodology is developed and implemented to mitigate the lengthy software development cycle typically associated with constructing a discrete adjoint solver for aerodynamic simulations. The approach is based on a complex-variable formulation that enables straightforward differentiation of complicated real-valued functions. An automated scripting process is used to create the complex-variable form of the set of discrete equations. An efficient method for assembling the residual and cost function linearizations is developed. The accuracy of the implementation is verified through comparisons with a discrete direct method as well as a previously developed handcoded discrete adjoint approach. Comparisons are also shown for a large-scale configuration to establish the computational efficiency of the present scheme. To ultimately demonstrate the power of the approach, the implementation is extended to high temperature gas flows in chemical nonequilibrium. Finally, several fruitful research and development avenues enabled by the current work are suggested.

Nielsen, Eric J.↗

NASA NTRS2006

Uncertainty Analysis for a Jet Flap Airfoil

An analysis of variance (ANOVA) study was performed to quantify the potential uncertainties of lift and pitching moment coefficient calculations from a computational fluid dynamics code, relative to an experiment, for a jet flap airfoil configuration. Uncertainties due to a number of factors including grid density, angle of attack and jet flap blowing coefficient were examined. The ANOVA software produced a numerical model of the input coefficient data, as functions of the selected factors, to a user-specified order (linear, 2-factor interference, quadratic, or cubic). Residuals between the model and actual data were also produced at each of the input conditions, and uncertainty confidence intervals (in the form of Least Significant Differences or LSD) for experimental, computational, and combined experimental / computational data sets were computed. The LSD bars indicate the smallest resolvable differences in the functional values (lift or pitching moment coefficient) attributable solely to changes in independent variable, given just the input data points from selected data sets. The software also provided a collection of diagnostics which evaluate the suitability of the input data set for use within the ANOVA process, and which examine the behavior of the resultant data, possibly suggesting transformations which should be applied to the data to reduce the LSD. The results illustrate some of the key features of, and results from, the uncertainty analysis studies, including the use of both numerical (continuous) and categorical (discrete) factors, the effects of the number and range of the input data points, and the effects of the number of factors considered simultaneously.

Green, Lawrence L.↗

NASA NTRS2006

Learning User Preferences for Sets of Objects

Most work on preference learning has focused on pairwise preferences or rankings over individual items. In this paper, we present a method for learning preferences over sets of items. Our learning method takes as input a collection of positive examples--that is, one or more sets that have been identified by a user as desirable. Kernel density estimation is used to estimate the value function for individual items, and the desired set diversity is estimated from the average set diversity observed in the collection. Since this is a new learning problem, we introduce a new evaluation methodology and evaluate the learning method on two data collections: synthetic blocks-world data and a new real-world music data collection that we have gathered.

preferences↗

NASA NTRS2005

Efficient Construction of Discrete Adjoint Operators on Unstructured Grids by Using Complex Variables

Nielsen, Eric J.↗

NASA NTRS2009

Evolutionary Computing Methods for Spectral Retrieval

A methodology for processing spectral images to retrieve information on underlying physical, chemical, and/or biological phenomena is based on evolutionary and related computational methods implemented in software. In a typical case, the solution (the information that one seeks to retrieve) consists of parameters of a mathematical model that represents one or more of the phenomena of interest. The methodology was developed for the initial purpose of retrieving the desired information from spectral image data acquired by remote-sensing instruments aimed at planets (including the Earth). Examples of information desired in such applications include trace gas concentrations, temperature profiles, surface types, day/night fractions, cloud/aerosol fractions, seasons, and viewing angles. The methodology is also potentially useful for retrieving information on chemical and/or biological hazards in terrestrial settings. In this methodology, one utilizes an iterative process that minimizes a fitness function indicative of the degree of dissimilarity between observed and synthetic spectral and angular data. The evolutionary computing methods that lie at the heart of this process yield a population of solutions (sets of the desired parameters) within an accuracy represented by a fitness-function value specified by the user. The evolutionary computing methods (ECM) used in this methodology are Genetic Algorithms and Simulated Annealing, both of which are well-established optimization techniques and have also been described in previous NASA Tech Briefs articles. These are embedded in a conceptual framework, represented in the architecture of the implementing software, that enables automatic retrieval of spectral and angular data and analysis of the retrieved solutions for uniqueness.

Terrile, Richard↗

NASA NTRS2009

Steady-State ALPS for Real-Valued Problems

The two objectives of this paper are to describe a steady-state version of the Age-Layered Population Structure (ALPS) Evolutionary Algorithm (EA) and to compare it against other GAs on real-valued problems. Motivation for this work comes from our previous success in demonstrating that a generational version of ALPS greatly improves search performance on a Genetic Programming problem. In making steady-state ALPS some modifications were made to the method for calculating age and the method for moving individuals up layers. To demonstrate that ALPS works well on real-valued problems we compare it against CMA-ES and Differential Evolution (DE) on five challenging, real-valued functions and on one real-world problem. While CMA-ES and DE outperform ALPS on the two unimodal test functions, ALPS is much better on the three multimodal test problems and on the real-world problem. Further examination shows that, unlike the other GAs, ALPS maintains a genotypically diverse population throughout the entire search process. These findings strongly suggest that the ALPS paradigm is better able to avoid premature convergence then the other GAs.

Hornby, Gregory S.↗

NASA NTRS2013

Impact of Gender on Pharmocokinetics of Intranasal Scopolamine

Introduction: An intranasal gel dosage formulation of scopolamine (INSCOP) was developed for the treatment of Space Motion Sickness (SMS), which is commonly experienced by astronauts during space missions. The bioavailability and pharmacokinetics (PK) were evaluated under IND guidelines. Since information is lacking on the effect of gender on the PK of Scopolamine, we examined gender differences in PK parameters of INSCOP at three dose levels of 0.1, 0.2 and 0.4 mg. Methods: Plasma scopolamine concentrations as a function of time data were collected from twelve normal healthy human subjects (6 male/6 female) who participated in a fully randomized double blind crossover study. The PK parameters were derived using WinNonlin. Covariate analysis of PK profiles was performed using NONMEN and statistically compared using a likelihood ratio test on the difference of objective function value (OFV). Statistical significance for covariate analysis was set at P<0.05(ΔOFV=3.84). Results: No significant difference in PK parameters between male and female subjects was observed with 0.1 and 0.2 mg doses. However, CL and Vd were significantly different between male and female subjects at the 0.4 mg dose. Results from population covariate modeling analysis indicate that a onecompartment PK model with first-order elimination rate offers best fit for describing INSCOP concentration-time profiles. The inclusion of sex as a covariate enhanced the model fitting (ΔOFV=-4.1) owing to the genderdependent CL and Vd differences after the 0.4 mg dose. Conclusion: Statistical modeling of scopolamine concentration-time data suggests gender-dependent pharmacokinetics of scopolamine at the high dose level of 0.4 mg. Clearance of the parent compound was significantly faster and the volume of distribution was significantly higher in males than in females, As a result, including gender as a covariate to the pharmacokinetic model of scopolamine offers the best fit for PK modeling of the drug at dose of 0.4 mg or higher.

Putcha, L.↗

NASA NTRS2013

The NASA-Goddard Multi-Scale Modeling Framework - Land Information System: Global Land/atmosphere Interaction with Resolved Convection

The present generation of general circulation models (GCM) use parameterized cumulus schemes and run at hydrostatic grid resolutions. To improve the representation of cloud-scale moist processes and landeatmosphere interactions, a global, Multi-scale Modeling Framework (MMF) coupled to the Land Information System (LIS) has been developed at NASA-Goddard Space Flight Center. The MMFeLIS has three components, a finite-volume (fv) GCM (Goddard Earth Observing System Ver. 4, GEOS-4), a 2D cloud-resolving model (Goddard Cumulus Ensemble, GCE), and the LIS, representing the large-scale atmospheric circulation, cloud processes, and land surface processes, respectively. The non-hydrostatic GCE model replaces the single-column cumulus parameterization of fvGCM. The model grid is composed of an array of fvGCM gridcells each with a series of embedded GCE models. A horizontal coupling strategy, GCE4fvGCM4Coupler4LIS, offered significant computational efficiency, with the scalability and I/O capabilities of LIS permitting landeatmosphere interactions at cloud-scale. Global simulations of 2007e2008 and comparisons to observations and reanalysis products were conducted. Using two different versions of the same land surface model but the same initial conditions, divergence in regional, synoptic-scale surface pressure patterns emerged within two weeks. The sensitivity of largescale circulations to land surface model physics revealed significant functional value to using a scalable, multi-model land surface modeling system in global weather and climate prediction.

cumulus schemes↗

NASA NTRS2012

Evaluation of Precipitation Detection over Various Surfaces from Passive Microwave Imagers and Sounders

During the middle part of this decade a wide variety of passive microwave imagers and sounders will be unified in the Global Precipitation Measurement (GPM) mission to provide a common basis for frequent (3 hr), global precipitation monitoring. The ability of these sensors to detect precipitation by discerning it from non-precipitating background depends upon the channels available and characteristics of the surface and atmosphere. This study quantifies the minimum detectable precipitation rate and fraction of precipitation detected for four representative instruments (TMI, GMI, AMSU-A, and AMSU-B) that will be part of the GPM constellation. Observations for these instruments were constructed from equivalent channels on the SSMIS instrument on DMSP satellites F16 and F17 and matched to precipitation data from NOAA's National Mosaic and QPE (NMQ) during 2009 over the continuous United States. A variational optimal estimation retrieval of non-precipitation surface and atmosphere parameters was used to determine the consistency between the observed brightness temperatures and these parameters, with high cost function values shown to be related to precipitation. The minimum detectable precipitation rate, defined as the lowest rate for which probability of detection exceeds 50%, and the detected fraction of precipitation, are reported for each sensor, surface type (ocean, coast, bare land, snow cover) and precipitation type (rain, mix, snow). The best sensors over ocean and bare land were GMI (0.22 mm/hr minimum threshold and 90% of precipitation detected) and AMSU (0.26 mm/hr minimum threshold and 81% of precipitation detected), respectively. Over coasts (0.74 mm/hr threshold and 12% detected) and snow-covered surfaces (0.44 mm/hr threshold and 23% detected), AMSU again performed best but with much lower detection skill, whereas TMI had no skill over these surfaces. The sounders (particularly over water) benefited from the use of re-analysis data (vs. climatology) to set the a-priori atmospheric state and all instruments benefit from the use of a conditional snow cover emissivity database over land. It is recommended that real-time sources of these data be used in the operational GPM precipitation algorithms.

GPM↗

NASA NTRS2014

Interval Predictor Models with a Formal Characterization of Uncertainty and Reliability

This paper develops techniques for constructing empirical predictor models based on observations. By contrast to standard models, which yield a single predicted output at each value of the model's inputs, Interval Predictors Models (IPM) yield an interval into which the unobserved output is predicted to fall. The IPMs proposed prescribe the output as an interval valued function of the model's inputs, render a formal description of both the uncertainty in the model's parameters and of the spread in the predicted output. Uncertainty is prescribed as a hyper-rectangular set in the space of model's parameters. The propagation of this set through the empirical model yields a range of outputs of minimal spread containing all (or, depending on the formulation, most) of the observations. Optimization-based strategies for calculating IPMs and eliminating the effects of outliers are proposed. Outliers are identified by evaluating the extent by which they degrade the tightness of the prediction. This evaluation can be carried out while the IPM is calculated. When the data satisfies mild stochastic assumptions, and the optimization program used for calculating the IPM is convex (or, when its solution coincides with the solution to an auxiliary convex program), the model's reliability (that is, the probability that a future observation would be within the predicted range of outputs) can be bounded rigorously by a non-asymptotic formula.

Crespo, Luis G.↗

NASA NTRS2016

Application of Interval Predictor Models to Space Radiation Shielding

This paper develops techniques for predicting the uncertainty range of an output variable given input-output data. These models are called Interval Predictor Models (IPM) because they yield an interval valued function of the input. This paper develops IPMs having a radial basis structure. This structure enables the formal description of (i) the uncertainty in the models parameters, (ii) the predicted output interval, and (iii) the probability that a future observation would fall in such an interval. In contrast to other metamodeling techniques, this probabilistic certi cate of correctness does not require making any assumptions on the structure of the mechanism from which data are drawn. Optimization-based strategies for calculating IPMs having minimal spread while containing all the data are developed. Constraints for bounding the minimum interval spread over the continuum of inputs, regulating the IPMs variation/oscillation, and centering its spread about a target point, are used to prevent data over tting. Furthermore, we develop an approach for using expert opinion during extrapolation. This metamodeling technique is illustrated using a radiation shielding application for space exploration. In this application, we use IPMs to describe the error incurred in predicting the ux of particles resulting from the interaction between a high-energy incident beam and a target.

Crespo, Luis G.↗

NASA NTRS2017

Interval Predictor Models for Data with Measurement Uncertainty

An interval predictor model (IPM) is a computational model that predicts the range of an output variable given input-output data. This paper proposes strategies for constructing IPMs based on semidefinite programming and sum of squares (SOS). The models are optimal in the sense that they yield an interval valued function of minimal spread containing all the observations. Two different scenarios are considered. The first one is applicable to situations where the data is measured precisely whereas the second one is applicable to data subject to known biases and measurement error. In the latter case, the IPMs are designed to fully contain regions in the input-output space where the data is expected to fall. Moreover, we propose a strategy for reducing the computational cost associated with generating IPMs as well as means to simulate them. Numerical examples illustrate the usage and performance of the proposed formulations.

Lacerda, Marcio J.↗

NASA NTRSDate not supplied

Interval Predictor Models for Robust System Identification

This paper proposes a framework for the identification and uncertainty quantification of plant models according to multivariable data. The only restriction imposed upon such models is for their outputs to depend continuously on their parameters. An Interval Predictor Model (IPM) prescribes the parameters of a computational model as a path-connected set thereby making each predicted output an interval-valued function of its inputs. The formulation proposed seeks the parameter set for which the predicted outputs tightly enclose the data. This set, which is modeled as a semi-algebraic set of low-degree polynomials, enables the characterization of possibly strong parameter dependencies commonly found in practice. This uncertainty characterization makes the resulting plant model amenable to robust control approaches using polynomial optimization. Furthermore, we use non-convex scenario theory to assess the reliability of the resulting IPM. This assessment yields a distribution-free upper bound on the probability that future data will fall outside the predicted intervals.

interval↗

NASA NTRSDate not supplied

Towards Formalization of Advanced Linear Algebra with Applications to Dynamical Systems using PVS

Linear Algebra is essential for numerous aerospace problems of interest. Formal reasoning about hybrid systems that contain variables modeled by differential equations rely on concepts from Linear Algebra such as eigenvalues, matrix decompositions, and matrix valued functions. For example, the long-term dynamics of a system of differential equations depend on the stability/instability of its equilibrium points, which often reduces to an eigenvalue problem. This talk will embark on a quest to formalize theorems and results about eigenvalues and eigenvectors using PVS. We shall start our journey with 2 x 2 complex matrices, where we will apply our PVS code to a simple example of a dynamical system. Since it can be difficult or impossible to give simple expressions of eigenvalues for larger matrices (i.e. 5 x 5 or higher), we then move towards specifying the power method for verified computation of eigenvalue approximations in PVS. This effort requires development of multivariate complex arithmetic. At the end of the day, having such additions to the PVS NASA libraries will help move towards the use of formal methods to verify concepts of control theory and system level verification.

Linear Algebra↗

NASA NTRSDate not supplied

A Look at the Truths and Misconceptions of the Variational Quantum Eigensolver and the Implications of Overparameterization

In this work, we investigate loss landscapes of the variational quantum eigensolver (VQE) by quantifying the number of local minima through empirical analyses. We focus on minimal models in chemistry and physics so that we can do a complete analysis using more computationally expensive tools. We employ Hessian eigenvalue calculations and the nudged elastic band algorithm to characterize these landscapes. Our results expand upon the existing literature by highlighting the optimization challenges faced by VQE. We find that, as the number of parameters in our ansatz increases, the number of basins increases while the corresponding loss function values converge toward the global minimum value. This observation implies that overparameterization may lead to an ``effective convexity'' in VQE loss landscapes, a phenomenon supported by theoretical and numerical work in classical machine learning.

quantum computing↗

NASA NTRSDate not supplied

Hamiltonian Control for Coupled Bending-Torsion Motion of a Swept Wing With R. T. Jones' Approximation

This paper presents distributed Hamiltonian control design approach for a Lagrangian infinite-dimensional system describing the coupled bending-torsion motion of a swept wing in unsteady airflow, which is taken into account using Theodorsen's complex valued function of reduced frequency. An implementable flutter suppressing control surface deflection commands are obtained using modes separation and R.T. Jones's approximation, and were validated in desktop simulations for flight conditions beyond the flutter airspeed.

Vahram Stepanyan↗

DOE OSTI2019

Quasi-Stochastic Approximation and Off-Policy Reinforcement Learning: Preprint

The Robbins-Monro stochastic approximation algorithm is a foundation of many algorithmic frameworks for reinforcement learning and often an efficient approach to solving (or approximating the solution to) complex optimal control problems. In many cases, however, practitioners are unable to apply these techniques because of an inherent high variance. This paper aims to provide a general foundation for 'quasi-stochastic approximation,' in which all of the processes under consideration are deterministic, much like quasi-Monte Carlo for variance reduction in simulation. The variance reduction can be substantial, subject to tuning of pertinent parameters in the algorithm. This paper introduces a new coupling argument to establish the optimal rate of convergence provided the gain is sufficiently large. These results are established for linear models and tested also in nonideal settings. A major application of these general results is a new class of reinforcement learning algorithms for deterministic state space models. In this setting, the main contribution is a class of algorithms for approximating the value function for a given policy, using a different policy designed to introduce exploration.

24 POWER TRANSMISSION AND DISTRIBUTION↗