SEARCH · Search NASA

Results for “RL”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

All sources NASA NTRS DOE OSTI

NASA NTRS2000

Adaptivity in Agent-Based Routing for Data Networks

Adaptivity, both of the individual agents and of the interaction structure among the agents, seems indispensable for scaling up multi-agent systems (MAS s) in noisy environments. One important consideration in designing adaptive agents is choosing their action spaces to be as amenable as possible to machine learning techniques, especially to reinforcement learning (RL) techniques. One important way to have the interaction structure connecting agents itself be adaptive is to have the intentions and/or actions of the agents be in the input spaces of the other agents, much as in Stackelberg games. We consider both kinds of adaptivity in the design of a MAS to control network packet routing. We demonstrate on the OPNET event-driven network simulator the perhaps surprising fact that simply changing the action space of the agents to be better suited to RL can result in very large improvements in their potential performance: at their best settings, our learning-amenable router agents achieve throughputs up to three and one half times better than that of the standard Bellman-Ford routing algorithm, even when the Bellman-Ford protocol traffic is maintained. We then demonstrate that much of that potential improvement can be realized by having the agents learn their settings when the agent interaction structure is itself adaptive.

Wolpert, David H.↗

NASA NTRS1998

Using Collective Intelligence to Route Internet Traffic

A Collective Intelligence (COIN) is a community of interacting reinforcement learning (RL) algorithms designed so that their collective behavior maximizes a global utility function. We introduce the theory of COINs, then present experiments using that theory to design COINs to control internet traffic routing. These experiments indicate that COINs outperform previous RL-based systems for such routing that have previously been investigated.

Wolpert, David H.↗

NASA NTRS2004

Conductivity of GRCop-42 Alloy Enhanced

GRCop-84, a material developed at the NASA Glenn Research Center, has shown considerable promise for staged combustion rocket engine cycles such as the Space Shuttle Main Engine. However, for an expander cycle rocket engine, the transfer of heat to the fuel is a paramount factor in determining the efficiency of the engine. Examples of current and potential future expander cycle rocket engines are the Pratt & Whitney RL-10, RL-60, and RLX engines. Development of a higher conductivity version of GRCop-84 was undertaken to meet these needs. All expander cycle engines need a main combustion chamber liner with the maximum possible thermal conductivity. In an effort at Glenn to trade some of the greatly increased mechanical properties of GRCop-84 for improved thermal conductivity, the amounts of chromium and niobium were halved. The new Cu-4 at.% Cr-2 at.% Nb alloy was designated GRCop-42.

Ellis, David L.↗

NASA NTRS2005

Ka-Band, MEMS Switched Line Phase Shifters Implemented in Finite Ground Coplanar Waveguide

Ka-band MEMS switched line phase shifters implemented in finite ground coplanar waveguide are described in this paper. The phase shifters are constructed of single-pole double-throw (SPDT) switches with additional reference and phase offset transmission line lengths. The one- and two-bit phase shifters are fabricated on high resistivity (HR) silicon with a dielectric constant, Epsilon(sub T) = 11.7 and a substrate thickness, t = 500microns. The switching architectures integrated within the phase shifters consist of MEMS switches that are doubly anchored cantilever beam capacitive switches with additional high inductive sections (MEMS LC device). The SPDT switch is composed of a T-junction with a MEMS LC device at each output port. The one-bit phase shifter described in this paper has an insertion loss (IL) and return loss (RL) of 0.9 dB and 30 dB while the two-bit described has an IL and RL of 1.8 dB and 30 dB respectively. The one-bit phase shifter's designed offset phase is 22.5deg and actual measured phase shift is 21.8deg. The two-bit phase shifter's designed offset phase is 22.5deg, 45deg, and 67.5deg and the actual measured phase shifts are 21.4deg, 44.2deg, and 65.8deg, respectively.

Scardelletti, Maximilian C.↗

NASA NTRS2007

High-Temperature Characterization of Alumina Substrates and Folded Slot Antenna

The characterization of 99.6% polycrystalline alumina substrates and folded slot antennas is presented in this paper. Coplanar Waveguide (CPW) calibration standards where fabricated on 99.6% polycrystalline alumina substrates. Thru-Reflect-Line (TRL) calibrations were performed at temperatures of 25 to 400 C over a frequency range from 0.5 to 50 GHz. The effective dielectric constant of the alumina substrates was determined to be relatively constant at 5.4 from 25 to 200 C and then increased steadily to approximately 6 at 400 C. The loss of the CPW lines increased as temperature and frequency increased as well noting that the increase in loss is even more profound as both the temperature and frequency increased simultaneously. A CPW fed folded slot antenna fabricated on the alumina substrates is also presented. The Return Loss (RL) of a CPW fed folded slot antenna exhibits a self impedance matching technique. The RL of the folded slot antenna at 25 C displays a resonance at 3.5 GHz and is approximately 16 dB in magnitude. At 300 C the resonance shifts down in frequency to approximately 3.3 dB and has roughly the same magnitude. However the loss of the antenna response also increases at this elevated temperature. This work is part of the initial stages of an effort at NASA Glenn research Center to develop wireless sensors for harsh environments. This work can reduce cost, space to house all the hard wired connections, and reduce the weight of the overall units such as aerospace engines and allow for the placement of the wireless sensors in the engine which are not possible with hard wired sensors.

Scardelletti, Maximilian C.↗

NASA NTRS2011

Current Sheets in the Heliosheath: Voyager 1, 2009

We identified all of the current sheets for which we have relatively complete and accurate magnetic field (B) data from Voyager 1 (V1) from days of year (DOYs) 1 to 331, 2009, which were obtained deep in the heliosheath between 108.5 and 111.8 AU. Three types of current sheets were found: (1) 15 proton boundary layers (PBLs), (2) 10 and 3 magnetic holes and magnetic humps, respectively, and (3) 3 sector boundaries. The magnetic field strength changes across PBL, and the profile B(t) is linearly related to the hyperbolic tangent function, but the direction of B does not change. For each of the three sector boundaries, B rotated in a plane normal to the minimum variance direction, and the component of B along the minimum variance direction was zero within the uncertainties, indicating that the sector boundaries were tangential discontinuities. The structure of the sector boundaries was not as simple as that for PBLs. The average thickness of magnetic holes and humps (approx.30 RL) was twice that of the PBLs (approx.15 RL). The average thickness of the current sheets associated with sector boundaries was close to the thickness of the PBLs. Our observations are consistent with the hypothesis that magnetic holes and humps are solitons, which are initiated by the mirror mode instability, and evolve by nonlinear kinetic plasma processes to pressure balanced structures maintained by magnetization currents and proton drift currents in the gradients of B.

Burlaga, L. F.↗

NASA NTRS2013

Nuclear Thermal Rocket Simulation in NPSS

Four nuclear thermal rocket (NTR) models have been created in the Numerical Propulsion System Simulation (NPSS) framework. The models are divided into two categories. One set is based upon the ZrC-graphite composite fuel element and tie tube-style reactor developed during the Nuclear Engine for Rocket Vehicle Application (NERVA) project in the late 1960s and early 1970s. The other reactor set is based upon a W-UO2 ceramic- metallic (CERMET) fuel element. Within each category, a small and a large thrust engine are modeled. The small engine models utilize RL-10 turbomachinery performance maps and have a thrust of approximately 33.4 kN (7,500 lbf ). The large engine models utilize scaled RL-60 turbomachinery performance maps and have a thrust of approximately 111.2 kN (25,000 lbf ). Power deposition profiles for each reactor were obtained from a detailed Monte Carlo N-Particle (MCNP5) model of the reactor cores. Performance factors such as thermodynamic state points, thrust, specific impulse, reactor power level, and maximum fuel temperature are analyzed for each engine design.

Belair, Michael L.↗

NASA NTRS2013

Nuclear Thermal Rocket Simulation in NPSS

Four nuclear thermal rocket (NTR) models have been created in the Numerical Propulsion System Simulation (NPSS) framework. The models are divided into two categories. One set is based upon the ZrC-graphite composite fuel element and tie tube-style reactor developed during the Nuclear Engine for Rocket Vehicle Application (NERVA) project in the late 1960s and early 1970s. The other reactor set is based upon a W-UO2 ceramic-metallic (CERMET) fuel element. Within each category, a small and a large thrust engine are modeled. The small engine models utilize RL-10 turbomachinery performance maps and have a thrust of approximately 33.4 kN (7,500 lbf ). The large engine models utilize scaled RL-60 turbomachinery performance maps and have a thrust of approximately 111.2 kN (25,000 lbf ). Power deposition profiles for each reactor were obtained from a detailed Monte Carlo N-Particle (MCNP5) model of the reactor cores. Performance factors such as thermodynamic state points, thrust, specific impulse, reactor power level, and maximum fuel temperature are analyzed for each engine design.

Belair, Michael L.↗

NASA NTRS2014

One Dimensional Analysis Model of a Condensing Spray Chamber Including Rocket Exhaust Using SINDA/FLUINT and CEA

Modeling droplet condensation via CFD codes can be very tedious, time consuming, and inaccurate. CFD codes may be tedious and time consuming in terms of using Lagrangian particle tracking approaches or particle sizing bins. Also since many codes ignore conduction through the droplet and or the degradating effect of heat and mass transfer if noncondensible species are present, the solutions may be inaccurate. The modeling of a condensing spray chamber where the significant size of the water droplets and the time and distance these droplets take to fall, can make the effect of droplet conduction a physical factor that needs to be considered in the model. Furthermore the presence of even a relatively small amount of noncondensible has been shown to reduce the amount of condensation. It is desirable then to create a modeling tool that addresses these issues. The path taken to create such a tool is illustrated. The application of this tool and subsequent results are based on the spray chamber in the Spacecraft Propulsion Research Facility (B2) located at NASA's Plum Brook Station that tested an RL-10 engine. The platform upon which the condensation physics is modeled is SINDAFLUINT. The use of SINDAFLUINT enables the ability to model various aspects of the entire testing facility, including the rocket exhaust duct flow and heat transfer to the exhaust duct wall. The ejector pumping system of the spray chamber is also easily implemented via SINDAFLUINT. The goal is to create a transient one dimensional flow and heat transfer model beginning at the rocket, continuing through the condensing spray chamber, and finally ending with the ejector pumping system. However the model of the condensing spray chamber may be run independently of the rocket and ejector systems detail, with only appropriate mass flow boundary conditions placed at the entrance and exit of the condensing spray chamber model. The model of the condensing spray chamber takes into account droplet conduction as well as the degrading effect of mass and heat transfer due to the presence of noncondensibles. The one dimension model of the condensing spray chamber makes no presupposition on the pressure profile within the chamber, allowing the implemented droplet physics of heat and mass transfer coupled to the SINDAFLUINT solver to determine a transient pressure profile of the condensing spray chamber. Model results compare well to the RL-10 engine pressure test data.

Chemical Equilibrium↗

NASA NTRS2014

One Dimensional Analysis Model of a Condensing Spray Chamber Including Rocket Exhaust Using SINDA/FLUINT and CEA

Modeling droplet condensation via CFD codes can be very tedious, time consuming, and inaccurate. CFD codes may be tedious and time consuming in terms of using Lagrangian particle tracking approaches or particle sizing bins. Also since many codes ignore conduction through the droplet and or the degradating effect of heat and mass transfer if noncondensible species are present, the solutions may be inaccurate. The modeling of a condensing spray chamber where the significant size of the water droplets and the time and distance these droplets take to fall, can make the effect of droplet conduction a physical factor that needs to be considered in the model. Furthermore the presence of even a relatively small amount of noncondensible has been shown to reduce the amount of condensation [Ref 1]. It is desirable then to create a modeling tool that addresses these issues. The path taken to create such a tool is illustrated. The application of this tool and subsequent results are based on the spray chamber in the Spacecraft Propulsion Research Facility (B2) located at NASA's Plum Brook Station that tested an RL-10 engine. The platform upon which the condensation physics is modeled is SINDAFLUINT. The use of SINDAFLUINT enables the ability to model various aspects of the entire testing facility, including the rocket exhaust duct flow and heat transfer to the exhaust duct wall. The ejector pumping system of the spray chamber is also easily implemented via SINDAFLUINT. The goal is to create a transient one dimensional flow and heat transfer model beginning at the rocket, continuing through the condensing spray chamber, and finally ending with the ejector pumping system. However the model of the condensing spray chamber may be run independently of the rocket and ejector systems detail, with only appropriate mass flow boundary conditions placed at the entrance and exit of the condensing spray chamber model. The model of the condensing spray chamber takes into account droplet conduction as well as the degrading effect of mass and heat transfer due to the presence of noncondensibles. The one dimension model of the condensing spray chamber makes no presupposition on the pressure profile within the chamber, allowing the implemented droplet physics of heat and mass transfer coupled to the SINDAFLUINT solver to determine a transient pressure profile of the condensing spray chamber. Model results compare well to the RL-10 engine pressure test data.

Condensing Spray Model↗

NASA NTRS2017

The Impact of Lidar Detection Sensitivity on Assessing Aerosol Direct Radiative Effects

Spaceborne lidar observations have great potential to provide accurate global estimates of the aerosol direct radiative effect (DRE) in both clear and cloudy conditions. However, comparisons between observations from the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite (CALIPSO) and multiple years of Atmospheric Radiation Measurement (ARM) program’s ground-based Raman lidars (RL) show that CALIPSO does not detect all radiatively significant aerosol, i.e. aerosol that directly modifies the Earth’s radiation budget. We estimated that using CALIPSO observations results in an underestimate of the magnitude of the global mean aerosol DRE by up to 54%. The ARM RL datasets along with NASA Langley airborne high spectral resolution lidar (HSRL) data from multiple field campaigns are used to compute the detection sensitivity required to accurately resolve the aerosol DRE. This shows that a lidar with a backscatter coefficient detection sensitivity of about 1−2x10(exp -4)km(exp -1)sr(exp -1) at 532nm would resolve all the aerosol needed to derive the DRE to within 1%.

Tyler J. Thorsen↗

NASA NTRSDate not supplied

Energy-Optimized Path Planning for Uas in Varying Winds Via Reinforcement Learning

In this paper we propose a reinforcement learning (RL) algorithm for path planning of Unmanned Aviation Vehicles (UAVs) under varying wind conditions. Solutions to UAV path planning problems are becoming increasingly necessary as autonomous UAVs continue to enter commercial and government spaces. Path-planning is inherently challenging, as UAVs need to account for dynamically changing flying conditions such as weather, obstacle or no-fly zones, degraded vehicle health, and off-nominal battery power consumption. Machine learning methods such as reinforcement learning (RL) have the potential to revolutionize how vehicles navigate in such uncertain environments. In this study, we compute UAV trajectories from a pre-determined starting position to a target cell within a 7X7 grid environment by optimizing parameters for mission assurance and safety limits in addition to the energy consumption and operation time. The UAV navigates the grid by taking actions to move in any of the eight cardinal and inter-cardinal directions, under constant thrust profile. The resultant UAV state is sampled from a probability distribution which accounts for the UAV’s action, local wind velocity, and the presence of obstacles or boundaries. As the unmanned airspace gets more complex due to multiple vehicles and environmental uncertainties, trade-offs between energy consumption, operation time, risk tolerance, and mission assurance need to be made. Our Markov Decision Process (MDP) environment model can capture any combination of these in the optimization objective, making it novel compared to other work in the field.

trajectory planning↗

DOE OSTI2024

Reinforcement Learning to Enhance Optimal Operation of Resilient Community Energy Systems

This paper presents a novel model-free multi-agent Reinforcement Learning (RL) control method to enhance the resilience of community energy systems in island mode, which coordinates multiple objectives without the necessity of identifying system models that require expert knowledge. Specifically, a community-level coordinator agent is designed to allocate renewable energy resources among different buildings, and multiple building-level agents are developed to optimize load schedules based on limited energy resources and requirements of building loads and occupants’ comfort. In a two-day evaluation, our RL approach demonstrated a similar performance against MPC without requiring system models and formulation of optimization problems as required in MPC.

ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION↗

DOE OSTI2026

CDRL: Certification-Driven Reinforcement Learning for Neutrino Flavor Model Discovery

Many scientific discovery problems require searching combinatorial hypothesis spaces under complex domain constraints. Reinforcement learning (RL) offers a promising approach, but existing methods rely on scalar rewards that provide limited information about why candidate solutions fail, leading agents to repeatedly explore invalid regions. We introduce Certification-Driven Reinforcement Learning (CDRL), a framework that leverages structured feedback from symbolic reasoning tools. When a candidate violates domain constraints, these tools produce certificates identifying the actions responsible for failure. CDRL converts these certificates into reusable constraints that eliminate classes of invalid solutions and guide exploration toward valid regions. We evaluate CDRL on neutrino flavor model discovery in theoretical particle physics, where the hypothesis space exceeds $10^{26}$ possible models, and compare it with the state-of-the-art RL approach previously used for this task. Across three theory spaces, CDRL achieves up to 1.95$\times$ higher valid model rates and up to 6.33$\times$ higher neutrino model rates while evaluating up to 4$\times$ fewer candidates. We further extract 40 interpretable rules from search trajectories using a post-hoc decision-tree framework and show that reusing them as soft constraints yields gains of up to 2$\times$ in valid model rates and 3$\times$ in neutrino model discovery across all three theory spaces. These results suggest that CDRL uncovers reusable structure in combinatorial search spaces and provides a general framework for scientific model discovery.

Jha, Piyush [Georgia Tech., Atlanta; Georgia Tech]↗

NASA NTRS1964