MULTI-OBJECTIVE REINFORCEMENT LEARNING FOR LOW-THRUST TRANSFER DESIGN BETWEEN LIBRATION POINT ORBITS

Christopher J Sullivan; Natasha Bosanac; Alinda K Mashiku; Rodney L Anderson

NASA NTRS · 20210014584

MULTI-OBJECTIVE REINFORCEMENT LEARNING FOR LOW-THRUST TRANSFER DESIGN BETWEEN LIBRATION POINT ORBITS

Christopher J Sullivan·Natasha Bosanac·Alinda K Mashiku·Rodney L Anderson

Abstract

Multi-Reward Proximal Policy Optimization (MRPPO) is a multi-objective reinforcement learning algorithm used to construct low-thrust transfers between periodic orbits in multi-body systems. Previous implementations of MRPPO have relied on a predefined reference transfer to successfully train each policy. In this paper, an algorithmic modification labeled the ‘moving reference’, is introduced to autonomously construct these reference trajectories during training. With this modification, MRPPO is used to recover various low-thrust transfers between two periodic orbits in the Earth-Moon circular restricted three-body problem to solve a multi-objective optimization problem. These results are then compared with the solutions recovered via a traditional optimization formulation.

Explore related subjects

algorithm periodic orbits

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Christopher J Sullivan, Natasha Bosanac, Alinda K Mashiku, Rodney L Anderson. MULTI-OBJECTIVE REINFORCEMENT LEARNING FOR LOW-THRUST TRANSFER DESIGN BETWEEN LIBRATION POINT ORBITS. https://ntrs.nasa.gov/citations/20210014584

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related reports

DOE OSTI2024

Demonstration of the rodeo algorithm on a quantum computer

The rodeo algorithm is an efficient algorithm for eigenstate preparation and eigenvalue estimation for any observable on a quantum computer. This makes it a promising tool for studying the spectrum and structure of atomic nuclei as well as other fields of quantum many-body physics. The only requirement is that the initial state has sufficient overlap probability with the desired eigenstate. While it is exponentially faster than well-known algorithms such as phase estimation and adiabatic evolution for eigenstate preparation, it has yet to be implemented on an actual quantum device. In this work, we apply the rodeo algorithm to determine the energy levels of a random one-qubit Hamiltonian, resulting in a relative error of 0.08% using mid-circuit measurements on the IBM Q device Casablanca. This surpasses the accuracy of directly-prepared eigenvector expectation values using the same quantum device. We take advantage of the high-accuracy energy determination and use the Hellmann-Feynman theorem to compute eigenvector expectation values for a different random one-qubit observable. For the Hellmann-Feynman calculations, we find a relative error of 0.7%. Here, we conclude by discussing possible future applications of the rodeo algorithm for multi-qubit Hamiltonians.

algorithm↗

NASA NTRS2021

Joint cloud water path and rainwater path retrievals from airborne ORACLES observations

This study presents a new algorithm that combines W-band reflectivity measurements from the Airborne Precipitation Radar – third generation (APR-3) passive radiometric cloud optical depth and effective radius retrievals from the Research Scanning Polarimeter (RSP) to estimate total liquid water path in warm clouds and identify the contributions from cloud water path (CWP) and rainwater path (RWP). The resulting CWP estimates are primarily determined by the optical depth input, although reflectivity measurements contribute ∼10 %–50 % of the uncertainty due to attenuation through the profile. Uncertainties in CWP estimates across all conditions are 25 % to 35 %, while RWP uncertainty estimates frequently exceed 100 %. Two-thirds of all radar-detected clouds observed during the ObseRvations of Aerosols above CLouds and their intEractionS (ORACLES) campaign that took place from 2016–2018 over the southeast Atlantic Ocean have CWP between 41 and 168 g/sq. m and almost all CWPs (99 %) between 6 to 445 g/sq. m. RWP, by contrast, typically makes up a much smaller fraction of total liquid water path (LWP), with more than 70 % of raining clouds having less than 10 g/sq. m of rainwater. In heavier warm rain (i.e., rain rate exceeding 40 mm/h or 1000 mm/d), however, RWP is observed to exceed 2500 g/sq. m. CWP (RWP) is found to be approximately 30 g/sq. m (7 g/sq. m) larger in unstable environments compared to stable environments. Surface precipitation is also more than twice as likely in unstable environments. Comparisons against in situ cloud microphysical probe data spanning the range of thermodynamic stability and meteorological conditions encountered across the southeast Atlantic basin demonstrate that the combined APR-3 and RSP dataset enable a robust joint cloud–precipitation retrieval algorithm to support future ORACLES precipitation susceptibility and cloud–aerosol–precipitation interaction studies.

algorithm↗

NASA NTRS2018

Drag De-Orbit Device (D3): A Retractable Device for CubeSat Attitude and Orbit Control using Aerodynamic Forces

The increasing number of CubeSats being launched has raised concerns about orbital debris since most of these satellites have no means of active orbit control. Some technologies exist to increase the surface area of a CubeSat and expedite de-orbit due to aerodynamic drag in low Earth orbit, but most of these devices cannot be retracted and hence cannot be used for orbital maneuvering. This paper discusses the De-Orbit Drag Device (D3) module that is capable of de-orbiting a 12U, 15kg CubeSat from a 700 km circular orbit in under 25 years and can be deployed and retracted to modulate the aerodynamic drag force experienced by the satellite. This facilitates orbital maneuvering using aerodynamic drag and the active targeting of a de-orbit location. In addition, the geometry of this drag device provides 3-axis attitude stabilization of the host CubeSat using aerodynamic and gravity gradient torques which is useful for many missions and provides a predictable aerodynamic profile for use in orbital maneuvering algorithms.

algorithm↗