Search NASA⌕ Search

SEARCH · Search NASA

Results for “Deep Q network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

15 records
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Microservice Architecture for Cognitive Networks

This develops the concept of a cognitive network and describes a microservice based architecture which could be used to implement such a system. Delay tolerant networking (DTN) influences the design of the architecture as well as the networking scenarios that the system attempts to address. A cognitive storage and fragmentation service is developed based on existing artificial intelligence techniques such as Advantage Actor Critic (A2C) and Deep Q-Networks. The system is simulated using OpenAI Gym in a custom developed DTN environment.

cognitive networks↗
NASA NTRSDate not supplied

Deep Reinforcement Learning based Routing in an Air-to-Air Ad-hoc Network

This paper studies the Multiple Sources and Multiple Destinations (MSMD) routing problem in a dynamic Air-to-Air Ad-hoc Network (AAAN). We consider a spectrum limited scenario where multiple links have to share the same frequency channel so that co-channel interference becomes inevitable. As a result, routing decisions and spectrum access are coupled and must be jointly considered. This paper proposes a deep Q-learning based algorithm to find an optimal routing and channel selection strategy that minimizes the end-to-end communication delay. Specifically, under the assumption that only local information is available to every node, the Deep Q-Network (DQN) is trained offline to learn the optimal routing and channel selection strategy. After the trained DQN is implemented in every node, multiple relay nodes can simultaneously determine their next-hop relay and channel selections in real-time. Simulation results demonstrate the efficacy of our proposed algorithm.

AAAN↗
NASA NTRSDate not supplied

Deep Reinforcement Learning based Routing in an Air-to-Air Ad-hoc Network

This paper studies the Multiple Sources and Multiple Destinations (MSMD) routing problem in a dynamic Air-to-Air Ad-hoc Network (AAAN). We consider a spectrum limited scenario where multiple links have to share the same frequency channel so that co-channel interference becomes inevitable. As a result, routing decisions and spectrum access are coupled and must be jointly considered. This paper proposes a deep Q-learning based algorithm to find an optimal routing and channel selection strategy that minimizes the end-to-end communication delay. Specifically, under the assumption that only local information is available to every node, the Deep Q-Network (DQN) is trained offline to learn the optimal routing and channel selection strategy. After the trained DQN is implemented in every node, multiple relay nodes can simultaneously determine their next-hop relay and channel selections in real-time. Simulation results demonstrate the efficacy of our proposed algorithm.

AAAN↗
NASA NTRS1988

Digital carrier demodulation for the DSN Advanced Receiver

The digital in-phase and quadrature (I and Q) carrier demodulation for the Deep Space Network's (DSN) Advanced Receiver is described and analyzed. The system provides coherent demodulation for a variety of modulation formats including Binary Phase Shift Keying (BPSK), BPSK with a carrier residual, Quadrature Phase Shift Keying (QPSK), Offset-QPSK (OQPSK), and Minimum Shift Keying (MSK). The focus is on the theory and hardware design of the halfband filters which are the integral part of the demodulator. The underlying theory of the filters is summarized, a breadboard hardware design is described, and a VLSI implementation is proposed which significantly decreases the hardware. A second important problem analyzed is DC-offset in the demodulator. This is a serious problem which translates into bias error in the residual carrier phase detector. The dynamic range of the complex mixer is analyzed using a probabilistic approach. It is deduced that the resulting static phase error is less than 0.2 deg when the ratio of carrier power to noise power in the demodulator input bandwidth is -72 dB or higher. Thus, the static phase error is negligible at a carrier power to noise spectral density of 0 dB-Hz for a 15 MHz bandwidth demodulator.

Sadr, R.↗
NASA NTRS2010

NASA Tech Briefs, December 2010

Topics include: Coherent Frequency Reference System for the NASA Deep Space Network; Diamond Heat-Spreader for Submillimeter-Wave Frequency Multipliers; 180-GHz I-Q Second Harmonic Resistive Mixer MMIC; Ultra-Low-Noise W-Band MMIC Detector Modules; 338-GHz Semiconductor Amplifier Module; Power Amplifier Module with 734-mW Continuous Wave Output Power; Multiple Differential-Amplifier MMICs Embedded in Waveguides; Rapid Corner Detection Using FPGAs; Special Component Designs for Differential-Amplifier MMICs; Multi-Stage System for Automatic Target Recognition; Single-Receiver GPS Phase Bias Resolution; Ultra-Wideband Angle-of-Arrival Tracking Systems; Update on Waveguide-Embedded Differential MMIC Amplifiers; Automation Framework for Flight Dynamics Products Generation; Product Operations Status Summary Metrics; Mars Terrain Generation; Application-Controlled Parallel Asynchronous Input/Output Utility; Planetary Image Geometry Library; Propulsion Design With Freeform Fabrication (PDFF); Economical Fabrication of Thick-Section Ceramic Matrix Composites; Process for Making a Noble Metal on Tin Oxide Catalyst; Stacked Corrugated Horn Rings; Refinements in an Mg/MgH2/H2O-Based Hydrogen Generator; Continuous/Batch Mg/MgH2/H2O-Based Hydrogen Generator; Strain System for the Motion Base Shuttle Mission Simulator; Ko Displacement Theory for Structural Shape Predictions; Pyrotechnic Actuator for Retracting Tubes Between MSL Subsystems; Surface-Enhanced X-Ray Fluorescence; Infrared Sensor on Unmanned Aircraft Transmits Time-Critical Wildfire Data; and Slopes To Prevent Trapping of Bubbles in Microfluidic Channels.

Source record↗
NASA NTRSDate not supplied

Learning Based Edge Computing in Air-to-Air Communication Network

This paper studies learning-based edge computing and communication in a dynamic Air-to-Air Ad-hoc Network (AAAN). Due to spectrum scarcity, we assume the number of Air-to-Air (A2A) communication links is greater than that of the available frequency channels, such that some communication links have to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the aggregate spectrum utilization efficiency under resource and fairness constraints. A distributed deep Q learning-based edge computing and communication algorithm is proposed to find the optimal solution. In particular, we design two different neural network structures and each communication link can converge to the optimal operation by exploiting only the local information from its neighbors, making it scalable to large networks. Finally, experimental results demonstrate the effectiveness of the proposed solution in various AAAN scenarios.

Zhe Wang↗
NASA NTRSDate not supplied

Joint Spectrum Access and Power Control in Air-Air Communications - A Deep Reinforcement Learning Based Approach

This paper considers the dynamic spectrum access and power control problem in a single-hop point-to-point Air-Air Communication Network (AACN). Due to spectrum scarcity, we assume the number of Aircraft-to-Aircraft (A2A) communication links is greater than that of the available channels, such that some communication links need to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the Weighted Sum Spectral Efficiency (WSSE). A distributed and dynamic deep Q learning-based algorithm is proposed to find the optimal solution. Specifically, we design two different policies that are trained by conducting a trial-and-error scheme. Each communication link can achieve the optimal policy by exploiting the local information from its neighbors, and this distributive approach make it scalable to large networks. Finally, our experimental results demonstrate the effectiveness of the proposed solution in various AACN scenarios.

Zhe Wang↗
NASA NTRSDate not supplied

Joint Spectrum Access and Power Control in Air-Air Communications - A Deep Reinforcement Learning Based Approach

This paper considers the dynamic spectrum access and power control problem in a single-hop point-to-point Air-Air Communication Network (AACN). Due to spectrum scarcity, we assume the number of Aircraft-to-Aircraft (A2A) communication links is greater than that of the available channels, such that some communication links need to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the Weighted Sum Spectral Efficiency (WSSE). A distributed and dynamic deep Q learning-based algorithm is proposed to find the optimal solution. Specifically, we design two different policies that are trained by conducting a trial-and-error scheme. Each communication link can achieve the optimal policy by exploiting the local information from its neighbors, and this distributive approach make it scalable to large networks. Finally, our experimental results demonstrate the effectiveness of the proposed solution in various AACN scenarios.

Zhe Wang↗
NASA NTRS1977

An improved digital algorithm for fast amplitude approximations of quadrature pairs

A computationally fast algorithm for approximating the amplitude (A) of a quadrature pair (I,Q) has been discovered. The piecewise linear formula, the unbiased estimate of A=X+1/8y when X or = 3Y and 7/8X + 1/2Y when X or = 3Y applies where X is congruent to the maximum absolute value of (I,Q) and Y is congruent to the minimum absolute value of (I,Q). This algorithm has been proven to be far more accurate then currently used approximations. An immediate application is the wideband digital spectrum analyzer under development for monitoring radio frequency interference at Deep Space Network stations.

Levitt, B. K.↗
NASA NTRS2007

NASA Tech Briefs, September 2007

Topics covered include; Rapid Fabrication of Carbide Matrix/Carbon Fiber Composites; Coating Thermoelectric Devices To Suppress Sublimation; Ultrahigh-Temperature Ceramics; Improved C/SiC Ceramic Composites Made Using PIP; Coating Carbon Fibers With Platinum; Two-Band, Low-Loss Microwave Window; MCM Polarimetric Radiometers for Planar Arrays; Aperture-Coupled Thin-Membrane L-Band Antenna; WGM-Based Photonic Local Oscillators and Modulators; Focal-Plane Arrays of Quantum-Dot Infrared Photodetectors; Laser Range and Bearing Finder With No Moving Parts; Microrectenna: A Terahertz Antenna and Rectifier on a Chip; Miniature L-Band Radar Transceiver; Robotic Vision-Based Localization in an Urban Environment; Programs for Testing an SSME-Monitoring System; Cathodoluminescent Source of Intense White Light; Displaying and Analyzing Antenna Radiation Patterns; Payload Operations Support Team Tools; Space-Shuttle Emulator Software; Soft Real-Time PID Control on a VME Computer; Analyzing Radio-Frequency Coverage for the ISS; Nanorod-Based Fast-Response Pressure-Sensitive Paints; Capacitors Would Help Protect Against Hypervelocity Impacts; Diaphragm Pump With Resonant Piezoelectric Drive; Improved Quick-Release Pin Mechanism; Designing Rolling-Element Bearings; Reverse-Tangent Injection in a Centrifugal Compressor; Inertial Measurements for Aero-assisted Navigation (IMAN); Analysis of Complex Valve and Feed Systems; Improved Path Planning Onboard the Mars Exploration Rovers; Robust, Flexible Motion Control for the Mars Explorer Rovers; Solar Sail Spaceflight Simulation; Fluorine-Based DRIE of Fused Silica; Mechanical Alloying for Making Thermoelectric Compounds; Process for High-Rate Fabrication of Alumina Nanotemplates; Electroform/Plasma-Spray Laminates for X-Ray Optics; An Automated Flying-Insect Detection System; Calligraphic Poling of Ferroelectric Material; Blackbody Cavity for Calibrations at 200 to 273 K; KML Super Overlay to WMS Translator; High-Performance Tiled WMS and KML Web Server; Modeling of Radiative Transfer in Protostellar Disks; Composite Pulse Tube; Photometric Calibration of Consumer Video Cameras; Criterion for Identifying Vortices in High- Pressure Flows; Amplified Thermionic Cooling Using Arrays of Nanowires; Delamination-Indicating Thermal Barrier Coatings; Preventing Raman Lasing in High-Q WGM Resonators; Procedures for Tuning a Multiresonator Photonic Filter; Robust Mapping of Incoherent Fiber-Optic Bundles; Extended-Range Ultrarefractive 1D Photonic Crystal Prisms; Rapid Analysis of Mass Distribution of Radiation Shielding; Modeling Magnetic Properties in EZTB; Deep Space Network Antenna Logic Controller; Modeling Carbon and Hydrocarbon Molecular Structures in EZTB; BigView Image Viewing on Tiled Displays; and Imaging Sensor Flight and Test Equipment Software.

Source record↗
NASA NTRS1983

Waveform inversion of mantle Love waves: The born seismogram approach

Normal mode theory, extended to the slightly laterally heterogeneous Earth by the first-order Born approximation, is applied to the waveform inversion of mantle Love waves (200-500 sec) for the Earth's lateral heterogeneity at l=2 and a spherically symmetric anelasticity (Q sub mu) structure. The data are from the Global Digital Seismograph Network (GDSN). The l=2 pattern is very similar to the results of other studies that used either different methods, such as phase velocity measurements and multiplet location measurements, or a different data set, such as mantle Rayleigh waves from different instruments. The results are carefully analyzed for variance reduction and are most naturally explained by heterogeneity in the upper 420 km. Because of the poor resolution of the data set for the deep interior, however, a fairly large heterogeneity in the transition zones, of the order of up to 3.5% in shear wave velocity, is allowed. It is noteworthy that Love waves of this period range can not constrain the structure below 420 km and thus any model presented by similar studies below this depth are likely to be constrained by Rayleigh waves (spheroidal modes) only.

Tanimoto, T.↗
NASA NTRS1984

Waveform inversion of mantle Love waves - The Born seismogram approach

Normal mode theory, extended to the slightly laterally heterogeneous earth by the first-order Born approximation, is applied to the waveform inversion of mantle Love waves (200-500 sec) for the earth's lateral heterogeneity at l = 2 and a spherically symmetric anelasticity (Q sub mu) structure. The data are from the Global Digital Seismograph Network (GDSN). The l = 2 pattern is very similar to the results of other studies that used either different methods, such as phase velocity measurements and multiplet location measurements, or a different data set, such as mantle Rayleigh waves from different instruments. The results are carefully analyzed for variance reduction and are most naturally explained by heterogeneity in the upper 420 km. Because of the poor resolution of the data set for the deep interior, however, a fairly large heterogeneity in the transition zones, of the order of up to 3.5 percent in shear wave velocity, is allowed. It is noteworthy that Love waves of this period range can not constrain the structure below 420 km and thus any model presented by similar studies below this depth are likely to be constrained by Rayleigh waves (spheroidal modes) only.

Tanimoto, T.↗
NASA NTRS1982

Noise temperature and noise figure concepts: DC to light

The Deep Space Network is investigating the use of higher operational frequencies for improved performance. Noise temperature and noise figure concepts are used to describe the noise performance of these receiving systems. It is proposed to modify present noise temperature definitions for linear amplifiers so they will be valid over the range (hf/kT) 1 (hf/kT). This is important for systems operating at high frequencies and low noise temperatures, or systems requiring very accurate calibrations. The suggested definitions are such that for an ideal amplifier, T sub e = (hg/k) = T sub q and F = 1. These definitions revert to the present definition for (hf/kT) 1. Noise temperature calibrations are illustrated with a detailed example. These concepts are applied to system signal-to-noise analysis. The fundamental limit to a receiving system sensitivity is determined by the thermal noise of the source and the quantum noise limit of the receiver. The sensitivity of a receiving system consisting of an ideal linear amplifier with a 2.7 K source, degrades significantly at higher frequencies.

Stelzried, C. T.↗
NASA NTRS2020

GROWTH on S190814bv: Deep Synoptic Limits on the Optical/Near-infrared Counterpart to a Neutron Star–Black Hole Merger

On 2019 August 14, the Advanced LIGO and Virgo interferometers detected the high-significance gravitational wave (GW) signal S190814bv. The GW data indicated that the event resulted from a neutron star–black hole (NSBH) merger, or potentially a low-mass binary BH merger. Due to the low false-alarm rate and the precise localization (23 deg^(2) at 90%), S190814bv presented the community with the best opportunity yet to directly observe an optical/near-infrared counterpart to an NSBH merger. To search for potential counterparts, the GROWTH Collaboration performed real-time image subtraction on six nights of public Dark Energy Camera images acquired in the 3 weeks following the merger, covering >98% of the localization probability. Using a worldwide network of follow-up facilities, we systematically undertook spectroscopy and imaging of optical counterpart candidates. Combining these data with a photometric redshift catalog, we ruled out each candidate as the counterpart to S190814bv and placed deep, uniform limits on the optical emission associated with S190814bv. For the nearest consistent GW distance, radiative transfer simulations of NSBH mergers constrain the ejecta mass of S190814bv to be M(ej) < 0.04 Mꙩ at polar viewing angles, or M(ej) < 0.03 Mꙩ if the opacity is κ < 2 sq.cm/g. Assuming a tidal deformability for the NS at the high end of the range compatible with GW170817 results, our limits would constrain the BH spin component aligned with the orbital momentum to be χ < 0.7 for mass ratios Q < 6, with weaker constraints for more compact NSs.

Igor Andreoni↗
NASA NTRS1998

Stability and Phase Noise Tests of Two Cryo-Cooled Sapphire Oscillators

A cryocooled Compensated Sapphire Oscillator (CSO), developed for the Cassini Ka-band Radio Science experiment, and operating in the 8K - 10K temperature range was previously demonstrated to show ultra-high stability of sigma(sub y) = 2.5 x 10 (exp -15) for measuring times 200 seconds less than or equal to tau less than or equal to 600 seconds using a hydrogen maser as reference. We present here test results for a second unit which allows CSO short-term stability and phase noise to be measured for the first time. Also included are design details of a new RF receiver and an intercomparison with the first CSO unit. Cryogenic oscillators operating below about 10K offer the highest possible short term stability of any frequency sources. However, their use has so far been restricted to research environments due to the limited operating periods associated with liquid helium consumption. The cryocooled CSO is being built in support of the Cassini Ka-band Radio Science experiment and is designed to operate continuously for periods of a year or more. Performance targets are a stability of 3-4 x 10 (exp -15) (1 second less than or equal to tau less than or equal to 100 seconds) and phase noise of -73dB/Hz @ 1Hz measured at 34 GHz. Installation in 5 stations of NASA's deep space network (DSN) is planned in the years 2000 - 2002. In the previous tests, actual stability of the CSO for measuring times tau less than or equal to 200 seconds could not be directly measured, being masked by short-term fluctuations of the H-maser reference. Excellent short-term performance, however, could be inferred by the success of an application of the CSO as local oscillator (L.O.) to the JPL LITS passive atomic standard, where medium-term stability showed no degradation due to L.O. instabilities at a level of (sigma)y = 3 x 10 (exp -14)/square root of tau. A second CSO has now been constructed, and all cryogenic aspects have been verified, including a resonator turn-over temperature of 7.907 K, and Q of 7.4 x 10 (exp 8). These values compare to a turn-over of 8.821 K and Q of 1.0 x 10 (exp 9) for the first resonator. Operation of this second unit provides a capability to directly verify for the first time the short-term (1 second less than or equal to tau less than or equal to 200 seconds) stability and the phase noise of the CSO units. The RF receiver used in earlier tests was sufficient to meet Cassini requirements for tau greater than or equal to 10 seconds but had short-term stability limited to 2-4 x 10 (exp -14) at tau = 1 second, a value 10 times too high to meet our requirements. A new low-noise receiver has been designed to provide approximately equal to 10-15 performance at 1 second, and one receiver is now operational, demonstrating again short-term CSO performance with H maser-limited stability. Short-term performance was degraded in the old receiver due to insufficient tuning bandwidth in a 100MHZ quartz VCO that was frequency-locked to the cryogenic sapphire resonator. The new receivers are designed for sufficient bandwidth, loop gain and low noise to achieve the required performance.

Dick, G. John↗