Search NASA⌕ Search

SEARCH · Search NASA

Results for “Network Layer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Deep Neural Network Algorithm for CMC Microstructure Characterization and Variability Quantification

Microstructure characterization and variability quantification are crucial for understanding ceramic matrix composites (CMCs) mechanical behavior and deformation mechanisms across length scales. Traditionally, analyses of the micrographs obtained from microscopy are labor-intensive. However, with the vast improvement in computer vision (CV) and deep learning (DL), an automated algorithm can be designed to extract essential microstructure variability from micrographs which can then be used to construct a statistically representative volume element (SRVE). The DL-based algorithm spans the taxonomy of microstructure analyses, including semantic segmentation of microstructure constituents, secondary phases, matrix/fiber interface, and defects, and quantifying the microstructure variability in terms of probability distributions. In this work, C/SiNC and SiC/SiNC CMCs microstructures are semantically segmented through a deep convolutional neural network, followed by variability quantification through the implementation of a fully connected regression layer, hence forming a deep regression network. The deep regression network operates in a feedforward regime, in which the neuron output signal traverses through the network in a unidirectional manner. The weight tensor associated with each layer is updated through a backpropagation stochastic gradient descent approach. The input gray-scale image obtained through in-house scanning electron microscope and confocal microscope micrographs is augmented through affine transformations to increase the training set size, which is then processed through four strided convolutional layers. This compresses the image resolution by half at each layer while increasing the image depth by applying different filters (image encoding). The class activation maps (CAMs) corresponding to the applied filters highlight the key architectural features and assist with the semantic segmentation of the microstructure.

Hamza, Mohamed H.↗
DOE OSTI2021

Freely scalable and reconfigurable optical hardware for deep learning

Abstract As deep neural network (DNN) models grow ever-larger, they can achieve higher accuracy and solve more complex problems. This trend has been enabled by an increase in available compute power; however, efforts to continue to scale electronic processors are impeded by the costs of communication, thermal management, power delivery and clocking. To improve scalability, we propose a digital optical neural network (DONN) with intralayer optical interconnects and reconfigurable input values. The path-length-independence of optical energy consumption enables information locality between a transmitter and a large number of arbitrarily arranged receivers, which allows greater flexibility in architecture design to circumvent scaling limitations. In a proof-of-concept experiment, we demonstrate optical multicast in the classification of 500 MNIST images with a 3-layer, fully-connected network. We also analyze the energy consumption of the DONN and find that digital optical data transfer is beneficial over electronics when the spacing of computational units is on the order of $$>10\,\upmu $$ > 10 μ m.

42 ENGINEERING↗
DOE OSTI2024

ReLU, Sparseness, and the Encoding of Optic Flow in Neural Networks

Accurate self-motion estimation is critical for various navigational tasks in mobile robotics. Optic flow provides a means to estimate self-motion using a camera sensor and is particularly valuable in GPS- and radio-denied environments. The present study investigates the influence of different activation functions—ReLU, leaky ReLU, GELU, and Mish—on the accuracy, robustness, and encoding properties of convolutional neural networks (CNNs) and multi-layer perceptrons (MLPs) trained to estimate self-motion from optic flow. Our results demonstrate that networks with ReLU and leaky ReLU activation functions not only achieved superior accuracy in self-motion estimation from novel optic flow patterns but also exhibited greater robustness under challenging conditions. The advantages offered by ReLU and leaky ReLU may stem from their ability to induce sparser representations than GELU and Mish do. Our work characterizes the encoding of optic flow in neural networks and highlights how the sparseness induced by ReLU may enhance robust and accurate self-motion estimation from optic flow.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Neural Network‐Based Methods for Ocean Surface Wave Measurement Using Submarine Distributed Acoustic Sensing (DAS)

Two new data-driven models for estimating ocean surface waves from distributed acoustic sensing (DAS) submarine cable strain rate are developed using supervised machine learning on a 10-day data set collected offshore of Oliktok Point, Alaska. The new models were trained on target data from seafloor pressure moorings at three sites spaced evenly along 27.1 km of cable and were benchmarked against an empirical transfer function method previously used to estimate waves from DAS. A model which uses convolutional neural networks to transform 2-km frequency-wavenumber strain spectra to seafloor pressure spectra outperforms the benchmark in wave height prediction (RMSE of 0.15 vs. 0.41 m) and period prediction (0.29 vs. 0.37 s) when evaluated on a held-out test data set. When applied to a DAS data set collected on the same cable 2 years prior, the CNN-based model maintained similar significant wave height performance (RMSE = 0.23 m) relative to available satellite altimetry data. A two-hidden-layer, fully connected neural network which transforms 1-D strain spectra to seafloor pressure spectra also outperforms the benchmark in wave height prediction (RMSE of 0.19 vs. 0.41 m), but does not generalize as well to the prior data. Regression-based machine learning is useful for estimating waves from DAS data when the pressure-strain relationship varies temporally and spatially across different wave conditions. Models can be applied to DAS data to measure waves with higher spatial resolution and longer temporal coverage than traditional methods, which often measure waves only at a single point.

Davis, Jacob R. [Univ. of Washington, Seattle, WA ↗
DOE OSTI2021

Validation of the Sentinel-5 Precursor TROPOMI cloud data with Cloudnet, Aura OMI O 2 –O 2 , MODIS, and Suomi-NPP VIIRS

Accurate knowledge of cloud properties is essential to the measurement of atmospheric composition from space. In this work we assess the quality of the cloud data from three Copernicus Sentinel-5 Precursor (S5P) TROPOMI cloud products: (i) S5P OCRA/ROCINN_CAL (Optical Cloud Recognition Algorithm/Retrieval of Cloud Information using Neural Networks;Clouds-As-Layers), (ii) S5P OCRA/ROCINN_CRB (Clouds-as-Reflecting Boundaries), and (iii) S5P FRESCO-S (Fast Retrieval Scheme for Clouds from Oxygen absorption bands – Sentinel). Target properties of this work are cloud-top height and cloud optical thickness (OCRA/ROCINN_CAL), cloud height (OCRA/ROCINN_CRB and FRESCO-S), and radiometric cloud fraction (all three algorithms). The analysis combines (i) the examination of cloud maps for artificial geographical patterns, (ii) the comparison to other satellite cloud data (MODIS, NPP-VIIRS, and OMI O2–O2), and (iii) ground-based validation with respect to correlative observations (30 April 2018 to 27 February 2020) from the Cloudnet network of ceilometers, lidars, and radars. Zonal mean latitudinal variation of S5P cloud properties is similar to that of other satellite data. S5P OCRA/ROCINN_CAL agrees well with NPP VIIRS cloud-top height and cloud optical thickness and with Cloudnet cloud-top height, especially for the low (mostly liquid) clouds. For the high clouds, S5P OCRA/ROCINN_CAL cloud-top height is below the cloud-top height of VIIRS and of Cloudnet, while its cloud optical thickness is higher than that of VIIRS. S5P OCRA/ROCINN_CRB and S5P FRESCO cloud height are well below the Cloudnet cloud mean height for the low clouds but match on average better with the Cloudnet cloud mean height for the higher clouds. As opposed to S5P OCRA/ROCINN_CRB and S5P FRESCO, S5P OCRA/ROCINN_CAL is well able to match the lowest CTH mode of the Cloudnet observations. Peculiar geographical patterns are identified in the cloud products and will be mitigated in future releases of the cloud data products.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

A Classification of G -invariant Shallow Neural Networks

When trying to fit a deep neural network (DNN) to a G-invariant target function with G a group, it only makes sense to constrain the DNN to be G-invariant as well. However, there can be many different ways to do this, thus raising the problem of “G-invariant neural architecture design”: What is the optimal Ginvariant architecture for a given problem? Before we can consider the optimization problem itself, we must understand the search space, the architectures in it, and how they relate to one another. In this paper, we take a first step towards this goal; we prove a theorem that gives a classification of all G-invariant single-hidden-layer or “shallow” neural network (G-SNN) architectures with ReLU activation for any finite orthogonal group G, and we prove a second theorem that characterizes the inclusion maps or “network morphisms” between the architectures that can be leveraged during neural architecture search (NAS). The proof is based on a correspondence of every G-SNN to a signed permutation representation of G acting on the hidden neurons; the classification is equivalently given in terms of the first cohomology classes of G, thus admitting a topological interpretation. The G-SNN architectures corresponding to nontrivial cohomology classes have, to our knowledge, never been explicitly identified in the literature previously. Using a code implementation, we enumerate the G-SNN architectures for some example groups G and visualize their structure. Lastly, we prove that architectures corresponding to inequivalent cohomology classes coincide in function space only when their weight matrices are zero, and we discuss the implications of this for NAS.

Agrawal, Devanshu↗
DOE OSTI2022

Variational encoder geostatistical analysis (VEGAS) with an application to large scale riverine bathymetry

Estimation of riverbed profiles, also known as bathymetry, plays a vital role in many applications, such as safe and efficient inland navigation, prediction of bank erosion, land subsidence, and flood risk management. The high cost and complex logistics of direct bathymetry surveys, i.e, depth imaging, have encouraged the use of indirect measurements such as surface flow velocities. However, estimating high-resolution bathymetry from indirect measurements is an inverse problem that can be computationally challenging. Here, we propose a reduced-order model (ROM) based approach that utilizes a variational autoencoder (VAE), a type of deep neural network with a narrow layer in the middle, to compress bathymetry and flow velocity information and accelerate bathymetry inverse problems from flow velocity measurements. In our application, the shallow-water equations (SWE) with appropriate boundary conditions (BCs), e.g., the discharge and/or the free surface elevation, constitute the forward problem, to predict flow velocity. Then, ROMs of the SWEs are constructed on a nonlinear manifold of low dimensionality through a variational encoder and the bathymetry inversion problem is derived on the low-dimensional latent space in a Hierarchical Bayesian setting. Further, the reformulation allows variational inference with a small number (e.g., $\mathscr{O}$ (100) of ROM runs and efficient uncertainty quantification. We have tested our inversion approach on a one-mile reach of the Savannah River, GA, USA. Once the neural network is trained (offline stage), the proposed technique can perform the inversion operation orders of magnitude faster than traditional inversion methods that are commonly based on linear projections, such as principal component analysis (PCA), or the principal component geostatistical approach (PCGA). Furthermore, tests show that the algorithm can estimate the bathymetry with good accuracy even with sparse flow velocity measurements.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Evaluation of data driven low-rank matrix factorization for accelerated solutions of the Vlasov equation

Low-rank methods have shown success in accelerating simulations of a collisionless plasma described by the Vlasov equation, but still rely on computationally costly linear algebra every time step. We propose a data-driven factorization method using artificial neural networks, specifically with convolutional layer architecture, that trains on existing simulation data. At inference time, the model outputs a low-rank decomposition of the distribution field of the charged particles, and we demonstrate that this step is faster than the standard linear algebra technique. Numerical experiments show that the method achieves comparable reconstruction accuracy for interpolation tasks, generalizing to unseen test data in a manner beyond just memorizing training data; patterns in factorization also inherently followed the same numerical trend as those within algebraic methods (e.g., truncated singular-value decomposition). However, when training on the first 70% of a time-series data and testing on the remaining 30%, the method fails to meaningfully extrapolate. Despite this limiting result, the technique may have benefits for simulations in a statistical steady-state or otherwise showing temporal stability. These results suggest that while the model offers a computationally efficient alternative for datasets with temporal stability, its current formulation is best suited for interpolation rather than for predicting future states in time-evolving systems. This study thus lays the groundwork for further refinement of neural network-based approaches to low-rank matrix factorization in high-dimensional plasma simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Illinois Express Quantum Network (IEQNET): metropolitan-scale experimental quantum networking over deployed optical fiber

The Illinois Express Quantum Network (IEQNET) is a program to realize metro-scale quantum networking over deployed optical fiber using currently available technology. IEQNET consists of multiple sites that are geographically dispersed in the Chicago metropolitan area. Each site has one or more quantum nodes (Qnodes) representing the communication parties in a quantum network. Q-nodes generate or measure quantum signals such as entangled photons and communicate the results via standard, classical, means. The entangled photons in IEQNET nodes are generated at multiple wavelengths, and are selectively distributed to the desired users via optical switches. Here we describe the network architecture of IEQNET, including the Internet-inspired layered hierarchy that leverages software-defined-networking (SDN) technology to perform traditional wavelength routing and assignment between the Q-nodes. Specifically, SDN decouples the control and data planes, with the control plane being entirely classical. Issues associated with synchronization, calibration, network monitoring, and scheduling will be discussed. An important goal of IEQNET is demonstrating the extent to which the control plane can coexist with the data plane using the same fiber lines. This goal is furthered by the use of tunable narrow-band optical filtering at the receivers and, at least in some cases, a wide wavelength separation between the quantum and classical channels. We envision IEQNET to aid in developing robust and practical quantum networks by demonstrating metro-scale quantum communication tasks such as entanglement distribution and quantum-state teleportation.

Chung, Joaquin↗
DOE OSTI2022

Implementation Aspects of Smart Grids Cyber-Security Cross-Layered Framework for Critical Infrastructure Operation

Communication networks in power systems are a major part of the smart grid paradigm. It enables and facilitates the automation of power grid operation as well as self-healing in contingencies. Such dependencies on communication networks, though, create a roam for cyber-threats. An adversary can launch an attack on the communication network, which in turn reflects on power grid operation. Attacks could be in the form of false data injection into system measurements, flooding the communication channels with unnecessary data, or intercepting messages. Using machine learning-based processing on data gathered from communication networks and the power grid is a promising solution for detecting cyber threats. In this paper, a co-simulation of cyber-security for cross-layer strategy is presented. The advantage of such a framework is the augmentation of valuable data that enhances the detection as well as identification of anomalies in the operation of the power grid. The framework is implemented on the IEEE 118-bus system. The system is constructed in Mininet to simulate a communication network and obtain data for analysis. A distributed three controller software-defined networking (SDN) framework is proposed that utilizes the Open Network Operating System (ONOS) cluster. According to the findings of our suggested architecture, it outperforms a single SDN controller framework by a factor of more than ten times the throughput. This provides for a higher flow of data throughout the network while decreasing congestion caused by a single controller’s processing restrictions. Furthermore, our CECD-AS approach outperforms state-of-the-art physics and machine learning-based techniques in terms of attack classification. The performance of the framework is investigated under various types of communication attacks.

cross-layered↗
DOE OSTI2021

Low-Power Deep Learning Inference using the SpiNNaker Neuromorphic Platform

n this presentation we will discuss recent results on using the SpiNNaker neuromorphic platform (48-chip model) for deep learning neural network inference. We use the Sandia Labs developed Whet stone spiking deep learning library to train deep multi-layer perceptrons and convolutional neural networks suitable for the spiking substrate on the neural hardware architecture. By using the massively parallel nature of SpiNNaker, we are able to achieve, under certain network topologies, substantial network tiling and consequentially impressive inference throughput. Such high-throughput systems may have eventual application in remote sensing applications where large images need to be chipped, scanned, and processed quickly. Additionally, we explore complex topologies that push the limits of the SpiNNaker routing hardware and investigate how that impacts mapping software-implemented networks to on-hardware instantiations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Probing for Artifacts: Detecting Imagenet Model Evasions

While deep learning models have made incredible progress across a variety of machine learning tasks, they remain vulnerable to adversarial examples crafted to fool otherwise trustworthy models. In this work we approach this problem through the lens of a detection framework. We propose a classification network that uses the hidden layer activations of a trained model as inputs to detect adversarial artifacts in an input. We train this classification network simultaneously against multiple adversarial algorithms to create a more robust detector and show higher detection rates than several alternatives. The novelty of our approach is in the scale and scope of probing Imagenet models for adversarial artifacts. In addition, we propose an improvement to feature squeezing, another common adversarial example detection method.

Rounds, Jeremiah↗
DOE OSTI2022

DeepZipper: A Novel Deep-learning Architecture for Lensed Supernovae Identification

Large-scale astronomical surveys have the potential to capture data on large numbers of strongly gravitationally lensed supernovae (LSNe). To facilitate timely analysis and spectroscopic follow-up before the supernova fades, an LSN needs to be identified soon after it begins. To quickly identify LSNe in optical survey data sets, we designed ZipperNet, a multibranch deep neural network that combines convolutional layers (traditionally used for images) with long short-term memory layers (traditionally used for time series). We tested ZipperNet on the task of classifying objects from four categories—no lens, galaxy-galaxy lens, lensed Type-Ia supernova, lensed core-collapse supernova—within high-fidelity simulations of three cosmic survey data sets: the Dark Energy Survey, Rubin Observatory’s Legacy Survey of Space and Time (LSST), and a Dark Energy Spectroscopic Instrument (DESI) imaging survey. Among our results, we find that for the LSST-like data set, ZipperNet classifies LSNe with a receiver operating characteristic area under the curve of 0.97, predicts the spectroscopic type of the lensed supernovae with 79% accuracy, and demonstrates similarly high performance for LSNe 1–2 epochs after first detection. We anticipate that a model like ZipperNet, which simultaneously incorporates spatial and temporal information, can play a significant role in the rapid identification of lensed transient systems in cosmic survey experiments.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Tempering kinetics during multilayer laser additive manufacturing of a ferritic steel

Grade 91 steel forms martensite during additive manufacturing and the extent of tempering of martensite significantly affects the mechanical properties of parts. Currently, there is a lack of quantitative understanding of the tempering kinetics for Grade 91 steel, and as a result, the effects of repeated thermal cycles on properties for different processing conditions cannot be determined. Here we evaluate the tempering kinetics by determining the constant terms in the Johnson Mehl Avrami kinetic equation from the tempering data available in the literature and the thermal cycles computed using a rigorously-tested heat and fluid flow model of multi-layer additive manufacturing. The raw tempering data are cleaned using a neural network to enhance accuracy. The lower layers experience repeating cycles of heating and cooling when the upper layers are added. As a result, the hardness is reduced owing to the tempering of martensite. In contrast, martensite formed in the upper layers is not tempered to the same extent and the hardness remains high. Therefore, the hardness of the part increases with the distance from the substrate. Variations in the heat input at different laser powers and scanning speeds significantly affect the extent of tempering. Finally, since the method used here can provide a quantitative understanding of the tempering of martensite and the spatial variation in hardness, it can be used to tailor the microstructure and hardness of heat treatable printed metallic parts.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Regulating the Third Metal to Design and Engineer Multilayered NiFeM (M: Co, Mn, and Cu) Nanofoam Anode Catalysts for Anion-Exchange Membrane Water Electrolyzers

In this study, alkaline anion-exchange membrane water electrolyzers (AEMWEs) for green hydrogen production have received intensive attention due to their feasibility of using earth-abundant platinum group metal (PGM)-free catalysts. Herein, the third metal is incorporated into NiFe-based catalysts to regulate their electronic structures and morphologies, aiming to achieve sufficient oxygen evolution reaction (OER) activity and performance in AEMWEs. The ternary NiFeM (M: Cu, Co, or Mn) catalysts are featured with multiple layered structures and nanofoam network morphologies, consisting of highly OER-active amorphous Ni-rich oxide shells and electrically conductive metallic alloy cores. The physical and electronic perturbations to the NiFe induced by a third element lead to a fine-tuning of the redox ability of the metal sites at the reaction centers, which breaks the scaling relationship between OH* and O* intermediates at the reaction centers. Thus, the unique structural configuration and electronic regulation simultaneously benefit catalytic activity and performance improvements. These NiFeM nanofoam catalysts demonstrated promising anode performance in actual AEMWEs, comparable to the IrO 2 reference, especially at high current densities. Notably, using various electrolytes (e.g., KOH solution or pure water) for AEMWEs exhibited a different performance trend among studied NiFeM catalysts, likely due to dynamic changes of catalysts under various OER environments.

25 ENERGY STORAGE↗
DOE OSTI2024

Hydrogen embrittlement of Zircaloy-4 fabricated by ultrasonic additive manufacturing

Ultrasonic additive manufacturing (UAM) was successfully applied to the zirconium material system to create a planar geometry. Following fabrication, SS-J3 type tensile specimens of the UAM and wrought Zircaloy-4 with a nominal gage section of 5×1.2×0.75 mm were machined for hydriding studies and mechanical testing. The SS-J3 type tensile specimens were gas-charged with hydrogen using a custom system that precisely controls hydrogen gas flow rate, hydrogen partial pressure, and temperature. To avoid altering the UAM material, the maximum process temperature was limited to 550°C. Using different initial hydrogen gas pressures and flow rates, various hydrogen contents (70–1755 wppm) were achieved for Zircaloy-4 specimens. Tensile testing shows that, regardless of hydrogen content, all UAM specimens measured yield strengths in the range of 557±16 MPa and ultimate tensile strengths of 660±4MPa. However, total elongation clearly decreased as a function of increasing hydrogen content. Further, at the lowest hydrogen content, the total plastic elongation measured 21.5%, and this value decreased to less than 2% when the hydrogen content was increased to 1000 wppm. Cross-sectional optical microscopy images revealed that the hydride distributions are randomly oriented. For the low hydrogen content specimen, these randomly distributed hydrides are isolated from each other. A sandwiched structure, consisting of high-density and low-density layers, was also observed for the specimens with hydrogen content between 200 and 400 wppm. As the hydrogen content increases, the hydrides diffuse into the low-density hydride layers to form a network across the whole specimen. Although the tensile properties of UAM and wrought Zircaloy-4 exhibit the same behavior with increasing hydrogen content, the distinctions in grain orientations led to differences in the hydride orientations at low and intermediate hydrogen concentrations.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Hybrid Machine Learning for Scanning Near-Field Optical Spectroscopy

The underlying physics behind an experimental observation often lacks a simple analytical description. This is especially the case for scanning probe microscopy techniques, where the interaction between the probe and the sample is nontrivial. Realistic modeling to include the details of the probe is always exponentially more difficult than its "spherical cow" counterparts. On the other hand, a well-trained artificial neural network based on real data can grasp the hidden correlation between the signal and sample properties. In this work, we show that, via a combination of model calculation and experimental data acquisition, a physics-infused hybrid neural network can predict the tip-sample interaction in the widely used scattering-type scanning near-field optical microscope. This hybrid network provides a long-sought solution for accurate extraction of material properties from tip-specific raw data. The methodology can be extended to other scanning probe microscopy techniques as well as other data-oriented physical problems in general.

36 MATERIALS SCIENCE↗
DOE OSTI2026

From Reproducible Edge–Cloud Experimentation to Real-World Practice: The E2Clab Experience

Reproducibility is already difficult in distributed systems; on the computing continuum, it becomes substantially harder. Applications that span sensing devices, edge and fog resources, and cloud platforms must be evaluated across heterogeneous hardware, variable network conditions, cross-layer orchestration decisions, and long-running workflow lifecycles. We use E2Clab as a case study to examine these challenges and their implications for experimental methodology. We explain why reproducible experimentation is harder on the continuum, then revisit E2Clab as an initial response based on explicit modeling of infrastructure, workflow lifecycle, and artifacts. Lastly, we discuss how its evolution toward more realistic application settings can be understood through the lens of Translational Computer Science. We argue that reproducible continuum experimentation requires methods that are rigorous enough for research while remaining adaptable to real-world practice.

42 ENGINEERING↗