Search NASA⌕ Search

SEARCH · Search NASA

Results for “network architecture”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Preprocessing for Unintended Conducted Emissions Classification with ResNet

Characterization of Unintended Conducted Emissions (UCE) from electronic devices is important when diagnosing electromagnetic interference, performing nonintrusive load monitoring (NILM) of power systems, and monitoring electronic device health, among other applications. Prior work has demonstrated that UCE analysis can serve as a diagnostic tool for energy efficiency investigations and detailed load analysis. While explaining the feature selection of deep networks with certainty is often not fully comprehensive, or in other applications, quite lacking, additional tools/methods for further corroboration and confirmation can help further the understanding of the researcher. This is true especially in the subject application of the study in this paper. Often the focus of such efforts is the selected features themselves, and there is not as much understanding gained about the noise in the collected data. If selected feature and noise characteristics are known, it can be used to further shape the design of the deep network or associated preprocessing. This is additionally difficult when the available data are limited, as in the case which the authors investigated in this study. Here, the authors present a novel work (which is a proposed complementary portion of the overall solution to the deep network classification explainability problem for this application) by applying a systematic progression of preprocessing and a deep neural network (ResNet architecture) to classify UCE data obtained via current transformers. By using a methodical application of preprocessing techniques prior to a deep classifier, hypotheses can be produced concerning what features the deep network deems important relative to what it perceives as noise. For instance, it is hypothesized in this particular study as a result of execution of the proposed method and periodic inspection of the classifier output that the UCE spectral features are relatively close to each other or to the interferers, as systematically reducing the beta parameter of the Kaiser window produced progressively better classification performance, but only to a point, as going below the Beta of eight produced decreased classifier performance, as well as the hypothesis that further spectral feature resolution was not as important to the classifier as rejection of the leakage from a spectrally distant interference. This can be very important in unpredictable low-FNR applications, where knowing the difference between features and noise is difficult. As a side-benefit, much was learned regarding the best preprocessing to use with the selected deep network for the UCE collected from these low power consumer devices obtained via current transformers. Baseline rectangular windowed FFT preprocessing provided a 62% classification increase versus using raw samples. After performing a more optimal preprocessing, more than 90% classification accuracy was achieved across 18 low-power consumer devices for scenarios in which the in-band features-to-noise ratio (FNR) was very poor.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Densely Connected G-invariant Deep Neural Networks with Signed Permutation Representations

We introduce and investigate, for finite groups G, G-invariant deep neural network (GDNN) architectures with ReLU activation that are densely connected- i.e., include all possible skip connections. In contrast to other G-invariant architectures in the literature, the preactivations of theG-DNNs presented here are able to transform by signed permutation representations (signed perm-reps) of G. Moreover, the individual layers of the G-DNNs are not required to be G-equivariant; instead, the preactivations are constrained to be G-equivariant functions of the network input in a way that couples weights across all layers. The result is a richer family of G-invariant architectures never seen previously. We derive an efficient implementation of G-DNNs after a reparameterization of weights, as well as necessary and sufficient conditions for an architecture to be "admissible"- i.e., nondegenerate and inequivalent to smaller architectures. We include code that allows a user to build a G-DNN interactively layer-by-layer, with the final architecture guaranteed to be admissible. We show that there are far more admissible G-DNN architectures than those accessible with the "concatenated ReLU" activation function from the literature. Finally, we apply G-DNNs to two example problems--(1) multiplication in --1, 1} (with theoretical guarantees) and (2) 3D object classification--finding that the inclusion of signed perm-reps significantly boosts predictive performance compared to baselines with only ordinary (i.e., unsigned) perm-reps.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Decentralized Filtering Adaptive Neural Network Control for Uncertain Switched Interconnected Nonlinear Systems

This article presents a novel decentralized filtering adaptive neural network control framework for uncertain switched interconnected nonlinear systems. Each subsystem has its own decentralized controller based on the established decentralized state predictor. For each subsystem, the nonlinear uncertainties are approximated by a Gaussian radial basis function (GRBF) neural network incorporated with a piecewise constant adaptive law, where the adaptive law will update adaptive parameters from the error dynamics between the host system and the decentralized state predictor by discarding the unknowns, whereas a decentralized filtering control law is derived to cancel both local and mismatched uncertainties from other subsystems, as well as achieve the local objective tracking of the host system. The achievement of global objective depends on the achievement of local objective for each subsystem. The matched uncertainties are canceled directly by adopting their opposite in the control signal, whereas a dynamic inversion of the system is required to eliminate the effect of the mismatched uncertainties on the output. By exploiting the average dwell time principle, the error bounds between the real system and the virtual reference system, which defines the best performance that can be achieved by the closed-loop system, are derived. A numerical example is given to illustrate the effectiveness of the decentralized filtering adaptive neural network control architecture by comparing against the model reference adaptive control (MRAC).

Average dwell time, decentralized, filtering adapt↗
DOE OSTI2024

Graph neural networks for CO 2 solubility predictions in Deep Eutectic Solvents

Deep Eutectic Solvents (DESs) are a promising class of solvents for CO 2 capture. DESs are complex mixtures that can be designed to optimize CO solubility and overall capture process efficiency. However, the vast design landscape of DES mixtures makes experimental investigation prohibitive; as such, there is a need for computational models that can quickly and efficiently navigate the design space and inform data collection efforts. In this work, we propose Graph Neural Network (GNN) models for predicting CO 2 solubility for DESs; the GNN leverages a mixture graph representation that captures the molecular structure of the DES components as well as their intermolecular interactions. Here, we compare the GNN framework against alternative architectures (neural networks, graph convolution networks, and random forests) and data representations (molecular fingerprints, sigma profiles, and graphs). We show that the proposed approach offers superior predictive performance; specifically, we show that solubility can be predicted reliably directly from molecular structure (without the need of using sigma profiles as proposed in previous studies). This result is important, as obtaining sigma profiles requires expensive density functional theory computations. We also explored the ability of GNNs to predict solubility for new DES mixtures and operating conditions. We found that the model extrapolates across temperature reliably. However, we also found deficiencies in the ability of the model to predict solubility for DES mixtures, pressures, and molar ratio not included in the training sets; we show that this is due to an inherent lack of chemical diversity in datasets available in the literature. The proposed computational capabilities can thus help navigate the design space of DES and inform data collection efforts. Our models, data, and benchmarks are shared as Python code implemented in Jupyter notebooks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Implementing Spatio-Temporal 3D-Convolution Neural Networks and UAV Time Series Imagery to Better Predict Lodging Damage in Sorghum

Unmanned aerial vehicle (UAV)-based remote sensing is gaining momentum in a variety of agricultural and environmental applications. Very-high-resolution remote sensing image sets collected repeatedly throughout a crop growing season are becoming increasingly common. Analytical methods able to learn from both spatial and time dimensions of the data may allow for an improved estimation of crop traits, as well as the effects of genetics and the environment on these traits. Multispectral and geometric time series imagery was collected by UAV on 11 dates, along with ground-truth data, in a field trial of 866 genetically diverse biomass sorghum accessions. We compared the performance of Convolution Neural Network (CNN) architectures that used image data from single dates (two spatial dimensions, 2D) versus multiple dates (two spatial dimensions + temporal dimension, 3D) to estimate lodging detection and severity. Lodging was detected with 3D-CNN analysis of time series imagery with 0.88 accuracy, 0.92 Precision, and 0.83 Recall. This outperformed the best 2D-CNN on a single date with 0.85 accuracy, 0.84 Precision, and 0.76 Recall. The variation in lodging severity was estimated by the best 3D-CNN analysis with 9.4% mean absolute error (MAE), 11.9% root mean square error (RMSE), and goodness-of-fit (R2) of 0.76. This was a significant improvement over the best 2D-CNN analysis with 11.84% MAE, 14.91% RMSE, and 0.63 R2. The success of the improved 3D-CNN analysis approach depended on the inclusion of “before and after” data, i.e., images collected on dates before and after the lodging event. The integration of geometric and spectral features with 3D-CNN architecture was also key to the improved assessment of lodging severity, which is an important and difficult-to-assess phenomenon in bioenergy feedstocks such as biomass sorghum. This demonstrates that spatio-temporal CNN architectures based on UAV time series imagery have significant potential to enhance plant phenotyping capabilities in crop breeding and Precision agriculture applications.

3D-convolution neural networks↗
DOE OSTI2021

INTEGRATE - Inverse Network Transformations for Efficient Generation of Robust Airfoil and Turbine Enhancements

The INTEGRATE (Inverse Network Transformations for Efficient Generation of Robust Airfoil and Turbine Enhancements) project is developing a new inverse-design capability for the aerodynamic design of wind turbine rotors using invertible neural networks. This AI-based design technology can capture complex non-linear aerodynamic effects while being 100 times faster than design approaches based on computational fluid dynamics. This project enables innovation in wind turbine design by accelerating time to market through higher-accuracy early design iterations to reduce the levelized cost of energy. INVERTIBLE NEURAL NETWORKS Researchers are leveraging a specialized invertible neural network (INN) architecture along with the novel dimension-reduction methods and airfoil/blade shape representations developed by collaborators at the National Institute of Standards and Technology (NIST) learns complex relationships between airfoil or blade shapes and their associated aerodynamic and structural properties. This INN architecture will accelerate designs by providing a cost-effective alternative to current industrial aerodynamic design processes, including: - Blade element momentum (BEM) theory models: limited effectiveness for design of offshore rotors with large, flexible blades where nonlinear aerodynamic effects dominate - Direct design using computational fluid dynamics (CFD): cost-prohibitive - Inverse-design models based on deep neural networks (DNNs): attractive alternative to CFD for 2D design problems, but quickly overwhelmed by the increased number of design variables in 3D problems AUTOMATED COMPUTATIONAL FLUID DYNAMICS FOR TRAINING DATA GENERATION - MERCURY FRAMEWORK The INN is trained on data obtained using the University of Marylands (UMD) Mercury Framework, which has with robust automated mesh generation capabilities and advanced turbulence and transition models validated for wind energy applications. Mercury is a multi-mesh paradigm, heterogeneous CPU-GPU framework. The framework incorporates three flow solvers at UMD, 1) OverTURNS, a structured solver on CPUs, 2) HAMSTR, a line based unstructured solver on CPUs, and 3) GARFIELD, a structured solver on GPUs. The framework is based on Python, that is often used to wrap C or Fortran codes for interoperability with other solvers. Communication between multiple solvers is accomplished with a Topology Independent Overset Grid Assembler (TIOGA). NOVEL AIRFOIL SHAPE REPRESENTATIONS USING GRASSMAN SPACES We developed a novel representation of shapes which decouples affine-style deformations from a rich set of data-driven deformations over a submanifold of the Grassmannian. The Grassmannian representation as an analytic generative model, informed by a database of physically relevant airfoils, offers (i) a rich set of novel 2D airfoil deformations not previously captured in the data , (ii) improved low-dimensional parameter domain for inferential statistics informing design/manufacturing, and (iii) consistent 3D blade representation and perturbation over a sequence of nominal shapes. TECHNOLOGY TRANSFER DEMONSTRATION - COUPLING WITH NREL WISDEM Researchers have integrated the inverse-design tool for 2D airfoils (INN-Airfoil) into WISDEM (Wind Plant Integrated Systems Design and Engineering Model), a multidisciplinary design and optimization framework for assessing the cost of energy, as part of tech-transfer demonstration. The integration of INN-Airfoil into WISDEM allows for the design of airfoils along with the blades that meet the dynamic design constraints on cost of energy, annual energy production, and the capital costs. Through preliminary studies, researchers have shown that the coupled INN-Airfoil + WISDEM approach reduces the cost of energy by around 1% compared to the conventional design approach. This page will serve as a place to easily access all the publications from this work and the repositories for the software developed and released through this pr...

aerodynamics↗
DOE OSTI2021

Deep Cellular Recurrent Network for Efficient Analysis of Time-Series Data With Spatial Information

Efficient processing of large-scale time series data is an intricate problem in machine learning. Conventional sensor signal processing pipelines with hand engineered feature extraction often involve huge computational cost with high dimensional data. Deep recurrent neural networks have shown promise in automated feature learning for improved time-series processing. However, generic deep recurrent models grow in scale and depth with increased complexity of the data. This is particularly challenging in presence of high dimensional data with temporal and spatial characteristics. Consequently, this work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to efficiently process complex multi-dimensional time series data with spatial information. Here, the cellular recurrent architecture in the proposed model allows for location-aware synchronous processing of time series data from spatially distributed sensor signal sources. Extensive trainable parameter sharing due to cellularity in the proposed architecture ensures efficiency in the use of recurrent processing units with high-dimensional inputs. This study also investigates the versatility of the proposed DCRNN model for classification of multi-class time series data from different application domains. Consequently, the proposed DCRNN architecture is evaluated using two time-series datasets: a multichannel scalp EEG dataset for seizure detection, and a machine fault detection dataset obtained in-house. The results suggest that the proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Ambassador Project (Final Report)

SEL, in collaboration with its partners, performed research, development and demonstration of a trust, data, and resource management software layer enabling multiple software applications from different suppliers to operate in a Software Defined Infrastructure. The Energy sector has successfully deployed Software Defined Networking (SDN) technology and is rapidly scaling deployments. One major benefit of SDN is the abstracted centralized nature of the orchestration and management of the network. This architecture enables end users to deploy new technology and capabilities as quickly as they perform firmware upgrades to field devices. When an end user needs to roll trucks to install new equipment, they simply deploy new software to run on top of this SD-Infrastructure. This has attracted many software companies to develop value added features and introduce new capabilities, which is all good for the industry but has also introduced new challenges, including how to manage trust between these software applications, how to manage the data integrity and access, and how to manage access to the network resources each of these software applications are demanding. Completion of this project developed and demonstrated a safe and reliable resource scheduler and cybersecurity trust management communication platform allowing multiple software applications to operate in a SD-Infrastructure resulting in the commercialization of a software eco-system and architecture that maintains interoperability, cybersecurity, and reliability where the end user can safely scale out their software to leverage the powerful benefits of SDN.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Sub-millisecond keyhole pore detection in laser powder bed fusion using sound and light sensors and machine learning

Laser powder bed fusion is a mainstream additive manufacturing technology widely used to manufacture complex parts in prominent sectors, including aerospace, biomedical, and automotive industries. However, during the printing process, the presence of an unstable vapor depression can lead to a type of defect called keyhole porosity, which is detrimental to the part quality. In this study, we developed an effective approach to locally detect the generation of keyhole pores during the printing process by leveraging machine learning and a suite of optical and acoustic sensors. Simultaneous synchrotron x-ray imaging allows the direct visualization of pore generation events inside the sample, offering high-fidelity ground truth. A neural network model adopting SqueezeNet architecture using single-sensor data was developed to evaluate the fidelity of each sensor for capturing keyhole pore generation events. Our comparative study shows that the near infrared images gave the highest prediction accuracy, followed by 100 kHz and 20 kHz microphones, and the photodiode sensitive to processing laser wavelength had the lowest accuracy. Using a single sensor, over 90% prediction accuracy can be achieved with a temporal resolution as short as 0.1 ms. A data fusion scheme was also developed with features extracted using SqueezeNet neural network architecture and classification using different machine learning algorithms. Our work demonstrates the correlation between the characteristic optical and acoustic emissions and the keyhole oscillation behavior, and thereby provides strong physics support for the machine learning approach.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Equivariant, safe and sensitive — graph networks for new physics

This study introduces a novel Graph Neural Network (GNN) architecture that leverages infrared and collinear (IRC) safety and equivariance to enhance the analysis of collider data for Beyond the Standard Model (BSM) discoveries. By integrating equivariance in the rapidity-azimuth plane with IRC-safe principles, our model significantly reduces computational overhead while ensuring theoretical consistency in identifying BSM scenarios amidst Quantum Chromodynamics backgrounds. The proposed GNN architecture demonstrates superior performance in tagging semi-visible jets, highlighting its potential as a robust tool for advancing BSM search strategies at high-energy colliders.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Evaluation of artificial neural network performance for classification of potato plants infected with potato virus Y using spectral data on multiple varieties and genotypes

Potato virus Y (Potyviridae, PVY) is a plant virus that poses a significant threat to potato producers on a global basis. The pathogen has disrupted seed potato supplies and negatively impacted yield and quality of commercial potato crops. The potato industry currently manages PVY infection levels via insecticide applications, regional seed certification programs that rely on field scouting to visually assess individual plants for infection status, and destructive and costly tissue sampling coupled with laboratory assays. Despite these efforts, PVY continues to confound potato industry stakeholders resulting in economic harm. Remote sensing and machine learning provide for the development of new tools to more accurately detect and spatially quantify PVY-infected plants versus the current state of the art. However, there is a need to understand how the occurrence of many different potato varieties impact the dynamics of developing models to detect potato plants impacted with PVY and their potential effectiveness. This study evaluates classification modelling outcomes using spectral datasets collected in different temporal and spatial environments (greenhouse and a production field) on multiple potato varieties consisting of labelled instances of plants infected with PVY and those not infected with the virus. A modelling framework was developed to support iterative modelling runs using artificial neural network (ANN) architectures configured as binary classifiers to develop sample populations to support statistical analysis on model performance using specific spectral subsets. When using spectral data to detect PVY-infected plants, ANN models achieved the highest mean accuracy of 0.894 on a single variety. Conversely, the same ANN model architecture only achieved a mean accuracy of 0.575 on a spectral data set representing 29 potato breeding lines. Additionally, statistical analysis indicates spectral regions including the red edge, near infrared and shortwave infrared contain more important spectral features for the ANN classifier introduced in this research.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

3D-ReG: A 3D ReRAM-based Heterogeneous Architecture for Training Deep Neural Networks

Deep neural network (DNN) models are being expanded to a broader range of applications. The computational capability of traditional hardware platforms cannot accommodate the growth of model complexity. Among recent technologies to accelerate DNN, resistive memory (ReRAM)-based processing-in-memory (PIM) emerged as a promising solution for DNN inference due to its high efficiency for matrix-based computation. We face two major technical challenges in extending the use of ReRAM-based accelerators for training: (1) full-precision data is essential in back-propagation; (2) the need to support both feed-forward and back-propagation aggravates the data-movement burden. We propose a heterogeneous architecture named as 3D-ReG, which leverages full-precision GPU to ensure training accuracy and low-overhead 3D integration to provide low-cost data movements. Moreover, we introduce conservative and aggressive task-mapping schemes, which partition the computation phases in different ways to balance execution efficiency and training accuracy. We evaluate 3D-ReG implemented with two 3D integration technologies, through-silicon vias (TSVs) and monolithic inter-tier vias (MIVs), and compare them with GPU-only and PIM-only counterparts. Various GPU-only platforms using two main-memory technologies (DRAM, ReRAM) and three interconnect technologies (2D, TSV, MIV) are evaluated as well. Experimental results show that 3D-ReG can achieve on average 5.64× training speedup and 3.56× higher energy efficiency compared with the GPU with DRAM as main memory, at the cost of 0.05%–3.39% accuracy drop. We define a new metric, gain-loss ratio (GLR), which quantitatively evaluates the capability of a DNN training hardware in terms of the model accuracy and hardware efficiency. The results of our comparison show that the aggressive task-mapping scheme on MIV-based 3D-ReG outperforms the other methods.

Computer Science↗
DOE OSTI2022

Secure Control Regions for Distributed Stochastic Systems with Application to Distributed Energy Resource Dispatch

With the increasing connectedness and interdependence of systems that are stochastic in nature, the issue of how to manage and coordinate them for safe operation has evidently become more important. In many networked system architectures, the system-wide output must be delicately managed, often within a prescribed set of bounds. In this paper, a novel control framework is proposed where the bounds on the outputs are translated into independent bounds on the controllable inputs of each subsystem. The main benefit of this framework is that respecting the individual control bounds suffices to guarantee that the system-wide outputs will remain within safe boundaries. Because the systems are assumed to be stochastic, the bounds on the output are introduced as probabilistic chance constraints. The benefits of this framework are demonstrated by applying it to the control of distributed energy resources in a distribution network where the main goal is to keep the voltage magnitudes within their prescribed bounds. The control bounds are evaluated using real data on an IEEE test system.

chance constrained optimization↗
DOE OSTI2022

Secure Control Regions for Distributed Stochastic Systems with Application to Distributed Energy Resource Dispatch: Preprint

With the increasing connectedness and interdependence of systems that are stochastic in nature, the issue of how to manage and coordinate them for safe operation has evidently become more important. In many networked system architectures, the system-wide output has to be delicately managed; often within a prescribed set of bounds. In this paper, a novel control framework is proposed where the bounds on the outputs are translated into independent bounds on the controllable inputs of each subsystem. The main benefit of this framework is that respecting the individual control bounds suffices to guarantee that the system-wide outputs will remain within safe boundaries. Since the systems are assumed to be stochastic, the bounds on the output are introduced as probabilistic chance constraints. The benefits of this framework are demonstrated by applying it to the control of distributed energy resources in a distribution networks where main goal is to keep the voltage magnitudes with their prescribed bounds. The control bounds are evaluated using real data on an IEEE test system.

chance constrained optimization↗
DOE OSTI2023

Machine-Learning-Based Rotating Detonation Engine Diagnostics: Evaluation for Application in Experimental Facilities

Real-time monitoring of combustion behavior is a crucial step toward actively controlled rotating detonation engine (RDE) operation in laboratory and industrial environments. Various machine learning methods have been developed to advance diagnostic efficiencies from conventional postprocessing efforts to real-time methods. Here this work evaluates and compares conventional techniques alongside convolutional neural network (CNN) architectures trained in previous studies, including image classification, object detection, and time series classification, according to metrics affecting diagnostic feasibility, external applicability, and performance. Real-time, capable diagnostics are deployed and evaluated using an altered experimental setup. Image-based CNNs are applied to externally provided images to approximate dataset restrictions. Image classification using high-speed chemiluminescence images and time series classification using high-speed flame ionization and pressure measurements achieve classification speeds enabling real-time diagnostic capabilities, averaging laboratory-deployed diagnostic feedback rates of 4–5 Hz. Object detection achieves the most refined resolution of 20 μs in postprocessing. Image and time series classification require the additional correlation of sensor data, extending their time-step resolutions to 80 ms. Comparisons show that no single diagnostic approach outperforms its competitors across all metrics. This finding justifies the need for a machine learning portfolio containing a host of networks to address specific needs throughout the RDE research community.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2020

An Authentication Vulnerability Assessment of Connected Lighting Systems

Emerging connected lighting systems (CLS) that incorporate distributed intelligence, network interfaces, and sensors can become data-collection platforms that enable a wide range of valuable new capabilities as well as greater energy savings in buildings and cities. However, CLS technology is currently at an early stage of development, and its increased connectivity introduces cybersecurity risks that are new to the lighting industry and that must be addressed for successful integration with other systems. While a number of existing frameworks, guidelines, and tests for evaluating cybersecurity vulnerability may apply to CLS in whole or in part, there is currently no mandatory requirement for cybersecurity testing or certification. The lighting industry, including technology developers and specification organizations, is currently evaluating the suitability of existing frameworks and guidelines for CLS. To support these efforts, Pacific Northwest National Laboratory (PNNL) is conducting a series of studies intended to educate lighting industry stakeholders on specific cybersecurity practices and characterize their implementation in commercially available CLS with varying system architectures, network-communication technologies, and degrees of maturity. This study demonstrates that tests for authentication vulnerabilities can be developed with objective pass/fail criteria, therby facilitating comparisons between CLS. Based on the limited results of this study, it appears that the CLS that are being brought to market have varying levels of authentication vulnerability. It is hoped that these evaluations will support and perhaps accelerate industry discussions on the risks of specific security vulnerabilities, what vulnerabilities should be addressed by in-development of future lighting-specific best practices, and whether any such practices should be included in voluntary lighting standards.

42 ENGINEERING↗
DOE OSTI2022

Deep Convolutional Neural Networks Exploit High-Spatial- and -Temporal-Resolution Aerial Imagery to Phenotype Key Traits in Miscanthus

Miscanthus is one of the most promising perennial crops for bioenergy production, with high yield potential and a low environmental footprint. The increasing interest in this crop requires accelerated selection and the development of new screening techniques. New analytical methods that are more accurate and less labor-intensive are needed to better characterize the effects of genetics and the environment on key traits under field conditions. We used persistent multispectral and photogrammetric UAV time-series imagery collected 10 times over the season, together with ground-truth data for thousands of Miscanthus genotypes, to determine the flowering time, culm length, and biomass yield traits. We compared the performance of convolutional neural network (CNN) architectures that used image data from single dates (2D-spatial) versus the integration of multiple dates by 3D-spatiotemporal architectures. The ability of UAV-based remote sensing to rapidly and non-destructively assess large-scale genetic variation in flowering time, height, and biomass production was improved through the use of 3D-spatiotemporal CNN architectures versus 2D-spatial CNN architectures. The performance gains of the best 3D-spatiotemporal analyses compared to the best 2D-spatial architectures manifested in up to 23% improvements in R2, 17% reductions in RMSE, and 20% reductions in MAE. The integration of photogrammetric and spectral features with 3D architectures was crucial to the improved assessment of all traits. In conclusion, our findings demonstrate that the integration of high-spatiotemporal-resolution UAV imagery with 3D-CNNs enables more accurate monitoring of the dynamics of key phenological and yield-related crop traits. This is especially valuable in highly productive, perennial grass crops such as Miscanthus, where in-field phenotyping is especially challenging and traditionally limits the rate of crop improvement through breeding.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Optimizing the optimizer for physics-informed neural networks and Kolmogorov-Arnold networks

Physics-Informed Neural Networks (PINNs) have revolutionized the computation of PDE solutions by integrating partial differential equations (PDEs) into the neural network’s training process as soft constraints, becoming an important component of the scientific machine learning (SciML) ecosystem. More recently, physics-informed Kolmogorv-Arnold networks (PIKANs) have also shown to be effective and comparable in accuracy with PINNs. In their current implementation, both PINNs and PIKANs are mainly optimized using first-order methods like Adam, as well as quasi-Newton methods such as BFGS and its low-memory variant, L-BFGS. However, these optimizers often struggle with highly nonlinear and non-convex loss landscapes, leading to challenges such as slow convergence, local minima entrapment, and (non)degenerate saddle points. In this study, we investigate the performance of Self- Scaled BFGS (SSBFGS), Self-Scaled Broyden (SSBroyden) methods and other advanced quasi-Newton schemes, including BFGS and L-BFGS with different line search strategies. These methods dynamically rescale updates based on historical gradient information, thus enhancing training efficiency and accuracy. We systematically compare these optimizers – using both PINNs and PIKANs – on key challenging PDEs, including the Burgers, Allen-Cahn, Kuramoto-Sivashinsky, Ginzburg-Landau, and Stokes equations. Additionally, we evaluate the performance of SSBFGS and SSBroyden for Deep Operator Network (DeepONet) architectures, demonstrating their effectiveness for data-driven operator learning. Our findings provide state-of-the-art results with orders-of-magnitude accuracy improvements without the use of adaptive weights or any other enhancements typically employed in PINNs. More broadly, our work reveal insights into the effectiveness of quasi-Newton optimization strategies in significantly improving the convergence and accurate generalization of PINNs and PIKANs.

97 MATHEMATICS AND COMPUTING↗