Search NASA⌕ Search

SEARCH · Search NASA

Results for “network regularization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Structure-preserving neural networks for the regularized entropy-based closure of a linear, kinetic, radiative transport equation

The main challenge of large-scale numerical simulation of radiation transport is the high memory and computation time requirements of discretization methods for kinetic equations. In this work, we derive and investigate a neural network-based approximation to the entropy-based closure method to accurately compute the solution of the multi-dimensional moment system with a low memory footprint and competitive computational time. We extend methods developed for the standard entropy-based closure to the regularized entropy-based closures. The main idea is to interpret structure-preserving neural network approximations of the regularized entropy-based closure as a two-stage approximation to the original entropy-based closure. We conduct a numerical analysis of this approximation and investigate optimal parameter choices. Our numerical experiments demonstrate that the method has a much lower memory footprint than traditional methods with competitive computation times and simulation accuracy. The code and all trained networks are provided on GitHub.

entropy closure↗

Effects of input gradient regularization on neural networks time-series forecasting of thermal power systems

This study proposes using neural networks, specifically gated recurrent unit (GRU), long-short-term memory (LSTM), and transformer networks, to improve control strategies in a 450 MW coal-fired power plant. However, neural networks face issues of becoming overly dependent on just a few variables to make predictions, which negatively impacts control decisions that rely on the model to determine the value of all manipulated variables. The paper introduces regularization techniques, including noise injection and input gradient regularization, during the training phase. Here, the work presents novel contributions in adapting neural networks to control industrial systems and applying regularization techniques from computer vision to industrial process control. Results demonstrate the effectiveness of input gradient regularization in reducing model dependence on subsets of variables, emphasizing the balance between fidelity and controllability. Further exploration is recommended, including the development of recurrent transformers, closed-loop control testing, and a sensitivity analysis on computer models to provide further insight.

20 FOSSIL-FUELED POWER PLANTS↗

Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimation

Deep learning has exhibited remarkable results across diverse areas. To understand its success, substantial research has been directed towards its theoretical foundations. Nev- ertheless, the majority of these studies examine how well deep neural networks can model functions with uniform regularities. In this paper, we explore a different angle: how deep neural networks can adapt to varying degrees of smoothness in functions and nonuni- form data distributions across different locations and scales. More precisely, we focus on a broad class of functions defined by nonlinear tree-based approximation methods. This class encompasses a range of function types, such as functions with uniform regularities and discontinuous functions. We develop nonparametric approximation and estimation theories for this class using deep ReLU networks. Our results show that deep neural networks are adaptive to the nonuniform smoothness of functions and nonuniform data distributions at different locations and scales. We apply our results to several function classes, and derive the corresponding approximation and generalization errors. The validity of our results is demonstrated through numerical experiments.

97 MATHEMATICS AND COMPUTING↗

Monomer-dimer tensor-network basis for qubit-regularized lattice gauge theories

Traditional SU⁡(𝑁) lattice gauge theories (LGTs) can be formulated using an orthonormal basis constructed from the irreducible representations (irreps) 𝑉 𝜆 of the SU⁡(𝑁) gauge symmetry. On a lattice, the elements of this basis are tensor networks comprising dimer tensors on the links labeled by a set of irreps {𝜆 ℓ } and monomer tensors on sites labeled by {𝜆 𝑠 }. These tensors naturally define a local site Hilbert space, ℋ$^𝑔_𝑠$, on which gauge transformations act. Gauss’s law introduces an additional index 𝛼 𝑠 =1,2,…,𝒟⁡(ℋ$^𝑔_𝑠$) that labels an orthonormal basis of the gauge-invariant subspace of ℋ$^𝑔_𝑠$. This monomer-dimer tensor-network (MDTN) basis, |{𝜆 𝑠 },{𝜆 ℓ },{𝛼 𝑠 }⟩, of the physical Hilbert space enables the construction of new qubit-regularized SU⁡(𝑁) gauge theories that are free of sign problems while preserving key features of traditional LGTs. Here, we investigate finite-temperature confinement-deconfinement transitions in a simple qubit-regularized SU(2) and SU(3) gauge theory in 𝑑 =2 and 𝑑 =3 spatial dimensions, formulated using the MDTN basis, and show that they reproduce the universal results of traditional LGTs at these transitions. Additionally, in 𝑑 =1, we demonstrate using a plaquette chain that the string tension at zero temperature can be continuously tuned to zero by adjusting a model parameter that plays the role of the gauge coupling in traditional LGTs.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Electrooptical adaptive switching network for the hypercube computer

An all-optical network design for the hyperswitch network using regular free-space interconnects between electronic processor nodes is presented. The adaptive routing model used is described, and an adaptive routing control example is presented. The design demonstrates that existing electrooptical techniques are sufficient for implementing efficient parallel architectures without the need for more complex means of implementing arbitrary interconnection schemes. The electrooptical hyperswitch network significantly improves the communication performance of the hypercube computer.

Chow, E.↗

Degrees of Rate Control in Interconnected Reaction Networks

Overall reactions in interconnected networks exhibit net, forward, and reverse rates that are governed by both constitutive elementary steps in the pathway of interest and branching elementary steps that lead to alternative products. Accordingly, steps in branching pathways exhibit negative net, forward, and reverse degrees of rate control, as they reduce reaction flux to the desired product. We here contextualize the forward and reverse degrees of rate control in terms of kinetic resistances (inverse of rates) and leverage the additive nature of kinetic resistance to decouple kinetic driving forces contributed by constitutive elementary steps and branching points (nodal species) in interconnected networks. Regardless of the network connectivity, forward and reverse degrees of rate control are shown to converge at equilibrium. Away from equilibrium, we identify two critical features of interconnected networks: stoichiometric regularity─condition where all stoichiometric numbers are unity─and pathway symmetry around nodal species─condition where branching pathways share the same rate constants, stoichiometry, and species concentrations/activities─that result in (i) equal forward, reverse, and consequently net degrees of rate control and (ii) forward and reverse degrees of rate control that exhibit constant offsets, respectively, across all extents of reaction. Furthermore, our discourse further provides a mathematical description for the influence of stoichiometric irregularity and pathway asymmetry on forward and reverse degrees of rate control. Altogether, the presented work details the effects of network (inter)connectivity and stoichiometry on reaction kinetics and, in doing so, establishes general protocols for capturing these effects as additive terms in the formulation of forward and reverse degrees of rate control.

10 SYNTHETIC FUELS↗

Next-Level Energy Management in Manufacturing: Facility-Level Energy Digital Twin Framework Based on Machine Learning and Automated Data Collection

This research introduces an energy prediction framework at the facility level supported by automated data collection and machine learning models. It investigates whether reducing the prediction time scale allows for applying more complex machine learning techniques and if those techniques improve the prediction accuracy. The primary advantages of this framework lie in its automation of the energy prediction process and its provision of real-time energy data suitable for use in energy dashboards or digital twins. A sitewide dataset was created by combining 15 min energy and daily production data of five shops—assembly, battery, body (electric), body (gas), and paint—from a globally recognized electric vehicle manufacturer. Various machine learning models were evaluated on daily, weekly, and monthly datasets, including, in increasingly complex order: naïve, simple linear regression, net regularized generalized linear regression, principal component regression, k-nearest neighbor, random forest, and Bayesian regularized neural network. Compared to the current state-of-the-art energy consumption prediction for the industrial facility level, this research investigates more complex models and smaller time intervals for higher accuracy. The findings revealed that the more complex monthly models require a minimum of a year and a half of data to operate, while weekly models demand a year of data to achieve improved accuracy. Daily models can operate with only six months of data but exhibit poor performance due to reduced prediction accuracy of production. Key challenges identified include access to reliable, high-quality energy and production data and the initial demand for human labor.

digital twin↗

Antarctic ice topography and 150 mb circulation from the Nimbus-6 TWERL Experiment

Many TWERLE balloons floating at an altitude of about 12.5 km, traversed the Antarctic during its 1975/6 summer. The balloons' positions were accurately determined by the Nimbus-6 satellite, which also received their sensors' data: radio altimeter, pressure and temperature. Subtracting the altimeter readings from the calculated balloon altitude above sea-level (a.s.l.) provides the Antarctic surface elevation at the balloon subtrack. The balloon altitude a.s.l. is calculated from daily synoptic maps covering latitudes south of 45 deg S. These maps are prepared using the regular radiosonde network supplemented by four times as many TWERLE balloons.

Levanon, N.↗

Satellite-tracking and Earth dynamics research programs

The major focus for operations during this period was the preliminary MERIT Campaign and its intensive tracking of LAGEOS for polar motion and Earth rotation studies. The data acquired from LAGEOS were used for other geophysical investigations, including studies of crustal dynamics, and Earth and ocean tides, and for the general development of precision orbit determination. The network performed regular tracking of several other retroreflector satellites including GEOS-1, GEOS-3, BE-C, and Starlette for refined determinations of station coordinates and Earth's gravity field and for studies of solid Earth dynamics.

Source record↗

Satellite-tracking and Earth dynamics research programs

The activities carried out by the Smithsonian Astrophysical Observatory (SAO) are described. The SAO network continued to track LAGEOS at highest priority for polar motion and Earth rotation studies, and for other geophysical investigations, including crustal dynamics, Earth and ocean tides, and the general development of precision orbit determination. The network performed regular tracking of several other retroreflector satellites including GEOS-1, GEOS-3, BE-C, and Starlette for refined determinations of station coordinates and the Earth's gravity field and for studies of solid Earth dynamics. A major program in laser upgrading continued to improve ranging accuracy and data yield. This program includes an increase in pulse repetition rate from 8 ppm to 30 ppm, a reduction in laser pulse width from 6 nsec to 2 to 3 nsec, improvements in the photoreceiver and the electronics to improve daylight ranging, and an analog pulse detection system to improve range noise and accuracy. Data processing hardware and software are discussed.

Source record↗

Satellite tracking and earth dynamics research programs

The SAO laser site in Arequipa continued routine operations throughout the reporting period except for the months of March and April when upgrading was underway. The laser in Orroral Valley was operational through March. Together with the cooperating stations in Wettzell, Grasse, Kootwikj, San Fernando, Helwan, and Metsahove the laser stations obtained a total of 37,099 quick-look observations on 978 passes of BE-C, Starlette, and LAGEOS. The Network continued to track LAGEOS at highest priority for polar motion and Earth rotation studies, and for other geophysical investigations, including crustal dynamics, Earth and ocean tides, and the general development of precision orbit determination. The Network performed regular tracking of BE-C and Starlette for refined determinations of station coordinate and the Earth's gravity field and for studies of solid earth dynamics. Monthly statistics of the passes and points are given by station and by satellite.

Source record↗

Predictive Modeling for Differential Diagnosis and Mortality Risk Assessment

The prevalence of electronic health record (EHR) systems has brought prodigious biomedical informatics opportunity. Automated machine learning methods can effectively utilize such data and have become common tools for healthcare predictive modeling. Researches in medical informatics have explored the potential of deep learning and classical models in emergent care scenarios. In particular, predicting differential diagnoses for admissions have proven useful in decreasing unnecessary lab tests and improving inpatient triage decision-making. Moreover, identification of high-risk patients for in-hospital mortality is vitally important to maximize allocation of medical resources.The Medical Information Mart for Intensive Care (MIMIC-III) database, containing de-identified critical care inpatient was used in our study. This data set captures hospital patient laboratory measurements, pharmacologic prescriptions, diagnostic data and procedure event recordings. When considering adult patients and discounting admissions with ICU length of stay less than 24 hours, there were 37,787 unique admissions and 30,414 total patients. We examined the top 25 most prevalent ICD-9 group-level disease specificities in MIMIC-III using a multi-label classification model. In-hospital mortality was modeled as binary classification with 4,155 (13%) adult patients that expired, of which 3,138 (75.5%) were in the ICU setting. The metrics AUC, F1 score, sensitivity and specificity values calculated for each disease label measured prediction performance.The usage of ICD-9 group codes reduced feature dimension from 14,567 to 942 and greatly improved distribution of patient diagnostic categories. Disease temporal patterns were captured by considering the most frequently sampled 6 vital signs and 13 laboratory values. Missing data were imputed at each time-stamp. Time-series raw hourly average values were converted into 5 summary features (mean, standard deviation, number of observations, min & max values). Patient demographic variables such as age, gender, marital status and ethnicity were also factored into the modeling. Choi et al showed that contextual embedding of medical data, diagnostic and procedural codes alone can predict future diagnoses with sensitivity as high as 0.79. We utilized an embedding technique called word2vec which allowed sparse representations of medical history to be transformed into dense word vectors. The mappings captured contextual information by treating each admission as a sentence and learning the most likely neighboring words in a sliding window fashion. Binary and multi-label classification was achieved via collapse models, which do not consider temporal information, as well as recurrent neural networks with regularization, Softmax output layer activation together with categorical cross-entropy as the loss function.

US Army collaboration↗

Sentinel

Network intrusion detection systems (NIDS) are commonplace in network security but they frequently employ algorithms that are computational demanding requiring hardware and software with significant power requirements. Two examples of such resource-intensive algorithms used for network security are regular expression matching and broader signature pattern matching which are commonly used in deep packet inspection (DPI). Network security algorithms that have large power requirements may be a challenge for low-power internet-of-things (IoT) environments, which generally lack the power resources to implement complex security measures like computationally expensive DPI at the edge. Furthermore, IoT environments incorporating 5G standalone networks have network latency constraints beyond just power that make DPI at the edge even more difficult. Programmable logic is ideally suited for machine learning inference for DPI because of its deep instruction level parallelism and single-cycle memory access. Machine learning approaches for DPI have been explored before using the programmable logic of field programmable gate arrays (FPGA) as a potential solution for NIDS approaches that would be power-suitable for IoT. However, those previous programmable logic NIDS approaches utilize either a supervised or unsupervised learning model. Sentinel utilizes the ensemble of these two machine learning approaches known as a semi-supervised approach which has shown promise in NIDS implementations. Sentinel provides a programmable logic implementation of a semi-supervised approach for DPI which operates at much lower power and latency than a GPU implementation with negligible loss of accuracy due to quantization through a logistic regressor.

Anderson, MatthewW [Idaho National Laboratory (INL↗

Observing large-scale solar surface flows with GONG: Investigation of a key element in solar activity buildup

The Global Oscillation Network Group (GONG) solar telescope network has begun regular operations, and will provide continuous Doppler images of large-scale nearly-steady motions at the solar surface, primarily those due to supergranulation. Not only the Sun's well-known magnetic network, but also flux diffusion, dispersal, and concentration at the surface appear to be controlled by supergranulation. Through such magnetoconvective interactions, magnetic stresses develop, leading to solar activity. We show a Doppler movie made from a 45.5 hr time series obtained 1995 May 9-10 using data from three of the six GONG sites (Learmonth, Tenerife, Tucson), to demonstrate the capability of this system.

Beck, John G.↗

Regular Topologies for Gigabit Wide-Area Networks: Congestion Avoidance Testbed Experiments

This document is Volume 3 of the final technical report on the work performed by SRI International (SRI) on SRI Project 8600. The document includes source listings for all software developed by SRI under this effort. Since some of our work involved the use of ST-II and the Sun Microsystems, Inc. (Sun) High-Speed Serial Interface (HSI/S) driver, we have included some of the source developed by LBL and BBN as well. In most cases, our decision to include source developed by other contractors depended on whether it was necessary to modify the original code. If we have modified the software in any way, it is included in this document. In the case of the Traffic Generator (TG), however, we have included all the ST-II software, even though BBN performed the integration, because the ST-II software is part of the standard TG release. It is important to note that all the code developed by other contractors is in the public domain, so that all software developed under this effort can be re-created from the source included here.

Denny, Barbara A.↗

Regular Topologies for Gigabit Wide-Area Networks

In general terms, this project aimed at the analysis and design of techniques for very high-speed networking. The formal objectives of the project were to: (1) Identify switch and network technologies for wide-area networks that interconnect a large number of users and can provide individual data paths at gigabit/s rates; (2) Quantitatively evaluate and compare existing and proposed architectures and protocols, identify their strength and growth potentials, and ascertain the compatibility of competing technologies; and (3) Propose new approaches to existing architectures and protocols, and identify opportunities for research to overcome deficiencies and enhance performance. The project was organized into two parts: 1. The design, analysis, and specification of techniques and protocols for very-high-speed network environments. In this part, SRI has focused on several key high-speed networking areas, including Forward Error Control (FEC) for high-speed networks in which data distortion is the result of packet loss, and the distribution of broadband, real-time traffic in multiple user sessions. 2. Congestion Avoidance Testbed Experiment (CATE). This part of the project was done within the framework of the DARTnet experimental T1 national network. The aim of the work was to advance the state of the art in benchmarking DARTnet's performance and traffic control by developing support tools for network experimentation, by designing benchmarks that allow various algorithms to be meaningfully compared, and by investigating new queueing techniques that better satisfy the needs of best-effort and reserved-resource traffic. This document is the final technical report describing the results obtained by SRI under this project. The report consists of three volumes: Volume 1 contains a technical description of the network techniques developed by SRI in the areas of FEC and multicast of real-time traffic. Volume 2 describes the work performed under CATE. Volume 3 contains the source code of all software developed under CATE.

Shacham, Nachum↗

NNFDivergence

The code implements f divergence regularization for neural networks in the Python-based Pytorch framework. The methods are the main focus but the repository will also contain examples that operate on purely synthetic "toy" data or on openly available, public data from NASA.

Klein, Natalie [@lanl]↗

Dynamic Low-Rank Training with Spectral Regularization: Achieving Robustness in Compressed Representations

Deployment of neural networks on resource-constrained devices demands models that are both compact and robust to adversarial inputs. However, compression and adversarial robustness often conflict. In this work, we introduce a dynamical low-rank training scheme enhanced with a novel spectral regularizer that controls the condition number of the low-rank core in each layer. This approach mitigates the sensitivity of compressed models to adversarial perturbations without sacrificing clean accuracy. The method is model- and data-agnostic, computationally efficient, and supports rank adaptivity to automatically compress the network at hand. Extensive experiments across standard architectures, datasets, and adversarial attacks show the regularized networks can achieve over 94\% compression while recovering or improving adversarial accuracy relative to uncompressed baselines.

Schotthoefer, Steffen [ORNL] (ORCID:00000002156965↗