Search NASA⌕ Search

SEARCH · Search NASA

Results for “DATA NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,333 records · Page 74

Hyperplane decision trees as piecewise linear surrogate models for chemical process design

Recent trends in chemical engineering research point towards an increasing reliance on data-driven modeling approaches. Neural networks, for instance, have proven to be accurate when data is plentiful and high-dimensional, but in many cases, they require computationally-intensive training procedures. Here, in this work, we describe hyperplane decision trees (HT) as a highly expressive and low-compute machine learning model architecture. These models are locally linear and have linear decision boundaries, resulting in a piecewise linear model of the data. This property allows them to be converted into mixed-integer linear constraints which can be globally optimized. Our open-source PyTorch implementation of this method is a fast, flexible, and accessible way to build accurate piecewise linear models of data.

Decision trees↗

pvlib iotools—Open-source Python functions for seamless access to solar irradiance data

Access to accurate solar resource data is critical for numerous applications, including estimating the yield of solar energy systems, developing radiation models, and validating irradiance datasets. However, lack of standardization in data formats and access interfaces across providers constitutes a major barrier to entry for new users. pvlib python’s iotools subpackage aims to solve this issue by providing standardized Python functions for reading local files and retrieving data from external providers. All functions follow a uniform pattern and return convenient data outputs, allowing users to seamlessly switch between data providers and explore alternative datasets. The pvlib package is community-developed on GitHub: https://github.com/pvlib/pvlib-python. As of pvlib python version 0.9.5, the iotools subpackage supports 12 different datasets, including ground measurement, reanalysis, and satellite-derived irradiance data. The supported ground measurement networks include the Baseline Surface Radiation Network (BSRN), NREL MIDC, SRML, SOLRAD, SURFRAD, and the US Climate Reference Network (CRN). Additionally, satellite-derived and reanalysis irradiance data from the following sources are supported: PVGIS (SARAH & ERA5), NSRDB PSM3, and CAMS Radiation Service (including McClear clear-sky irradiance).

14 SOLAR ENERGY↗

dCache: Inter-disciplinary storage system

The dCache project provides open-source software deployed internationally to satisfy ever more demanding storage requirements. Its multifaceted approach provides an integrated way of supporting different use-cases with the same storage, from high throughput data ingest, data sharing over wide area networks, efficient access from HPC clusters and long term data persistence on a tertiary storage. Though it was originally developed for the HEP experiments, today it is used by various scientific communities, including astrophysics, biomed, life science, which have their specific requirements. In this paper we describe some of the new requirements as well as demonstrate how dCache developers are addressing them.

Mkrtchyan, Tigran↗

Datashare

Datashare facilitates communication and data sharing within local networks in potentially dangerous situations such as an explosive ordnance disposal. During such events, there is a need to transmit information rapidly around the incident area. It is a distributed database that does not require an internet connection for operation. In addition, Datashare interfaces with XTK and other software applications, allowing for seamless integration and data management. Datashare supports video calls over the network, enabling real-time communication among users. This software serves to organize, package, and share between responders on location and export data to those off location. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Eldridge, Bryce [Sandia National Lab. (SNL-CA), Li↗

Real-Time Visualization of Network Behaviors for Situational Awareness

Plentiful, complex, and dynamic data make understanding the state of an enterprise network difficult. Although visualization can help analysts understand baseline behaviors in network traffic and identify off-normal events, visual analysis systems often do not scale well to operational data volumes (in the hundreds of millions to billions of transactions per day) nor to analysis of emergent trends in real-time data. We present a system that combines multiple, complementary visualization techniques coupled with in-stream analytics, behavioral modeling of network actors, and a high-throughput processing platform called MeDICi. This system provides situational understanding of real-time network activity to help analysts take proactive response steps. We have developed these techniques using requirements gathered from the government users for which the tools are being developed. By linking multiple visualization tools to a streaming analytic pipeline, and designing each tool to support a particular kind of analysis (from high-level awareness to detailed investigation), analysts can understand the behavior of a network across multiple levels of abstraction.

97 MATHEMATICS AND COMPUTING↗

Deep Hierarchical Super Resolution for Scientific Data

We present a novel technique for hierarchical super resolution (SR) with neural networks (NNs), which upscales volumetric data represented with an octree data structure to a high-resolution uniform gridwith minimal seam artifacts on octree node boundaries. Our method uses existing state-of-the-art SR models and adds flexibility to upscale input data with varying levels of detail across the domain, instead of only uniform grid data that are supported in previous approaches.The key is to use a hierarchy of SR NNs, each trained to perform 2x SR between two levels of detail, with a hierarchical SR algorithm that minimizes seam artifacts by starting from the coarsest level of detail and working up.We show that our hierarchical approach outperforms baseline interpolation and hierarchical upscaling methods, and demonstrate the usefulness of our proposed approach across three use cases including data reduction using hierarchical downsampling+SR instead of uniform downsampling+SR, computation savings for hierarchical finite-time Lyapunov exponent field calculation, and super-resolving low-resolution simulation results for a high-resolution approximation visualization.

97 MATHEMATICS AND COMPUTING↗

Pycheron: A Python-Based Seismic Waveform Data Quality Control Software Package

Supplementing an existing high-quality seismic monitoring network with openly available station data could improve coverage and decrease magnitudes of completeness; however, this can present challenges when varying levels of data quality exist. Without discerning the quality of openly available data, using it poses significant data management, analysis, and interpretation issues. Incorporating additional stations without properly identifying and mitigating data quality problems can degrade overall monitoring capability. If openly available stations are to be used routinely, a robust, automated data quality assessment for a wide range of quality control (QC) issues is essential. To meet this need, we developed Pycheron, a Python-based library for QC of seismic waveform data. Pycheron was initially based on the Incorporated Research Institutions for Seismology’s Modular Utility for STAtistical kNowledge Gathering but has been expanded to include more functionality. Pycheron can be implemented at the beginning of a data processing pipeline or can process stand-alone data sets. Its objectives are to (1) identify specific QC issues; (2) automatically assess data quality and instrumentation health; (3) serve as a basic service that all data processing builds on by alerting downstream processing algorithms to any quality degradation; and (4) improve our ability to process orders of magnitudes more data through performance optimizations. This article provides an overview of Pycheron, its features, basic workflow, and an example application using a synthetic QC data set.

58 GEOSCIENCES↗

Editorial [Special issue on software defined networks and infrastructures, network function virtualisation, autonomous systems and network management]

For last few years, there has been a tremendous growth in data traffic due to high adoption rate of mobile devices and cloud computing. Internet of things (IoT) will stimulate even further growth. This is increasing scale and complexity of telecom/internet service provider (SP) and enterprise data centre (DC) compute and network infrastructures. As a result, managing these large network-compute converged infrastructures is becoming complex and cumbersome. To cope up, network and DC operators are trying to automate network and system operations, administrations and management (OAM) functions. OAM includes all non-functional mechanisms which keep the network running.

97 MATHEMATICS AND COMPUTING↗

Towards a Scalable and Adaptive Application Support Platform for Large-Scale Distributed E-Sciences in High-Performance Network Environments

The advent of large-scale collaborative scientific applications has demonstrated the potential for broad scientific communities to pool globally distributed resources to produce unprecedented data acquisition, movement, and analysis. System resources including supercomputers, data repositories, computing facilities, network infrastructures, storage systems, and display devices have been increasingly deployed at national laboratories and academic institutes. These resources are typically shared by large communities of users over Internet or dedicated networks and hence exhibit an inherent dynamic nature in their availability, accessibility, capacity, and stability. Scientific applications using either experimental facilities or computation-based simulations with various physical, chemical, climatic, and biological models feature diverse scientific workflows as simple as linear pipelines or as complex as a directed acyclic graphs, which must be executed and supported over wide-area networks with massively distributed resources. Application users oftentimes need to manually configure their computing tasks over networks in an ad hoc manner, hence significantly limiting the productivity of scientists and constraining the utilization of resources. The success of these large-scale distributed applications requires a highly adaptive and massively scalable workflow platform that provides automated and optimized computing and networking services. This project is to design and develop a generic Scientific Workflow Automation and Management Platform (SWAMP), which contains a web-based user interface specially tailored for a target application, a set of user libraries, and several easy-to-use computing and networking toolkits for application scientists to conveniently assemble, execute, monitor, and control complex computing workflows in heterogeneous high-performance network environments. SWAMP will enable the automation and management of the entire process of scientific workflows with the convenience of a few mouse clicks while hiding the implementation and technical details from end users. Particularly, we will consider two types of applications with distinct performance requirements: data-centric and service-centric applications. For data-centric applications, the main workflow task involves large-volume data generation, catalog, storage, and movement typically from supercomputers or experimental facilities to a team of geographically distributed users; while for service-centric applications, the main focus of workflow is on data archiving, preprocessing, filtering, synthesis, visualization, and other application-specific analysis. We will conduct a comprehensive comparison of existing workflow systems and choose the best suited one with open-source code, a flexible system structure, and a large user base as the starting point for our development. Based on the chosen system, we will develop and integrate new components including a black box design of computing modules, performance monitoring and prediction, and workflow optimization and reconfiguration, which are missing from existing workflow systems. A modular design for separating specification, execution, and monitoring aspects will be adopted to establish a common generic infrastructure suited for a wide spectrum of science applications. We will further design and develop efficient workflow mapping and scheduling algorithms to optimize the workflow performance in terms of minimum end-to-end delay, maximum frame rate, and highest reliability. We will develop and demonstrate the SWAMP system in a local environment, the grid network, and the 100Gpbs Advanced Network Initiative (ANI) testbed. The demonstration will target scientific applications in climate modeling and high energy physics and the functions to be demonstrated include workflow deployment, execution, steering, and reconfiguration. Throughout the project period, we will work closely with the science communities in the fields of climate modeling and high energy physics including Spallation Neutron Source (SNS) and Large Hadron Collider (LHC) projects to mature the system for production use.

97 MATHEMATICS AND COMPUTING↗

Deep learning for computational chemistry

The rise and fall of artificial neural networks is well documented in the scientific literature of both the fields of computer science and computational chemistry. Yet almost two decades later, we are now seeing a resurgence of interest in deep learning, a machine learning algorithm based on “deep” neural networks. Within the last few years, we have seen the transformative impact of deep learning the computer science domain, notably in speech recognition and computer vision, to the extent that the majority of practitioners in those field are now regularly eschewing prior established models in favor of deep learning models. In this review, we provide an introductory overview into the theory of deep neural networks and their unique properties as compared to traditional machine learning algorithms used in cheminformatics. By providing an overview of the variety of emerging applications of deep neural networks, we highlight its ubiquity and broad applicability to a wide range of challenges in the field, including QSAR, virtual screening, protein structure modeling, QM calculations, materials synthesis and property prediction. In reviewing the performance of deep neural networks, we observed a consistent outperformance against non neural networks state-of-the-art models across disparate research topics, and deep neural network based models often exceeded the “glass ceiling” expectations of their respective tasks. Coupled with the maturity of GPU-accelerated computing for training deep neural networks and the exponential growth of chemical data on which to train these networks on, we anticipate that deep learning algorithms will be a useful tool and may grow into a pivotal role for various challenges in the computational chemistry field.

Computational Chemistry↗

Idaho National Laboratory Quality Of Service Dataset

The code is designed to run tests to generate and collect data from a Wi-Fi network using OPENWRT or a simulated a 5G network using Open5gs and UERANSIM. The tests simulate the network performing downloads or uploads of various files with a varying number of concurrent users. The tests use tcpdump to collect the network traffic but only stores the summarized data. The summarized datasets will be included.

Krome, Cameron [Idaho National Laboratory (INL), I↗

Unique Building Identifier (UBID): Public Sector Implementation Guide

Buildings generate data throughout their lifecycle – about ownership & taxation, usage, zoning, code compliance, energy use, and retrofits. State and local governments collect this data after it flows through growing networks of people and systems. But collecting data is only half the battle; what’s really needed is information – the actionable insights that lead to successful policy outcomes.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Evader Interdiction and Collateral Damage

In network interdiction problems, evaders (hostile agents or data packets) are moving through a network towards their targets and we wish to choose sensor placement locations in order to intercept them before they reach their destinations. Sensor locations should be chosen economically, balancing security gains with cost, including the inconvenience sensors inflict upon innocent travelers. We give optimal sensor allocation algorithms for several classes of special graphs and hardness and optimal approximation results for general graphs, including for deterministic or Markov chain-based and oblivious or reactive evaders. In a similar-sounding but much different problem setting posed by [10] where the innocent travelers can also be reactive, we again give optimal algorithms for special cases and hardness and (essentially) optimal approximation results on general graphs.

ALGORITHMS↗

Deadlock-free class routes for collective communications embedded in a multi-dimensional torus network

A computer implemented method and a system for routing data packets in a multi-dimensional computer network. The method comprises routing a data packet among nodes along one dimension towards a root node, each node having input and output communication links, said root node not having any outgoing uplinks, and determining at each node if the data packet has reached a predefined coordinate for the dimension or an edge of the subrectangle for the dimension, and if the data packet has reached the predefined coordinate for the dimension or the edge of the subrectangle for the dimension, determining if the data packet has reached the root node, and if the data packet has not reached the root node, routing the data packet among nodes along another dimension towards the root node.

97 MATHEMATICS AND COMPUTING↗

Enhancing Drinking Water Quality Modeling: Leveraging Physics Informed Neural Networks for Learning with Imperfect Reaction Models and Partial Data

Chemical kinetics models, typically formulated as systems of ordinary or partial differential equations, are valuable tools for simulating drinking water quality. However, these models often face inaccuracies due to discrepancies between the laboratory and the real-world conditions, as well as limitations in experimental analytical methods, hindering the accurate representation of the true underlying chemical mechanisms. In this study, we propose a Physics Informed Neural Network (PINN), using the eXtreme Theory of Functional Connections, to improve the prediction of chemical concentrations over time. The PINN method accounts for imperfect chemical models and incorporates partial data to improve predictions. Focusing on reactions describing water disinfection residual and disinfectant byproduct formation, which are crucial for public health and regulatory compliance, we demonstrate that the PINN model is able to accurately predict the concentrations of chemical species across various pH values. Notably, the model extends its accuracy to predict concentrations of chemical species not originally included in its training data. The developed method can be extended to a variety of chemical systems, offering a wide array of potential applications.

13 HYDRO ENERGY↗

Multiscale Data-Driven Seismic Full-Waveform Inversion With Field Data Study

Seismic full-waveform inversion (FWI), which uses iterative methods to estimate high-resolution subsurface models from seismograms, is a powerful imaging technique in exploration geophysics. In recent years, the computational cost of FWI has grown exponentially due to the increasing size and resolution of seismic data. Moreover, it is a nonconvex problem and can encounter local minima due to the limited accuracy of the initial velocity models or the absence of low frequencies in the measurements. To overcome these computational issues, we develop a multiscale data-driven FWI method based on fully convolutional networks (FCNs). In preparing the training data, we first develop a real-time style transform method to create a large set of synthetic subsurface velocity models from natural images. We then develop two convolutional neural networks with encoder-decoder structures to reconstruct the low- and high-frequency components of the subsurface velocity models, separately. To validate the performance of our data-driven inversion method and the effectiveness of the synthesized training set, we compare it with conventional physics-based waveform inversion approaches using both synthetic and field data. Finally, these numerical results demonstrate that, once our model is fully trained, it can significantly reduce the computation time and yield more accurate subsurface velocity models in comparison with conventional FWI.

58 GEOSCIENCES↗

B/PS bulges in DESI Legacy edge-on galaxies – I. Sample building

ABSTRACT We present the biggest up-to-date sample of edge-on galaxies with boxy/peanut-shaped (B/PS) bulges and X-structures. The sample was prepared using images from the Dark Energy Spectroscopic Instrument (DESI) Legacy catalogue and contains about 2000 galaxies. To find suitable candidates in catalogue, we made the assumption that the residues (original images minus model) of galaxies with B/PS bulges should exhibit a characteristic X-shape. Galaxies with such features were selected by eye and then used as input data for a neural network training, which was applied to a bigger sample of edge-on galaxies. Using the available data and the photometric models from the literature, we investigated the observational and statistical properties of the sample created. Comparing the B/D ratios for galaxies with and without B/PS bulges, we found that the B/D ratio for galaxies from our sample is statistically higher, with typical values in the range ≈0.2–0.5 depending on the decomposition procedure. We studied how the opening angles φ of the X-structure and the length of its rays are distributed in the formed sample and found them to be consistent with previous measurements and predictions from N-body models, e.g. $\varphi \gtrsim 25{^\circ }$, but measured here for a much larger number of galaxies. We found a sharp increase in the B/PS bulge fraction for stellar masses log M⋆ ≳ 10.4, but for edge-on galaxies, which complements the results of previous works. The sample can be used in future work to test various bar models and their relationship with B/PS bulges, as well as to study their stability and evolution.

79 ASTRONOMY AND ASTROPHYSICS↗

Hybrid PDES Simulation of HPC Networks Using Zombie Packets

Although high-fidelity network simulations have proven to be reliable and cost-effective tools to peer into architectural questions for high-performance computing (HPC) networks, they incur a high resource cost. The time spent in simulating a single millisecond of network traffic in the highest detail can take hours, even for static, well-behaved traffic patterns such as uniform random. Surrogate models offer a significant reduction in runtime, yet they cannot serve as complete replacements and should only be used when appropriate. Thus, there is a need for hybrid modeling, where high-fidelity simulation and surrogates run side-by-side. Here, we present a surrogate model for HPC networks in which: packets bypass the network, while the network state is left untouched, i.e., suspended. To bypass the network, we use historical data to estimate the arrival time at which every packet should be scheduled at; to suspend the network, all in-flight packets are scheduled to arrive at their destinations, and are kept in the system to awaken as zombies when switching back to high-fidelity. Speedup for a hybrid model is relative to the proportion of surrogate to high-fidelity. This light-weight surrogate obtained up to 76× speedup. Keeping the zombies in the network showed an increase in the accuracy of the high-fidelity simulation on restart when compared to restarting the network from an empty state.

HPC networks↗