Search NASA⌕ Search

SEARCH · Search NASA

Results for “network data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,045 records · Page 58

Large Scale Caching and Streaming of Training Data for Online Deep Learning

The training of deep neural network models on large data remains a difficult problem, despite progress towards scalable techniques. In particular, there is a mismatch between the random but predetermined order in which AI flows select training samples and the streaming I/O patterns for which traditional HPC data storage (e.g., parallel file systems) are designed. In addition, as more data are obtained, it is feasible neither simply to train learning models incrementally, due to catastrophic forgetting (i.e., bias towards new samples), nor to train frequently from scratch, due to prohibitive time and/or resource constraints. In this paper, we study data management techniques that combine caching and streaming with rehearsal support in order to enable efficient access to training samples in both offline training and continual learning. We revisit state-of-art streaming approaches based on data pipelines that transparently handle prefetching, caching, shuffling, and data augmentation, and discuss the challenges and opportunities that arise when combining these methods with data-parallel training techniques. We also report on preliminary experiments that evaluate the I/O overheads involved in accessing the training samples from a parallel file system (PFS) under several concurrency scenarios, highlighting the impact of the PFS on the design of the data pipelines.

data pipelines↗

LBNL DDoS Detection on Science Networks v0.1

This software contains a set of analysis tools to identify certain types of DDoS attacks and disambiguate those attacks from large scientific data transfers over networks as well as ordinary Internet network traffic. It also contains a piece of "driver" code that allows individual scripts that detect certain types of attacks to selectively be run.

Peisert, SeanP.↗

Data and Code for: Observation-constrained agroecosystem model inversion reveals continental-scale variation of winter wheat traits

This repository contains the simulation outputs and processing scripts associated with the study of winter wheat traits across the United States, utilizing the Ecosys agroecosystem model. The dataset includes model results for both rainfed and irrigated winter wheat systems, supporting the findings presented in the manuscript titled "Observation-constrained agroecosystem model inversion reveals continental-scale variation of winter wheat traits." Data includes the original Ecosys simulation outputs (archived in .db format within the compressed .zip files) and extracted analysis data (stored in .pkl files for efficient processing). Python code for data processing and figure generation is provided in a Jupyter notebook. External Observational Datasets should refer to the following official repositories for the input and validation data used in this study. The eddy covariance data from the AmeriFlux network (https://ameriflux.lbl.gov/). Climate-forcing data of NLDAS-2 from NASA LDAS (https://ldas.gsfc.nasa.gov/nldas/nldas-2-forcing-data). Soil data from the Gridded Soil Survey Geographic Database (gSSURGO), available at (https://www.nrcs.usda.gov/resources/data-and-reports/gridded-soil-survey-geographic-gssurgo-database). Crop yields, planting and harvest dates from the USDA public databases (https://quickstats.nass.usda.gov/; https://webapp.rma.usda.gov/apps/actuarialinformationbrowser/CropCriteria.aspx). Satellite-derived SLOPE GPP data from ORNL DAAC (https://daac.ornl.gov/cgi-bin/dsviewer.pl?ds_id=1786). Land use and crop progress information from the USDA Crop Data Layer and Crop Progress and Condition Gridded Layers (https://www.nass.usda.gov/Research_and_Science/). The Ecosys model code is available online at https://github.com/jinyun1tang/ECOSYS.

Wheat↗

Seismic Analysis of Spatio-Temporal Fracture Generation During EGS Resource Development

This DOE GTO funded project was a collaborative effort between Array Information Technology and the University of California at Berkeley. The project goal is to improve technology to assess in-situ stress magnitude and orientation, kinematic fracture parameters, rupture size, rupture orientation as well as temporal changes and volumetric distribution of the injected fluid during Enhanced Geothermal System (EGS) resource development. The project leveraged high-frequency seismic data recorded by the LBNL 34-station permanent geophone network and seismic broadband data recorded by a temporary 33-station seismometer network that operated in The Geysers during the injection phase of the DOE GTO funded EGS demonstration project at the well Prati-32. The operation of the broadband network with high station density and good azimuthal coverage to monitor the injection phase produced an unprecedented dataset that is typically unavailable for EGS operations. The two datasets offer the advantages of studying the parameters under investigation over a broader frequency band and the higher sensitivity of the broadband seismometers resulted in a richer dataset with a lower magnitude of completeness. Moment tensor (MT) inversion have been employed to develop a catalog of earthquake source solutions including moment tensor and finite source rupture area. This resulted in a 168-event waveform-based seismic moment tensor catalog for events 0.7≤Mw≤3.9 based on a semi-automated moment tensor analysis technique. The MT solutions were used for in-situ stress estimation during the EGS injection phases. We found an approximate 15-degree counterclockwise rotation of the least compressive stress sigma-3, and a rotation of the maximum compressive stress sigma-1 toward the vertical as the injected volume of water increased. We developed a rupture-area magnitude scaling relationship of The Geysers earthquakes obtained from finite-source inversion for fault slip that reveals a very high correlation to published results. Based on the rupture area-magnitude relationship we mapped Mw from the LBNL earthquake catalog to estimate the activated fracture area, its location and orientation within the injection volume of Prati-32. The double-difference Wadati (DDW) technique was applied to high-accuracy P- and S-wave differential travel times derived from waveform cross correlation to image fluid saturation in the subsurface based on Vp/Vs-ratio. The recently developed technique has been applied to estimate temporal changes of fluid distribution in fault zones and in volcanic regions but not yet in engineered geo-reservoirs. The improvement of the technology includes high-resolution waveform cross correlation applied to the high-quality datasets with lower magnitude of completeness resulting in a higher number of recorded events. The improvement allowed to estimate both, the spatial and temporal distribution of injected fluids and the state of the injected fluid during reservoir stimulation. Results of the DDW analysis revealed the development of an injection-derived steam plume during the early injection phase, particularly in the northern part of the Prati-32 study area.

15 GEOTHERMAL ENERGY↗

Hybrid Machine Learning for Scanning Near-Field Optical Spectroscopy

The underlying physics behind an experimental observation often lacks a simple analytical description. This is especially the case for scanning probe microscopy techniques, where the interaction between the probe and the sample is nontrivial. Realistic modeling to include the details of the probe is always exponentially more difficult than its "spherical cow" counterparts. On the other hand, a well-trained artificial neural network based on real data can grasp the hidden correlation between the signal and sample properties. In this work, we show that, via a combination of model calculation and experimental data acquisition, a physics-infused hybrid neural network can predict the tip-sample interaction in the widely used scattering-type scanning near-field optical microscope. This hybrid network provides a long-sought solution for accurate extraction of material properties from tip-specific raw data. The methodology can be extended to other scanning probe microscopy techniques as well as other data-oriented physical problems in general.

36 MATERIALS SCIENCE↗

Scalable Memory Registration for High-Performance Networks Using Helper Threads

Remote DMA (RDMA) enables high performance networks to reduce data copying between an application and the operating system (OS). However RDMA operations in some high performance networks require communication memory explicitly registered with the network adapter and pinned by the OS. Memory registration and pinning limits the flexibility of the memory system and reduces the amount of memory that user processes can allocate. These issues become more significant on multicore platforms, since registered memory demand grows linearly with the number of processor cores. In this paper we propose a new memory registration/deregistration strategy to reduce registered memory on multicore architectures for HPC applications. We hide the cost of dynamic memory management by offloading all dynamic memory registration and deregistration requests to a dedicated memory management helper thread. We investigate design policies and performance implications of the helper thread approach. We evaluate our framework with the NAS parallel benchmarks, for which our registration scheme significantly reduces the registered memory (23.62% on average and up to 49.39%) and avoids memory registration/deregistration costs for reused communication memory. We show that our system enables the execution of problem sizes that could not complete under existing memory registration strategies.

99 GENERAL AND MISCELLANEOUS↗

Implementation Aspects of Smart Grids Cyber-Security Cross-Layered Framework for Critical Infrastructure Operation

Communication networks in power systems are a major part of the smart grid paradigm. It enables and facilitates the automation of power grid operation as well as self-healing in contingencies. Such dependencies on communication networks, though, create a roam for cyber-threats. An adversary can launch an attack on the communication network, which in turn reflects on power grid operation. Attacks could be in the form of false data injection into system measurements, flooding the communication channels with unnecessary data, or intercepting messages. Using machine learning-based processing on data gathered from communication networks and the power grid is a promising solution for detecting cyber threats. In this paper, a co-simulation of cyber-security for cross-layer strategy is presented. The advantage of such a framework is the augmentation of valuable data that enhances the detection as well as identification of anomalies in the operation of the power grid. The framework is implemented on the IEEE 118-bus system. The system is constructed in Mininet to simulate a communication network and obtain data for analysis. A distributed three controller software-defined networking (SDN) framework is proposed that utilizes the Open Network Operating System (ONOS) cluster. According to the findings of our suggested architecture, it outperforms a single SDN controller framework by a factor of more than ten times the throughput. This provides for a higher flow of data throughout the network while decreasing congestion caused by a single controller’s processing restrictions. Furthermore, our CECD-AS approach outperforms state-of-the-art physics and machine learning-based techniques in terms of attack classification. The performance of the framework is investigated under various types of communication attacks.

cross-layered↗

End-to-End Network QoS via Scheduling of Flexible Resource Reservation Requests

Modern data-intensive applications move vast amounts of data between multiple locations around the world. To enable predictable and reliable data transfer, next generation networks allow such applications to reserve network resources for exclusive use. In this paper, we solve an important problem (called SMR3) to accommodate multiple and concurrent network reservation requests between a pair of end-sites. Given the varying availability of bandwidth within the network, our goal is to accommodate as many reservation requests as possible while minimizing the total time needed to complete the data transfers. We first prove that SMR3 is an NP-hard problem. Then we solve it by developing a polynomial-time heuristic, called RRA. The RRA algorithm hinges on an efficient mechanism to accommodate large number of requests by minimizing the bandwidth wastage. Finally, via numerical results, we show that RRA constructs schedules that accommodate significantly larger number of requests compared to other, seemingly efficient, heuristics.

97 MATHEMATICS AND COMPUTING↗

Meteorological Tower Network Expansion Pilot Project Siting Considerations to Ensure Data Quality and Representativeness of Three New Meteorological Towers at Los Alamos National Laboratory

Associated with the Meteorological Tower Network Expansion Pilot Project by the Environmental Protection and Compliance (EPC) Compliance Programs (CP) Meteorology and Air Quality (MAQ) team, the purpose of this technical report is to establish requisite justification for the siting of three 10-meter high instrumented meteorological towers at three new locations on the Los Alamos National Laboratory (LANL) site. These towers and their associated instrumentation are similar in scope to the existing 10-meter meteorological tower that is collecting meteorological data in Mortandad Canyon (MDCN). The conclusions and recommendations in this technical report will inform LANL management, EPC-CP MAQ staff, and other LANL divisions of the criteria employed for the selection of the final location for siting these three new towers. Meeting such comprehensive criteria will ensure that all data acquired will be spatially representative for meeting the diverse Safety Basis (SB), emergency management, and operations organizations programmatic needs and analytical data input requirements to its codes.

54 ENVIRONMENTAL SCIENCES↗

SU-E-T-23: A Developing Australian Network for Datamining and Modelling Routine Radiotherapy Clinical Data and Radiomics Information for Rapid Learning and Clinical Decision Support

Purpose: Large amounts of routine radiotherapy (RT) data are available, which can potentially add clinical evidence to support better decisions. A developing collaborative Australian network, with a leading European partner, aims to validate, implement and extend European predictive models (PMs) for Australian practice and assess their impact on future patient decisions. Wider objectives include: developing multi-institutional rapid learning, using distributed learning approaches; and assessing and incorporating radiomics information into PMs. Methods: Two initial standalone pilots were conducted; one on NSCLC, the other on larynx, patient datasets in two different centres. Open-source rapid learning systems were installed, for data extraction and mining to collect relevant clinical parameters from the centres’ databases. The European DSSs were learned (“training cohort”) and validated against local data sets (“clinical cohort”). Further NSCLC studies are underway in three more centres to pilot a wider distributed learning network. Initial radiomics work is underway. Results: For the NSCLC pilot, 159/419 patient datasets were identified meeting the PM criteria, and hence eligible for inclusion in the curative clinical cohort (for the larynx pilot, 109/125). Some missing data were imputed using Bayesian methods. For both, the European PMs successfully predicted prognosis groups, but with some differences in practice reflected. For example, the PM-predicted good prognosis NSCLC group was differentiated from a combined medium/poor prognosis group (2YOS 69% vs. 27%, p<0.001). Stage was less discriminatory in identifying prognostic groups. In the good prognosis group two-year overall survival was 65% in curatively and 18% in palliatively treated patients. Conclusion: The technical infrastructure and basic European PMs support prognosis prediction for these Australian patient groups, showing promise for supporting future personalized treatment decisions, improved treatment quality and potential practice changes. The early indications from the distributed learning and radiomics pilots strengthen this. Improved routine patient data quality should strengthen such rapid learning systems.

60 APPLIED LIFE SCIENCES↗

Condensed Phase Water Molecular Multipole Moments from Deep Neural Network Models Trained on Ab Initio Simulation Data

Ionic solvation phenomena in liquids involve intense interactions in the inner solvation shell. For interactions beyond the first shell, the ion–solvent interaction energies result from the sum of many smaller-magnitude contributions that can still include polarization effects. Deep neural network (DNN) methods have recently found wide application in developing efficient molecular models that maintain near-quantum accuracy. In this work, we extend the DeePMD-kit code to produce accurate molecular multipole moments in the bulk and near interfaces. The new method is validated by comparing the DNN moments with those generated by ab initio simulations. The moments are used to compute the electrostatic potential at the center of a molecular-sized hydrophobic cavity in water. The results show that the fields produced by the DNN models are in quantitative agreement with the AIMD-derived values. These efficient methods will open the door to more accurate solvation models for large solutes such as proteins.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Compact Graph Representations and Parallel Connectivity Algorithms for Massive Dynamic Network Analysis

Graph-theoretic abstractions are extensively used to analyze massive data sets. Temporal data streams from socioeconomic interactions, social networking web sites, communication traffic, and scientific computing can be intuitively modeled as graphs. We present the first study of novel high-performance combinatorial techniques for analyzing large-scale information networks, encapsulating dynamic interaction data in the order of billions of entities. We present new data structures to represent dynamic interaction networks, and discuss algorithms for processing parallel insertions and deletions of edges in small-world networks. With these new approaches, we achieve an average performance rate of 25 million structural updates per second and a parallel speedup of nearly28 on a 64-way Sun UltraSPARC T2 multicore processor, for insertions and deletions to a small-world network of 33.5 million vertices and 268 million edges. We also design parallel implementations of fundamental dynamic graph kernels related to connectivity and centrality queries. Our implementations are freely distributed as part of the open-source SNAP (Small-world Network Analysis and Partitioning) complex network analysis framework.

97 MATHEMATICS AND COMPUTING↗

AN ARTIFICIAL NEURAL NETWORK EVALUATION OF TUBERCULOSIS USING GENETIC AND PHYSIOLOGICAL PATIENT DATA

When doctors see more cases of patients with tell-tale symptoms of a disease, it is hoped that they will be able to recognize an infection administer treatment appropriately, thereby speeding up recovery for sick patients. We hope that our studies can aid in the detection of tuberculosis by using a computer model called an artificial neural network. Our model looks at patients with and without tuberculosis (TB). The data that the neural network examined came from the following: patient' age, gender, place, of birth, blood type, Rhesus (Rh) factor, and genes of the human Leukocyte Antigens (HLA) system (9q34.1) present in the Major Histocompatibility Complex. With availability in genetic data and good research, we hope to give them an advantage in the detection of tuberculosis. We try to mimic the doctor's experience with a computer test, which will learn from patient data the factors that contribute to TB.

60 APPLIED LIFE SCIENCES↗

Search for $t\bar{t}H/A \rightarrow t\bar{t}t\bar{t}$ production in proton–proton collisions at $\sqrt{s}=13$ $\text {TeV}$ with the ATLAS detector

A search is presented for a heavy scalar (H) or pseudo-scalar (A) predicted by the two-Higgs-doublet models, where the H/A is produced in association with a top-quark pair $(t\bar{t}H/A),$ and with the H/A decaying into a $t\bar{t}$ pair. The full LHC Run 2 proton–proton collision data collected by the ATLAS experiment is used, corresponding to an integrated luminosity of $139~\text {fb}^{-1}.$ Events are selected requiring exactly one or two opposite-charge electrons or muons. Data-driven corrections are applied to improve the modelling of the $t\bar{t}$ +jets background in the regime with high jet and b-jet multiplicities. These include a novel multi-dimensional kinematic reweighting based on a neural network trained using data and simulations. An H/A-mass parameterised graph neural network is trained to optimise the signal-to-background discrimination. In combination with the previous search performed by the ATLAS Collaboration in the multilepton final state, the observed upper limits on the $t\bar{t}H/A \rightarrow t\bar{t}t\bar{t}$ production cross-section at 95% confidence level range between 14 fb and 5.0 fb for an H/A with mass between 400 GeV and 1000 GeV , respectively. Assuming that both the H and A contribute to the $t\bar{t}t\bar{t}$ cross-section, tan β values below 1.7 or 0.7 are excluded for a mass of 400 GeV or 1000 GeV , respectively. The results are also used to constrain a model predicting the pair production of a colour-octet scalar, with the scalar decaying into a $t\bar{t}$ pair.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Improving Quality of Observational Streaming Medical Data by Using Long Short-Term Memory Networks (LSTMs)

We present an exploration of the encoder-decoder structured Long Short-Term Memory Network (LSTM) as a detector of the anomalous missing observations in streaming medical data by using the difference between the LSTM-reconstructed and observed values as the anomaly detector. We experiment with time-series data from bedside monitoring devices from the available Medical Information Mart for Intensive Care Database (MIMIC). Our results show that not only encoder-decoder LSTM approach works well for detecting the difference between anomalous and normal missing observations in streaming medical data, but also has an imputation potential for the missing data.

Bowie, Michael B.↗

Detailed Statistical Models of Host-Based Data for Detection of Malicious Activity

The cybersecurity research community has focused primarily on the analysis and automation of intrusion detection systems by examining network traffic behaviors. Expanding on this expertise, advanced cyber defense analysis is turning to host-based data to use in research and development to produce the next generation network defense tools. The ability to perform deep packet inspection of network traffic is increasingly harder with most boundary network traffic moving to HTTPS. Additionally, network data alone does not provide a full picture of end-to-end activity. These are some of the reasons that necessitate looking at other data sources such as host data. We outline our investigation into the processing, formatting, and storing of the data along with the preliminary results from our exploratory data analysis. In writing this report, it is our goal to aid in guiding future research by providing foundational understanding for an area of cybersecurity that is rich with a variety of complex, categorical, and sparse data, with a strong human influence component. Including suggestions for guiding potential directions for future research.

97 MATHEMATICS AND COMPUTING↗

Non-unimodal and non-concave relationships in the network Macroscopic Fundamental Diagram caused by hierarchical streets

Unimodal, concave relationships between average network productivity and accumulation or density aggregated across spatially compact regions of urban networks—so called network Macroscopic Fundamental Diagrams (MFDs)—have recently been shown to exist on homogeneous street networks. When present, MFD relationships facilitate the modeling of traffic congestion at a regional level and have led to the development of various regional traffic control strategies. However, real street networks are not homogeneous—they generally have a hierarchical structure where some streets (e.g., arterials) promote higher mobility than others (e.g., local roads). Here, this paper examines how the presence of hierarchical roadway structures may potentially cause non-unimodal patterns in a network's MFD. These are observed using three types of tools: analytical models of simple network structures, simulations of various idealized roadway networks, and empirical data. The impacts of street hierarchy depend on how vehicles use different roadway types to move within the network; i.e., their routing strategy. The findings suggest that the presence of roadway hierarchies may lead to MFDs that have non-unimodal or non-concave patterns on the free-flow branch when vehicles route themselves according to user equilibrium principles, which is closest to what would be observed in realistic situations. Such patterns are contrary to what is traditionally assumed in most MFD-based modeling frameworks. However, the unimodal and concave MFD should be expected under system optimal routing conditions that maximize network productivity for a given traffic state.

42 ENGINEERING↗

Spatially explicit network analysis reveals multi‐species annual cycle movement patterns of sea ducks

Abstract Conservation of long‐distance migratory species poses unique challenges. Migratory connectivity, that is, the extent to which groupings of individuals at breeding sites are maintained in wintering areas, is frequently used to evaluate population structure and assess use of key habitat areas. However, for species with complex or variable annual cycle movements, this traditional bimodal framework of migratory connectivity may be overly simplistic. Like many other waterfowl, sea ducks often travel to specific pre‐ and post‐breeding sites outside their nesting and wintering areas to prepare for migration by feeding extensively and, in some cases, molting their flight feathers. These additional migrations may play a key role in population structure, but are not included in traditional models of migratory connectivity. Network analysis, which applies graph theory to assess linkages between discrete locations or entities, offers a powerful tool for quantitatively assessing the contributions of different sites used throughout the annual cycle to complex spatial networks. We collected satellite telemetry data on annual cycle movements of 672 individual sea ducks of five species from throughout eastern North America and the Great Lakes. From these data, we constructed a multi‐species network model of migratory patterns and site use over the course of breeding, molting, wintering, and migratory staging. Our results highlight inter‐ and intra‐specific differences in the patterns and complexity of annual cycle movement patterns, including the central importance of staging and molting sites in James Bay, the St. Lawrence River, and southern New England to multi‐species annual cycle habitat linkages, and highlight the value of Long‐tailed Ducks ( Calengula haemalis ) as an umbrella species to represent the movement patterns of multiple sea duck species. We also discuss potential applications of network migration models to conservation prioritization, identification of population units, and integrating different data streams.

54 ENVIRONMENTAL SCIENCES↗