Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data transfer node”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

75 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

SMALE: Enhancing Scalability of Machine Learning Algorithms on Extreme-Scale Computing Platforms

Deployment and execution of machine learning tasks on extreme-scale computing platforms face several significant technical challenges: 1) High computing cost incurred by dense networks – The computing workload of deep networks with densely-connected topology increases rapidly with the network size, imposing a non-scalable computing model of extreme-scale computing platforms; 2) Non-optimized workload distribution – Many advanced deep learning algorithms, e.g., sparsification and irregular net-work topology, produce very unbalanced workload distribution on extreme-scale computing platforms. The computation efficiency is greatly hindered by the incurred data and computation redundancies as well as long tails of the node with extensive workload; 3) Constraints in data movement and I/O bottle-neck – Inter-node data movement in extreme-scale computing platforms are associated with high energy and latency costs, and subject to the constraints of I/O bandwidth; and 4) Generalization of algorithm realization and acceleration on computing platforms – The large varieties of machine learning algorithms and structures of extreme-scale computing platforms make the derivation of a generalized algorithm realization and acceleration method very challenging, which, however, is the requirement by domain scientists and interested users. We call the above challenges Smale’s Problems in Machine Learning and Understanding for High-Performance Computing Scientific Discovery. The objective of our three-year research project is to develop a holistic innovation set at structure, assembly, and acceleration layers of machine learning algorithms to address the above challenges in algorithm deployment and execution. Three tasks are particularly performed, including: At the algorithm structure level, we investigate the techniques that can structurally sparsify on the topology of deep networks for computing workload reduction. We also study clustering and pruning techniques that can optimize the workload distributions over the extreme-scale computing platforms; At the algorithm assembly level, we derive a unified learning framework for unsupervised transfer learning and dynamic growing capabilities. Novel training methods are also exploited to enhance the training efficiency of the proposed framework; At the algorithm acceleration level, we will develop a series of techniques that can accelerate the computation of sparse matrix operations, which are one of the core executions in deep learning and optimize memory access of the concerned platforms. Our proposed techniques attack the fundamental problems in machine learning algorithms running on extreme-scale computing platforms by vertically integrating the solutions at three closely entangled layers, paving the long-term scaling path of machine learning applications under DOE context. Three tasks corresponding to the above respective research orientations are performed during the three-year project period with our collaborators at ORNL. The outcome of the proposed project is anticipated to form a holistic solution set of novel algorithms and network topologies, efficient training techniques, and fast acceleration methods to promote the computing scalability of the machine learning applications of particular interest to DOE.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

LLNL Kimberlina 1.2 NUFT Simulations June 2018 (v2)

This dataset contains the output 6,000, 3-dimensional reactive multi-phase flow and transport aquifer simulations of brine and CO2 leakage into a protective aquiver in California’s San Joaquin Valley and input data files detailing the geologic mesh, aquifer physical properties and CO2 and brine injection rates. This data set was generated as an ongoing effort with the US DOE National Risk Assessment Partnership (NRAP) to evaluate the effectiveness of monitoring techniques to detect brine and CO2 leakage from legacy wells into underground sources of drinking water overlaying a CO2 storage reservoir. Each simulation contains a unique set of input parameters, generated stochastically. The outputs consist of these upper three geologic layers (from top): the Etchegoin, Macoma-Chanac, Santa Margarita-McLure formations. These simulations span the several distances (1, 3 and 6 km or wells W31-0.2, W31-0.5 and W31-1.0, respectively) from the CO2 injector, initiated from bottom hole pressure and saturation to calculate wellbore leakage from the storage reservoir, with low and high regional groundwater gradients and wellbore leakage into 5 leaky nodes. The dataset includes 1,000 unique simulations for each distance, which each contain a unique aquifer heterogeneity, aquifer and caprock permeability, and two model generations are included with a high permeability (prod07) and hybrid permeability (prod09). The range of permeability distributions is listed in Table 1. Each model generation consists of 3,000 simulations. Included in the dataset are the leakage rates determined from 2D wellbore models which utilize the pressure and CO2 saturation from LBL's reservoir simulations, NUFT mesh files with distributed lithology, NUFT rocktab files which describe the material properties for the geologic layers and the NUFT input files and post-processed output 'ntab' files. Each ntab file contains spatial (rows) and temporal (columns) model output tables for each model cell, the locations (x,y,z) and dimensions for each cells (dx, dy, dz). Table 1. Permeability distribution ranges for prod07 and prod09 model generations Geologic Layer: Permeability Range (log10 m^2) prod07 prod09 Etchegoin -12.92 to -10.92 -13.70 to -11.44 Macoma-Chanac -12.72 to -10.72 -13.50 to -11.24 Santa Margarita-McLure -12.70 to -10.70 -13.48 to -11.22 The input files used to generate the model include which are included in the dataset are: Time series of CO2 leakage input into the model (ex: Q_brn.W31-0.2.sim1000.layers123.tab) Time series of CO2 leakage input into the model (ex: Q_CO2.W31-0.2.sim1000.layers123.tab) Physical properties of the aquifer materials detailing the aquifer porosity, solid density, partitioning coefficients, permeabilities and van-Genuchten parameters detailed in a NUFT rocktab file: (ex: sim1000.usnt.rocktab) Numerical mesh and geologic data assigned to each model cell detailed in a NUFT genmsh format (ex: sim1000.mesh_k16.prod07.trans.genmsh) The primary output parameters are: pH (use absolute value) Change in TDS (mg/kg) Change in Pressure (Pa) Change CO2 gas saturation (fraction range 0.0-1.0) for example, the directory /p/lscratchh/mansoor1/nrap/kimberlina/prod09/mainfiles/sim1000/W31- 0.2 contains: sim1000.W31-0.2.trans.pH.red.ntab sim1000.W31-0.2.no_bg.trans.TDS.red.ntab sim1000.W31-0.2.usnt.P.deltabg.red.ntab sim1000.W31-0.2.usnt.CO2_sat.deltabg.red.ntab Each row in the NTAB files consist of model output per numerical grid cell. Each output file contains 33 columns (variables), including the information of numerical records, geologic location and sizes and the simulated parameter values over time. The first 13 variables are about numerical records and relative geologic information for a simulation grid: 1. index: simulation index 2. i: the ith grid of x-axis 3. j: the ith grid of y-axis 4. k: the ith grid of z-axis 5. element_ref: element reference 6. nuft_ind: nuft index 7. x: grid location in the x axis direction 8. y: grid location in the y axis direction 9. z: grid location in the z axis direction 10. dx: grid length in the x axis direction 11. dy: grid length in the y axis direction 12. dz: grid length in the z axis direction 13. volume: volume of the simulation grid The remainder (14, 15, 16...) variables are the simulated parameter values over time, take Pressure as an example, are: 14. 0.0y: initial pressure per cell. 15. 10.0y: simulated pressure at the end of the 10th year. 16. 20.0y: simulated pressure at the end of the 20th year. ... (repeated for every 10 years until 200 years)... The model extends 10,000 m, 5,000 m and 1,411 m in the x,y and z dimensions, respectively. The mesh consists of 164,832 cells with mesh dimensions of 101 x 51 x 32 (nx, ny, nz), with cell dimensions ranging from 100 m laterally (along x and y-axis) and model layers are as designated in the z-axis: Layer 1: atmosphere (1e-30 m thick) Layer 2: upper caprock (10 m thick) Layers 3-13: Etchegoin (536.23 m thck) Layers 14-27: Macoma-Chanac (679.04 m thick) Layers 28-32: Santa Margarita-McLure (185.94 m thick) The wellbore is placed along node i=51, j=26, and extends vertically along 5 nodes from the top to the bottom of the model. Special instructions when extracting files: Each Gzip archive (ex: prod07.sim1000-sim00099.tar.gz) contains 100 simulations. Gzip archives should be transferred into base directories (ie. In Linux: mkdir prod07; mv prod07.*.tar.gz prod07/.) before extracting, or files will be overwritten. Each sub-simulation tree should have the following file structure pattern (using the linux 'tree' command): |-- prod07 | |-- sim0001 | |-- W31-0.2 | | |-- Q_brn.W31-0.2.sim0001.layers123.tab | | |-- Q_co2.W31-0.2.sim0001.layers123.tab | | |-- sim0001.W31-0.2.no_bg.trans.TDS.red.ntab | | |-- sim0001.W31-0.2.trans.pH.red.ntab | | |-- sim0001.W31-0.2.usnt.CO2_sat.deltabg.red.ntab | | |-- sim0001.W31-0.2.usnt.P.deltabg.red.ntab | |-- W31-0.5 | | |-- Q_brn.W31-0.5.sim0001.layers123.tab | | |-- Q_co2.W31-0.5.sim0001.layers123.tab | | |-- sim0001.W31-0.5.no_bg.trans.TDS.red.ntab | | |-- sim0001.W31-0.5.trans.pH.red.ntab | | |-- sim0001.W31-0.5.usnt.CO2_sat.deltabg.red.ntab | | |-- sim0001.W31-0.5.usnt.P.deltabg.red.ntab | |-- W31-1.0 | | |-- Q_brn.W31-1.0.sim0001.layers123.tab | | |-- Q_co2.W31-1.0.sim0001.layers123.tab | | |-- sim0001.W31-1.0.no_bg.trans.TDS.red.ntab | | |-- sim0001.W31-1.0.trans.pH.red.ntab | | |-- sim0001.W31-1.0.usnt.CO2_sat.deltabg.red.ntab | | |-- sim0001.W31-1.0.usnt.P.deltabg.red.ntab | |-- sim0001.mesh_k16.prod07.trans.genmsh Disclaimer This document was prepared as an account of work sponsored by an agency of the United States government. Neither the United States government nor Lawrence Livermore National Security, LLC, nor any of their employees makes any warranty, expressed or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States government or Lawrence Livermore National Security, LLC. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States government or Lawrence Livermore National Security, LLC, and shall not be used for advertising or product endorsement purposes. Lawrence Livermore National Laboratory is operated by Lawrence Livermore National Security, LLC, for the U.S. Department of Energy, National Nuclear Security Administration under Contract DE-AC52-07NA27344. This report was reviewed and released as LLNL-MI-753464.

aquifer↗
DOE OSTI2020

Description of Sensor Assignment Optimization Method as Deployed on a Multi-Node Cluster

Data analytic methods are being developed to address the problem of how to assign a sensor set in a nuclear facility such that a requisite level of process monitoring capability is realized and that the sensor set is sufficiently rich to determine the status of the individual sensors with respect to need for calibration. There is an awareness in the nuclear industry that data analytics combined with rich sensor sets represent a means to improve operations and reduce costs. In the industry the calibration problem has been previously approached as an empirical data-driven problem with several methods having been developed. However, the experience of the utilities over the past ten years with these methods indicates that the absence of physics-based information renders the data-driven approach less reliable. Complicating factors such as the inherent variability of operation (both equipment alignment and operating condition) can confound a pure data-driven approach while there are no rigorous guidelines for determining what constitutes an adequate sensor set. The solution under development to overcome these shortcomings supplements the data analytic method with process information in a so-called process-constrained data-analytic approach. Simple balance equations are written for generic components (e.g., mechanical pump, valve, and heat exchanger). These do not require a priori knowledge of process parameters, such as heat transfer coefficients or friction factors. All that is needed on the part of the utility user is to identify the components and how they are connected. This report describes the development of a parallel computing capability for determining the optimal sensor set. The optimal sensor set problem suffers from the curse of dimensionality. Computation time increases exponentially as the size of the system grows. To overcome this difficulty a pre-conditioner algorithm is developed to find an approximate solution close the actual solution. This serves as a seed for the full-blown algorithm and acts to constrain the space that must searched. The optimization algorithms are described and the implementation on a parallel computing platform is described. The application of the method to a use case we are solving in collaboration with our utility partner served to illustrate how the default sensor set in a nuclear plant may not provide sufficient coverage to infer sensor calibration status.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗