Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data analytics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 919 records · Page 51

Accelerating the rate of discovery: toward high-repetition-rate HED science

As high-intensity short-pulse lasers that can operate at high-repetition-rate (HRR) (>10 H z ) come online around the world, the high energy density (HED) science they enable will experience a radical paradigm shift. The >10 3 increase in shot rate over today's shot-per-hour drivers translates into dramatically faster data acquisition, more experiments, and the ability to exploit machine learning, and thus the potential to significantly accelerate the advancement of HED science. A wide range of HED experiments, from opacity investigations to secondary source generation to plasma nuclear physics, will benefit from the increased statistics, precision, and exploration of phase space. Besides increasing the rate at which scientific experiments can be performed, HRR also allows for the rapid delivery of optimal experiments supported by simulations and modeling augmented by close coupling to empirical data. To fully realize such an HRR framework, numerous subsystems must be developed and brought together, including feedback laser control loops, high-throughput targetry and diagnostics, cognitive simulation, enhanced HED codes, and advanced data analytics. This paper describes the vision for an integrated HRR laser experimental HED system and outlines some of the major considerations and challenges for realizing it.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Defining the Synthetic Biology Supply Chain

In this article, a team of experts in synthetic biology, data analytics, and national security describe the overall supply chain surrounding synthetic biology. The team analyzes selected interactions within that network to better understand the risks raised by synthetic biology and identifies opportunities for risk mitigation. To introduce the concept, the article will briefly describe how an understanding of supply chains has been important in promoting nuclear nonproliferation objectives. The article concludes by assessing the structure and networks identified in the supply chains to reveal potential opportunities for future biodefense research and development; options for additional information exchange; and means to interdict, detect, or deter suspicious activity.

nuclear supply chain↗

Human-in-the-Loop: The Future of Machine Learning in Automated Electron Microscopy

Machine learning (ML) methods are progressively gaining acceptance in the electron microscopy community for de-noising, semantic segmentation, and dimensionality reduction of data post-acquisition. The introduction of the application programming interfaces (APIs) by major instrument manufacturers now allows the deployment of ML workflows in microscopes, not only for data analytics but also for real-time decision-making and feedback for microscope operation. However, the number of use cases for real-time ML remains remarkably small. Furthermore, we discuss some considerations in designing ML-based active experiments and pose that the likely strategy for the next several years will be human-in-the-loop automated experiments (hAE). In this paradigm, the ML learning agent directly controls beam position and image and spectroscopy acquisition functions, and a human operator monitors experiment progression in real and feature space of the system and tunes the policies of the ML agent to steer the experiment toward specific objectives.

47 OTHER INSTRUMENTATION↗

A Distribution Level Wide Area Monitoring System for the Electric Power Grid–FNET/GridEye

The wide area monitoring system (WAMS) is considered a pivotal component of future electric power grids. As a pilot WAMS that has been operated for more than a decade, the frequency monitoring network FNET/GridEye makes use of hundreds of global positioning system-synchronized phasor measurement sensors to capture the increasingly complicated grid behaviors across the interconnected power systems. In this paper, the FNET/GridEye system is overviewed and its operation experiences in electric power grid wide area monitoring are presented. Particularly, the implementation of a number of data analytics applications will be discussed in details. FNET/GridEye lays a firm foundation for the later WAMS operation in the electric power industry.

24 POWER TRANSMISSION AND DISTRIBUTION↗

EGBTER: Capturing Degree Distribution, Clustering Coefficients, and Community Structure in a Single Random Graph Model

Random graph models are important constructs for data analytic applications as well as pure mathematical developments, as they provide capabilities for network synthesis and principled analysis. Several models have been developed with the aim of faithfully preserving important graph metrics and substructures. With the goal of capturing degree distribution, clustering coefficient, and communities in a single random graph model, we propose a new model to address shortcomings in a progression of network modeling capabilities. The Block Two-Level Erdos-Rényi (BTER) model of Seshadhri et aI., designed to allow prescription of expected degree and clustering coefficient distributions, neglects community modeling, while the Generalized BTER (GBTER) model of Bridges et aI., designed to add community modeling capabilities to BTER, struggles to faithfully represent all three characteristics simultaneously. In this work, we fit BTER and two GBTER configurations to several real-world networks and compare the results with that of our new model, the Extended GBTER (EGBTER) model. Our results support that EBGTER adds a community-modeling flexibility to BTER, while retaining a satisfactory level of accuracy in terms of degree and clustering coefficient. Our insights and empirical testing of previous models as well as the new model are novel contributions to the literature.

El-Daghar, Omar↗

A Flexible-blocking Based Approach for Performance Tuning of Matrix Multiplication Routines for Large Matrices with Edge Cases

Efficient and scalable matrix operations are being highly demanding in the recent era of Machine Learning, Deep Learning, and Big Data Analytics. The two commonly used matrix-matrix operations in the Basic Linear Algebra Subprograms (BLAS) specification are General Matrix-Matrix multiplication (GEMM) and Symmetric Rank-k update (SYRK). The SYRK routine is a specialization of the GEMM routine, where half of the multiplications are skipped as the resultant matrix is known to be symmetric. Fortunately, several linear algebra libraries implement these BLAS routines quite efficiently. The libraries usually partition the input matrices into blocks and place them in processor caches, thus improving performance by leveraging the caches. However, the contemporary libraries are highly optimized for squarish matrices, but the performance degrades significantly for the matrices with edge case (strictly thin or strictly fat shapes) in the multicore machine. The primary reason is that the current state-of-the-art libraries make fixed block shapes based on a processor architecture, and do not consider the shape of the input matrices. In this paper, we propose a new blocking approach, we name it Flexible-blocking, to mitigate the scalability issues. In contrast to the contemporary libraries, our approach formulates the blocks of the input matrices based on the shapes of the matrices as well as the number of threads used in the implementation. Our proposed technique shows noticeable performance improvement on multicore shared-memory machines for the edge case matrices.

Hossain, Md Mosharaf↗

A Lakehouse Architecture for the Management and Analysis of Heterogeneous Data for Biomedical Research and Mega-biobanks

Data Lakehouse is a new paradigm in data architectures that embodies and integrates already established concepts for the systematic management of disparate, large-scale data – a data lake for heterogeneous data management, use of open standards for high-performance querying, and systematic maintenance of the data "freshness". In addition to being a new concept, the data lakehouse is also still a conceptual construct. Many projects that use the lakehouse require maturing, empirical studies, and specific implementations. In this paper, we present our implementation of the data lakehouse concept in a biomedical research and health data analytics domain, and we discuss the implementation of some unique and novel features such as support for specialized access controls in support of HIPAA regulation and IRB protocols, and support for the FAIR standard.

Begoli, Edmon↗

AnalyzeThat: A Programmable Shared-Memory System for an Array of Processing-In-Memory Devices

Processing In Memory (PIM), the concept of integrating processing directly with memory, has been attracting a lot of attention since PIM can assist in overcoming the throughput limitation caused by data movement between CPU and memory. The challenge, however, is that it requires the programmers to have a deep understanding of the PIM architecture to maximize the benefits such as data locality and parallel thread execution on multiple PIM devices. In this study, we present AnalyzeThat, a programmable shared-memory system for parallel data processing with PIM devices. Thematic to AnalyzeThat is a rich PIM-Aware Data Structure (PADS), which is an encapsulation that integrally ties together the data, the analysis tasks and the runtime needed to interface with the PIM device array. The PADS abstraction provides (i) a key-value data container that allows programmers to easily store data on multiple PIMs, (ii) a suite of parallel operations with which users can easily implement data analysis applications, and (iii) a runtime, hidden to programmers, which provides the mechanisms needed to overlay both the data and the tasks on the PIM device array in an intelligent fashion, based on PIM-specific information collected from the hardware. We have developed a PIM emulation framework called AnalyzeThat. Our experimental evaluation with representative data analytics applications suggests that the proposed system can significantly reduce the PIM programming effort without losing its technology benefits.

97 MATHEMATICS AND COMPUTING↗

Parallel I/O Evaluation Techniques and Emerging HPC Workloads: A Perspective

Emerging workloads such as artificial intelligence, big data analytics and complex multi-step workflows alongside future exascale applications are anticipated future HPC workloads, which will result in a more diverse I/O system workload and even less predictable I/O behavior and access patterns. Along with the ever increasing gap between the compute and storage performance capabilities, the in-depth understanding of extreme-scale I/O behavior and the I/O performance modeling and prediction are essential tools of the large-scale I/O evaluation process for addressing the needs of extreme-scale hybrid workloads. In this survey article, we focus on the state-of-the-art of the I/O behavior and performance analysis process for HPC systems in a 5-year time window and identify future research challenges.

Neuwirth, Sarah↗

CoSim: A Simulator for Co-Scheduling of Batch and On-Demand Jobs in HPC Datacenters

The increasing scale and complexity of scientific applications are rapidly transforming the ecosystem of tools, methods, and workflows adopted by the high-performance computing (HPC) community. Big data analytics and deep learning are gaining traction as essential components in this ecosystem in a variety of scenarios, such as, steering of experimental instruments , acceleration of high-fidelity simulations through surrogate computations, and guided ensemble searches. In this context, the batch job model traditionally adopted by the supercomputing in-frastructures needs to be complemented with support to schedule opportunistic on-demand analytics jobs, leading to the problem of efficient preemption of batch jobs with minimum loss of progress. In this paper, we design and implement a simulator, CoSim, that enables on-the-fly analysis of the trade-offs arising between delaying the start of opportunistic on-demand jobs, which leads to longer analytics latency, and loss of progress due to preemption of batch jobs, which is necessary to make room for on-demand jobs. To this end, we propose an algorithm based on dynamic programming with predictable performance and scalability that enables supercomputing infrastructure schedulers to analyze the aforementioned trade-off and take decisions in near real-time. Compared with other state-of-art approaches using traces of the Theta pre-Exascale machine, our approach is capable of finding the optimal solution, while achieving high performance and scalability.

batch job preemption↗

Parallel Algorithms for Efficient Computation of High-Order Line Graphs of Hypergraphs

This paper considers structures of systems beyond dyadic (pairwise) interactions and investigates mathematical modeling of multi-way interactions and connections as hypergraphs, where captured relationships among system entities are set-valued. To date, in most situations, entities in a hypergraph are considered connected as long as there is at least one common ``neighbor''. However, minimal commonality sometimes discards the ``strength'' of connections and interactions among groups. To this end, considering the ``width'' of a connection, referred to as the \emph{$s$-overlap} of neighbors, provides more meaningful insights into how closely the communities or entities interact with each other. In addition, $s$-overlap computation is the fundamental kernel to construct the line graph of a hypergraph, a low-order approximation of the hypergraph which can carry significant information about the original hypergraph. Subsequent stages of a data analytics pipeline then can apply highly-tuned graph algorithms on the line graph to reveal important features. Given a hypergraph, computing the $s$-overlaps by exhaustively considering all pairwise entities can be computationally prohibitive. To tackle this challenge, we develop efficient algorithms to compute $s$-overlaps and the corresponding line graph of a hypergraph. We propose several heuristics to avoid execution of redundant work and improve performance of the $s$-overlap computation. Our parallel algorithm, combined with these heuristics, is orders of magnitude (more than $10\times$) faster than the naive algorithm in all cases and the SpGEMM algorithm with filtration in most cases (especially with large $s$ value).

hypergraph algorithms, graph algorithms, parallel ↗

Exploring the Optimal Platform Configuration for Power-Constrained HPC Workflows

In high-performance computing (HPC) workflows, data analytics is typically utilized to gain insights from scientific simulations. Approaching the era of exascale, online analysis is gaining popularity due to the savings of I/O to persistent storage. As computing capability keeps growing, power consumption is becoming critical to HPC facilities. Enforcing power limits is emerging as a practical trend for power-constrained HPC facilities. However, it remains unclear how to choose the appropriate power limits for various HPC workflows and how to distribute the power limit of a workflow between simulation and analysis. In addition, given a power limit, it is unclear what the optimal scales and power capping levels are for various workflows, especially when taking reliability into account. In order to resolve these issues in power-constrained HPC, in this paper, we propose a reliability-aware model to determine the aforementioned platform configurations for HPC workflows. We also validate our model and present model-driven studies for a wide range of real-system scenarios. Our study reveals interesting insights about how platform configuration affects the performance and energy efficiency of HPC workflows under power constraints.

Tang, Kun↗

Strategies for Integrating Deep Learning Surrogate Models with HPC Simulation Applications

The emerging trend of the convergence of high performance computing (HPC), machine learning/deep learning (ML/DL), and big data analytics presents a host of challenges for large-scale computing campaigns that seek best practices to interleave traditional scientific simulation-based workloads with ML/DL models. A portfolio of systematic approaches to incorporate deep learning into modeling and simulation serves a vital need when we support AI for science at a computing facility. In this paper, we evaluate several strategies for deploying deep learning surrogate models in a representative physics application on supercomputers at the Oak Ridge Leadership Computing Facility (OLCF). We discuss a set of recommended deployment architectures and implementation approaches. We analyze and evaluate these alternatives and show their performance and scalability up to 1000 GPUs on two mainstream platforms equipped with different deep learning hardware and software stacks.

Yin, Junqi↗

Proactive Posturing of Large Power Grid for Mitigating Hurricane Impacts

In the past decade, natural disasters such as hurricanes have challenged the operation and control of U.S. power grid more. It is crucial to develop proactive strategies to assist grid operators for better emergency response and minimized electricity service interruptions. In this paper, we propose a proactive posturing methodology of power system elements, and formulated a Security-Constrained Optimal Power Flow (SCOPF) that informed by cross-domain hurricane modeling and its potential impacts on the grid elements. Simulation results based on real-world power grid and historical hurricane event validated the applicability of the proposed optimization formulation, and show potential to enable grid operators and planners with interactive cross-domain data analytics in mitigating hurricane impacts.

power system emergency response, security constrai↗

Social energy: mining energy from the society

The inherent nature of energy, i.e., physicality, sociality and informatization, implies the inevitable and intensive interaction between energy systems and social systems. From this perspective, we define 'social energy' as a complex sociotechnical system of energy systems, social systems and the derived artificial virtual systems which characterize the intense intersystem and intra-system interactions. The recent advancement in intelligent technology, including artificial intelligence and machine learning technologies, sensing and communication in Internet of Things technologies, and massive high performance computing and extreme-scale data analytics technologies, enables the possibility of substantial advancement in socio-technical system optimization, scheduling, control and management. In this paper, we provide a discussion on the nature of energy, and then propose the concept and intention of social energy systems for electrical power. A general methodology of establishing and investigating social energy is proposed, which is based on the ACP approach, i.e., 'artificial systems' (A), 'computational experiments' (C) and 'parallel execution' (P), and parallel system methodology. A case study on the University of Denver (DU) campus grid is provided and studied to demonstrate the social energy concept. In the concluding remarks, we discuss the technical pathway, in both social and nature sciences, to social energy, and our vision on its future.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Chemical Sensing Strategies for Real-Time Monitoring of Transformer Oil: A Review

Power transformers are a central component in the field of energy distribution and transmission. The early recognition of incipient faults in operating transformers is substantially cost effective by lessening impromptu blackouts. A standout amongst the most responsive and dependable strategies utilized for assessing the health of oil filled electrical equipment is dissolved gas analysis (DGA). Nowadays, there is an expanding requirement for better nonintrusive diagnostic and online monitoring tools to survey the internal state of the transformers. Chemical sensors are viewed as a key innovation for condition monitoring of transformer health, coordinating the non-invasiveness with typical sensor features, such as cost, usability, portability, and the integration with the data networks. Low-cost chemical sensors-based DGA techniques are expected to drastically augment the diagnostic abilities empowering the deployment on a broader range of oil filled power assets. The recent development involves both specific sensors designed to detect individual dissolved gas in transformer oil and non-specific sensors, operated in near ambient conditions, with the potential to be applied in a DGA system. In this paper, general background and operating guidelines of DGA are presented to address the origin of the gas formation, methods for their detection and the interpretation of the results by data analytics. The recent significant interest and advancements in chemical sensors to DGA applications are reviewed. Lastly, future research perspectives and challenges for the development of novel DGA chemical sensors are also discussed.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Distributed-Memory Parallel Symmetric Nonnegative Matrix Factorization

We develop the first distributed-memory parallel implementation of Symmetric Nonnegative Matrix Factorization (SymNMF), a key data analytics kernel for clustering and dimensionality reduction. Our implementation includes two different algorithms for SymNMF, which give comparable results in terms of time and accuracy. The first algorithm is a parallelization of an existing sequential approach that uses solvers for non symmetric NMF. The second algorithm is a novel approach based on the Gauss-Newton method. It exploits second-order information without incurring large computational and memory costs. We evaluate the scalability of our algorithms on the Summit system at Oak Ridge National Laboratory, scaling up to 128 nodes (4,096 cores) with 70% efficiency. Additionally, we demonstrate our software on an image segmentation task.

Eswar, Srinivas↗

Enabling HPC Scientific Workflows for Serverless

The convergence of edge computing, big data analytics, and AI with traditional scientific calculations is increasingly being adopted in HPC workflows. Workflow management systems are crucial for managing and orchestrating these complex computational tasks. However, it is difficult to identify patterns within the growing population of HPC workflows. Serverless has emerged as a novel computing paradigm, offering dynamic resource allocation, quick response time, fine-grained resource management and auto-scaling. In this paper, we propose a framework to enable HPC scientific workflows on serverless. Our approach integrates a widely used traditional HPC workflow generator with an HPC serverless workflow management system to create benchmark suites of scientific workflows with diverse characteristics. These workflows can be executed on different serverless platforms. We comprehensively compare executing workflows on traditional local containers and serverless computing platforms. Our results show that serverless can reduce CPU and memory usage respectively by 78.11% and 73.92% without compromising performance.

Andrei da silva, Anderson↗