Search NASA⌕ Search

SEARCH · Search NASA

Results for “high performance visualization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

TU-CD-207-08: Intrinsic Image Quality Comparison of Synthesized 2-D and FFDM Images

Purpose: With the combined interest of managing patient dose, maintaining or improving image quality, and maintaining or improving the diagnostic utility of mammographic data, this study aims to compare the intrinsic image quality of Hologic’s synthesized 2-D (C-View) and 2-D FFDM images in terms of resolution, contrast, and noise. Methods: This study utilized a novel 3-D printed anthropomorphic breast phantom in addition to the American College of Radiology (ACR) mammography accreditation phantom. Analysis of the 3-D anthropomorphic phantom included visual assessment of resolution and analysis of the normalized noise power spectrum. Analysis of the ACR phantom included both visual inspection and objective automated analysis using in-house software. The software incorporates image- and object-specific CNR visibility thresholds which account for image characteristics such as noise texture which affect object visualization. T- test statistical analysis was also performed on ACR phantom scores. Results: The spatial resolution of C-View images is markedly lower (at least 50% worse) than that of FFDM. And while this is generally associated with the benefit of reduced relative noise magnitude, the noise in C-View images tends to have a more mottled (predominantly low-frequency) texture. In general, for high contrast objects, C-View provides superior visualization over FFDM; however this benefit diminishes for low contrast objects and is applicable only to objects that are sufficiently larger than the spatial resolution threshold. Based on both observer and automated ACR phantom analysis, between 50–70% of C-View images failed to meet ACR minimum accreditation requirements – primarily due to insufficient (unbroken) fiber visibility. Conclusion: Compared to FFDM, C-View offers better depiction of objects of certain size and contrast, but provides poorer overall resolution and noise properties. Based on these findings, the utilization of C-View images in the clinical setting requires careful consideration, especially if considering the discontinuation of FFDM imaging.

60 APPLIED LIFE SCIENCES↗

Understanding failure in austenitic steels: Key considerations for molten salt storage in CSP applications

Owing to their excellent properties, austenitic stainless steels are extensively used in boilers, furnaces, molten salt tanks, and other applications that are subjected to extreme mechanical loads and high-temperature conditions. Their high corrosion and creep resistance make them suitable for high-temperature operating environments. Additionally, good fatigue resistance and favorable mechanical and visual properties are essential. However, the premature failure of several components at elevated temperatures has been previously reported. Although the failure analysis of components in service is complex, processes such as cold work and welding have been identified as contributing factors to the performance degradation of these steels. This study aims to analyze the various documented failure modes and mechanisms in austenitic steels, including creep, cracking, stress relaxation cracking, and fatigue, to better understand the multi-objective design requirements for these alloys as structural materials for high-temperature molten salt tanks in Concentrating Solar Power (CSP) plants. Stabilized austenitic grades, such as AISI 347H, demonstrate superior resistance to creep and corrosion-related degradation when compared to non-stabilized grades like AISI 316L at temperatures relevant to concentrated solar power (CSP) applications. In contrast, nickel-based alloys offer enhanced corrosion resistance, albeit at a higher cost. This review underscores that creep, stress relaxation cracking, and thermo-mechanical fatigue are the predominant long-term failure risks in CSP hot tanks.

14 SOLAR ENERGY↗

Evaluation of OpenAI Codex for HPC Parallel Programming Models Kernel Generation

We evaluate AI-assisted generative capabilities on fundamental numerical kernels in high-performance computing (HPC), including AXPY, GEMV, GEMM, SpMV, Jacobi Stencil, and CG. We test the generated kernel codes for a variety of language-supported programming models, including (1) C++ (e.g., OpenMP [including offload], OpenACC, Kokkos, SyCL, CUDA, and HIP), (2) Fortran (e.g., OpenMP [including offload] and OpenACC), (3) Python (e.g., numpy, Numba, cuPy, and pyCUDA), and (4) Julia (e.g., Threads, CUDA.jl, AMDGPU.jl, and KernelAbstractions.jl). We use the GitHub Copilot capabilities powered by the GPT-based OpenAI Codex available in Visual Studio Code as of April 2023 to generate a vast amount of implementations given simple + + prompt variants. To quantify and compare the results, we propose a proficiency metric around the initial 10 suggestions given for each prompt. Results suggest that the OpenAI Codex outputs for C++ correlate with the adoption and maturity of programming models. For example, OpenMP and CUDA score really high, whereas HIP is still lacking. We found that prompts from either a targeted language such as Fortran or the more general purpose Python can benefit from adding code keywords, while Julia prompts perform acceptably well for its mature programming models (e.g., Threads and CUDA.jl). We expect for these benchmarks to provide a point of reference for each programming model's community. Overall, understanding the convergence of large language models, AI, and HPC is crucial due to its rapidly evolving nature and how it is redefining human-computer interactions.

Godoy, William↗

Dynamic functional imaging of brain glucose utilization using fPET-FDG

We report that glucose is the principal source of energy for the brain and yet the dynamic response of glucose utilization to changes in brain activity is still not fully understood. Positron emission tomography (PET) allows quantitative measurement of glucose metabolism using 2-[18F]-fluorodeoxyglucose (FDG). However, FDG PET in its current form provides an integral (or average) of glucose consumption over tens of minutes and lacks the temporal information to capture physiological alterations associated with changes in brain activity induced by tasks or drug challenges. Traditionally, changes in glucose utilization are inferred by comparing two separate scans, which significantly limits the utility of the method. We report a novel method to track changes in FDG metabolism dynamically, with higher temporal resolution than exists to date and within a single session. Using a constant infusion of FDG, we demonstrate that our technique (termed fPET-FDG) can be used in an analysis pipeline similar to fMRI to define within-session differential metabolic responses. We use visual stimulation to demonstrate the feasibility of this method. Ultimately, this new method has a great potential to be used in research protocols and clinical settings since fPET-FDG imaging can be performed with most PET scanners and data acquisition and analysis are straightforward. fPET-FDG is a highly complementary technique to MRI and provides a rich new way to observe functional changes in brain metabolism.

2-[18F]-fluorodeoxyglucose (FDG)↗

Galaxy morphological classification in deep-wide surveys via unsupervised machine learning

ABSTRACT Galaxy morphology is a fundamental quantity, which is essential not only for the full spectrum of galaxy-evolution studies, but also for a plethora of science in observational cosmology (e.g. as a prior for photometric-redshift measurements and as contextual data for transient light-curve classifications). While a rich literature exists on morphological-classification techniques, the unprecedented data volumes, coupled, in some cases, with the short cadences of forthcoming ‘Big-Data’ surveys (e.g. from the LSST), present novel challenges for this field. Large data volumes make such data sets intractable for visual inspection (even via massively distributed platforms like Galaxy Zoo), while short cadences make it difficult to employ techniques like supervised machine learning, since it may be impractical to repeatedly produce training sets on short time-scales. Unsupervised machine learning, which does not require training sets, is ideally suited to the morphological analysis of new and forthcoming surveys. Here, we employ an algorithm that performs clustering of graph representations, in order to group image patches with similar visual properties and objects constructed from those patches, like galaxies. We implement the algorithm on the Hyper-Suprime-Cam Subaru-Strategic-Program Ultra-Deep survey, to autonomously reduce the galaxy population to a small number (160) of ‘morphological clusters’, populated by galaxies with similar morphologies, which are then benchmarked using visual inspection. The morphological classifications (which we release publicly) exhibit a high level of purity, and reproduce known trends in key galaxy properties as a function of morphological type at z < 1 (e.g. stellar-mass functions, rest-frame colours, and the position of galaxies on the star-formation main sequence). Our study demonstrates the power of unsupervised machine learning in performing accurate morphological analysis, which will become indispensable in this new era of deep-wide surveys.

Martin, G.↗

Machine learning elastic constants of multi-component alloys

The present manuscript explores application of machine learning methods for determining elastic constants and other derived mechanical properties of multi-component alloys. Here, a number of machine learning models, including linear regression, neural network and random forest based models, are trained and tested on a dataset of binary alloys generated using density functional theory (DFT) calculations and spanning over a large number of elemental species in the periodic table. Starting with a wide range of simple and easily accessible compositionally-averaged elemental features, a correlation-based feature selection strategy was used to systematically down-select a set of most relevant features towards the prediction of the elasticity tensor components. The true predictive performance and the associated uncertainties of the models were established by testing on unseen data and bootstrapping, respectively. A single and pair-wise feature partial dependence analysis was performed to visualize the average property trends in the multi-dimensional feature space in order to further understand the achieved predictive performance. The utility of the trained model is further demonstrated by obtaining sufficiently accurate yet highly efficient approximations for bulk modulus, Young’s modulus, shear modulus and Poisson’s ratio for alloys beyond the binary space (i.e., two-component alloys) on which the model was originally trained. More importantly, we test and validate the predictive performance of the developed model directly against the experimentally measured elastic constants of technologically relevant multi-component alloys (such as, Ni- and Ti-based alloys). Finally, utility of such a data-enabled route is demonstrated by predicting the possible range of various elastic properties for vast composition space available within the five component Ni-Cr-Fe-Mo-W alloy system in a high-throughput manner.

36 MATERIALS SCIENCE↗

Probing ternary solvent effect in high V oc polymer solar cells using advanced AFM techniques

This work describes a simple method to develop a high V oc low band gap PSCs. In addition, two new atomic force microscopy (AFM)-based nanoscale characterization techniques to study the surface morphology and physical properties of the structured active layer are introduced. With the help of ternary solvent processing of the active layer and C 60 buffer layer, a bulk heterojunction PSC with V oc more than 0.9 V and conversion efficiency 7.5% is developed. In order to understand the fundamental properties of the materials ruling the performance of the PSCs tested, AFM-based nanoscale characterization techniques including Pulsed-Force-Mode AFM (PFM-AFM) and Mode-Synthesizing AFM (MSAFM) are introduced. Interestingly, MSAFM exhibits high sensitivity for direct visualization of the donor–acceptor phases in the active layer of the PSCs. Lastly, conductive-AFM (cAFM) studies reveal local variations in conductivity in the donor and acceptor phases as well as a significant increase in photocurrent in the PTB7:ICBA sample obtained with the ternary solvent processing.

25 ENERGY STORAGE↗

Framework for Analysis and Visualization of Simulation Data (VizAly)

As supercomputing resources increase, cosmological scientists are able to run more detailed and larger simulations generating massive amounts of data. Analyzing these simulations with an available open-source toolkit is important for collaborative Department of Energy scientific discovery across labs, universities, and other partners. Developed software as a part of this collection include: comparing data with other existing simulations, verifying and validating results with observation databases, new halo finder algorithms, and using analytical tools to get insights into the physics of the cosmological universe. The goal of this software project is to provide a set of open-source libraries, tools, and packages for large-scale cosmology that allows scientists to visualize, analyze, and compare large-scale simulation and observational data sets. Developed software will provide a variety of methods for processing, visualization, and analysis of astronomical observation and cosmological simulation data. These tools are intended for deployment on multiple scientific computing platforms, including but not limited to personal computers, cloud computing, experimental sites (telescopes) and high performance supercomputers.

Wendelberger, Joanne↗

Spectral Peak Enhancement by Combining Trusted Response Elements via Machine Learning (SPECTRE-ML) v0.8.0

SPECTRE-ML (Spectral Peak Enhancement by Combining Trusted Response Elements via Machine Learning) is a machine learning based program for finding optimal clusters of radiation detector segments (i.e., pixels or voxels) in order to improve spectral performance. It provides facilities for pre-processing and analyzing training datasets, running ML algorithms, and evaluating and visualizing outputs. SPECTRE-ML outperforms simpler ad-hoc segmentation methods such as uniform depth clusters, learning detector performance trends such as dead layers, edge effects, and gain shifts. Although extensible to arbitrary highly-segmented spectroscopic radiation detectors, SPECTRE-ML currently focuses on improving spectral performance in highly-segmented CdZnTe (CZT) detectors for International Atomic Energy Agency (IAEA) non-destructive assay (NDA) safeguards tasks.

Vavrek, Jayson↗

ATDM AD projects to ECP Report (Q3 FY2020)

The SPARC team has completed much of the workflow development needed for the coupled and uncoupled analyses needed to support our flight test validation work. The SPARC team has continued to make progress on development activities to support unsteady, full reentry vehicle aero analysis, which have focused on turbulence modeling, uniform mesh refinement, in-situ visualization (with the Paraview/Catalyst team), and performance analysis. The SPARC team is making solid progress towards our Q4 goal of demonstrating an improved Wall Modeled Large Eddy Simulation (WMLES) capability for standard geometries using high-order finite difference, discontinuous Galerkin, and low-dissipation finite volume methods. The SPARC team has met the minimum completion criteria for parts of the L1, including documenting performance and scaling of SPARC on Trinity, Sierra and Astra, and performing runs for the flight test validation case.

97 MATHEMATICS AND COMPUTING↗

Research Update: Reactively sputtered nanometer-thin ZrN film as a diffusion barrier between Al and boron layers for radiation detector applications

In this paper, optimization of the process flow for PureB detectors is investigated. Diffusion barrier layers between a boron layer and the aluminum interconnect can be used to enhance the performance and visual appearance of radiation detectors. Few nanometers-thin Zirconium Nitride (ZrN) layer deposited by reactive sputtering in a mixture of Ar/N{sub 2}, is identified as a reliable diffusion barrier with better fabrication process compatibility than others. The barrier properties of this layer have been tested for different boron layers deposited at low and high temperatures with extensive optical microscopy analyses, electron beam induced current, SEM, and electrical measurements. This study demonstrated that spiking behavior of pure Al on Si can be prevented by the thin ZrN layer thus improving the performance of the radiation detectors fabricated using boron layer.

36 MATERIALS SCIENCE↗

Electrical Analysis of Pulsed Laser Annealed Poly-Si:Ga/SiOx Passivating Contacts

High-efficiency single crystalline silicon (c-Si) solar cells require precise control of dopant diffusion profiles and highly active doping concentrations through thermal annealing. Conventional furnace annealing has been successfully employed for dopant diffusion in aluminum back-surface field (Al-BSF), passivated emitter and rear contact (PERC), and Topcon cells. However, furnace annealing limits some next-generation polycrystalline silicon (poly-Si) on SiOx passivating contacts. This limitation largely affects p-type passivating contacts by not being able to: 1) control the dopant diffusion accurately to prevent B segregation at the SiOx and c-Si interface; and 2) provide highly activated doping concentrations for low contact resistivity. In this contribution, we use a nanosecond excimer laser to examine the passivation quality and electrical performance of both B- and Ga-doped poly-Si/SiOx passivating contacts. The core of this method is to take advantage of the nonequilibrium nature of the anneal through rapid melting and recrystallizing the poly-Si in a short timescale and to achieve doping concentrations above the solid solubility limit. Simulations were performed on polished surfaces to visualize the doping diffusion profiles under different laser conditions, and secondary ion mass spectrometry (SIMS) was used to verify the experimental diffusion profiles post pulsed laser melting (PLM). The results show the dopant diffusion profiles can be tuned precisely through PLM. The electrical analysis using VdP-Hall measurements reveals that the active doping concentration reached 10^21 cm-3 for B and ~2×10^21 cm-3 for Ga in poly-Si, far exceeding their solid solubility limit in Si, with a nearly 100% dopant activation achieved for B, and 20% for Ga. This highly activated dopant profile results in a low contact resistivity of <15 mO·cm^2 for B-doped contacts and <40 mO·cm^2 for Ga-doped contacts. We demonstrate that the passivation quality of the laser annealed contacts shows a linear dependency with the laser energy density and the number of pulses. PLM was also performed on random pyramid textured samples, and we found that preferential melting exists for the tips of pyramids, which can be pursued further for selective pinhole opening. Additionally, we perform thermal stability tests of these hyperdoped passivating contacts and show that under certain laser conditions, the B-doped samples exhibit good thermal stability with near full activation for temperatures up to 800°C. Lastly, we transfer this technique to a larger area by overlapping laser spots and show uniform passivation quality and electrical performance. We are currently implementing this PLM annealed p-type passivating contacts into double-side back-junction poly-Si/SiOx passivating contact devices. Our results show that PLM can not only enable the next generation of high-efficiency c-Si photovoltaics technology but can also benefit the c-Si integrated circuit industry through hyperdoping other semiconductors with atoms that may otherwise have a low solubility.

doping diffusion profile↗

Integrated Research Infrastructure Architecture Blueprint Activity (Final Report 2023)

The complexity of scientific pursuits is increasing rapidly with aspects that require dynamic integration of experiment, observation, theory, modeling, simulation, visualization, machine learning (ML), artificial intelligence (AI), and analysis. Research projects across the Department of Energy (DOE) are increasingly data and compute intensive. Innovative research teams are accelerating the pace of discovery by using high-performance computational and data tools in their research workflows and leveraging multiple research infrastructures. Additionally, several recent high-level U.S. government reports underscore the necessity of a new advanced computing ecosystem for international competitiveness and national security. International competitors are moving forward with major research infrastructure integration efforts that seek to capture a competitive advantage in the global innovation race. Owing to its unparalleled constellation of world-class experimental and observational facilities and high-performance and extreme-scale computational, data, and networking infrastructure, DOE is positioned to be a global leader in this new era of integrated science. However, this new integration paradigm will demand continuing evolution to ensure the U.S. remains a global leader in research and innovation. The DOE Office of Science (SC) has seized on the strategic importance of integration and has adopted a vision for Integrated Research Infrastructure (IRI): To empower researchers to meld DOE’s world-class research tools, infrastructure, and user facilities seamlessly and securely in novel ways to radically accelerate discovery and innovation. To respond to the evolving computational requirements of research and the competitive international innovation landscape, experimental facilities could be connected with high performance computing resources for near real-time analysis, and resources should be provided for merging enormous and diverse data for AI/ML techniques and analysis.

97 MATHEMATICS AND COMPUTING↗

Feasible Actuator Range Modifier (FARM), a Tool Aiding the Solution of Unit Dispatch Problems for Advanced Energy Systems

Integrated energy systems (IESs) seek to minimize power generating costs in future power grids through the coupling of different energy technologies. To accommodate fluctuations in load demand due to the penetration of renewable energy sources, flexible operation capabilities must be fully exploited, and even power plants that are traditionally considered as base-load units need to be operated according to unconventional paradigms. Thermomechanical loads induced by frequent power adjustments can accelerate the wear and tear. If a unit is flexibly operated without respecting limits on materials, the risk of failures of expensive components will eventually increase, nullifying the additional profits ensured by flexible operation. In addition to the bounds on power variations (explicit constraints),the solution of the unit dispatch problem needs to meet the limits on the variation of key process variables, including temperature, pressure and flow rate (implicit constraints).The FARM (Feasible Actuator Range Modifier) module was developed to enable existing optimization algorithms to identify solutions to the unit dispatch problem that are both economically favorable and technologically sustainable. Thanks to the iterative dispatcher–validator scheme, FARM permits addressing all the imposed constraints without excessively increasing the computational costs. In this work, the algorithms constituting the module are described, and the performance was assessed by solving the unit dispatch problem for an IES composed of three units, i.e., balance of plant, gas turbine, and high-temperature steam electrolysis. Finally, the FARM module provides dedicated tools for visualizing the response of the constrained variables of interest during operational transients and a tool aiding the operator at making decisions. These techniques might represent the first step towards the deployment of an ecological interface design (EID) for IES units.

47 OTHER INSTRUMENTATION↗

WarpVisit

WarpVisit is an insitu simulation application that integrates the Warp laser plasma accelerator simulation framework with Visit a parallel visualization application. WarpVisit is written in python and supports interactive or live mode where user can connect to Warp with the Visit GUI and batch mode for batch for non-interactive use on high-performance computing resources.

Loring, Burlen↗

Focused Ion Beam Tomography of Alloy 617 Corroded in Molten Chloride Salt

Materials qualification of reactor structural materials is a critical step in rapid implementation of advanced nuclear reactor technologies, particularly to assess the corrosion performance in these designs. Accelerated qualification of reactor structural materials requires incorporating powerful computational toolsets, such as phase field modelling in the Multiphysics Object-Oriented Simulation Environment (MOOSE) framework, to predict the evolution of structural materials due to corrosion. Accordingly, computational toolsets will require experimental data generated at appropriate length scales to validate accuracy. Focused ion beam (FIB) provides a high degree of control over manipulation of materials for analytical purposes, including capturing data on the evolution in the microstructure and elemental composition of materials at the mesoscale, an appropriate length scale for phase field modelling of intergranular diffusion phenomena using the MOOSE framework. For instance, the FEI Helios G4 UX dual beam plasma FIB microscope at the Irradiated Materials Characterization Laboratory (IMCL) is capable of backscatter diffraction (EBSD) and energy-dispersive x-ray spectroscopy (EDS) documenting the evolution in the microstructure and elemental composition, respectively. The Helios can perform EDS and EBSD three-dimensionally (3D) using tomography, which is then combined using different software packages to visualize 3D volumes correlating elemental composition to microstructural data. The purpose of this investigation was to develop a streamlined characterization and data processing workflow for 3D tomography studies on the FEI Helios G4 plasma FIB. The investigation is segmented into three parts: 1) Optimizing the data collection workflow, 2) identifying appropriate data processing and visualization software (i.e. DREAM.3D, MIPAR, and VGStudioMax), and 3) establishing an infrastructure for public release. The optimization of the data collection workflow is in collaboration with members of the U220 department to setup formal training on the tomography operation of the G4, through ThermoFisher Scientific, and exploring DREAM.3D, MIPAR, and VGStudioMax data processing/visualization software packages. VGStudioMax currently demonstrates the most promise for future use. Optimization of the data collection and processing workflow is still ongoing. A collaboration with INL High Performance Computing (HPC) established an open-source license for expediting the public release of FIB tomography datasets through HPC. FIB tomography data generated by the G4 will provide comprehensive data for validating 3D phase field mesoscale modelling tools within the MOOSE framework for accelerated qualification of reactor structural materials.

Copeland-Johnson, Trishelle↗

Interactive Visualization of Multi-Group Cross Sections on High-Fidelity Spatial Meshes

Core simulation tools utilizing Monte Carlo transport methods represent the 'Holy Grail' for nuclear reactor analysis. However, deterministic transport methods pose a number of advantages to Monte Carlo from a computational perspective, which seems likely to make these methods more viable in the near to intermediate term. Deterministic methods almost universally discretize the energy domain in a few to hundreds of energy groups. One of the key challenges for deterministic transport-based methods to be viable is accurate, reactor agnostic multi-group cross section generation. This paper describes tools for data processing and visualization of multigroup cross sections generated on high-fidelity spatial meshes from the OpenMC Monte Carlo code. Next generation deterministic transport reactor core modeling tools will require multi-group cross section (MGXS) generation on high-fidelity spatial meshes. High-fidelity MGXS are necessary for the accurate simulation of neutron physics in nuclear reactors with a high degree of spectral heterogeneity - e.g., large localized neutron flux gradients throughout a reactor core. Although many local (e.g., pin cell) approximations have been developed to model the scalar flux to compute spectral-averaged MGXS, these techniques cannot account for complex intra-pin and intra-assembly spatial self-shielding effects. Monte Carlo presents the most accurate method for reactor agnostic multi-group cross section generation since it does not require the use of local approximations to the flux. Monte Carlo has been proposed as a highly desirable framework for reactor agnostic cross section generation for many years. Monte Carlo methods can effectively embed all of the relevant spectral effects into the MGXS through the random particle tracking and interaction sampling processes. Previous work has applied MCNP and Serpent to compute few-group constants for diffusion calculations, and OpenMC to compute multi-group cross sections for transport-based methods. The primary difficulty with using Monte Carlo for MGXS generation is the tremendous computational resources required to adequately converge multi-group cross sections in each spatial region and energy group. Recent development of the OpenMC Monte Carlo code for high-performance computing platforms has made multigroup cross section generation from Monte Carlo more tractable in the foreseeable future. The recent release of OpenMC v0.7.1 includes a new Python-based module for MGXS generation. The MGXS generation module leverages many of the algorithms and features for 'big data' analysis of large tally datasets on complex spatial meshes. This paper describes the data management workflow and visualization tools which have been developed to explore the plethora of MGXS data generated with OpenMC. It is the authors' hope that the development of the visualization tool described in this paper will ultimately enable the design of machine learning algorithms to accelerate the convergence rate of high-fidelity multi-group cross section generated with Monte Carlo. (authors)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Big data analytics in smart grids: state-of-the-art, challenges, opportunities, and future directions

Big data has a potential to unlock novel groundbreaking opportunities in the power grid sector that enhances a multitude of technical, social, and economic gains. The currently untapped potential of applying the science of big data for better planning and operation of the power grid is a very challenging task and needs significant efforts all-around. As power grid technologies evolve in conjunction with measurement and communication technologies, this results in unprecedented amount of heterogeneous big data sets from diverse sources. In particular, computational complexity, data security, and operational integration of big data into utility decision frameworks are the key challenges to transform the heterogeneous large dataset into actionable outcomes. Moreover, due to the complex nature of power grids along with the need to balance power in real time, seamless integration of big data into utility operations is very critical. In this context, big data analytics combined with grid visualization can lead to better situational awareness and predictive decisions. This paper presents a comprehensive state-of-the-art review of big data analytics and its applications in power grids, and also identifies challenges and opportunities from utility, industry, and research perspectives. The paper analyzes research gaps and presents insights on future research directions to integrate big data analytics into electric utility decision framework. Detailed information for utilities looking to apply big data analytics and details insights on how utilities can enhance revenue streams and bring disruptive innovation in the industry are discussed. More importantly, general guidelines for utilities to make the right investment in the adoption of big data analytics by unveiling interdependencies among critical infrastructures and operations are also provided.

13 HYDRO ENERGY↗