Search NASA⌕ Search

SEARCH · Search NASA

Results for “big data applications”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Introduction to NASA Goddard Workshop on Artificial Intelligence

Artificial Intelligence (AI) is a collection of advanced technologies that allows machines to think and act, both humanly and rationally, through sensing, comprehending, acting and learning. AI's foundations lie at the intersection of several traditional fields Philosophy, Mathematics, Economics, Neuroscience, Psychology and Computer Science. Although the inception of AI started in the 1950's, it has recently made a strong comeback in all aspects of society and all over the world; this is mainly due to the timely combination of increased data volumes, advanced and mature algorithms, and improvements in computing power and storage. Current AI applications include big data analytics, robotics, intelligent sensing, assisted decision making, and speech recognition just to name a few.This workshop will be investigating how AI technologies can be adapted or developed to address the following challenges: Discover events of interest and correlations in large amounts of science data; improve the outcomes of science modeling and data assimilation using improved data processing, integration, and analysis. Design advisors for mission planning and operations, including anomaly detection and spacecraft health monitoring. Develop tools for engineering support, including advanced manufacturing, orbit determination, new component design and system engineering. Customize intelligent user interfaces, including visual analytics and natural language processing.

Le Moigne, Jacqueline↗

Overview of Artificial Intelligence (AI) at NASA Goddard

Artificial Intelligence (AI) is a collection of advanced technologies that allows machines to think and act, both humanly and rationally, through sensing, comprehending, acting and learning. AI's foundations lie at the intersection of several traditional fields Philosophy, Mathematics, Economics, Neuroscience, Psychology and Computer Science. Although the inception of AI started in the 1950's, it has recently made a strong comeback in all aspects of society and all over the world; this is mainly due to the timely combination of increased data volumes, advanced and mature algorithms, and improvements in computing power and storage. Current AI applications include big data analytics, robotics, intelligent sensing, assisted decision making, and speech recognition just to name a few. During the Tour, we will show a few examples of the current AI activities at NASA Goddard.

Le Moigne, Jacqueline↗

Big-Data Analytics for Electric Grid and Demand-Side Management

This report describes research to characterize the status of, and trends in, big-data analytics for the electricity grid. The research focused on 1) identifying power-grid big-data sources, types, and characteristics; and 2) characterizing big-data architecture, analytic methods, technology applications, and challenges. The first part of this report describes the main sources and characteristics of big data for the smart grid and comprehensively reviews big-data architecture, technologies, and applications in the power sector. The second part of this report presents case studies of big data applications in the power industry: (1) a smart-meter data and predictive analytics method for demand response (DR), (2) synchrophasor data analytics for the distribution grid, and (3) utility data for peak-demand management. For the predictive analytics case study, smart-meter-data-driven and physical models were developed to predict the potential kilowatt (kW) capacity reduction from DR. The DR estimation framework that was developed works for both small and large-scale customers. The synchrophasor case study demonstrates use of an algorithm applied to time-series data to detect events that appear as significant changes, known as “edges,” in voltage magnitude measurements. The synchrophasor case study also introduces an approach for clustering sets of events to reveal unique features that distinguish them (e.g., distinguishing capacitor bank switching from transformer tap changes). The peak-demand management case study describes the use of the data analytics to enable DR programs to limit forecasted peak demand, resulting in cost savings to the utility. The findings from the research described in this report support identification of opportunities and technologies for big-data and analytics applications for demand-side management in the power sector as well as other approaches to modernizing the electricity grid.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Digital archives, big data and image-based culturomics for social impact assessment: Opportunities and challenges

Social impact assessment (SIA) is well-established but uses conventional approaches that have become less effective in recent decades, particularly in relation to declining survey response rates and a lack of youth engagement. Images from digital archives and social media sources are poised to advance the research and practice of SIA by transcending text-based methods with insights into changing landscapes, and human engagement with them. This viewpoint describes progress, challenges and cautions toward the development of such tools (defined as culturomics), using hydroelectricity cases to illustrate potential approaches. These tools build on foundational work in a range of disciplines, including the humanities and computer science. We describe necessary advances in machine learning, image digitization, and data aggregation and visualization techniques, as well as ways to ensure that such tools are carefully tested, applied and interpreted. Challenges include the automation, acquisition and management of datasets, and using these tools appropriately and equitably. Critically, culturomics of any kind must not be used as a replacement for engagement with people, but as complementary to inclusive stakeholder engagement. - Highlights: •Image-based digital archives and social media present opportunities for new SIA tools. •Application-ready big data approaches are emerging across many fields. •SIA scholars, practitioners and stakeholders should engage with culturomics. •Challenges include automation, digitization, interpretation and justice. •Culturomics of any kind must not replace engagement with people.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Methods and Models for Analyzing Methane Sorption Capacity of Coal Based on Its Physicochemical Characteristics

The authors study the influence of physicochemical parameters on methane adsorption capacity of coal and offer the analytical method for the methane adsorption capacity for three-phased condition of methane. It is found that in the depth interval to 300 m below surface, methane adsorption capacity measured in lab can exceed natural gas content of coal obtained from geological exploration data by 30%, and the change in the thermodynamic condition of coal–methane system brings irreversible physicochemical consequences in terms of the altered ratios of physical states of the main components. There is no linear connection between natural gas content of a coal bed and its methane adsorption capacity with respect to occurrence depth. The application of Big Data in treatment and interpretation of large data flows is described. The theoretical data predicted using the proposed method and the experimental data on methane content of Kuzbass coal agree.

01 COAL, LIGNITE, AND PEAT↗

Scanning Probe Microscopy in the Information Age

Scanning probe microscopy (SPM) has brought scientific exploration, visualization, and control to nanoscale. A plethora of measurement approaches and techniques devised around SPM enabled probing structure and functionalities crucial for virtually all areas of science. While high-resolution electron microscopy techniques preceded scanning probes, SPM effectively opened the door to probing the structure–property relationship on length scales spanning 100s of micrometers down to a single atom. As the complexity of the material systems and the scientific questions increased, so too did the SPM techniques. Substantial efforts have been dedicated toward the development of better instrumentation platforms, probes, and novel spectroscopic measurements; however, many opportunities related to data processing and acquisition in SPM are just being realized now. This chapter will describe the recent progress in data-driven SPM by giving a broad overview of the field, the foundation of the scanning tunneling microscope (STM) and the atomic force microscope (AFM), as well as a description behind the operating principles and dynamic modes. A comprehensive overview of the multi-frequency techniques, Band excitation, and general acquisition mode SPM is given, setting the context for the application of big data, machine learning, and artificial intelligence in the field of dynamic SPM.

Collins, Liam↗

Nonnegative canonical tensor decomposition with linear constraints: nnCANDELINC

Abstract There is an emerging interest for tensor factorization applications in big‐data analytics and machine learning. To speed up the factorization of extra‐large datasets, organized in multidimensional arrays (also known as tensors), easy to compute compression‐based tensor representations, such as, Tucker and tensor train formats, are used to approximate the initial large‐tensor. Further, tensor factorization is used to extract latent features that can facilitate discoveries of new mechanisms and signatures hidden in the data, where the explainability of the latent features is of principal importance. Nonnegative tensor factorization extracts latent features that are naturally sparse and parts of the data, which makes them easily interpretable. However, to take into account available domain knowledge and subject matter expertise, often additional constraints need to be imposed, which lead us to canonical decomposition with linear constraints (CANDELINC), a canonical polyadic decomposition with rank deficient factors. In CANDELINC, Tucker compression is used as a preprocessing step, which lead to a larger residual error but to more explainable latent features. Here, we propose a nonnegative CANDELINC (nnCANDELINC) accomplished via a specific nonnegative Tucker decomposition; we refer to as minimal or canonical nonnegative Tucker. We derive several results required to understand the specificity of nnCANDELINC, focusing on the difficulties of preserving the nonnegative rank of a tensor to its Tucker core and comparing the real valued to nonnegative case. Finally, we demonstrate nnCANDELINC performance on synthetic and real‐world examples.

97 MATHEMATICS AND COMPUTING↗

Learning Global Proliferation Expertise Evolution Using AI-Driven Analytics and Public Information

Detecting and anticipating global proliferation expertise and capability evolution from unstructured, noisy, and incomplete public data streams is a highly desired, but extremely challenging task. Here, in this article, we present our pioneering data-driven approach to support the non-proliferation mission to detect and explain the evolution of proliferation expertise and capability development globally from terabytes of publicly available information (PAI), focusing on our knowledge extraction pipeline and descriptive analytics. We first discuss how we fuse nine open-source data streams, including multilingual data, to convert 4 TB of unstructured data to structured knowledge and encode dynamically evolving proliferation expertise representations—content and context graphs. For this, we rely on natural language processing (NLP) and deep learning (DL) models to perform information extraction, topic modeling, and distributed text representation (aka embedding) learning. We then present interactive, usable, and explainable descriptive analytics to refine domain knowledge and present it in a human-understandable form. Finally, we introduce future work avenues that will leverage our dynamic knowledge representations and descriptive analytics to enable predictive and prescriptive inferences to achieve real-time domain understanding and contextual reasoning about global proliferation expertise and capability evolution.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Fault Detection Utilizing Convolution Neural Network on Timeseries Synchrophasor Data From Phasor Measurement Units

An end-to-end supervised learning method is proposed for fault detection in the electric grid using Big Data from multiple Phasor Measurement Units (PMUs). The approach consists of preprocessing steps aimed at reducing data noise and dimensionality, followed by utilization of six classification models considered for detecting faults. Three of the models were variants of Convolutional Neural Network (CNN) architectures that consider a single type of measurement (voltage, current or frequency) at all PMUs or all types together also at all PMUs. CNN based models were compared to traditional methods of Logistic Regression (LR), Multi-layer Perceptron (MLP) and Support Vector Machine (SVM). Evaluation was conducted on two-year data measured by PMUs at 37 locations in a large electric grid. Here, the response variable for classification were extracted from the grid-wide outage event log. Experiments show that CNN-based models outperformed traditional methods on one year out-of-sample outage detection over the entire grid.

42 ENGINEERING↗

Predictive Big Data Analytics: A Study of Parkinson’s Disease Using Large, Complex, Heterogeneous, Incongruent, Multi-Source and Incomplete Observations

Background A unique archive of Big Data on Parkinson’s Disease is collected, managed and disseminated by the Parkinson’s Progression Markers Initiative (PPMI). The integration of such complex and heterogeneous Big Data from multiple sources offers unparalleled opportunities to study the early stages of prevalent neurodegenerative processes, track their progression and quickly identify the efficacies of alternative treatments. Many previous human and animal studies have examined the relationship of Parkinson’s disease (PD) risk to trauma, genetics, environment, co-morbidities, or life style. The defining characteristics of Big Data–large size, incongruency, incompleteness, complexity, multiplicity of scales, and heterogeneity of information-generating sources–all pose challenges to the classical techniques for data management, processing, visualization and interpretation. We propose, implement, test and validate complementary model-based and model-free approaches for PD classification and prediction. To explore PD risk using Big Data methodology, we jointly processed complex PPMI imaging, genetics, clinical and demographic data. Methods and Findings Collective representation of the multi-source data facilitates the aggregation and harmonization of complex data elements. This enables joint modeling of the complete data, leading to the development of Big Data analytics, predictive synthesis, and statistical validation. Using heterogeneous PPMI data, we developed a comprehensive protocol for end-to-end data characterization, manipulation, processing, cleaning, analysis and validation. Specifically, we (i) introduce methods for rebalancing imbalanced cohorts, (ii) utilize a wide spectrum of classification methods to generate consistent and powerful phenotypic predictions, and (iii) generate reproducible machine-learning based classification that enables the reporting of model parameters and diagnostic forecasting based on new data. We evaluated several complementary model-based predictive approaches, which failed to generate accurate and reliable diagnostic predictions. However, the results of several machine-learning based classification methods indicated significant power to predict Parkinson’s disease in the PPMI subjects (consistent accuracy, sensitivity, and specificity exceeding 96%, confirmed using statistical n-fold cross-validation). Clinical (e.g., Unified Parkinson's Disease Rating Scale (UPDRS) scores), demographic (e.g., age), genetics (e.g., rs34637584, chr12), and derived neuroimaging biomarker (e.g., cerebellum shape index) data all contributed to the predictive analytics and diagnostic forecasting. Conclusions Model-free Big Data machine learning-based classification methods (e.g., adaptive boosting, support vector machines) can outperform model-based techniques in terms of predictive precision and reliability (e.g., forecasting patient diagnosis). We observed that statistical rebalancing of cohort sizes yields better discrimination of group differences, specifically for predictive analytics based on heterogeneous and incomplete PPMI data. UPDRS scores play a critical role in predicting diagnosis, which is expected based on the clinical definition of Parkinson’s disease. Even without longitudinal UPDRS data, however, the accuracy of model-free machine learning based classification is over 80%. The methods, software and protocols developed here are openly shared and can be employed to study other neurodegenerative disorders (e.g., Alzheimer’s, Huntington’s, amyotrophic lateral sclerosis), as well as for other predictive Big Data analytics applications.

Science & Technology - Other Topics↗

Evaluating Offshore Infrastructure Integrity

Drilling in the offshore environment involves a complex network of infrastructure including pipelines, platforms, rigs, subsea installations, ports, and terminals. Government and industry partners have developed this network over many decades and it remains a critical part of the United States (U.S.) energy portfolio. Many of the major components of this system have been designed with a 20- to 30-year lifespan, yet consistent and growing energy demands support the need to extend the design life of existing infrastructure or repurpose it for secondary needs (i.e. enhanced oil recovery, carbon storage, and new wells). As a result, a growing portion of the offshore infrastructure in the U.S. is approaching or has exceeded its original design life. A critical step in ensuring the continued safe and effective operation of offshore infrastructure is developing a comprehensive understanding of the state of offshore infrastructure and the factors that effect it. The purpose of this project is to assess the current state of existing infrastructure and identify the factors involved in infrastructure degradation through the development and application of big data analytics, machine learning, and advanced spatio-temporal analysis. The project leverages existing data at NETL and combines it with new information on offshore oil and gas structures and the ambient offshore environment in an effort to identify patterns associated with infrastructure integrity. Building on the identified trends and patterns, this project incorporates exploratory analytics and spatial analysis tools in conjunction with machine learning and statistical models to characterize the condition of existing platforms in the offshore environment and predict their risk of failure.

02 PETROLEUM↗

Big data for ecologists: Highlighting the ORNL DAAC

Ecologists are increasingly confronted by questions that can be addressed only by integrating data from numerous sources, often across large geographic areas and broad time periods. The supply of ecological big data is increasing at a rapid pace as researchers are publishing their data sets and large, public science and data infrastructures (such as NEON, DataONE, LTER, & NCEAS) are producing and curating extensive volumes of complex data and metadata. While supply of, and demand for, ecological data is on the rise, many ecologists now face a new challenge in locating and synthesizing the data relevant for their particular question. Here we highlight selected popular big data products applicable to ecological research available from the NASA Distributed Active Archive Center (DAAC) located at Oak Ridge National Laboratory (ORNL).

Boyer, Alison G↗

Considerations on the Use of Custom Accelerators for Big Data Analytics

Accelerators, including Graphic Processing Units (GPUs) for gen- eral purpose computation, many-core designs with wide vector units (e.g., Intel Phi), have become a common component of many high performance clusters. The appearance of more stable and reliable tools tools that can automatically convert code written in high-level specifications with annotations (such as C or C++) to hardware de- scription languages (High-Level Synthesis - HLS), is also setting the stage for a broader use of reconfigurable devices (e.g., Field Pro- grammable Gate Arrays - FPGAs) in high performance system for the implementation of custom accelerators, helped by the fact that new processors include advanced cache-coherent interconnects for these components. In this chapter, we briefly survey the status of the use of accelerators in high performance systems targeted at big data analytics applications. We argue that, although the progress in the use of accelerators for this class of applications has been sig- nificant, differently from scientific simulations there still are gaps to close. This is particularly true for the ”irregular” behaviors exhibited by no-SQL, graph databases. We focus our attention on the limits of HLS tools for data analytics and graph methods, and discuss a new architectural template that better fits the requirement of this class of applications. We validate the new architectural templates by mod- ifying the Graph Engine for Multithreaded System (GEMS) frame- work to support accelerators generated with such a methodology, and testing with queries coming from the Lehigh University Benchmark (LUBM). The architectural template enables better supporting the task and memory level parallelism present in graph methods by sup- porting a new control model and a enhanced memory interface. We show that out solution allows generating parallel accelerators, pro- viding speed ups with respect to conventional HLS flows. We finally draw conclusions and present a perspective on the use of reconfig- urable devices and Design Automation tools for data analytics.

Data Analytics↗

Roadmap on artificial intelligence and big data techniques for superconductivity

This paper presents a roadmap to the application of AI techniques and big data (BD) for different modelling, design, monitoring, manufacturing and operation purposes of different superconducting applications. To help superconductivity researchers, engineers, and manufacturers understand the viability of using AI and BD techniques as future solutions for challenges in superconductivity, a series of short articles are presented to outline some of the potential applications and solutions. These potential futuristic routes and their materials/technologies are considered for a 10–20 yr time-frame.

machine learning, neural network↗

Union: An Automatic Workload Manager for Accelerating Network Simulation

With the rapid growth of the machine learning applications, the workloads of future HPC systems are anticipated to be a mix of scientific simulation, big data analytics, and machine learning applications. Simulation is a great research vehicle to understand the performance implications of co-running scientific applications with big data and machine learning workloads on large-scale systems. In this paper, we present Union, a workload manager that provides an automatic framework to facilitate hybrid workload simulation in CODES. Furthermore, we use Union, along with CODES, to investigate various hybrid workloads composed of traditional simulation applications and emerging learning applications on two dragonfly systems. The experiment results show that both message latency and communication time are important performance metrics to evaluate network interference. Network interference on HPC applications is more reflected by the message latency variation, whereas ML application performance depends more on the communication time.

heterogeneous workloads↗