Search NASA⌕ Search

SEARCH · Search NASA

Results for “big data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Tutorial: Lessons Learned for Behavior Analysts from Data Scientists

Big data is a computing term used to refer to large and complex data sets, typically consisting of terabytes or more of diverse data that is produced rapidly. The analysis of such complex data sets requires advanced analysis techniques with the capacity to identify patterns and abstract meanings from the vast data. The field of data science combines computer science with mathematics/statistics and leverages artificial intelligence, in particular machine learning, to analyze big data. This field holds great promise for behavior analysis, where both clinical and research studies produce large volumes of diverse data at a rapid pace (i.e., big data). This article presents basic lessons for the behavior analytic researchers and clinicians regarding integration of data science into the field of behavior analysis. We provide guidance on how to collect, protect, and process the data, while highlighting the importance of collaborating with data scientists to select a proper machine learning model that aligns with the project goals and develop models with input from human experts. Here, we hope this serves as a guide to support the behavior analysts interested in the field of data science to advance their practice or research, and helps them avoid some common pitfalls.

42 ENGINEERING↗
DOE OSTI2020

Open Source Visualization and Analysis Platform for 3D Reconstructions of Materials by Transmission Electron Microscopy

Three-dimensional characterization of materials at the nano- and meso-scale has become possible with transmission and scanning transmission electron microscopes (S/TEM). Its importance has extended to a wide class of nanomaterials such as hydrogen fuel cells, solar cells, industrial catalysts, new battery materials and semiconductor devices, as well as spanning high-tech industry, universities, and national labs. While capable instrumentation is abundant, this rapidly expanding demand for high-resolution tomography is bottlenecked by software that is instead tailored for lower-dose, biological applications and not optimized for higher-resolution materials applications. Existing tomography fails to utilize the chemical information provided by S/TEM spectrometers. To address this problem, this project delivered a scalable, fully functional, freely-distributable, open Source materials tomography package with a modern user interface that enables automated acquisition, alignment, and real-time reconstruction of raw tomography data, and provides advanced segmentation, three-dimensional chemical visualization and analysis optimized for materials applications. It has established an extendable framework capable of automation for high-throughput for the tomography of materials from data acquisition to visualization. Phase I and II delivered a full-featured, cross-platform, clean and integrated application for S/TEM materials tomography. It can read projection data from the microscope, with graphical tools for alignment of data, tomographic reconstruction, segmentation, and visualization of the reconstructed 3D volume. The entire pipeline can be saved to an XML state file, enabling fully reproducible data processing and analysis with a full Python environment for the development of custom algorithms, processing, and analysis requirements. Phase IIB extended the application to provide real-time tomographic reconstruction, "live updates" as data is processed, analysis of big data, and multi-channel tomography. Here, quantitative assessment of nanomaterials can occur as data is being recorded on a 3D visualization platform that accommodates additional information from multiple channels. We provide unmatched high-throughput tomography, where the complete tomographic pipeline from measurement to 3D visualization can occur rapidly. The project was extended to offer capabilities for micro-CT, X-ray, neutron, focused ion beam, atomic electron tomography and atom probe tomography. With over 600 transmission electron microscopes worldwide and approximately 50 coming online each year, the demand and impact of an open-source tomography tool is large. Significant opportunities exist in high-tech industry, universities, and national labs to enable or enhance three-dimensional imaging at the nanoscale and bring automated high-throughput approaches that will accelerate progress in materials characterization and metrology. The project supports a service based business model by enabling lab-specific acquisition and processing customization and integration-support and development that will be provided into Phase III and beyond.

36 MATERIALS SCIENCE↗
NASA NTRS2017

Big lunar data visualization and analysis

NASA's earth and planetary spacecraft return large amounts of remote sensing data, such as imagery and raw science measurements, in support of remarkable research. Not only does the data lead to new scientific discoveries about our planet and the solar system, it provides a wealth of information to educate, inspire, and engage the public at large. To leverage this rich data for mission planning, scientific research, public outreach and education, it is essential to make it accessible and understandable, analyzable, all while appealing to their interests. This presentation will highlight web-based capabilities that showcase NASA's large volume of lunar data collected from past and current Moon missions. It is particularly relevant as the new Administration has more plans for the Moon. We will illustrate big data visualization and analysis in easy-touse and interactive mediums for diverse use.

Malhotra, Shan↗
NASA NTRS2015

High Resolution Nature Runs and the Big Data Challenge

NASA's Global Modeling and Assimilation Office at Goddard Space Flight Center is undertaking a series of very computationally intensive Nature Runs and a downscaled reanalysis. The nature runs use the GEOS-5 as an Atmospheric General Circulation Model (AGCM) while the reanalysis uses the GEOS-5 in Data Assimilation mode. This paper will present computational challenges from three runs, two of which are AGCM and one is downscaled reanalysis using the full DAS. The nature runs will be completed at two surface grid resolutions, 7 and 3 kilometers and 72 vertical levels. The 7 km run spanned 2 years (2005-2006) and produced 4 PB of data while the 3 km run will span one year and generate 4 BP of data. The downscaled reanalysis (MERRA-II Modern-Era Reanalysis for Research and Applications) will cover 15 years and generate 1 PB of data. Our efforts to address the big data challenges of climate science, we are moving toward a notion of Climate Analytics-as-a-Service (CAaaS), a specialization of the concept of business process-as-a-service that is an evolving extension of IaaS, PaaS, and SaaS enabled by cloud computing. In this presentation, we will describe two projects that demonstrate this shift. MERRA Analytic Services (MERRA/AS) is an example of cloud-enabled CAaaS. MERRA/AS enables MapReduce analytics over MERRA reanalysis data collection by bringing together the high-performance computing, scalable data management, and a domain-specific climate data services API. NASA's High-Performance Science Cloud (HPSC) is an example of the type of compute-storage fabric required to support CAaaS. The HPSC comprises a high speed Infinib and network, high performance file systems and object storage, and a virtual system environments specific for data intensive, science applications. These technologies are providing a new tier in the data and analytic services stack that helps connect earthbound, enterprise-level data and computational resources to new customers and new mobility-driven applications and modes of work. In our experience, CAaaS lowers the barriers and risk to organizational change, fosters innovation and experimentation, and provides the agility required to meet our customers' increasing and changing needs

big data analysis↗
NASA NTRS2018

Introduction to Analysis Methods for Big Earth Data

Big Earth Data are too big to be tractable to simple data inspection and require models to make sense of all the data. Useful models for Big Earth Data may be physical, statistical, or machine learning based. In many cases, hybrid models combine attributes of two or more of these types.

parallel processing (computers)↗
NASA NTRS2022

Introduction to Analysis Methods for Big Earth Data

Big Earth Data are too big to be tractable to simple data inspection. Thus, they typically require models to make sense of all the data. Useful models for Big Earth Data may be physical, statistical, or machine learning based. While physical models are ideal for understanding the data, they are not always feasible, particularly when our ability to observe at finer scales exceeds our ability to incorporate the physics. Statistical models are more generalized, but computationally intensive for many Earth Observation datasets. Machine Learning models generally scale well but are sometimes limited in the physical understanding they can offer. Hybrid models combine attributes—and advantages—of two or more of these types.

Christopher Lynnes↗
DOE OSTI2021

Exploratory analysis and performance prediction of big data transfer in High-performance Networks

Big data transfer in large-scale scientific and business applications is increasingly carried out over connections with guaranteed bandwidth provisioned in High-performance Networks (HPNs) via advance bandwidth reservation. Provisioning agents need to carefully schedule data transfer requests, compute network paths, and allocate appropriate bandwidths. Such reserved bandwidths, if not fully utilized, could be simply wasted due to the exclusive access during the approved time window, and cause extra overhead and complexity for resource management. This calls for accurate performance prediction to reserve bandwidths that match actual needs and avoid over-provisioning. We employ machine learning algorithms to predict big data transfer performance based on extensive performance measurements collected in the past several years from data transfer tests using different protocols and toolkits between various end sites on several real-life physical or emulated testbeds. We first analyze the performance patterns in response to a comprehensive list of parameters in end-host systems, network connections, and data transfer applications, which motivate the use of machine learning and also help us identify the effects of latent factors. We then propose threshold- and clustering-based methods to eliminate negative effects of latent factors in data preprocessing and build a robust performance predictor based on customized domain-oriented loss functions. The performance of the proposed methods is verified by extensive experiments using SVR and RFR as well as theoretical analysis of the general performance bound.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Inverse size-dependent Stokes shift in strongly quantum confined CsPbBr 3 perovskite nanoplates

Colloidal semiconductor nanocrystals (NCs) are used as bright chromatic fluorophores for energy-efficient displays. Here we focus here on the size-dependent Stokes shift for CsPbBr 3 nanocrystals. The Stokes shift, i.e., the difference between the wavelengths of absorption and emission maxima, is crucial for display application, as it controls the degree to which light is reabsorbed by the emitting material reducing the energetic efficiency. One major impediment to the industrial adoption of NCs is that slight deviations in manufacturing conditions may result in a wide dispersion of the product's properties. A data-driven analysis of over 2000 reactions comparing two data sets, one produced via standard colloidal synthesis and the other via high-throughput automated synthesis is discussed. We show that differences in the reaction conditions of colloidal CsPbBr 3 nanocrystals yield nanocrystals with opposite Stokes shift size-dependent trends. These match the morphologies of two-dimensional nanoplatelets (NPLs) and nanocrystal cubes. The Stokes shift size dependence trend of NPLs and nanocubes is non-monotonic indicating different physics is at play for the two nanocrystal morphologies. For nanocrystals with cubic shape, with the increase of edge length, there is a significant decrease in Stokes shift values. However, for NPLs with the increase of thickness (1–4 ML), Stokes shift values will increase. The study emphasizes the transition from a spectroscopic point of view and relates the two Stokes shift trends to 2D and 0D exciton dimensionalities for the two morphologies. Our findings highlight the importance of CsPbBr 3 nanocrystal morphology for Stokes shift prediction.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Street context of various demographic groups in their daily mobility

Abstract We present an urban science framework to characterize phone users’ exposure to different street context types based on network science, geographical information systems (GIS), daily individual trajectories, and street imagery. We consider street context as the inferred usage of the street, based on its buildings and construction, categorized in nine possible labels. The labels define whether the street is residential, commercial or downtown, throughway or not, and other special categories. We apply the analysis to the City of Boston, considering daily trajectories synthetically generated with a model based on call detail records (CDR) and images from Google Street View. Images are categorized both manually and using artificial intelligence (AI). We focus on the city’s four main racial/ethnic demographic groups (White, Black, Hispanic and Asian), aiming to characterize the differences in what these groups of people see during their daily activities. Based on daily trajectories, we reconstruct most common paths over the street network. We use street demand (number of times a street is included in a trajectory) to detect each group’s most relevant streets and regions. Based on their street demand, we measure the street context distribution for each group. The inclusion of images allows us to quantitatively measure the prevalence of each context and points to qualitative differences on where that context takes place. Other AI methodologies can further exploit these differences. This approach presents the building blocks to further studies that relate mobile devices’ dynamic records with the differences in urban exposure by demographic groups. The addition of AI-based image analysis to street demand can power up the capabilities of urban planning methodologies, compare multiple cities under a unified framework, and reduce the crudeness of GIS-only mobility analysis. Shortening the gap between big data-driven analysis and traditional human classification analysis can help build smarter and more equal cities while reducing the efforts necessary to study a city’s characteristics.

Salgado, Ariel (ORCID:0000000177015372)↗
DOE OSTI2024

Secondary Ion Mass Spectral Imaging of Metals and Alloys

Secondary Ion Mass Spectrometry (SIMS) is an outstanding technique for Mass Spectral Imaging (MSI) due to its notable advantages, including high sensitivity, selectivity, and high dynamic range. As a result, SIMS has been employed across many domains of science. In this review, we provide an in-depth overview of the fundamental principles underlying SIMS, followed by an account of the recent development of SIMS instruments. The review encompasses various applications of specific SIMS instruments, notably static SIMS with time-of-flight SIMS (ToF-SIMS) as a widely used platform and dynamic SIMS with Nano SIMS and large geometry SIMS as successful instruments. We particularly focus on SIMS utility in microanalysis and imaging of metals and alloys as materials of interest. Additionally, we discuss the challenges in big SIMS data analysis and give examples of machine leaning (ML) and Artificial Intelligence (AI) for effective MSI data analysis. Finally, we recommend the outlook of SIMS development. It is anticipated that in situ and operando SIMS has the potential to significantly enhance the investigation of metals and alloys by enabling real-time examinations of material surfaces and interfaces during dynamic transformations.

36 MATERIALS SCIENCE↗
NASA NTRS2016

Self-Aware Vehicles: Mission and Performance Adaptation to System Health

Advances in sensing (miniaturization, distributed sensor networks) combined with improvements in computational power leading to significant gains in perception, real-time decision making/reasoning and dynamic planning under uncertainty as well as big data predictive analysis have set the stage for realization of autonomous system capability. These advances open the design and operating space for self-aware vehicles that are able to assess their own capabilities and adjust their behavior to either complete the assigned mission or to modify the mission to reflect their current capabilities. This paper discusses the self-aware vehicle concept and associated technologies necessary for full exploitation of the concept. A self-aware aircraft, spacecraft or system is one that is aware of its internal state, has situational awareness of its environment, can assess its capabilities currently and project them into the future, understands its mission objectives, and can make decisions under uncertainty regarding its ability to achieve its mission objectives.

Gregory, Irene M.↗
NASA NTRS2018

NASA Pilot-Engaged Expert Response Using IBM Watson Technology: Prototype Evaluation of Knowledge Retrieval System

NASA Langley Research Center and IBM have been investigating the use of IBM Watson technology in aerospace research and development. One application of Watson technology is the Pilot-Engaged Expert Response (PEER) use case. The PEER system is envisioned as an in-cockpit advisor that will act as a source of situationally-relevant information for pilots and other flight crew members to assist in decision making about real-time events and situations that arise in the course of aircraft operations. PEER will make available vast stores of knowledge and information quickly and directly, putting important informational resources where they are needed most. IBM has worked with NASA to develop an architecture and articulate a roadmap for the development of the PEER system. That vision is built around Watson Discovery Advisor (WDA) software solution, derived from IBM's Jeopardy!-winning automatic question answering system. PEER makes use of WDA's sophisticated question-answering capabilities as its core, adding important User Interface components and other customizations for the cockpit environment, including communication with flight systems and other external data sources. The development plan for PEER includes four development stages, with the current project constituting the first phase. In this project, a prototype instance of PEER was successfully adapted to the aviation domain, enabling users to ask questions about aviation topics and receive useful and accurate answers to these questions. Major tasks accomplished include the development of procedures for domain adaptation through automatic lexicon extraction from domain glossaries; generation of question-answer training data which was used to train the system; and assessment of the effectiveness of domain adaptation, which showed a dramatic improvement in the ability of the PEER system to answer domain-relevant questions. In addition, the vision for the PEER system was pushed forward by the articulation of a plan for the automatic enhancement of question-answering with contextual information. This initial phase focused on two main goals: 1) the targeted domain adaptation of the underlying WDA system to the aviation domain; and, 2) the design of the software systems needed to leverage flight-contextual data. Domain adaptation of the WDA system proceeds via three main activities: Domain data ingestion, lexical customization and model training. A textual corpus consisting of 1,147 individual documents with more than 7.5 million words of text was ingested into the system and this served as the basis of all further development. A domain lexicon of over 3,500 aviation-domain terms was semi-automatically generated from domain documents and used to train the system. In addition, a set of over 500 question-answer (QA) pairs relevant to the PEER use case was developed; these were used to train and assess the system. These important first steps established the basis for the PEER system. In addition, steps were taken towards the integration of the PEER system into the cockpit environment with the development of a functional design for the Contextual Data Augmentation (CDA) subsystem. This subsystem brings to bear contextual data to improve system responses. It has three main submodules: the Contextual Data Collection module, the Contextual Data Selection module, and the Contextual QA Augmentation module. These modules form a processing pipeline that addresses the problems associated with automatically integrating information from external resources into the knowledge-retrieval mechanism.

Machine learning↗
DOE OSTI2020

On the Marriage of Asynchronous Many Task Runtimes and Big Data: A Glance

The rise of the accelerator-based architectures and reconfigurable computing have showcased the weakness of software stack toolchains that still maintain a static view of the hardware instead of relying on a symbiotic relationship between static (e.g., compilers) and dynamic tools (e.g., runtimes). In the past decades, this need has given rise to adaptive runtimes with increasingly finer computational tasks. These finer tasks help to take advantage of the hardware by switching out when a long latency operation is encountered (because of the deeper memory hierarchies and new memory technologies that might target streaming instead of random access), thus trading off idle time for unrelated work. Examples of these finer task runtimes are Asynchronous Many Task (AMT) runtimes, in which highly efficient computational graphs run on a variety of hardware. Due to its inherent latency tolerant characteristics, Latency-sensitive applications, such as Graph Analytics and Big Data can effectively use these runtimes. This paper aims to present an example of how the careful design of an AMT can exploit the hardware substrate when faced with high latency applications such as the ones given in the Big Data domain. Moreover, with its introspection and adaptive capabilities, we aim to show the power of these runtimes when facing the changing requirements of the application workloads. We use the Performance Open Community Runtime (P-OCR) as our vehicle to demonstrate the concepts presented here.

adaptive runtime, big data analysis↗
DOE OSTI2020

Big-Data-Driven Geo-Spatiotemporal Correlation Analysis between Precursor Pollen and Influenza and its Implication to Novel Coronavirus Outbreak

Although studies of many respiratory viruses and pollens are often framed by both seasonal and health related perspectives, pollen has yet to be extensively examined as an important covariate to seasonal respiratory viruses (SRVs) in any context, including a causal one. This study contributes to those goals through an investigation of SRVs and pollen counts at selected regions across the Western Hemisphere. Two complementary decadal-scaled geospatial profiles were developed. One laterally spanned the US and was anchored by detailed pollen information for Albuquerque, New Mexico. The other straddled the equator to include Fortaleza, Brazil. We found that the geospatial and climatological patterns of pollen advancement and decline across the US every year presented a statistically significant correlation to the subsequent emergence and decline of SRVs. Other significant covariates included winds, temperatures, and atmospheric moisture. Our study indicates that areas of the US with lower geostrophic wind baselines are typically areas of persistently higher and earlier influenza like illness (ILI) cases. In addition to that continental- scaled contrast, many sites indicated seasonal highs of geostrophic winds and ILI which were closely aligned. These observations suggest extensive scale-dependent connectivity of viruses to geostrophic circulation. Pollen emergence and its own scale-dependent circulation may contribute to the geospatial and seasonal patterns of ILI. We explore some uncertainties associated with this investigation, and consider the possibility that in a temperate climate, following a Spring pollen emergence, a resulting increase in pollen triggered human Immunoglobulin E (IgE) antibodies may suppress ILIs for several months.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

A mathematical assessment of the isolation random forest method for anomaly detection in big data

We present the mathematical analysis of the Isolation Random Forest Method (IRF Method) for anomaly detection, proposed by Liu F.T., Ting K.M. and Zhou Z. H. in their seminal work as a heuristic method for anomaly detection in Big Data. We prove that the IRF space can be endowed with a probability induced by the Isolation Tree algorithm (iTree). In this setting, the convergence of the IRF method is proved, using the Law of Large Numbers. Here, a couple of counterexamples are presented to show that the method is inconclusive and no certificate of quality can be given, when using it as a means to detect anomalies. Hence, an alternative version of the method is proposed whose mathematical foundation is fully justified. Furthermore, a criterion for choosing the number of sampled trees needed to guarantee confidence intervals of the numerical results is presented. Finally, numerical experiments are presented to compare the performance of the classic method with the proposed one.

97 MATHEMATICS AND COMPUTING↗