Search NASA⌕ Search

SEARCH · Search NASA

Results for “big data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

Earth Science Data Analytics: Preparing for Extracting Knowledge from Information

Data analytics is the process of examining large amounts of data of a variety of types to uncover hidden patterns, unknown correlations and other useful information. Data analytics is a broad term that includes data analysis, as well as an understanding of the cognitive processes an analyst uses to understand problems and explore data in meaningful ways. Analytics also include data extraction, transformation, and reduction, utilizing specific tools, techniques, and methods. Turning to data science, definitions of data science sound very similar to those of data analytics (which leads to a lot of the confusion between the two). But the skills needed for both, co-analyzing large amounts of heterogeneous data, understanding and utilizing relevant tools and techniques, and subject matter expertise, although similar, serve different purposes. Data Analytics takes on a practitioners approach to applying expertise and skills to solve issues and gain subject knowledge. Data Science, is more theoretical (research in itself) in nature, providing strategic actionable insights and new innovative methodologies. Earth Science Data Analytics (ESDA) is the process of examining, preparing, reducing, and analyzing large amounts of spatial (multi-dimensional), temporal, or spectral data using a variety of data types to uncover patterns, correlations and other information, to better understand our Earth. The large variety of datasets (temporal spatial differences, data types, formats, etc.) invite the need for data analytics skills that understand the science domain, and data preparation, reduction, and analysis techniques, from a practitioners point of view. The application of these skills to ESDA is the focus of this presentation. The Earth Science Information Partners (ESIP) Federation Earth Science Data Analytics (ESDA) Cluster was created in recognition of the practical need to facilitate the co-analysis of large amounts of data and information for Earth science. Thus, from a to advance science point of view: On the continuum of ever evolving data management systems, we need to understand and develop ways that allow for the variety of data relationships to be examined, and information to be manipulated, such that knowledge can be enhanced, to facilitate science. Recognizing the importance and potential impacts of the unlimited ways to co-analyze heterogeneous datasets, now and especially in the future, one of the objectives of the ESDA cluster is to facilitate the preparation of individuals to understand and apply needed skills to Earth science data analytics. Pinpointing and communicating the needed skills and expertise is new, and not easy. Information technology is just beginning to provide the tools for advancing the analysis of heterogeneous datasets in a big way, thus, providing opportunity to discover unobvious scientific relationships, previously invisible to the science eye. And it is not easy It takes individuals, or teams of individuals, with just the right combination of skills to understand the data and develop the methods to glean knowledge out of data and information. In addition, whereas definitions of data science and big data are (more or less) available (summarized in Reference 5), Earth science data analytics is virtually ignored in the literature, (barring a few excellent sources).

data analytics↗
NASA NTRS1981

LANDSAT demonstration/application and GIS integration in south central Alaska

Automated geographic information systems were developed for two sites in Southcentral Alaska to serve as tests for both the process of integrating classified LANDSAT data into a comprehensive environmental data base and the process of using automated information in land capability/suitability analysis and environmental planning. The Big Lake test site, located approximately 20 miles north of the City of Anchorage, comprises an area of approximately 150 square miles. The Anchorage Hillside test site, lying approximately 5 miles southeast of the central part of the city, extends over an area of some 25 square miles. Map construction and content is described.

Burns, A. W.↗
NASA NTRS2016

Climate Analytics as a Service

Exascale computing, big data, and cloud computing are driving the evolution of large-scale information systems toward a model of data-proximal analysis. In response, we are developing a concept of climate analytics as a service (CAaaS) that represents a convergence of data analytics and archive management. With this approach, high-performance compute-storage implemented as an analytic system is part of a dynamic archive comprising both static and computationally realized objects. It is a system whose capabilities are framed as behaviors over a static data collection, but where queries cause results to be created, not found and retrieved. Those results can be the product of a complex analysis, but, importantly, they also can be tailored responses to the simplest of requests. NASA's MERRA Analytic Service and associated Climate Data Services API provide a real-world example of climate analytics delivered as a service in this way. Our experiences reveal several advantages to this approach, not the least of which is orders-of-magnitude time reduction in the data assembly task common to many scientific workflows.

big data↗
NASA NTRS1974

Experiment to evaluate feasibility of utilizing Skylab-EREP remote sensing data for tectonic analysis of the Bighorn Mountains region, Wyoming-Montana

The author has identified the following significant results. S-190A color transparencies from SL-2 of the Big Horn basin region provide the best format to date for geologic study of that region; red beds are quite mappable and resistant key beds sharply outlined. An S-190B color frame from SL-3 of the Pryor-Bighorn mountains provides no indication that the Nye-Bowler lineament extends east of East Pryor Mountain. This has important implications regarding the role of this and other lineaments (which also appear to be of restricted length) in the tectonics of the region. Extensions of these lineaments for great distances does not seem warranted on the basis of surface evidence.

Hoppin, R. A.↗
NASA NTRS2011

Coupling Between Doppler Radar Signatures and Tornado Damage Tracks

On April 27, 2011, the southeastern United States was raked with several episodes of severe weather. Numerous tornadoes caused extensive damage, and tragically, the deaths of over 300 people. In Alabama alone, there were 61 confirmed tornados, 4 of them produced EF5 damage, and several were on the ground an hour or more with continuous damage tracks exceeding 80km. The use of Doppler radars covering the region provided reflectivity and velocity signatures that allowed forecasters to monitors the severe storms from beginning to end issuing hundreds of severe weather warnings throughout the day. Meteorologists from the the NWS performed extensive surveys to assess the intensity, duration, and ground track of tornadoes reported during the event. Survey activities included site visits to the affected locations, analysis of radar and satellite data, aerial surveys, and interviews with eyewitnesses. Satellite data from NASA's MODIS and ASTER instruments played a helpful role in determining the location of tornado damage paths and in the assessment. High resolution multispectral and temporal composites helped forecasters corroborate their damage assessments, determine starting and ending points for tornado touchdowns, and helped to provide forecasters with a better big-picture view of the damage region. The imagery also helped to separate damage from the April 27th tornados from severe weather that occurred earlier that month. In a post analysis of the outbreak, tornado damage path signatures observed in the NASA satellite data have been correlated to "debris ball" signatures in the NWS Doppler radars and a special ARMOR dual-polarization radar operated by the University of Alabama Huntsville during the event. The Doppler radar data indicates a circular enhanced reflectivity signal and rotational couplet in the radial velocity likely associated with the tornado that is spatially correlated with the damage tracks in the observed satellite data. An algorithm to detect and isolate the "debris ball" from precipitation signatures in the dual polarization radar data has been developed and verified using the NASA damage track data.

Jedlovec, Gary J.↗
NASA NTRS2017

Investigating Access Performance of Long Time Series with Restructured Big Model Data

Data sets generated by models are substantially increasing in volume, due to increases in spatial and temporal resolution, and the number of output variables. Many users wish to download subsetted data in preferred data formats and structures, as it is getting increasingly difficult to handle the original full-size data files. For example, application research users such as those involved with wind or solar energy, or extreme weather events are likely only interested in daily or hourly model data at a single point (or for a small area) for a long time period, and prefer to have the data downloaded in a single file. With native model file structures, such as hourly data from NASA Modern-Era Retrospective analysis for Research and Applications Version-2 (MERRA-2), it may take over 10 hours for the extraction of parameters-of-interest at a single point for 30 years. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is exploring methods to address this particular user need. One approach is to create value-added data by reconstructing the data files. Taking MERRA-2 data as an example, we have tested converting hourly data from one-day-per-file into different data cubes, such as one-month, or one-year. Performance is compared for reading local data files and accessing data through interoperable services, such as OPeNDAP. Results show that, compared to the original file structure, the new data cubes offer much better performance for accessing long time series. We have noticed that performance is associated with the cube size and structure, the compression method, and how the data are accessed. An optimized data cube structure will not only improve data access, but also may enable better online analysis services

reanalysis↗
NASA NTRS2019

NASA GES DISC Giovanni: Current and Future

Giovanni (Geospatial Interactive Online Visualization and Analysis Infrastructure), developed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), has established a reputation among NASA users for easy access, analysis, and visualization of NASA Earth science data. Currently, Giovanni supports over 1900 variables in eight disciplinary areas. Like any other enterprise application, Giovanni faces big data challenges, such as servicing increasingly large data volumes and more complex data types, while at the same time addressing the demands of a more diverse user community, e.g., placing requests for long-term time series from multiple spatially and temporally dense data records. I will present how Giovanni has been evolving from an on-premises, monolithic software application towards a cloud-enabled implementation to address these challenges.

Analytics↗
NASA NTRS1978

Measurement and analysis of magnetic field variation during a class 2b flare

A digital analysis of high-time-resolution videomagnetograms taken during a class 2b flare that occurred at 60 deg east is reported. The data were obtained at the Big Bear Observatory and calibrated by a Mt. Wilson magnetogram. Changes of weak magnetic fields (less than 100 G) with an amplitude of 30% to 100% have been detected over 55% of the optical flare region, apparently taking place at the initial phase of the flare. Statistical considerations suggest a real flare association with most of these changes. H-alpha observations show that large changes took place over the footpoints of heavily inclined structures like penumbral fibrils, while smaller changes took place over the plage region. An apparent polarity reversal was found at the feet of erupted fibrils. Based on force-free field calculations these changes can be reasonably explained as a transformation of the current-carrying fields to potential fields which produced large changes in the field-line inclination and rotation.

Tanaka, K.↗
NASA NTRS1998

Emission Lines and the High Energy Continuum

Quasars show many striking relationships between line and continuum radiation whose origins remain a mystery. FeII, [OIII], Hbeta, and HeII emission line properties correlate with high energy continuum properties such as the relative strength of X-ray emission, and X-ray continuum slope. At the same time, the shape of the high energy continuum may vary with luminosity. An important tool for studying global properties of Quasi Stellar Objects (QSOs) is the co-addition of data for samples of QSOS. We use this to show that X-ray bright (XB) QSOs show stronger emission lines in general, but particularly from the narrow line region. The difference in the [OIII]/Hbeta ratio is particularly striking, and even more so when blended FeII emission is properly subtracted. Weaker narrow forbidden lines ([OII] and NeV) are enhanced by factors of 2 to 3 in both UV and optical XB composite spectra. The physical origin of these diverse and interrelated correlations has yet to be determined. Unfortunately, many physically informative trends intrinsic to QSOs may be masked by dispersion in the data due to either low signal-to-noise or variability. An important tool for studying global properties of QSOs is the co-addition of data for samples of QSOS. We use this to show that X-ray bright (XB) QSOs show stronger emission lines in general, but particularly from the narrow line region. The difference in the [OIII]/Hbeta ratio is particularly striking, and even more so when blended Fell emission is properly subtracted. Weaker narrow forbidden lines ([OII] and NeV) are enhanced by factors of 2 to 3 in both UV and optical XB composite spectra. We describe a large-scale effort now underway to probe these effects in large samples, using both data and analysis as homogeneous as possible. Using an HST FOS Atlas of QSO spectra, with primary comparison to ROSAT PSPC spectral constraints, we will model the Big Blue Bump, its relationship to luminosity and QSO type, and we will analyze and contrast line emission and UV/X-ray continuum properties. Absorption of the continuum near the broad emission line region may play a profound role, which we will be able to constrain by direct analysis of observed UV/X-ray spectral absorption.

Green, Paul↗
NASA NTRS2014

MERRA Analytic Services: Meeting the Big Data Challenges of Climate Science Through Cloud-enabled Climate Analytics-as-a-service

Climate science is a Big Data domain that is experiencing unprecedented growth. In our efforts to address the Big Data challenges of climate science, we are moving toward a notion of Climate Analytics-as-a-Service (CAaaS). We focus on analytics, because it is the knowledge gained from our interactions with Big Data that ultimately produce societal benefits. We focus on CAaaS because we believe it provides a useful way of thinking about the problem: a specialization of the concept of business process-as-a-service, which is an evolving extension of IaaS, PaaS, and SaaS enabled by Cloud Computing. Within this framework, Cloud Computing plays an important role; however, we it see it as only one element in a constellation of capabilities that are essential to delivering climate analytics as a service. These elements are essential because in the aggregate they lead to generativity, a capacity for self-assembly that we feel is the key to solving many of the Big Data challenges in this domain. MERRA Analytic Services (MERRAAS) is an example of cloud-enabled CAaaS built on this principle. MERRAAS enables MapReduce analytics over NASAs Modern-Era Retrospective Analysis for Research and Applications (MERRA) data collection. The MERRA reanalysis integrates observational data with numerical models to produce a global temporally and spatially consistent synthesis of 26 key climate variables. It represents a type of data product that is of growing importance to scientists doing climate change research and a wide range of decision support applications. MERRAAS brings together the following generative elements in a full, end-to-end demonstration of CAaaS capabilities: (1) high-performance, data proximal analytics, (2) scalable data management, (3) software appliance virtualization, (4) adaptive analytics, and (5) a domain-harmonized API. The effectiveness of MERRAAS has been demonstrated in several applications. In our experience, Cloud Computing lowers the barriers and risk to organizational change, fosters innovation and experimentation, facilitates technology transfer, and provides the agility required to meet our customers' increasing and changing needs. Cloud Computing is providing a new tier in the data services stack that helps connect earthbound, enterprise-level data and computational resources to new customers and new mobility-driven applications and modes of work. For climate science, Cloud Computing's capacity to engage communities in the construction of new capabilies is perhaps the most important link between Cloud Computing and Big Data.

Data Analytics↗
NASA NTRSDate not supplied

NASA’s Prototype Spectral Water Inversion Processor and Emulator (SWIPE): Towards Global Coastal and Inland Water Quality and Algal Biodiversity Monitoring

Degradation of Earth’s inland water resources due to anthropogenic perturbations and climate anomalies at both local and global scales continues to place human health at substantial risk. There is now a growing necessity to develop pragmatic approaches that allow timely and effective extrapolation of local processes, to spatially resolved global products, and to promote operational and sustainable resource policy management. This presentation will provide updates on NASA’s prototype open-source aquatic modeling platform, Spectral Water Inversion Processor and Emulator (SWIPE), which is a comprehensive, multi-faceted modeling platform for both forward and inverse modeling of diverse aquatic ecosystems from the benthos to top-of-atmosphere (TOA). SWIPE provides a cohesive application which leverages recent advancements in particle modeling, Big Data analytics, and machine learning to develop a high-fidelity synthetic training ground for sensitivity studies and algorithm development for multispectral or upcoming hyperspectral missions. Some of the prominent features of SWIPE to be discussed include: 1. Advanced hyperspectral modeling of globally diverse algal and non-algal particles using a novel two-layer coated sphere scattering model and radiative transfer modeling, 2. Massive, highly detailed synthetic spectral libraries of Analysis-Ready-Data (ARD) which include spectral libraries of particle microphysics, water biogeophysical and optical properties, as well as surface and TOA reflectances at 1 nm resolution, 3. An ensemble of pre-built analytic, machine learning, and deep learning inversion algorithms for various water quality and biodiversity related retrieval parameters and uncertainty quantification, 4. Sensor-agnostic water quality inversion at wide ranging spatial and spectral resolutions including a codebase for seamless application in the Google Earth Engine and NASA Earth Exchange (NEX) for planetary scale analysis. SWIPE will be a fully open-source platform based in python with comprehensive documentation, tutorials, and options for distributed computing on high performance computing clusters or on single, local machines. Further, we will discuss how we envision SWIPE contributing towards a global analysis of coastal and inland water quality dynamics.

top-of-atmosphere (TOA)↗
NASA NTRS2016

An Integrated Gate Turnaround Management Concept Leveraging Big Data/Analytics for NAS Performance Improvements

The Integrated Gate Turnaround Management (IGTM) concept was developed to improve the gate turnaround performance at the airport by leveraging relevant historical data to support optimization of airport gate operations, which include: taxi to the gate, gate services, push back, taxi to the runway, and takeoff, based on available resources, constraints, and uncertainties. By analyzing events of gate operations, primary performance dependent attributes of these events were identified for the historical data analysis such that performance models can be developed based on uncertainties to support descriptive, predictive, and prescriptive functions. A system architecture was developed to examine system requirements in support of such a concept. An IGTM prototype was developed to demonstrate the concept using a distributed network and collaborative decision tools for stakeholders to meet on time pushback performance under uncertainties.

Big Data and Net-enabled ATM↗
NASA NTRS2016

Achieving Fast Operational Intelligence in NASA's Deep Space Network Through Complex Event Processing

NASA’s Deep Space Network (DSN) is a complex, global project, in which the expertise of human operators remain crucial for its successful operation. To find ways to save costs in operations and to improve its services, a number of modernization efforts are underway in the DSN. One such effort is a research and technology development task at the Jet Propulsion Laboratory that is investigating the use of complex event processing (CEP) for intelligent assessment of situations, trend analysis, and advanced automation. The technology leverages the significant business intelligence (BI) and data science advancements made in the enterprise industries over the last several years. The open source big data processing engine Apache SparkTM and the high-throughput, distributed messaging system Apache Kafka form the core of the DSN Complex Event Processing (DCEP) framework. This paper discusses the system engineering perspective of why achieving efficient, lower-cost operations in the DSN is a challenging problem, how the DCEP system handles the use cases that help realize intelligent operations, and how this solution fits into the overall model of the planned DSN Follow-the- Sun Operations (FtSO).

Choi, Joshua S.↗
NASA NTRS1986

AIS spectra of desert shrub canopies

Airborne Imaging Spectrometer (AIS) data were collected 30 August 1985 from a desert shrub community in central Oregon. Spectra from artificial targets placed on the test site and from bare soil, big sagebrush (Artemesia tridentata wyomingensis), silver sagebrush (Artemesia cana bolander), and exposed volcanic rocks were studied. Spectral data from grating position 3 (tree mode) were selected from 25 ground positions for analysis by Principal Factor Analysis (PFA). In this grating position, as many as six factors were identified as significant in contributing to spectral structure. Channels 74 through 84 (tree mode) best characterized between-class differences. Other channels were identified as nondiscriminating and as associated with such errors as excessive atmospheric absorption and grating positin changes. The test site was relatively simple with the two species (A. tridentata and A. cana) representing nearly 95% of biomass and with only two mineral backgrounds, a montmorillonitic soil and volcanic rocks. If, as in this study, six factors of spectral structure can be extracted from a single grating position from data acquired over a simple vegetation community, then AIS data must be considered rich in information-gathering potential.

Murray, R.↗
NASA NTRS2001

Estimating Expressed Temperature and Fractional Area of Hot Lava at the Kilauea Vent with AVIRIS Spectral Measurements

Imaging spectroscopy offers a framework based in physics and chemistry for scientific investigation of a wide range of phenomena of interest in the Earth environment. In the scientific discipline of volcanology knowledge of lava temperature and distribution at the surface provides insight into the volcano status and subsurface processes. A remote sensing strategy to measure surface lava temperatures and distribution would support volcanology research. Hot targets such as molten lava emit spectral radiance as a function of temperature. A figure shows a series of Planck functions calculated radiance spectra for hot targets at different temperatures. A maximum Lambertian solar reflected radiance spectrum is shown as well. While similar in form, each hot target spectrum has a unique spectral shape and is distinct from the solar reflected radiance spectrum. Based on this temperature-dependent signature, imaging spectroscopy provides an innovative approach for the remote-sensing-based measurement of lava temperature. A natural site for investigation of the measurement of lava temperature is the Big Island of Hawaii where molten lava from the Kilauea vent is present at the surface. In the past, Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) data sets have been used for the analysis of hot volcanic targets and hot burning fires. The research presented here builds upon and extends this earlier work. The year 2000 Hawaii AVIRIS data set has been analyzed to derive lava temperatures taking into account factors of fractional fill, solar reflected radiance, and atmospheric attenuation of the surface emitted radiance. The measurements, analyses, and current results for this research are presented here.

Green, Robert O.↗
NASA NTRS1974

Jupiter after Pioneer - A progress report

In December 1973, Pioneer 10 became the first spacecraft to reach the vicinity of Jupiter. The spacecraft passed through the Jovian magnetosphere in two weeks and sent back more than 300 pictures of the big planet. Measurements were conducted of EM fields, energetic particles, and micrometeoroids. Radio occultations observed are discussed along with observations in the infrared and ultraviolet range, magnetic measurements, questions of trajectory analysis, and data obtained with the aid of a plasma analyzer. Pioneer 10 has confirmed as inescapable the fact that Jupiter radiates more energy than it receives from the sun.

Mcdonough, T. R.↗
NASA NTRS2019

Integrated Analysis of Multiple User Metrics - A “Sequel”; and Introducing the Google Analytic

For decades, the Goddard Earth Sciences Data and Information Services Center (GES DISC) has archived and distributed enormous volumes of NASA Earth science data (accompanied with many developed tools and services) to various research/applications communities and the general public. Being “immersed” in the Big Data era, we have inevitably faced the challenges of our continually increasing archived data in both volume and variety, as well as enhanced user needs and demands. In recent years, we have actively analyzed different types of user metrics, such as operational distribution metrics (recording numbers of distinct users and downloaded data files, size of distributed data volume): user publication metrics (mining info from our Giovanni users’ publications): and Bugzilla metrics (collecting info from user questions or feedback from user assistance tickets). Such metrics have helped us achieve a better understanding of user needs, demands, characteristics, and behaviors, which has then helped us improve our user services. Now we will present a “Sequel” of integrated analysis of multiple metrics at the GES DISC by introducing and adding one new kind of metrics acquired via utilizing our recently implemented Google Analytic 360 suite. Several “newer” reports, e.g., “What web site features and links are the most popular (and least)?” and “What are the top 25 dataset Keyword searches?” retrieved from this new metrics set will be presented, along with the aforementioned “traditional” metrics results.

Shie, Chung-Lin↗
NASA NTRS2014

Research Data Alliance: Understanding Big Data Analytics Applications in Earth Science

The Research Data Alliance (RDA) enables data to be shared across barriers through focused working groups and interest groups, formed of experts from around the world - from academia, industry and government. Its Big Data Analytics (BDA) interest groups seeks to develop community based recommendations on feasible data analytics approaches to address scientific community needs of utilizing large quantities of data. BDA seeks to analyze different scientific domain applications (e.g. earth science use cases) and their potential use of various big data analytics techniques. These techniques reach from hardware deployment models up to various different algorithms (e.g. machine learning algorithms such as support vector machines for classification). A systematic classification of feasible combinations of analysis algorithms, analytical tools, data and resource characteristics and scientific queries will be covered in these recommendations. This contribution will outline initial parts of such a classification and recommendations in the specific context of the field of Earth Sciences. Given lessons learned and experiences are based on a survey of use cases and also providing insights in a few use cases in detail.

Riedel, Morris↗