Search NASA⌕ Search

SEARCH · Search NASA

Results for “big data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

What's My Prior? Baby Steps Toward Big Data

After four decades of single-event effects testing, radiation engineers face pressure to find alternatives to reduce or eliminate testing costs. Many of these alternatives focus on use of archival test data to bound SEE risk for similar parts. This presentation examines the challenges and limitations of such analyses and suggests analysis techniques and goals that are likely to be most appropriate for such efforts.

statistics and probability↗

QCGAT mixer compound exhaust system design and static big model test report

A mixer exhaust system was designed to meet the proposed performance and exhaust jet noise goals for the AiResearch QCGAT engine. Some 0.35 scale models of the various nozzles were fabricated and aerodynamically and acoustically tested. Preliminary optimization, engine cycle matching, model test data and analysis are presented. A final mixer exhaust system is selected for optimum performance for the overall flight regime.

Blackmore, W. L.↗

Engine Icing Data - An Analytics Approach

Engine icing researchers at the NASA Glenn Research Center use the Escort data acquisition system in the Propulsion Systems Laboratory (PSL) to generate and collect a tremendous amount of data every day. Currently these researchers spend countless hours processing and formatting their data, selecting important variables, and plotting relationships between variables, all by hand, generally analyzing data in a spreadsheet-style program (such as Microsoft Excel). Though spreadsheet-style analysis is familiar and intuitive to many, processing data in spreadsheets is often unreproducible and small mistakes are easily overlooked. Spreadsheet-style analysis is also time inefficient. The same formatting, processing, and plotting procedure has to be repeated for every dataset, which leads to researchers performing the same tedious data munging process over and over instead of making discoveries within their data. This paper documents a data analysis tool written in Python hosted in a Jupyter notebook that vastly simplifies the analysis process. From the file path of any folder containing time series datasets, this tool batch loads every dataset in the folder, processes the datasets in parallel, and ingests them into a widget where users can search for and interactively plot subsets of columns in a number of ways with a click of a button, easily and intuitively comparing their data and discovering interesting dynamics. Furthermore, comparing variables across data sets and integrating video data (while extremely difficult with spreadsheet-style programs) is quite simplified in this tool. This tool has also gathered interest outside the engine icing branch, and will be used by researchers across NASA Glenn Research Center. This project exemplifies the enormous benefit of automating data processing, analysis, and visualization, and will help researchers move from raw data to insight in a much smaller time frame.

Engine Icing↗

Supercomputing in the Age of Discovering Superearths, Earths and Exoplanet Systems

With the Kepler Mission, exoplanet science has entered the era of Big Data and has benefited tremendously from the availability of the NAS (NASA Advanced Supercomputing Division) Pleiades supercomputer. This presentation provides an overview of the history of exoplanet discoveries and highlights the key enabling technologies for Kepler. I provide an overview of the Kepler science pipeline and the algorithms used to analyze the data and detect transiting planets, all of which run routinely on the NAS Pleiades system. The presentation also provides a description of the Blender analysis program used to validate Kepler planet discoveries statistically, and the effort to discover exomoons lurking in Kepler data, both of which also run on Pleiades. Finally, I describe NASA's TESS (Transiting Exoplanet Survey Satellite) Mission, which leverages Kepler heritage to provide a science pipeline similar to that for Kepler and that will also run routinely on the NAS Pleiades supercomputer.

big data↗

Time and Measurement Days

Questions in data analysis involving the concepts of time and measurement are often pushed into the background or reserved for a philosophical discussion. Some examples are: a) Is causality a consequence of the laws of physics, or can the arrow of time be reversed? b) Can we determine the arrow of time of an event? c) Do we need the continuum hypothesis for the underlying function in any measurement process? d) Can we say anything about the analyticity of the underlying process of an event? e) Would it be valid to model a non-analytical process as function of time? f) What are the implications of all these questions for classical Fourier techniques? However, in the age of big data gathered either from space missions supplying ultra-precise long time series, or e.g. LIGO data from the ground, the moment to bring these questions to the foreground seems arrived. The limitations of our understanding of some fundamental processes is emphasized by the lack of solution for problems open for more than 2 decades, such as the non-detection of solar g-modes, or the modal identification of main sequence stellar pulsators like delta Scuti stars. Flicker noise or 1/f noise, for example, attributed in solar-like stars to granulation, is analyzed mostly only to apply noise reduction techniques, neither considering the classical problem of 1/f noise that was introduced a 100 years ago, nor taking into account ergodic or non-ergodic solutions that make inapplicable spectral analysis techniques in practice. This topic was discussed by Nicholas W. Watkins during the ITISE meeting held in Granada in 2016. There he presented preliminary results of his research on Mandelbrot's related work. We reproduce here his quotation of Mandelbrot (1999) "There is a sharp contrast between a highly anomalous ("non-white") noise that proceeds in ordinary clock time and a noise whose principal anomaly is that it is restricted to fractal time", suggesting a connection with the above proposed topics that could be phrased as the following additional questions:a) Is self-organized criticality (SOC) frequent in astrophysical phenomena? b) Could all fractals in nature be considered stochastic? c) Could we establish mathematical/physical relationships between chaotic and fractal behaviors in time series? d) Could the differences between fractals and chaos in terms of analyticity be used to understand the residuals of the fitting of stellar light curves? In this meeting we would like to approximate these problems from a holistic and multidisciplinary perspective, taking into account not only technical issues but also the deeper implications. In particular the concept of connectivity (introduced in Pascual-Granado et al. A&A, 2015) could be used to implement, within the framework of ARMA processes, an "arrow of time" (see attached document), and so studying the possible implications in the concept of time as envisaged by Watkins.

data analysis↗

Coevolution of Machine Learning and Process-Based Modelling to Revolutionize Earth and Environmental Sciences: A Perspective

Machine learning (ML) applications in Earth and environmental sciences (EES) have gained incredible momentum in recent years. However, these ML applications have largely evolved in ‘isolation’ from the mechanistic, process-based modelling (PBM) paradigms, which have historically been the cornerstone of scientific discovery and policy support. In this perspective, we assert that the cultural barriers between the ML and PBM communities limit the potential of ML, and even its ‘hybridization’ with PBM, for EES applications. Fundamental, but often ignored, differences between ML and PBM are discussed as well as their strengths and weaknesses in light of three overarching modelling objectives in EES, (1) nowcasting and prediction, (2) scenario analysis, and (3) diagnostic learning. The paper ponders over a ‘coevolutionary’ approach to model building, shifting away from a borrowing to a co-creation culture, to develop a generation of models that leverage the unique strengths of ML such as scalability to big data and high-dimensional mapping, while remaining faithful to process-based knowledge base and principles of model explainability and interpretability, and therefore, falsifiability.

Saman Razavi↗

An Integrated Gate Turnaround Management Concept Leveraging Big Data Analytics for NAS Performance Improvements

"Gate Turnaround" plays a key role in the National Air Space (NAS) gate-to-gate performance by receiving aircraft when they reach their destination airport, and delivering aircraft into the NAS upon departing from the gate and subsequent takeoff. The time spent at the gate in meeting the planned departure time is influenced by many factors and often with considerable uncertainties. Uncertainties such as weather, early or late arrivals, disembarking and boarding passengers, unloading/reloading cargo, aircraft logistics/maintenance services and ground handling, traffic in ramp and movement areas for taxi-in and taxi-out, and departure queue management for takeoff are likely encountered on the daily basis. The Integrated Gate Turnaround Management (IGTM) concept is leveraging relevant historical data to support optimization of the gate operations, which include arrival, at the gate, departure based on constraints (e.g., available gates at the arrival, ground crew and equipment for the gate turnaround, and over capacity demand upon departure), and collaborative decision-making. The IGTM concept provides effective information services and decision tools to the stakeholders, such as airline dispatchers, gate agents, airport operators, ramp controllers, and air traffic control (ATC) traffic managers and ground controllers to mitigate uncertainties arising from both nominal and off-nominal airport gate operations. IGTM will provide NAS stakeholders customized decision making tools through a User Interface (UI) by leveraging historical data (Big Data), net-enabled Air Traffic Management (ATM) live data, and analytics according to dependencies among NAS parameters for the stakeholders to manage and optimize the NAS performance in the gate turnaround domain. The application will give stakeholders predictable results based on the past and current NAS performance according to selected decision trees through the UI. The predictable results are generated based on analysis of the unique airport attributes (e.g., runway, taxiway, terminal, and gate configurations and tenants), and combined statistics from past data and live data based on a specific set of ATM concept-of-operations (ConOps) and operational parameters via systems analysis using an analytic network learning model. The IGTM tool will then bound the uncertainties that arise from nominal and off-nominal operational conditions with direct assessment of the gate turnaround status and the impact of a certain operational decision on the NAS performance, and provide a set of recommended actions to optimize the NAS performance by allowing stakeholders to take mitigation actions to reduce uncertainty and time deviation of planned operational events. An IGTM prototype was developed at NASA Ames Simulation Laboratories (SimLabs) to demonstrate the benefits and applicability of the concept. A data network, using the System Wide Information Management (SWIM)-like messaging application using the ActiveMQ message service, was connected to the simulated data warehouse, scheduled flight plans, a fast-time airport simulator, and a graphic UI. A fast-time simulation was integrated with the data warehouse or Big Data/Analytics (BAI), scheduled flight plans from Aeronautical Operational Control AOC, IGTM Controller, and a UI via a SWIM-like data messaging network using the ActiveMQ message service, illustrated in Figure 1, to demonstrate selected use-cases showing the benefits of the IGTM concept on the NAS performance.

Efficent ATM systems↗

New Era, New Opportunity, Is GES DISC Ready for Big Data Challenge?

The new era of Big Data has opened doors for many new opportunities, as well as new challenges, for both Earth science research/application and data communities. As one of the twelve NASA data centers - Goddard Earth Sciences Data and Information Services Center (GES DISC), one of our great challenges has been how to help research/application community efficiently (quickly and properly) accessing, visualizing and analyzing the massive and diverse data in natural hazard research, management, or even prediction. GES DISC has archived over 2000 TB data on premises and distributed over 23,000 TB of data since 2010. Our data has been widely used in every phase of natural hazard management and research, i.e. long term risk assessment and reduction, forecasting and predicting, monitoring and detection, early warning, damage assessment and response. The big data challenge is not just about data storage, but also about data discoverability and accessibility, and even more, about data migration/mirroring in the cloud. This paper is going to demonstrate GES DISC’s efforts and approaches of evolving our overall Web services and powerful Giovanni (Geospatial Interactive Online Visualization ANd aNalysis Infrastructure) tool into further improving data discoverability and accessibility. Prototype works will also be presented.

Li, A.↗

Western energy related overhead monitoring project. Phase 2: Summary

Assistance by NASA to EPA in the establishment and maintenance of a fully operational energy-related monitoring system included: (1) regional analysis applications based on LANDSAT and auxiliary data; (2) development of techniques for using aircraft MSS data to rapidly monitor site specific surface coal mine activities; and (3) registration of aircraft MSS data to a map base. The coal strip mines used in the site specific task were in Campbell County, Wyoming; Big Horn County, Montana; and the Navajo mine in San Juan County, New Mexico. The procedures and software used to accomplish these tasks are described.

Anderson, J. E.↗

Spinoff 2005

Topics covered include: Lighting the Way for Quicker, Safer Healing; Discovering New Drugs on the Cellular Level; Hydrogen Sensors Boost Hybrids; Today s Models Losing Gas?; 3-D Highway in the Sky; Popping a Hole in High-Speed Pursuits; Monitoring Wake Vortices for More Efficient Airports; From Rockets to Racecars; All-Terrain Intelligent Robot Braves Battlefront to Save Lives; Keeping the Air Clean and Safe--An Anthrax Smoke Detector; Lightning Often Strikes Twice; Technology That's Ready and Able to Inspect Those Cables; Secure Networks for First Responders and Special Forces; Space Suit Spins; Cooking Dinner at Home--From the Office; Nanoscale Materials Make for Large-Scale Applications; NASA s Growing Commitment: The Space Garden; Bringing Thunder and Lightning Indoors; Forty-Year-Old Foam Springs Back With New Benefits; Experiments With Small Animals Rarely Go This Well; NASA, the Fisherman's Friend; Crystal-Clear Communication a Sweet-Sounding Success; Inertial Motion-Tracking Technology for Virtual 3-D; Then Why Do They Call Earth the Blue Planet?; Valiant 'Zero-Valent' Effort Restores Contaminated Grounds; Harnessing the Power of the Sun; Water and Air Measures That Make 'PureSense'; Remote Sensing for Farmers and Flood Watching; Pesticide-Free Device a Fatal Attraction for Mosquitoes Making the Most of Waste Energy Washing Away the Worries About Germs Celestial Software Scratches More Than the Surface A Search Engine That's Aware of Your Needs Fault-Detection Tool Has Companies 'Mining' Own Business; Software to Manage the Unmanageable; Tracking Electromagnetic Energy With SQUIDs; Taking the Risk Out of Risk Assessment; Satellite and Ground System Solutions at Your Fingertips; Structural Analysis Made 'NESSUSary'; Software of Seismic Proportions Promotes Enjoyable Learning; Making a Reliable Actuator Faster and More Affordable; Cost-Cutting Powdered Lubricant NASA s Radio Frequency Bolt Monitor: A Lifetime of Spinoffs Going End to End to Deliver High-Speed Data; Advanced Joining Technology: Simple, Strong, and Secure; Big Results From a Smaller Gearbox; Low-Pressure Generator Makes Cleanrooms Cleaner; and The Space Laser Business Model.

Source record↗

Nine-Year Wilkinson Microwave Anisotropy Probe (WMAP) Observations: Cosmological Parameter Results

We present cosmological parameter constraints based on the final nine-year Wilkinson Microwave Anisotropy Probe (WMAP) data, in conjunction with a number of additional cosmological data sets. The WMAP data alone, and in combination, continue to be remarkably well fit by a six-parameter Lambda-CDM model. When WMAP data are combined with measurements of the high-l cosmic microwave background anisotropy, the baryon acoustic oscillation scale, and the Hubble constant, the matter and energy densities Omega(sub b)h(exp 2), Omega(sub c)h(exp 2)and Omega(sub Lambda), are each determined to a precision of approx. 1.5%. The amplitude of the primordial spectrum is measured to within 3%, and there is now evidence for a tilt in the primordial spectrum at the 5 sigma level, confirming the first detection of tilt based on the five-year WMAP data. At the end of the WMAP mission, the nine-year data decrease the allowable volume of the six-dimensional Lambda-CDM parameter space by a factor of 68,000 relative to pre-WMAP measurements. We investigate a number of data combinations and show that their Lambda-CDM parameter fits are consistent. New limits on deviations from the six-parameter model are presented, for example: the fractional contribution of tensor modes is limited to r < 0.13 (95% CL); the spatial curvature parameter is limited to Omega(sub kappa) = (0.0027 (sub +0.0039) (sup −0.0038;) the summed mass of neutrinos is limited to Sigma M(sub nu) < 0.44 eV (95% CL); and the number of relativistic species is found to lie within N(sub eff) = 3.84 +/- 0+/-40, when the full data are analyzed. The joint constraint on N(sub eff) and the primordial helium abundance, Y(sub He), agrees with the prediction of standard big bang nucleosynthesis. We compare recent Planck measurements of the Sunyaev-Zel'dovich effect with our seven-year measurements, and show their mutual agreement. Our analysis of the polarization pattern around temperature extrema is updated. This confirms a fundamental prediction of the standard cosmological model and provides a striking illustration of acoustic oscillations and adiabatic initial conditions in the early universe.

cosmology↗

Discreditation of Bobdownsite and the Establishment of Criteria for the Identification of Minerals with Essential Monofluorophosphate (PO3F2-)

Bobdownsite, IMA number 2008-037, was approved as a new mineral by the Commission on New Minerals, Nomenclature and Classification (CNMNC) as the fluorine endmember of the mineral whitlockite. The type locality of bobdownsite is in Big Fish River, Yukon Canada, and bobdownsite was reported to be the first naturally occurring mineral with essential monofluorophosphate (PO3F2-). The type specimen of bobdownsite has been reinvestigated by electron probe microanalysis (EPMA), and our data indicate that fluorine abundances are below detection in the mineral. In addition, we conducted detailed analysis of bobdownsite from the type locality by gas chromatography isotope ratio mass spectrometry, Raman spectroscopy, EPMA, and NMR spectroscopy. These data were compared with previously published data on synthetic monofluorophosphate salts. Collectively, these data indicate that bobdownsite is indistinguishable from whitlockite with a composition along the whitlockite-merrillite solid solution. Bobdownsite is therefore discredited as a valid mineral species. An additional mineral, krásnoite, has been purported to have monofluorophosphate components in its structure, but reexamination of those data indicate that F- in krásnoite forms bonds with Al, similar to OH- bonded to Al in perhamite. Consequently, krásnoite also lacks monofluorophosphate groups, and there are currently no valid mineral species with monofluorophosphate in their structure. We recommend that any future reports of new minerals that contain essential monofluorophosphate anions be vetted by abundance measurements of fluorine, vibrational spectroscopy (both Raman and FTIR), and where paramagnetic components are permissibly low, NMR spectroscopy. Furthermore, we emphasize the importance of using synthetic compounds containing monofluorophosphate anions as a point of comparison in the identification of minerals with essential monofluorophosphate. Structural data that yield satisfactory P-F bond lengths determined by X-ray crystallography, coupled with direct chemical analyses of fluorine in a material do not constitute sufficient evidence alone to identify a new mineral with essential monofluorophosphate anions.

McCubbin, Francis M.↗

Metadata Evaluation and Improvement: Evolving Analysis and Reporting

ESIP Community members create and manage a large collection of environmental datasets that span multiple decades, the entire globe, and many parts of the solar system. Metadata are critical for discovering, accessing, using and understanding these data effectively and ESIP community members have successfully created large collections of metadata describing these data. As part of the White House Big Earth Data Initiative (BEDI), ESDIS has developed a suite of tools for evaluating these metadata in native dialects with respect to recommendations from many organizations. We will describe those tools and demonstrate evolving techniques for sharing results with data providers.

metadata recommendations↗

A Spatiotemporal Indexing Approach for Efficient Processing of Big Array-Based Climate Data with MapReduce

Climate observations and model simulations are producing vast amounts of array-based spatiotemporal data. Efficient processing of these data is essential for assessing global challenges such as climate change, natural disasters, and diseases. This is challenging not only because of the large data volume, but also because of the intrinsic high-dimensional nature of geoscience data. To tackle this challenge, we propose a spatiotemporal indexing approach to efficiently manage and process big climate data with MapReduce in a highly scalable environment. Using this approach, big climate data are directly stored in a Hadoop Distributed File System in its original, native file format. A spatiotemporal index is built to bridge the logical array-based data model and the physical data layout, which enables fast data retrieval when performing spatiotemporal queries. Based on the index, a data-partitioning algorithm is applied to enable MapReduce to achieve high data locality, as well as balancing the workload. The proposed indexing approach is evaluated using the National Aeronautics and Space Administration (NASA) Modern-Era Retrospective Analysis for Research and Applications (MERRA) climate reanalysis dataset. The experimental results show that the index can significantly accelerate querying and processing (10 speedup compared to the baseline test using the same computing cluster), while keeping the index-to-data ratio small (0.0328). The applicability of the indexing approach is demonstrated by a climate anomaly detection deployed on a NASA Hadoop cluster. This approach is also able to support efficient processing of general array-based spatiotemporal data in various geoscience domains without special configuration on a Hadoop cluster.

big data↗

Development of a digital geographic data base for resource planning in a wildland environment

Multiple resource planning requires the ability to access information for several parameters in a coordinated way. Attempts to do this manually, through the use of multiple transparent overlays or similar techniques can become awkward if there are more than a few parameters under consideration. One solution to this problem is to use a computer system to collect and organize the information into a data base that will make access and analysis easier, even for large numbers of parameters. The increase in the types and forms of remote sensing data and the decrease in costs for computer systems in the last decade has made this approach more popular than in the past. This paper describes the development of one such data base for the Big Basin Redwoods State Park in the Santa Cruz Mountains in California. The data base contains information for satellite spectral data, soil type, vegetation type, and hypsographic data and was developed for use in a cooperative project being conducted by personnel from the Remote Sensing Research Program and the California Department of Parks and Recreation

Ritter, P. R.↗

Big-data Efficient and Automated Science Transfer (BEAST): An Open-Source Software Architecture for Arc Jet Data Management, Modeling, and Automation

Big-data Efficient and Automated Science Transfer (BEAST) is a facility data management application developed for the NASA Ames arc jet facilities. The current decentralized data management practices limit statistical tracking, synchronization between video/time series, search capability, data throughput, and data processing speed/efficiency. Consequently, BEAST was developed to provide a new data infrastructure with streamlined data collection, processing, transfer, and analysis. This new framework also seeks to implement the FAIR principles of data stewardship: Findable, Accessible, Interoperable, and Reusable. The BEAST framework is based on a combination of the Python Django web framework and the Python data stack to provide a monolithic, open-source platform for data management, automation, and machine learning. This architecture was chosen for maintainability and scalability for a small, in-house development team. This paper will describe the application framework, deployment, and discuss the benefits and future plans for the system.

Data management↗

Observations of intermediate degree solar oscillations - 1989 April-June

Frequencies, splittings, and line widths from 85 d of full disk Doppler observations of solar p-modes taken between April 4 and June 30, 1989 are presented. Comparison of the present mode parameters with published Big Bear Solar Observatory (BBSO) results yields good agreement in general and is thus a confirmation of their work using an independent instrument and set of analysis routines. Average differences in p-mode frequencies measured by the two experiments in spring-summer 1989 are explained as a result of differences in the exact periods of data collection during a time of rapidly changing solar activity. It is shown that the present a(1) splitting coefficients for p-modes with nu/L less than 45 micro-Hz suffer from a significant systematic error. Evidence is presented to the effect that a detector distortion or alignment problem, not a problem with the power spectra analysis, is the most likely explanation of this a(1) anomaly.

Bachmann, Kurt T.↗

Mining Twitter Data to Augment NASA GPM Validation

The Twitter data stream is an important new source of real-time and historical global information for potentially augmenting the validation program of NASA's Global Precipitation Measurement (GPM) mission. There have been other similar uses of Twitter, though mostly related to natural hazards monitoring and management. The validation of satellite precipitation estimates is challenging, because many regions lack data or access to data, especially outside of the U.S. and in remote and developing areas. The time-varying set of "precipitation" tweets can be thought of as an organic network of rain gauges, potentially providing a widespread view of precipitation occurrence. Twitter provides a large source of crowd for crowdsourcing. During a 24-hour period in the middle of the snow storm this past March in the U.S. Northeast, we collected more than 13,000 relevant precipitation tweets with exact geolocation. The overall objective of our project is to determine the extent to which processed tweets can provide additional information that improves the validation of GPM data. Though our current effort focuses on tweets and precipitation, our approach is general and applicable to other social media and other geophysical measurements. Specifically, we have developed an operational infrastructure for processing tweets, in a format suitable for analysis with GPM data; engaged with potential participants, both passive and active, to "enrich" the Twitter stream; and inter-compared "precipitation" tweet data, ground station data, and GPM retrievals. In this presentation, we detail the technical capabilities of our tweet processing infrastructure, including data abstraction, feature extraction, search engine, context-awareness, real-time processing, and high volume (big) data processing; various means for "enriching" the Twitter stream; and results of inter-comparisons. Our project should bring a new kind of visibility to Twitter and engender a new kind of appreciation of the value of Twitter by the science research communities.

validatio↗