Search NASA⌕ Search

SEARCH · Search NASA

Results for “big data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS1974

Jupiter after Pioneer - A progress report

In December 1973, Pioneer 10 became the first spacecraft to reach the vicinity of Jupiter. The spacecraft passed through the Jovian magnetosphere in two weeks and sent back more than 300 pictures of the big planet. Measurements were conducted of EM fields, energetic particles, and micrometeoroids. Radio occultations observed are discussed along with observations in the infrared and ultraviolet range, magnetic measurements, questions of trajectory analysis, and data obtained with the aid of a plasma analyzer. Pioneer 10 has confirmed as inescapable the fact that Jupiter radiates more energy than it receives from the sun.

Mcdonough, T. R.↗
NASA NTRS2019

Integrated Analysis of Multiple User Metrics - A “Sequel”; and Introducing the Google Analytic

For decades, the Goddard Earth Sciences Data and Information Services Center (GES DISC) has archived and distributed enormous volumes of NASA Earth science data (accompanied with many developed tools and services) to various research/applications communities and the general public. Being “immersed” in the Big Data era, we have inevitably faced the challenges of our continually increasing archived data in both volume and variety, as well as enhanced user needs and demands. In recent years, we have actively analyzed different types of user metrics, such as operational distribution metrics (recording numbers of distinct users and downloaded data files, size of distributed data volume): user publication metrics (mining info from our Giovanni users’ publications): and Bugzilla metrics (collecting info from user questions or feedback from user assistance tickets). Such metrics have helped us achieve a better understanding of user needs, demands, characteristics, and behaviors, which has then helped us improve our user services. Now we will present a “Sequel” of integrated analysis of multiple metrics at the GES DISC by introducing and adding one new kind of metrics acquired via utilizing our recently implemented Google Analytic 360 suite. Several “newer” reports, e.g., “What web site features and links are the most popular (and least)?” and “What are the top 25 dataset Keyword searches?” retrieved from this new metrics set will be presented, along with the aforementioned “traditional” metrics results.

Shie, Chung-Lin↗
DOE OSTI2025

Enhancing DESI DR1 full-shape analyses using HOD-informed priors

We present an analysis of DESI Data Release 1 (DR1) that incorporates Halo Occupation Distribution (HOD)-informed priors into Full-Shape (FS) modeling of the power spectrum based on cosmological perturbation theory (PT). By leveraging physical insights from the galaxy-halo connection, these HOD-informed priors on nuisance parameters substantially mitigate projection effects in extended cosmological models that allow for dynamical dark energy. The resulting credible intervals now encompass the posterior maximum from the baseline analysis using gaussian priors, eliminating a significant posterior shift observed in baseline studies. In the ΛCDM framework, a combined DESI DR1 FS information and constraints from the DESI DR1 baryon acoustic oscillations (BAO) — including Big Bang Nucleosynthesis (BBN) constraints and a weak prior on the scalar spectral index — yields Ω m = 0.2994 ± 0.0090 and σ 8 = 0.836$^{+0.024}_{-0.027}$, representing improvements of approximately 4% and 23% over the baseline analysis, respectively. For the w 0 w a CDM model, our results from various data combinations are highly consistent, with all configurations converging to a region with w 0 > -1 and w a < 0. This convergence not only suggests intriguing hints of dynamical dark energy but also underscores the robustness of our HOD-informed prior approach in delivering reliable cosmological constraints.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Distributed Machine Learning Workflow with PanDA and iDDS in LHC ATLAS

Machine Learning (ML) has become one of the important tools for High Energy Physics analysis. As the size of the dataset increases at the Large Hadron Collider (LHC), and at the same time the search spaces become bigger and bigger in order to exploit the physics potentials, more and more computing resources are required for processing these ML tasks. In addition, complex advanced ML workflows are developed in which one task may depend on the results of previous tasks. How to make use of vast distributed CPUs/GPUs in WLCG for these big complex ML tasks has become a popular research area. In this paper, we present our efforts enabling the execution of distributed ML workflows on the Production and Distributed Analysis (PanDA) system and intelligent Data Delivery Service (iDDS). First, we describe how PanDA and iDDS deal with large-scale ML workflows, including the implementation to process workloads on diverse and geographically distributed computing resources. Next, we report real-world use cases, such as HyperParameter Optimization, Monte Carlo Toy confidence limits calculation, and Active Learning. Finally, we conclude with future plans.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2014

Research Data Alliance: Understanding Big Data Analytics Applications in Earth Science

The Research Data Alliance (RDA) enables data to be shared across barriers through focused working groups and interest groups, formed of experts from around the world - from academia, industry and government. Its Big Data Analytics (BDA) interest groups seeks to develop community based recommendations on feasible data analytics approaches to address scientific community needs of utilizing large quantities of data. BDA seeks to analyze different scientific domain applications (e.g. earth science use cases) and their potential use of various big data analytics techniques. These techniques reach from hardware deployment models up to various different algorithms (e.g. machine learning algorithms such as support vector machines for classification). A systematic classification of feasible combinations of analysis algorithms, analytical tools, data and resource characteristics and scientific queries will be covered in these recommendations. This contribution will outline initial parts of such a classification and recommendations in the specific context of the field of Earth Sciences. Given lessons learned and experiences are based on a survey of use cases and also providing insights in a few use cases in detail.

Riedel, Morris↗
DOE OSTI2025

DESI 2024 VI: cosmological constraints from the measurements of baryon acoustic oscillations

We present cosmological results from the measurement of baryon acoustic oscillations (BAO) in galaxy, quasar and Lyman-α forest tracers from the first year of observations from the Dark Energy Spectroscopic Instrument (DESI), to be released in the DESI Data Release 1. DESI BAO provide robust measurements of the transverse comoving distance and Hubble rate, or their combination, relative to the sound horizon, in seven redshift bins from over 6 million extragalactic objects in the redshift range 0.1 < z < 4.2. To mitigate confirmation bias, a blind analysis was implemented to measure the BAO scales. DESI BAO data alone are consistent with the standard flat ΛCDM cosmological model with a matter density Ω m =0.295±0.015. Paired with a baryon density prior from Big Bang Nucleosynthesis and the robustly measured acoustic angular scale from the cosmic microwave background (CMB), DESI requires H 0 =(68.52±0.62) km s -1 Mpc -1 . In conjunction with CMB anisotropies from Planck and CMB lensing data from Planck and ACT, we find Ω m =0.307± 0.005 and H 0 =(67.97±0.38) km s -1 Mpc -1 . Extending the baseline model with a constant dark energy equation of state parameter w, DESI BAO alone requirew=-0.99 +0.15 -0.13 . In models with a time-varying dark energy equation of state parametrised by w 0 and w a , combinations of DESI with CMB or with type Ia supernovae (SN Ia) individually prefer w 0 > -1 and w a < 0. This preference is 2.6σ for the DESI+CMB combination, and persists or grows when SN Ia are added in, giving results discrepant with the ΛCDM model at the 2.5σ, 3.5σ or 3.9σ levels for the addition of the Pantheon+, Union3, or DES-SN5YR supernova datasets respectively. For the flat ΛCDM model with the sum of neutrino mass ∑ m ν free, combining the DESI and CMB data yields an upper limit ∑ m ν < 0.072 (0.113) eV at 95% confidence for a ∑ m ν > 0 (∑ m ν > 0.059) eV prior. These neutrino-mass constraints are substantially relaxed if the background dynamics are allowed to deviate from flat ΛCDM.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Energy Dependence Analysis of Neutrino-Nucleus Interactions in Scintillator for MINERvA

This thesis is on the intrinsic energy dependence in neutrino-nucleus interaction models, especially quasielastic scattering and 2p2h process, and the implications for understanding data from MINERvA, NOvA, and DUNE. Neutrino-nucleus interactions are modeled using fve structure functions, W1, W2, W3, W4 and W5, and each structure function contributes diferently to the energy dependence once it enters the cross-section calculations. The energy independent term in the calculations contains only W2 and the next lower order term (of the order 1/E) contains W3 also. In this thesis, we show that W2 is good enough to describe MINERvA data and energy dependence coming from the cross-section directly is small. However, W3 will be signifcant for lower energy experiments such as NOvA and DUNE (especially at and below the oscillation maximum). We have also developed a method to extract structure functions directly from MINERvA data though we find that systematic uncertainties present in the data are too big to successfully determine 1/E Term. In addition to this, we have applied muon angle cuts on MINERvA data and MC, which have their own energy dependent effects. An apparent energy dependent discrepancy comes from known shortcomings especially in the pion production model. Finally, because the energy dependence analysis has similarities to the low-$\nu$ method, we have evaluated discrepancies between the data and MC in light of diferent fux constraint techniques. The results suggest including MINERvA s low-$\nu$ method constraint is an improvement.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS2019

What's My Prior? Baby Steps Toward Big Data

After four decades of single-event effects testing, radiation engineers face pressure to find alternatives to reduce or eliminate testing costs. Many of these alternatives focus on use of archival test data to bound SEE risk for similar parts. This presentation examines the challenges and limitations of such analyses and suggests analysis techniques and goals that are likely to be most appropriate for such efforts.

statistics and probability↗
NASA NTRS1978

QCGAT mixer compound exhaust system design and static big model test report

A mixer exhaust system was designed to meet the proposed performance and exhaust jet noise goals for the AiResearch QCGAT engine. Some 0.35 scale models of the various nozzles were fabricated and aerodynamically and acoustically tested. Preliminary optimization, engine cycle matching, model test data and analysis are presented. A final mixer exhaust system is selected for optimum performance for the overall flight regime.

Blackmore, W. L.↗
NASA NTRS2017

Engine Icing Data - An Analytics Approach

Engine icing researchers at the NASA Glenn Research Center use the Escort data acquisition system in the Propulsion Systems Laboratory (PSL) to generate and collect a tremendous amount of data every day. Currently these researchers spend countless hours processing and formatting their data, selecting important variables, and plotting relationships between variables, all by hand, generally analyzing data in a spreadsheet-style program (such as Microsoft Excel). Though spreadsheet-style analysis is familiar and intuitive to many, processing data in spreadsheets is often unreproducible and small mistakes are easily overlooked. Spreadsheet-style analysis is also time inefficient. The same formatting, processing, and plotting procedure has to be repeated for every dataset, which leads to researchers performing the same tedious data munging process over and over instead of making discoveries within their data. This paper documents a data analysis tool written in Python hosted in a Jupyter notebook that vastly simplifies the analysis process. From the file path of any folder containing time series datasets, this tool batch loads every dataset in the folder, processes the datasets in parallel, and ingests them into a widget where users can search for and interactively plot subsets of columns in a number of ways with a click of a button, easily and intuitively comparing their data and discovering interesting dynamics. Furthermore, comparing variables across data sets and integrating video data (while extremely difficult with spreadsheet-style programs) is quite simplified in this tool. This tool has also gathered interest outside the engine icing branch, and will be used by researchers across NASA Glenn Research Center. This project exemplifies the enormous benefit of automating data processing, analysis, and visualization, and will help researchers move from raw data to insight in a much smaller time frame.

Engine Icing↗
DOE OSTI2025

A Data Science and Machine Learning Platform Supporting Large Particle Accelerator Control and Diagnostics Applications Final Report: SBIR Initial Phase II DE-SC0022583

The Machine Learning Data Platform (MLDP) is a product providing full-stack support for data science, Machine Learning, and Artificial Intelligence (ML/AI) applications at particle accelerator and large experimental physics facilities. It supports ML/AI applications from front-end, high-speed acquisition of heterogeneous, time-series data, through data archiving and management, to back-end analysis. The MLDP embodies a “data-science ready” platform for data analysis and ML/AI applications in diagnosis, modelling, control, and optimization of these facilities. It provides data scientists and applications a consistent, datacentric interface to archive data standardizing implementation and deployment of ML/AI algorithms to different operations configurations within the same facility, or between facilities. Being an open-source, public-domain project, the MLDP is intended for broadest possible impact by increasing accessibility and minimizing the required expertise for installation and operation. The MLDP can also be deployed at user facilities for experimental data collection, archiving, and analysis. It is capable of acquisition and archiving of heterogeneous data from experimental equipment (e.g., images, arrays, structures, etc.) along with system hardware configurations (e.g., scalars, tables), control system process variables, and any metadata required for provenance. Thus, the MLDP can manage experimental data through its entire lifecycle, from acquisition and archiving, through analysis and investigation, to release and final publication.

43 PARTICLE ACCELERATORS↗
DOE OSTI2023

ML for microbiomes

The software provides machine learning analysis and visualization to detect patterns in microbiome data, including topic modeling, probabilistic graphical modeling, conventional machine learning methods, and deep learning. The software is written in python and R, it uses some python and R libraries as well as big open-source libraries like sklearn, networkX, pytorch (python), pgmpy (python), and bnlearn (R). It also has a script to use for MALLET and DTM (open-source packages for topic modeling, written in Java).

Kim, Anastasiia↗
NASA NTRS2015

Supercomputing in the Age of Discovering Superearths, Earths and Exoplanet Systems

With the Kepler Mission, exoplanet science has entered the era of Big Data and has benefited tremendously from the availability of the NAS (NASA Advanced Supercomputing Division) Pleiades supercomputer. This presentation provides an overview of the history of exoplanet discoveries and highlights the key enabling technologies for Kepler. I provide an overview of the Kepler science pipeline and the algorithms used to analyze the data and detect transiting planets, all of which run routinely on the NAS Pleiades system. The presentation also provides a description of the Blender analysis program used to validate Kepler planet discoveries statistically, and the effort to discover exomoons lurking in Kepler data, both of which also run on Pleiades. Finally, I describe NASA's TESS (Transiting Exoplanet Survey Satellite) Mission, which leverages Kepler heritage to provide a science pipeline similar to that for Kepler and that will also run routinely on the NAS Pleiades supercomputer.

big data↗
NASA NTRS2017

Time and Measurement Days

Questions in data analysis involving the concepts of time and measurement are often pushed into the background or reserved for a philosophical discussion. Some examples are: a) Is causality a consequence of the laws of physics, or can the arrow of time be reversed? b) Can we determine the arrow of time of an event? c) Do we need the continuum hypothesis for the underlying function in any measurement process? d) Can we say anything about the analyticity of the underlying process of an event? e) Would it be valid to model a non-analytical process as function of time? f) What are the implications of all these questions for classical Fourier techniques? However, in the age of big data gathered either from space missions supplying ultra-precise long time series, or e.g. LIGO data from the ground, the moment to bring these questions to the foreground seems arrived. The limitations of our understanding of some fundamental processes is emphasized by the lack of solution for problems open for more than 2 decades, such as the non-detection of solar g-modes, or the modal identification of main sequence stellar pulsators like delta Scuti stars. Flicker noise or 1/f noise, for example, attributed in solar-like stars to granulation, is analyzed mostly only to apply noise reduction techniques, neither considering the classical problem of 1/f noise that was introduced a 100 years ago, nor taking into account ergodic or non-ergodic solutions that make inapplicable spectral analysis techniques in practice. This topic was discussed by Nicholas W. Watkins during the ITISE meeting held in Granada in 2016. There he presented preliminary results of his research on Mandelbrot's related work. We reproduce here his quotation of Mandelbrot (1999) "There is a sharp contrast between a highly anomalous ("non-white") noise that proceeds in ordinary clock time and a noise whose principal anomaly is that it is restricted to fractal time", suggesting a connection with the above proposed topics that could be phrased as the following additional questions:a) Is self-organized criticality (SOC) frequent in astrophysical phenomena? b) Could all fractals in nature be considered stochastic? c) Could we establish mathematical/physical relationships between chaotic and fractal behaviors in time series? d) Could the differences between fractals and chaos in terms of analyticity be used to understand the residuals of the fitting of stellar light curves? In this meeting we would like to approximate these problems from a holistic and multidisciplinary perspective, taking into account not only technical issues but also the deeper implications. In particular the concept of connectivity (introduced in Pascual-Granado et al. A&A, 2015) could be used to implement, within the framework of ARMA processes, an "arrow of time" (see attached document), and so studying the possible implications in the concept of time as envisaged by Watkins.

data analysis↗
DOE OSTI2023

Automated classification of big X-ray diffraction data using deep learning models

Abstract In current in situ X-ray diffraction (XRD) techniques, data generation surpasses human analytical capabilities, potentially leading to the loss of insights. Automated techniques require human intervention, and lack the performance and adaptability required for material exploration. Given the critical need for high-throughput automated XRD pattern analysis, we present a generalized deep learning model to classify a diverse set of materials’ crystal systems and space groups. In our approach, we generate training data with a holistic representation of patterns that emerge from varying experimental conditions and crystal properties. We also employ an expedited learning technique to refine our model’s expertise to experimental conditions. In addition, we optimize model architecture to elicit classification based on Bragg’s Law and use evaluation data to interpret our model’s decision-making. We evaluate our models using experimental data, materials unseen in training, and altered cubic crystals, where we observe state-of-the-art performance and even greater advances in space group classification.

Chemistry↗
DOE OSTI2025

Total X-ray scattering and big-box modeling of pressure-induced local disorder and partial amorphization in CsPbBr 3

The mechanisms governing pressure-induced amorphization and its reversibility in halide perovskites are not yet fully understood, particularly the contribution of local disorder. We performed high-pressure synchrotron total X-ray scattering and reverse Monte Carlo (RMC) big-box modeling using CsPbBr 3 as a model system to investigate short-range structural evolution in both the ordered and partially amorphous phases. While diffraction data indicate that long-range order persists up to 2 GPa, pair distribution function (PDF) analysis reveals significant local distortions, including PbBr 6 octahedral tilting and Cs displacement, which influence the bandgap through a complex interplay between bond compression and angular tilting. Beyond 2 GPa, CsPbBr 3 undergoes partial amorphization, with significant disordering of Cs and Br, while the Pb sublattice remains preserved, allowing for structural recovery upon decompression. Our work, accounting for both short- and long-range structural evolution through RMC modeling, successfully captures how disorder shapes the structural response of halide perovskites under pressure.

critical phenomena↗
NASA NTRS2022

Coevolution of Machine Learning and Process-Based Modelling to Revolutionize Earth and Environmental Sciences: A Perspective

Machine learning (ML) applications in Earth and environmental sciences (EES) have gained incredible momentum in recent years. However, these ML applications have largely evolved in ‘isolation’ from the mechanistic, process-based modelling (PBM) paradigms, which have historically been the cornerstone of scientific discovery and policy support. In this perspective, we assert that the cultural barriers between the ML and PBM communities limit the potential of ML, and even its ‘hybridization’ with PBM, for EES applications. Fundamental, but often ignored, differences between ML and PBM are discussed as well as their strengths and weaknesses in light of three overarching modelling objectives in EES, (1) nowcasting and prediction, (2) scenario analysis, and (3) diagnostic learning. The paper ponders over a ‘coevolutionary’ approach to model building, shifting away from a borrowing to a co-creation culture, to develop a generation of models that leverage the unique strengths of ML such as scalability to big data and high-dimensional mapping, while remaining faithful to process-based knowledge base and principles of model explainability and interpretability, and therefore, falsifiability.

Saman Razavi↗