Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Intensive Science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

PSTN-019: The LSST Science Pipelines Software: Optical Survey Pipeline Reduction and Analysis Environment

The NSF-DOE Vera C. Rubin Observatory is executing the Legacy Survey of Space and Time (LSST) as its prime mission, producing a series of data releases over the ten-year survey. The LSST Science Pipelines Software will be used to create these data releases and to perform the nightly prompt processing and alert production. This paper provides an overview of the LSST Science Pipelines Software, describing the components and their integration into pipelines that generate science-ready data products.

79 ASTRONOMY AND ASTROPHYSICS↗

Detecting Nanophase Weathering Products with CheMin: Reference Intensity Ratios of Allophane, Aluminosilicate Gel, and Ferrihydrite

X-ray diffraction (XRD) data collected of the Rocknest samples by the CheMin instrument on Mars Science Laboratory suggest the presence of poorly crystalline or amorphous materials [1], such as nanophase weathering products or volcanic and impact glasses. The identification of the type(s) of X-ray amorphous material at Rocknest is important because it can elucidate past aqueous weathering processes. The presence of volcanic and impact glasses would indicate that little chemical weathering has occurred because glass is highly susceptible to aqueous alteration. The presence of nanophase weathering products, such as allophane, nanophase iron-oxides, and/or palagonite, would indicate incipient chemical weathering. Furthermore, the types of weathering products present could help constrain pH conditions and identify which primary phases altered to form the weathering products. Quantitative analysis of phases from CheMin data is achieved through Reference Intensity Ratios (RIRs) and Rietveld refinement. The RIR of a mineral (or mineraloid) that relates the scattering power of that mineral (typically the most intense diffraction line) to the scattering power of a separate mineral standard such as corundum [2]. RIRs can be calculated from XRD patterns measured in the laboratory by mixing a mineral with a standard in known abundances and comparing diffraction line intensities of the mineral to the standard. X-ray amorphous phases (e.g., nanophase weathering products) have broad scattering signatures rather than sharp diffraction lines. Thus, RIRs of X-ray amorphous materials are calculated by comparing the area under one of these broad scattering signals with the area under a diffraction line in the standard. Here, we measured XRD patterns of nanophase weathering products (allophane, aluminosilicate gel, and ferrihydrite) mixed with a mineral standard (beryl) in the CheMinIV laboratory instrument and calculated their RIRs to help constrain the abundances of these phases in the Rocknest samples.

Rampe, E. B.↗

Real-Time On-Board Processing Validation of MSPI Ground Camera Images

The Earth Sciences Decadal Survey identifies a multiangle, multispectral, high-accuracy polarization imager as one requirement for the Aerosol-Cloud-Ecosystem (ACE) mission. JPL has been developing a Multiangle SpectroPolarimetric Imager (MSPI) as a candidate to fill this need. A key technology development needed for MSPI is on-board signal processing to calculate polarimetry data as imaged by each of the 9 cameras forming the instrument. With funding from NASA's Advanced Information Systems Technology (AIST) Program, JPL is solving the real-time data processing requirements to demonstrate, for the first time, how signal data at 95 Mbytes/sec over 16-channels for each of the 9 multiangle cameras in the spaceborne instrument can be reduced on-board to 0.45 Mbytes/sec. This will produce the intensity and polarization data needed to characterize aerosol and cloud microphysical properties. Using the Xilinx Virtex-5 FPGA including PowerPC440 processors we have implemented a least squares fitting algorithm that extracts intensity and polarimetric parameters in real-time, thereby substantially reducing the image data volume for spacecraft downlink without loss of science information.

Earth Sciences Decadal Survey↗

Artificial Intelligence (AI) Methods for Automating the Impact Tool Evidence Library

INTRODUCTION: The development of the Evidence Library for use with the IMPACT (Informing Mission Planning via Analysis of Complex Tradespaces) probability risk assessment tool involved a multilayered, time intensive process of data collection and analysis by subject matter experts from the Exploration Medical Capability (ExMC) Element Clinical and Science Team to produce clinical findings forms (CliFFs) for 120 medical conditions. Artificial Intelligence Large Language Models (LLMs) can be leveraged to facilitate this process, thus reducing labor and time. TOPIC: CliFFs contain information about medical conditions as they pertain to spaceflight. This includes condition definitions, incidence data, crew task impairment estimates caused by conditions, treatment protocols and references to literature used for gathering condition evidence. Guided by the Evidence Library Methods document and the CliFF development instructions, a team has leveraged Microsoft Azure AI services and open-source documentation to construct an AI-assisted automated pipeline for CliFF development. This process is designed to search, retrieve, and evaluate the applicable data, and ultimately generate a completed CliFF. The LLM evaluates the relevance of each of the source materials to spaceflight, either as direct evidence or as an analog. The model extracts keywords and generates brief summaries to enhance search and retrieval in later stages of CliFF development. For instance, it can calculate epidemiological statistical data, such as incidence rates and the likelihood of best or worst-case scenarios. APPLICATION: Large Language Models (LLMs) can efficiently summarize large amounts of text. Leveraging this technology will automate data retrieval and evidence gathering for medical databases, like the IMPACT tool, by aiding in the labor-intensive process of analyzing large bodies of literature and organizing it into a formatted document like a CliFF. This added efficiency will enable expeditious expansion of the Evidence Library with additional medical conditions and update previous CLiFFs as new technology becomes available.

Ali Al↗

Brochure on the 2024 ASCR Workshop on Analog Computing for Science

Analog computing fundamentally differs from digital by representing data with fully continuous physical quantities, such as voltages, probabilities, chemical concentrations, or light intensities, rather than encoding values in discrete binary states. While digital computing has historically excelled in precision, scalability, and noise resistance, newer analog approaches are gaining interest for their potential to dramatically improve energy efficiency and processing speed. Analog systems can inherently solve mathematical problems through their physical behavior and offer distinct advantages in scenarios where continuous operations are more effective than Boolean logic.

97 MATHEMATICS AND COMPUTING↗

Earth Science Deep Learning: Applications and Lessons Learned

Deep Learning: A subfield of machine learning; Algorithms inspired by function of the brain; Scales with amount of training data; Powerful tool without the need for feature engineering; Suitable for Earth Science applications. Deep Learning for Earth science at MSFC (Marshall Space Flight Center): Phenomena identification; Hurricane intensity (wind speed) estimation; Severe storm (hailstorm) detection; Transverse bands detection; Entity extraction for knowledge graph creation; Ephemeral water detection.

Labeled Data↗

Models and Methods for Sparse (Hyper)Network Science in Business, Industry, and Government

The authors are hosting an AMS sponsored Mathematics Research Community (MRC) focusing on two themes that have garnered intense attention in network models of complex relational data: (1) how to faithfully model multi-way relations in hypergraphs, rather than only pairwise interactions in graphs; and (2) challenges posed by modelling networks with extreme sparsity. Here we introduce and explore these two themes and their challenges. In this work, we hope to generate interest from researchers in pure and applied mathematics and computer science.

97 MATHEMATICS AND COMPUTING↗

Uncertainty in inventories for life cycle assessment: State‐of‐the‐art, challenges, and new technologies

Uncertainty is a critical factor that can hinder the quality and potential applications of life cycle assessment (LCA) results. A prominent source of uncertainty stems from the life cycle inventory (LCI) data. Various methodologies exist to estimate the uncertainty associated with LCI data, primarily based on the widely used structured pedigree matrix approach or the computationally intensive Monte Carlo simulation. This perspective review explores how new technologies (e.g., computational algorithms and data collection methods) from data science and related fields can contribute to identifying, quantifying, and reducing uncertainty in LCI modeling. A brief overview of the sources of uncertainty in LCI modeling and how they are addressed in current LCA practice is provided. Additionally, several new technologies are identified, and the potential benefits of their implementation in reducing uncertainties in LCI modeling are discussed. This perspective review concludes by identifying potential areas that require further development for these technologies.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Mini-apps for high performance data analysis

Scaling-up scientific data analysis and machine learning algorithms for data-driven discovery is a grand challenge that we face today. Despite the growing need for analysis from science domains that are generating `Big Data' from instruments and simulations, building high-performance analytical workflows of data-intensive algorithms have been daunting because: (i) the `Big Data' hardware and software architecture landscape is constantly evolving, (ii) newer architectures impose new programming models, and (iii) data-parallel kernels of analysis algorithms and their performance facets on different architectures are poorly understood. To address these problems, we have: (i) identified scalable data-parallel kernels of popular data analysis algorithms, (ii) implemented `Mini-Apps' of those kernels using different programming models (e.g. Map Reduce, MPI, etc.), (iii) benchmarked and validated the performance of the kernels in diverse architectures. In this paper, we discuss two of those Mini-Apps and show the execution of principal component analysis built as a workflow of the Mini-Apps. We show that Mini-Apps enable scientists to (i) write domain-specific data analysis code that scales on most HPC hardware and (ii) and offers the ability (most times with over a 10x speed-up) to analyze data sizes 100 times the size of what off-the-shelf desktop/workstations of today can handle.

Sukumar, Sreenivas R.↗

On-demand data analytics in HPC environments at leadership computing facilities: Challenges and experiences

Scaling-up scientific data analysis and machine learning algorithms for data-driven discovery is a grand challenge that we face today. Despite the growing need for analysis from science domains that are generating `Big Data' from instruments and simulations, building high-performance analytical workflows of data-intensive algorithms have been daunting because: (i) the `Big Data' hardware and software architecture landscape is constantly evolving, (ii) newer architectures impose new programming models, and (iii) data-parallel kernels of analysis algorithms and their performance facets on different architectures are poorly understood. To address these problems, we have: (i) identified scalable data-parallel kernels of popular data analysis algorithms, (ii) implemented `Mini-Apps' of those kernels using different programming models (e.g. Map Reduce, MPI, etc.), (iii) benchmarked and validated the performance of the kernels in diverse architectures. In this paper, we discuss two of those Mini-Apps and show the execution of principal component analysis built as a workflow of the Mini-Apps. We show that Mini-Apps enable scientists to (i) write domain-specific data analysis code that scales on most HPC hardware and (ii) and offers the ability (most times with over a 10x speed-up) to analyze data sizes 100 times the size of what off-the-shelf desktop/workstations of today can handle.

Harney, John↗

Catching the whispers from Uranus

Sophisticated telecommunications techniques are described that were used to acquire images of Uranus, its 14 moons and ten narrow rings darker than coal. The images, equal in quality to those transmitted from Saturn several years earlier despite the signal being weaker by 6 dB due to the increased distance, were received from Voyager 2 during its January 24, 1986 flyby of Uranus. Solutions to the problem of the weakening signal were found in modifications to Voyager's image processing system and NASA's ground tracking network. In April 1985, Voyager's prime flight data computer was reconfigured to accept only nonimaging science data, and its backup, only imaging data; the latter was reprogrammed to determine only arithmetic differences between adjacent pixel intensities rather than absolute intensities. By image compression, equivalent imaging information could be sent at lower bit rates. Instead of Golay coding, Reed-Solomon onboard encoding was used. These techniques gained the equivalent of 4-dB in imaging yield. Additional improvements were gained by using earth station antennas in pairs (the Parkes radio telescope and the Canberra ground station antenna). Moves under way to prepare for the Voyager encounter with Neptune in 1989 are described (using additional antennas and arrays, scaling up the Deep Space Network antennas from 64 m to 70 m, etc.) to assure almost Saturn-equivalent pictures despite a further 3.5-dB drop in signal strength.

Bartok, C. D.↗

Evaluation of the Four-Dimensional Ensemble-Variational Hybrid Data Assimilation with Self-Consistent Regional Background Error Covariance for Improved Hurricane Intensity Forecasts

The feasibility of a hurricane initialization framework based on the Gridpoint Statistical Interpolation (GSI)-based four-dimensional ensemble-variational (GSI-4DEnVar) hybrid data assimilation system for the Hurricane Weather Research and Forecasting model (HWRF) model is evaluated in this study. The system considers the temporal evolution of error covariances via the use of four-dimensional ensemble perturbations that are provided by high-resolution, self-consistent HWRF ensemble forecasts. It is different from the configuration of the GSI-based three-dimensional ensemble-variational (GSI-3DEnVar) hybrid data assimilation system, similar to that used in the operational HWRF, which employs background error covariances provided by coarser-resolution global ensembles from the National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS) ensemble Kalman filtering data assimilation system. In addition, our proposed initialization framework discards the empirical intensity correction in the vortex initialization package that is employed by the GSI-3DEnVar initialization framework in operational HWRF. Data assimilation and numerical simulation experiments for Hurricanes Joaquin (2015), Patricia (2015), and Matthew (2016) are conducted during their intensity changes. The impacts of two initialization frameworks on the HWRF analyses and forecasts are compared. It is found that GSI-4DEnVar leads to a reduction in track, minimum sea level pressure (MSLP), and maximum surface wind (MSW) forecast errors in all of the HWRF simulations, compared with the GSI-3DEnVar initialization framework. With assimilating high-resolution observations within the hurricane inner-core region, GSI-4DEnVar can produce the initial hurricane intensity reasonably well without the empirical vortex intensity correction. Further diagnoses with Hurricane Joaquin indicate that GSI-4DEnVar can significantly alleviate the imbalances in the initial conditions and enhance the performance of the data assimilation and subsequent hurricane intensity and precipitation forecasts

54 ENVIRONMENTAL SCIENCES↗

User Guide for TROPICS Data Products

This document provides information for using the data products available from the “Time-Resolved Observations of Precipitation structure and storm Intensity with a Constellation of Smallsats” (TROPICS) mission. The TROPICS mission will produce a range of data products that will be available at the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC). The data products will be produced at the TROPICS Data Processing Center (UW-M SSEC), and consist of Level-1 radiances (antenna and brightness temperatures), Level-2a unified resolution radiance, Level-2b Atmospheric Vertical Temperature Profiles (AVTP), Level-2b Atmospheric Vertical Moisture Profiles (AVMP), Level-2b Instantaneous Surface Rain Rate (ISRR), and Level-2b Tropical Cyclone (TC) intensity algorithms to estimate two primary variables: Minimum Sea Level Pressure (MSLP) and Maximum Sustained Winds (MSW). Two independent intensity estimation methods are included: 1) the Tropical Cyclone Intensity Estimate algorithm (TCIE) developed at the University of Wisconsin/CIMSS using native microwave brightness temperatures and 2) the Hurricane Intensity and Structure Algorithm (HISA) developed at Colorado State University/CIRA using microwave retrievals of temperature, moisture, and integrated quantities. In addition to MSW and MSLP, HISA also provides estimates of surface wind radii and 2D winds at standard pressure levels. TROPICS is adapting the NOAA STAR Microwave Integrated Retrieval System (MIRS) to retrieve the AVTP and AVMP data products. The ISRR algorithm uses the NASA Goddard Precipitation Retrieval and Profiling Scheme (PRPS).

TROPICS↗

Data Albums: An Event Driven Search, Aggregation and Curation Tool for Earth Science

One of the largest continuing challenges in any Earth science investigation is the discovery and access of useful science content from the increasingly large volumes of Earth science data and related information available. Approaches used in Earth science research such as case study analysis and climatology studies involve gathering discovering and gathering diverse data sets and information to support the research goals. Research based on case studies involves a detailed description of specific weather events using data from different sources, to characterize physical processes in play for a specific event. Climatology-based research tends to focus on the representativeness of a given event, by studying the characteristics and distribution of a large number of events. This allows researchers to generalize characteristics such as spatio-temporal distribution, intensity, annual cycle, duration, etc. To gather relevant data and information for case studies and climatology analysis is both tedious and time consuming. Current Earth science data systems are designed with the assumption that researchers access data primarily by instrument or geophysical parameter. Those who know exactly the datasets of interest can obtain the specific files they need using these systems. However, in cases where researchers are interested in studying a significant event, they have to manually assemble a variety of datasets relevant to it by searching the different distributed data systems. In these cases, a search process needs to be organized around the event rather than observing instruments. In addition, the existing data systems assume users have sufficient knowledge regarding the domain vocabulary to be able to effectively utilize their catalogs. These systems do not support new or interdisciplinary researchers who may be unfamiliar with the domain terminology. This paper presents a specialized search, aggregation and curation tool for Earth science to address these existing challenges. The search tool automatically creates curated "Data Albums", aggregated collections of information related to a specific science topic or event, containing links to relevant data files (granules) from different instruments; tools and services for visualization and analysis; and information about the event contained in news reports, images or videos to supplement research analysis. Curation in the tool is driven via an ontology based relevancy ranking algorithm to filter out non-relevant information and data.

Ramachandran, Rahul↗

ASC Data Science Infrastructure Project June 2023 [Slides]

Data Science Infrastructure (DSI) Project offers a standardized but flexible way of storing data and associated metadata to support AI/ML and data intensive scientific workflows: Establish requirements via use case interviews; Develop DSI framework to support flexible, secure, and accessible data stores tailored to researcher needs; Augment existing institutional frameworks for code management, performance testing and file indexing via automated DSI interfaces; Deploy DSI framework to support key use cases in collaboration with domain scientists.

97 MATHEMATICS AND COMPUTING↗

Community Coordinated Modeling Center Support of Science Needs for Integrated Data Environment

Space science models are essential component of integrated data environment. Space science models are indispensable tools to facilitate effective use of wide variety of distributed scientific sources and to place multi-point local measurements into global context. The Community Coordinated Modeling Center (CCMC) hosts a set of state-of-the- art space science models ranging from the solar atmosphere to the Earth's upper atmosphere. The majority of models residing at CCMC are comprehensive computationally intensive physics-based models. To allow the models to be driven by data relevant to particular events, the CCMC developed an online data file generation tool that automatically downloads data from data providers and transforms them to required format. CCMC provides a tailored web-based visualization interface for the model output, as well as the capability to download simulations output in portable standard format with comprehensive metadata and user-friendly model output analysis library of routines that can be called from any C supporting language. CCMC is developing data interpolation tools that enable to present model output in the same format as observations. CCMC invite community comments and suggestions to better address science needs for the integrated data environment.

Kuznetsova, M. M.↗

FRACTURE MECHANICS APPROACH TO ESTIMATE FATIGUE LIVES OF WELDED LAP-SHEAR SPECIMENS

A full range of stress intensity factor solutions for a kinked crack is developed as a function of weld width and the sheet thickness. When used with the associated main crack solutions (global stress intensity factors) in terms of the applied load and specimen geometry, the fatigue lives can be estimated for the laser-welded lap-shear specimens. The estimations are in good agreement with the experimental data. A classical solution for an infinitesimal kink is also employed in the approach. However, the life predictions tend to overestimate the actual fatigue lives. The traditional life estimations with the structural stress along with the experimental stress-fatigue life data (S-N curve) are also provided. In this case, the estimations only agree with the experimental data under higher load conditions.

36 MATERIALS SCIENCE↗