Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Intensive Science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS2004

The Montage architecture for grid-enabled science processing of large, distributed datasets

Montage is an Earth Science Technology Office (ESTO) Computational Technologies (CT) Round III Grand Challenge investigation to deploy a portable, compute-intensive, custom astronomical image mosaicking service for the National Virtual Observatory (NVO). Although Montage is developing a compute- and data-intensive service for the astronomy community, we are also helping to address a problem that spans both Earth and Space science, namely how to efficiently access and process multi-terabyte, distributed datasets. In both communities, the datasets are massive, and are stored in distributed archives that are, in most cases, remote from the available Computational resources. Therefore, state of the art computational grid technologies are a key element of the Montage portal architecture. This paper describes the aspects of the Montage design that are applicable to both the Earth and Space science communities.

virtual observatory↗
DOE OSTI2022

Challenges for Implementing FAIR Digital Objects with High Performance Workflows

New types of workflows are being used in science that couple traditional distributed and high-performance computing (HPC) with data-intensive approaches, and orchestrate ensembles of numerical simulations and artificial intelligence (AI) models. Such workflows may use AI models to supplement computation where numerical simulations may be too computationally expensive, to automate trivial yet time consuming operations, to perform preliminary selections among intractable numbers of combinations in domains as diverse as protein binding, fine-grid climate simulations, and drug discovery.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Report of the 2026 Workshop on Next-Generation Ecosystems for Scientific Computing: Harnessing Community, Software, and AI for Cross-Disciplinary Team Science

Scientific computing is undergoing rapid transformation as advances in artificial intelligence, heterogeneous computing, automation, and data-intensive research reshape not only computational tools but also the institutions, workforce models, and collaborative practices that support scientific discovery. This report synthesizes insights from the 2026 Workshop on Next-Generation Ecosystems for Scientific Computing, the second in a three-year series focused on strengthening scientific computing ecosystems through socio-technical co-design. Workshop discussions identified four interdependent strategic themes: software ecosystems for AI-enabled scientific discovery; trust, validation, and traceability; human-AI teaming and paradigm shifts; and workforce, pedagogy, and governance. The report translates these themes into eight priorities for community action spanning shared research infrastructure, trust and traceability, user experience, human-AI teaming, workforce development, cross-sector coordination, stewardship and sustainability, and evaluation of scientific value. Together, these priorities outline directions for building scientific computing ecosystems that remain trustworthy, sustainable, innovative, and resilient as AI assumes a growing role in scientific work.

AI↗
DOE OSTI2019

The Power of Environmental Observatories for Advancing Multidisciplinary Research, Outreach, and Decision Support: The Case of the Minnesota River Basin

Observatory-scale data collection efforts allow unprecedented opportunities for integrative, multidisciplinary investigations in large, complex watersheds, which can affect management decisions and policy. Through the National Science Foundation-funded REACH (REsilience under Accelerated CHange) project, in collaboration with the Intensively Managed Landscapes-Critical Zone Observatory, we have collected a series of multidisciplinary data sets throughout the Minnesota River Basin in south-central Minnesota, USA, a 43,400-km 2 tributary to the Upper Mississippi River. Postglacial incision within the Minnesota River valley created an erosional landscape highly responsive to hydrologic change, allowing for transdisciplinary research into the complex cascade of environmental changes that occur due to hydrology and land use alterations from intensive agricultural management and climate change. Data sets collected include water chemistry and biogeochemical data, geochemical fingerprinting of major sediment sources, high-resolution monitoring of river bluff erosion, and repeat channel cross-sectional and bathymetry data following major floods. The data collection efforts led to development of a series of integrative reduced complexity models that provide deeper insight into how water, sediment, and nutrients route and transform through a large channel network and respond to change. These models represent the culmination of efforts to integrate interdisciplinary data sets and science to gain new insights into watershed-scale processes in order to advance management and decision making. Finally, the purpose of this paper is to present a synthesis of the data sets and models, disseminate them to the community for further research, and identify mechanisms used to expand the temporal and spatial extent of short-term observatory-scale data collection efforts.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

Computing environments for reproducibility: Capturing the “Whole Tale”

The act of sharing scientific knowledge is rapidly evolving away from traditional articles and presentations to the delivery of executable objects that integrate the data and computational details (e.g., scripts and workflows) upon which the findings rely. This envisioned coupling of data and process is essential to advancing science but faces technical and institutional barriers. The Whole Tale project aims to address these barriers by connecting computational, data-intensive research efforts with the larger research process transforming the knowledge discovery and dissemination process into one where data products are united with research articles to create "living publications" or tales. The Whole Tale focuses on the full spectrum of science, empowering users in the long tail of science, and power users with demands for access to big data and compute resources. We report here on the design, architecture, and implementation of the Whole Tale environment.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2011

NPOESS Preparatory Project (NPP) Science Overview

NPP Instruments are: (1) well understood thanks to instrument comprehensive test, characterization and calibration programs. (2) Government team ready for October 25 launch followed by instrument activation and Intensive Calibration/Validation (ICV). NPP Data Products preliminary work includes: (1) JPSS Center for Satellite Applications and Research (STAR) team ready to support NPP ICV and operational data products. (2) NASA NPP science team ready to support NPP ICV and EOS data continuity.

Butler, James J.↗
DOE OSTI2023

Neural network-based control of an ultrafast laser

With the recent advances in machine learning (ML) and data science (DS), the control, modeling, and analysis of these complex systems continues to improve. In this work, we report on the optimization of the intensity of a femtosecond laser using feedforward neural networks (FFNN) that model the input–output relationships of the data. The input parameters of the system were optimized to achieve the required performance of the femtosecond laser. We propose a neural network-based control system to model the relationship between the spectral amplitude and phase of the input laser pulse at the amplifier input and the shape of the output pulse. Low-jitter laser parameter inputs and the resulting laser pulse duration were modeled, and the resulting correlation between the input and output data was used to optimize the laser pulse. Here, we demonstrate improved processing and laser control performance.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2026

Data-Driven Atomic Physics: Harnessing Machine Learning and High-Repetition-Rate Experiments for Laser-driven HED

High-energy-density plasma experiments are central to progress in atomic physics, fusion energy, and national security science, but they have traditionally been constrained by slow data collection and manual, time-intensive analysis. This project targeted that bottleneck by enabling high-repetition-rate experiments to produce and interpret much larger volumes of data quickly enough to guide experiments while they run.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRSDate not supplied

Enabling Space Biological Knowledge Discovery Through Image and Video Data Sharing

Increased biomedical risks and challenges associated with deep space missions and experiments (cis-Lunar, Mars transit/surface) require new knowledge discovery and development of novel ecosystems. Supporting distant and long-duration missions and experiments requires biological data (from yeast, microbes, fruit flies, C. elegans, plants, crops, rodents, humans) be findable, accessible, interoperable, reusable (FAIR), and maximally open-access. As data-intensive, bioinformatic, meta-analytical, and computer-assisted approaches continue to be a centerpiece of modern research, the NASA Biological and Physical Sciences division is expanding its Open Science capabilities beyond NASA GeneLab. The NASA Ames Life Sciences Data Archive (ALSDA) is a repository which is responsible for collecting and access to space biological imagery and video, alongside tabular and environmental data. In this presentation, we will discuss strategies dealing with archiving, curating, and accessibility of images from very distinct imaging modalities (e.g., micro-computed tomography, magnetic resonance imaging, photographic images of plants, fluorescence microscopy, behavioral videos, etc.). There are two main challenges: 1. Open-source data storage and 2. Metadata related to the imagery-video. Both have been solved by leveraging two existing open-source systems. For data storage, ALSDA is utilizing components through the Open Microscopy Environment (OME), which can read most imaging proprietary formats and display on a web interface complex multidimensional images (Z stack, multi-channel, temporal, spectral). Most technical metadata from imaging modalities are captured seamlessly. For metadata capturing experimental details, ALSDA (like GeneLab) uses the ISA-Tab specification which relies on the ISA data model to order and classify metadata. The ISA data model uses a tree structure with three files to capture the metadata: The top layer is the Investigations file, the second layer is the Study file(s), and the last layer is the Assay file(s). We believe such an approach may be useful for other types of image research data from other investigators in the AGU community.

imaging↗
NASA NTRS2002

Application of Reconfigurable Computing Technology to Multi-KiloHertz Micro-Laser Altimeter (MMLA) Data Processing

The Multi-KiloHertz Micro-Laser Altimeter (MMLA) is an aircraft based instrument developed by NASA Goddard Space Flight Center with several potential spaceflight applications. This presentation describes how reconfigurable computing technology was employed to perform MMLA signal extraction in real-time under realistic operating constraints. The MMLA is a "single-photon-counting" airborne laser altimeter that is used to measure land surface features such as topography and vegetation canopy height. This instrument has to date flown a number of times aboard the NASA P3 aircraft acquiring data at a number of sites in the Mid-Atlantic region. This instrument pulses a relatively low-powered laser at a very high rate (10 kHz) and then measures the time-of-flight of discrete returns from the target surface. The instrument then bins these measurements into a two-dimensional array (vertical height vs. horizontal ground track) and selects the most likely signal path through the array. Return data that does not correspond to the selected signal path are classified as noise returns and are then discarded. The MMLA signal extraction algorithm is very compute intensive in that a score must be computed for every possible path through the two dimensional array in order to select the most likely signal path. Given a typical array size with 50 x 6, up to 33 arrays must be processed per second. And for each of these arrays, roughly 12,000 individual paths must be scored. Furthermore, the number of paths increases exponentially with the horizontal size of the array, and linearly with the vertical size. Yet, increasing the horizontal and vertical sizes of the array offer science advantages such as improved range, resolution, and noise rejection. Due to the volume of return data and the compute intensive signal extraction algorithm, the existing PC-based MMLA data system has been unable to perform signal extraction in real-time unless the array is limited in size to one column, This limits the ability of the MMLA to operate in environments with sparse signal returns and a high number of noise return. However, under an IR&D project, an FPGA-based, reconfigurable computing data system has been developed that has been demonstrated to perform real-time signal extraction under realistic operating constraints. This reconfigurable data system is based on the commercially available Firebird Board from Annapolis Microsystems. This PCI board consists of a Xilinx Virtex 2000E FPGA along with 36 MB of SRAM arranged in five separately addressable banks. This board is housed in a rackmount PC with dual 850MHz Pentium processors running the Windows 2000 operating system. This data system performs all signal extraction in hardware on the Firebird, but also runs the existing "software based" signal extraction in tandem for comparison purposes. Using a relatively small amount of the Virtex XCV2000E resources, the reconfigurable data system has demonstrated to improve performance improvement over the existing software based data system by an order of magnitude. Performance could be further improved by employing parallelism. Ground testing and a preliminary engineering test flight aboard the NASA P3 has been performed, during which the reconfigurable data system has been demonstrated to match the results of the existing data system.

Powell, Wesley↗
DOE OSTI2024

Employing artificial intelligence to steer exascale workflows with colmena

Computational workflows are a common class of application on supercomputers, yet the loosely coupled and heterogeneous nature of workflows often fails to take full advantage of their capabilities. We created Colmena to leverage the massive parallelism of a supercomputer by using Artificial Intelligence (AI) to learn from and adapt a workflow as it executes. Colmena allows scientists to define how their application should respond to events (e.g., task completion) as a series of cooperative agents. In this paper, we describe the design of Colmena, the challenges we overcame while deploying applications on exascale systems, and the science workflows we have enhanced through interweaving AI. The scaling challenges we discuss include developing steering strategies that maximize node utilization, introducing data fabrics that reduce communication overhead of data-intensive tasks, and implementing workflow tasks that cache costly operations between invocations. These innovations coupled with a variety of application patterns accessible through our agent-based steering model have enabled science advances in chemistry, biophysics, and materials science using different types of AI. In conclusion, our vision is that Colmena will spur creative solutions that harness AI across many domains of scientific computing.

Workflows↗
NASA NTRS2007

Modeling Malaria Transmission in Thailand and Indonesia

Malaria Modeling and Surveillance is a project in the NASA Applied Sciences Public Health Applications Program. The main objectives of this project are: 1) identification of the potential breeding sites for major vector species: 2) implementation of a malaria transmission model to identify they key factors that sustain or intensify malaria transmission; and 3) implementation of a risk algorithm to predict the occurrence of malaria and its transmission intensity. Remote sensing and GIs are the essential elements of this project. The NASA Earth science data sets used in this project include AVHRR Pathfinder, TRMM, MODIS, NSIPP and SIESIP. Textural-contextual classifications are used to identify small larval habitats. Neural network methods are used to model malaria cases as a function of precipitation, temperatures, humidity and vegetation. Hindcastings based on these environmental parameters have shown good agreement to epidemiological records. Examples for spatio-temporal modeling of malaria transmissions in Southeast Asia are given. Discrete event simulations were used for modeling the detailed interactions among the vector life cycle, sporogonic cycle and human infection cycle, under the explicit influences of selected extrinsic and intrinsic factors. The output of the model includes the individual infection status and the quantities normally observed in field studies, such as mosquito biting rates, sporozoite infection rates, gametocyte prevalence and incidence. Results are in good agreement with mosquito vector and human malaria data acquired by Coleman et al. over 4.5 years in Kong Mong Tha, a remote village in western Thailand. Application of our models is not restricted to Southeast Asia. The model and techniques are equally applicable to other regions of the world, when appropriate epidemiological and vector ecological parameters are used as input.

Kiang, Richard↗
NASA NTRS1996

Implementation of a Very Large Atmospheric Correction Lookup Table for ASTER Using a Relational Database Management System

The Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER)is designed to provide a high resolution map of the Earth in both visible, near-infrared, and thermal spectral regions of the electromagnetic spectrum. The ASTER Science Team has developed several standard data product algorithms, but the most complex aod computing-intensive of these is the estimation of surface radiance and reflectance values, which is done by modeling and correcting for the effects of the atmosphere. The algorithm for atmospheric correction in the visible bands sensed by ASTER calls for use of a very large Atmospheric Correction Look Up Table (ACLUT).

ASTER↗
DOE OSTI2025

Improving microstructures segmentation via pretraining with synthetic data

Image analysis of material microstructures through microscopy is an integral capability in the field of materials science. The topological and chemical information obtained through microscopy allow us to draw vital connections between material microstructures, properties, and processing. While scanning electron microscopy (SEM) is able to yield a considerable wealth of information interpretable by the intuition of experts, there has been considerable interest in using machine learning, convolutional neural networks (CNNs) in particular, for such image analysis task. Training CNNs for an image analysis task requires a large annotated dataset. However, in many materials science applications, obtaining a large annotated dataset is cost and labor intensive. In this work, we study the use of synthetic data to enlarge the available annotated experimental data of uranium oxide. We utilize a modified Potts model to simulate uranium oxide particles with morphologies similar to those observed experimentally. We then leverage an image-to-image translation model to synthesize the simulated particles as if they are acquired with SEM. Through this process, we obtain pairs of particle images and their corresponding SEM representations, which corresponds to pairs of annotations and images. Unlike previous works, we leverage synthetic data for pretraining a CNN model prior, and finetune that model further with experimental data. We experimentally demonstrate that using synthetic data as incremental learning process benefits the overall performance compared to training a model on combined synthetic and experimental data.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Multiple parameter dynamic photoresponse microscopy for data-intensive optoelectronic measurements of van der Waals heterostructures

Quantum devices made from van der Waals (vdW) heterostructures of two dimensional (2D) materials may herald a new frontier in designer materials that exhibit novel electronic properties and unusual electronic phases. However, due to the complexity of layered atomic structures and the physics that emerges, experimental realization of devices with tailored physical properties will require comprehensive measurements across a large domain of material and device parameters. Such multi-parameter measurements require new strategies that combine data-intensive techniques—often applied in astronomy and high energy physics—with the experimental tools of solid state physics and materials science. We discuss the challenges of comprehensive experimental science and present a technique, called Multi-Parameter Dynamic Photoresponse Microscopy (MPDPM), which utilizes ultrafast lasers, diffraction limited scanning beam optics, and hardware automation to characterize the photoresponse of 2D heterostructures in a time efficient manner. Using comprehensive methods on vdW heterostructures results in large and complicated data sets; in the case of MPDPM, we measure a large set of images requiring advanced image analysis to extract the underlying physics. We discuss how to approach such data sets in general and in the specific case of a graphene–boron nitride–graphite heterostructure photocell.

36 MATERIALS SCIENCE↗
DOE OSTI2025

PSTN-019: The LSST Science Pipelines Software: Optical Survey Pipeline Reduction and Analysis Environment

The NSF-DOE Vera C. Rubin Observatory is executing the Legacy Survey of Space and Time (LSST) as its prime mission, producing a series of data releases over the ten-year survey. The LSST Science Pipelines Software will be used to create these data releases and to perform the nightly prompt processing and alert production. This paper provides an overview of the LSST Science Pipelines Software, describing the components and their integration into pipelines that generate science-ready data products.

79 ASTRONOMY AND ASTROPHYSICS↗