Search NASA⌕ Search

SEARCH · Search NASA

Results for “Analysis Ready Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Ice-nucleating particles (INPs) concentrations from SAIL-Net

This data set contains ice-nucleating particle (INP) concentration spectra collected during the SAIL-Net sampling period, which complemented the Surface Atmosphere Integrated Field Laboratory (SAIL) campaign in the East River watershed near Crested Butte, Colorado. SAIL-Net was a distributed aerosol measurement network designed to investigate aerosol variability across complex mountainous terrain. The data set includes samples from multiple SAIL-Net sites, including AOS, Gothic, Snodgrass, Pumphouse, Irwin, and Top. INP concentrations are reported as a function of freezing temperature, together with confidence limits, sampling times, site location, elevation, sampled air volume, and treatment information. These data provide an analysis-ready record of the INPs across the SAIL-Net network.

activation temperature↗

An Overview of the Mock LISA Data Challenges

The LISA International Science Team Working Group on Data Analysis (LIST-WG1B) is sponsoring several rounds of mock data challenges, with the purpose of fostering the development of LISA data-analysis capabilities, and of demonstrating technical readiness for the maximum science exploitation of the LISA data. The first round of challenge data sets were released at this Symposium. We describe the objectives, structure, and timeline of this program.

black holes↗

How Can NASA Science Benefit Solar Energy Development and Assessment?

This is a keynote presentation for a workshop highlighting the use of remote sensing and modeling products derived with NASA missions and Earth system modeling for solar energy system development. We broadly introduce NASA's fleet of satellite producing publicly available data products that would be useful for the information needed solar energy development. We also introduce the work in Earth system science modeling that provides additional global gridded data products that are also relevant. We identify that one issue is obtaining and using this data is the level of expertise needed to find it. NASA has developed a new Earth Action program to address this gap and bring the data closer to the user. Now under the Energy Resources element, the NASA LaRC's POWER (Prediction of Worldwide Renewable Energy) project has developed a web services platform that provides key information needed for solar energy development as analysis-ready. This means that the data for particular locations are immediately useful for help provide the information needed by engineers and architects to plan for solar systems. Thus, the presentation shows why NASA products would be relevant and then shows examples of finding the needed data through the POWER web portal.

remote sensing↗

SatCORPS Global Cloud Composite (GCC): the Design and Delivery of A High Quality, High Resolution, Global Cloud Product Available in Near-Real Time

The NASA Satellite ClOud and Radiation Property retrieval System (SatCORPS) supports the development of an analysis ready and cloud-optimized data transformation pipeline and geospatial service enablement of a global cloud composite (GCC) product derived from global geostationary satellite imagery. This geospatial service will be available at high temporal and spatial resolution via the SatCORPS web mapping application for visualization and analysis as well as direct ingestion to common geospatial software and custom programming. The resulting global cloud composite products from the processing pipeline can then be geospatially-service enabled as ArcGIS Image Services and Open Geospatial Consortium (OGC) Web Mapping/Coverage Services for visualization and analysis via a web mapping application and common geospatial software. Near real time global observations are created through the composition of five geostationary satellites that provides modelling and forecasting communities with the capability to provide high quality and timely information to start the projection process. The Global Cloud Composite product combines information from geostationary satellites, GOES-16, GOES-17, Himawari-8, Meteosat-11 and Meteosat-9 to create a single global composite netcdf file and images using the different products within the netcdf file. The SatCORPS team, though our Global Cloud Composite (GCC) product and web-based visualization tools including Geographical Information System (GIS) services provide near real time global cloud product information to both automated processes and traditional web users that is timely and high quality derived from geostationary satellites. The Global Cloud Composite product takes advantage of the scalable processing resources provided by the AWS batch service to provide new composites every thirty minutes. Because information from each of the low earth orbiting satellites is available on schedules tuned to the specific satellite, the processing algorithm temporally composites the final dataset as each satellite’s information becomes available. The SatCORPS team has leveraged our experience using Amazon Web Services (AWS) to build a low latency high availability tool that allows end users both human and automated to acquire high quality and high-resolution Geostationary Earth Orbiting (GEO) information at zero cost to the end user. This presentation will describe how we architected and implemented the service as well as lessons learned based on our experiences both developing and operating the system. The lessons learned include how we integrated multiple services including Amazon Batch, Amazon S3 and Amazon Lambda service to create a low cost but high-performance processing system that is capable of identifying and processing the most appropriate satellite overpass information into global cloud composites. We will also describe our web-based tools including our Geographic Information System that can be used for visualization and analysis. The products from the processing can be geospatially-service enabled as ArcGIS Image Services and Open Geospatial Consortium (OGC) Web Mapping/Coverage Services for visualization and analysis via a web mapping application and common geospatial software. The SatCORPS Global Composite Cloud product provides sophisticated global composited cloud research products with very low latency that we see that as filling a rapidly growing need in the research and modelling community with no up-front nor ongoing costs associated with downloading or using the information.

AWS AMCE SMCE GCC SATCORPS GLOBAL CLOUD COMPOSITE ↗

SAXS Assistant: Automated SAXS analysis for structural discovery in biologics and polymeric nanoparticles

Small-angle x-ray scattering (SAXS) is a powerful technique for assessing macromolecular structure. High-throughput SAXS is limited by the time-consuming and, at times, subjective nature of SAXS data interpretation. Here, we present SAXS Assistant, a Python-based script that streamlines SAXS data analysis to extract features for machine learning (ML) and key structural parameters, including the Guinier radius of gyration (R g ), pair distance distribution function (PDDF)-derived R g , maximum particle dimension (D max ), and Kratky plots. The script builds upon BioXTAS RAW and validates reliability via Guinier/PDDF R g agreement, an important indicator of well-measured data sets. For assistance in D max estimation, a multilayer perceptron regressor was trained with 1940 data files from the Small Angle Scattering Biological Data Bank. The model achieved a test set performance R 2 = 0.90 and mean absolute error = 11.7 Å. Training exclusively with experimental data translates analyses from researchers, including experts in the field, to the ML model, which helps assess D max estimations from PDDF. Gaussian mixture model clustering was implemented to classify profiles into structural classes based on entries in the Small Angle Scattering Biological Data Bank. Users may therefore assess the similarity between experimental samples and known biomolecular shapes within the mapped repository entries. This probabilistic clustering aids in quantifying information from Kratky and generating shape-descriptive features. SAXS Assistant accelerates SAXS data analysis through enforced quality control, ML-ready outputs, and flags for low-confidence results. In addition to providing the ability to analyze large data sets at high throughput, this tool is versatile and may serve researchers in both biological and synthetic polymer research fields.

36 MATERIALS SCIENCE↗

Water Across Synthetic Aperture Radar Data (WASARD): SAR Water Body Classification for the Open Data Cube

The detection of inland water bodies from Synthetic Aperture Radar (SAR) data provides a great advantage over water detection with optical data, since SAR imaging is not impeded by cloud cover. Traditional methods of detecting water from SAR data involves using thresholding methods that can be labor intensive and imprecise. This paper describes Water Across Synthetic Aperture Radar Data (WASARD): a method of water detection from SAR data which automates and simplifies the thresholding process using machine learning on training data created from Geoscience Australia’s WOFS algorithm. Of the machine learning models tested, the Linear Support Vector Machine was determined to be optimal, with the option of training using solely the VH polarization or a combination of the VH and VV polarizations. WASARD was able to identify water in the target area with a correlation of 97% with WOFS. Sentinel-1, Open Data Cube, Earth Observations, Machine Learning, Water Detection 1. INTRODUCTION Water classification is an important function of Earth imaging satellites, as accurate remote classification of land and water can assist in land use analysis, flood prediction, climate change research, as well as a variety of agricultural applications [2]. The ability to identify bodies of water remotely via satellite is immensely cheaper than contracting surveys of the areas in question, meaning that an application that can accurately use satellite data towards this function can make valuable information available to nations which would not be able to afford it otherwise. Highly reliable applications for the remote detection of water currently exist for use with optical satellite data such as that provided by LANDSAT. One such application, Geoscience Australia’s Water Observations from Space (WOFS) has already been ported for use with the Open Data Cube [6]. However, water detection using optical data from Landsat is constrained by its relatively long revisit cycle of 16 days [5], and water detection using any optical data is constrained in that it lacks the ability to make accurate classifications through cloud cover [2]. The alternative solution which solves these problems is water detection using SAR data, which images the Earth using cloud-penetrating microwaves. Because of its advantages over optical data, much research has been done into water detection using SAR data. Traditionally, this has been done using the thresholding method, which involves picking a polarization band and labeling all pixels for which this band’s value is below a certain threshold as containing water. The thresholding method works since water tends to return a much lower backscatter value to the satellite than land [1]. However, this method can be flawed since estimating the proper threshold is often imprecise, complicated, and labor intensive for the end user. Thresholding also tends to use data from only one SAR polarization, when a combination of polarizations can provide insight into whether water is present. [2] In order to alleviate these problems, this paper presents an application for the Open Data Cube to detect water from SAR data using support vector machine (SVM) classification. 2. PLATFORM WASARD is an application for the Open Data Cube, a mechanism which provides a simple yet efficient means of ingesting, storing, and retrieving remote sensing data. Data can be ingested and made analysis ready according to whatever specifications the researcher chooses, and easily resampled to artificially alter a scene’s resolution. Currently WASARD supports water detection on scenes from ESA’s Sentinel-1 and JAXA’s ALOS. When testing WASARD, Sentinel-1 was most commonly used due to its relatively high spatial resolution and its rapid 6 day revisit cycle [5]. With minor alterations to the application's code, however, it could support data from other satellites. 3. METHODOLOGY Using supervised classification, WASARD compares SAR data to a dataset pre-classified by WOFS in order to train an SVM classifier. This classifier is then used to detect water in other SAR scenes outside the training set. Accuracy was measured according to the following metrics:  Precision: a measure of what percentage of the points WASARD labels as water are truly water  Recall: a measure of what percentage of the total water cover WASARD was able to identify.  F1 Score: a harmonic average of the precision and recall scores Both precision and recall are calculated at the end of the training phase, when the trained classifier is compared to a testing dataset. Because the WOFS algorithm’s classifications are used as the truth values when training a WASARD classifier, when precision and recall are mentioned in this paper, they are always with respect to the values produced by WOFS on a similar scene of Landsat data, which themselves have a classification accuracy of 97% [6]. Visual representations of water identified by WASARD in this paper were produced using the function wasard_plot(), which is included in WASARD. 3.1 Algorithm Selection The machine learning model used by WASARD is the Linear Support Vector Machine (SVM). This model uses a supervised learning algorithm to develop a classifier, meaning it creates a vector which can be multiplied by the vector formed by the relevant data bands to determine whether a pixel in a SAR scene contains water. This classifier is trained by comparing data points from selected bands in a SAR scene to their respective labels, which in this case are “water” or “not water” as given by the WOFS algorithm. The SVM was selected over the Random Forest model, which outperformed the SVM in training speed, but had a greater classification time and lower accuracy, and the Multilayer Perceptron Artificial Neural Network, which had a slightly higher average accuracy than the SVM, but much greater training and classification times. Figure 1: Visual representation of the SVM Classifier. Each white point represents a pixel in a SAR scene. In Figure 1, the diagonal line separating pixels determined to be water from those determined not to be water represents the actual classification vector produced by the SVM. It is worth noting that once the model has been trained, classification of pixels is done in a similar manner as in the thresholding method. This is especially true if only one band was used to train the model. 3.1 Feature Selection Sentinel-1 collects data from two bands: the Vertical/Vertical polarization (VV) and the Vertical/Horizontal polarization (VH). When 100 SVM classifiers were created for each polarization individually, and for the combination of the two, the following results were achieved: Figure 2: Accuracy of classifiers trained using different polarization bands. Precision and Recall were measured with respect to the values produced by WOFS. Figure 2 demonstrates that using both the VV and VH bands trades slightly lower recall for significantly greater precision when compared with the VH band alone, and that using the VV band alone is inferior in both metrics. WASARD therefore defaults to using both the VV and VH bands, and includes the option to use solely the VH band. The VV polarization’s lower precision compared to the VH polarization is in contrast to results from previous research and may merit further analysis [4]. 3.2 Training a Classifier The steps in training a classifier with WASARD are 1. Selecting two scenes (one SAR, one optical) with the same spatial extents, and acquired close to each other in time, with a preference that the scenes are taken on the same day. 2. Using the WOFS algorithm to produce an array of the detected water in the scene of optical data, to be used as the labels during supervised learning 3. Data points from the selected bands from the SAR acquisition are bundled together into an array with the corresponding labels gathered from WOFS. A random sample with an equal number of points labeled “Water” and “Not Water” is selected to be partitioned into a training and a testing dataset 4. Using Scikit-Learn’s LinearSVC object, the training dataset is used to produce a classifier, which is then tested against the testing dataset to determine its precision and recall The result is a wasard_classifier object, which has the following attributes: 1. f1, recall, and precision: 3 metrics used to determine the classifier’s accuracy 2. Coefficient: Vector which the SVM uses to make its predictions. The classifier detects water when the dot product of the coefficient and the vector formed by the SAR bands is positive 3. Save(): allows a user to save a classifier to the disk in order to use it without retraining 4. wasard_classify(): Classifies an entire xarray of SAR data using the SVM classifier All of the above steps are performed automatically when the user creates a wasard_classifier object. 3.3 Classifying a Dataset Once the classifier has been created, it can be used to detect water in an xarray of SAR data using wasard_classify(). By taking the dot product of the classifier’s coefficients and the vector formed by the selected bands of SAR data, an array of predictions is constructed. A classifier can effectively be used on the same spatial extents as the ones where it was trained, or on any area with a similar landscape. While

Kreiser, Zachary↗

Satellite Ocean Colour: Current Status and Future Perspective

Spectrally resolved water-leaving radiances (ocean colour) and inferred chlorophyll concentration are key to studying phytoplankton dynamics at seasonal and inter-annual scales, for a better understanding of the role of phytoplankton in marine biogeochemistry; the global carbon cycle; and the response of marine ecosystems to climate variability, change and feedback processes. Ocean colour data also have a critical role in operational observation systems monitoring coastal eutrophication, harmful algal blooms, and sediment plumes. The contiguous ocean-colour record reached 21 years in 2018; however, it is comprised of a number of one-off missions such that creating a consistent time-series of ocean-colour data requires merging of the individual sensors (including MERIS, Aqua-MODIS, SeaWiFS, VIIRS, and OLCI) with differing sensor characteristics, without introducing artefacts. By contrast, the next decade will see consistent observations from operational ocean colour series with sensors of similar design and with a replacement strategy. Also, by 2029 the record will start to be of sufficient duration to discriminate climate change impacts from natural variability, at least in some regions. This paper describes the current status and future prospects in the field of ocean colour focusing on large to medium resolution observations of oceans and coastal seas. It reviews the user requirements in terms of products and uncertainty characteristics and then describes features of current and future satellite ocean-colour sensors, both operational and innovative. The key role of in situ validation and calibration is highlighted as are ground segments that process the data received from the ocean-colour sensors and deliver analysis-ready products to end-users. Example applications of the ocean-colour data are presented, focusing on the climate data record and operational applications including water quality and assimilation into numerical models. Current capacity building and training activities pertinent to ocean colour are described and finally a summary of future perspectives is provided.

ocean colour↗

Operational Analytics Studies for ATLAS Distributed Computing: Data Popularity Forecast and Utilization of the WLCG Centers

Operational analytics is the direction of research related to the analysis of the current state of computing processes and the prediction of future states in order to anticipate imbalances and take timely measures to stabilize a complex system. There are two relevant areas in ATLAS Distributed Computing that are currently the focus of studies: user physics analysis including the forecast of popularity of data samples among users, and evaluating WLCG centers for their readiness to process user analysis payloads. Studying these areas is challenging due to the complexity involved, as it requires a comprehensive understanding of numerous boundary conditions typically found in large-scale distributed computing infrastructures. Forecasts of data popularity are problematic without the categorization of user tasks by their types (data transformation or physics analysis), which do not always appear on the surface but may induce noise, which introduces significant distortions for predictive analysis. Evaluating the WLCG resources by their analysis workloads is also a challenging task as it is necessary to find a balance between the workload of the resource, its performance, the waiting time for jobs on it, as well as the volume of jobs that it processes. This is especially difficult in a heterogeneous computing environment, where legacy resources are used along with modern high-performance machines. We will look at these areas of research in detail and discuss what tools and methods are used in our work, demonstrating results already obtained.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Biological Research and Space Health Enabled by Machine Learning to Support Deep Space Missions

A key science goal of the NASA “Moon to Mars” campaign is to understand how biology responds to the Lunar, Martian, and deep space environments in order to advance fundamental knowledge, reduce risk, and support safe, productive human space missions. Through the powerful emerging approaches of artificial intelligence (AI) and machine learning (ML), a paradigm shift has begun in biomedical science and engineered astronaut health systems, to enable Earth independence and autonomy of mission operations. Here we present an overview of AI/ML architecture to support deep space mission goals, developed with leaders in the field. First, we focus on the fundamental biological research that supports our understanding of physiological responses to spaceflight, and we describe current efforts to support AI/ML research including data standardization and data engineering through maximally open and FAIR (findable, accessible, interoperable, reusable) databases and the generation of AI-ready datasets for reuse and analysis. We also discuss remote data management frameworks for research data as well as environmental and health data that are generated during deep space missions. We highlight several research projects that leverage data standardization and management for fundamental biological discovery to uncover the complex effects of space travel on living systems. Next, we provide an overview of cutting-edge AI/ML approaches that can be integrated to support remote monitoring and analysis during deep space missions, including generative models and large language models to learn the underlying biomedical patterns and predict outcomes or answer questions during off world medical scenarios. We also describe current AI/ML methods to support this research and monitoring through automated cloud-based labs which enable limited human intervention and closed-loop experimentation in remote settings. These labs could support mission autonomy by analyzing environmental data streams, and would be facilitated through in situ analytics capabilities to avoid sending large raw data files through low bandwidth communications. Finally, in the context of deep space missions with limited communications or access to medical advice from Earth, we describe a solution for integrated, real-time mission biomonitoring across hierarchical levels from continuous environmental monitoring, to wearables and point-of-care devices, to molecular and physiological monitoring. We introduce a precision space health system that will ensure that the future of space health is predictive, preventative, participatory and personalized.

artificial intelligence↗

Heating effects on jack pine pyrogenic organic matter properties from a pyrocosm study in 2022

This dataset contains data associated with the preprint “Fire removes preexisting pyrogenic organic matter from the ecosystem through the mechanisms of both direct combustion and increasing mineralizability” (Luo et al., 2025b), which is the complementary study to the published paper “Reburning pyrogenic organic matter: a laboratory method for dosing dynamic heat fluxes from above” (Luo et al., 2025a). We designed a full-factorial experiment with different burial depths of jack pine (Pinus banksiana Lamb) pyrogenic organic matter (PyOM) (Surface, 1 cm, and 5 cm) and different heat-flux profiles (High, Low, and Control) to examine how subsequent fires affect the properties of preexisting PyOM. We measured total carbon (C), pH, dissolved organic carbon (DOC), dissolved inorganic carbon (DIC), and mineralized C (as CO₂-C, from a 12-week incubation).We found that high heat flux and/or surface placement resulted in substantial direct C losses through combustion. Intermediate heat exposure produced both combustion losses and increases in DOC and mineralizability, which may have complex long-term implications: an increased dissolved fraction of PyOM may promote downward transport into mineral soils and potentially contribute to deeper, longer-term C storage, but it may also make PyOM more susceptible to microbial decomposition. Under the lowest heat flux and deepest burial, most PyOM was retained, and changes in DOC and C mineralization were minimal. Finally, PyOM pH, an important chemical property, decreased under low-temperature heating but increased under higher temperatures.We uploaded pH data for all samples (“pH_of_all_samples.csv”); pH and temperature-related data (peak temperature and degree hours) for samples in High and Low heat-flux treatments (“pH_vs_peakT_and_degree_hours_only_for_heated_samples.csv”); total C data (“CN_pct_C_stock_C_loss_in_samples.csv”); DOC and DIC data (“doc_dic.csv”); and mineralized C (CO₂-C) data (“CO2-C_all_original.csv”). Additional details can be found in the Methods & Sampling section.All datasets uploaded to ESS-DIVE are clearly labeled, cleaned, and include both raw and derived data, ready for reuse in other analyses. All analysis code and raw datasets are also available on GitHub: https://github.com/MengmengLuo/Fire-removes-preexisting-pyrogenic-organic-matter-from-the-ecosystem.

54 ENVIRONMENTAL SCIENCES↗

Analysis and testing of high entrainment single nozzle jet pumps with variable mixing tubes

An analytical model was developed to predict the performance characteristics of axisymmetric single-nozzle jet pumps with variable area mixing tubes. The primary flow may be subsonic or supersonic. The computer program uses integral techniques to calculate the velocity profiles and the wall static pressures that result from the mixing of the supersonic primary jet and the subsonic secondary flow. An experimental program was conducted to measure mixing tube wall static pressure variations, velocity profiles, and temperature profiles in a variable area mixing tube with a supersonic primary jet. Static pressure variations were measured at four different secondary flow rates. These test results were used to evaluate the analytical model. The analytical results compared well to the experimental data. Therefore, the analysis is believed to be ready for use to relate jet pump performance characteristics to mixing tube design.

Hickman, K. E.↗

GRC MILab Software: Quick Start Guide

This document provides detailed installation and operating instructions for the GRC MILab Excel Add-In software developed at the NASA Glenn Research Center. The software described has been implemented to facilitate the process of importing into Microsoft Excel and analyzing materials test data from a wide range of materials tests. All resulting data is then ready for automated upload to the relevant table of the GRC Materials Intelligence (MI) database. This new software represents an update to the original MILab software developed by Granta Design Ltd.—a company specializing in materials software, data, and databases—for members of the Materials Data Management Consortium (MDMC), a collaboration between Granta, ASM International, NASA Glenn, and several other materials-oriented corporations and government agencies in the aerospace and defense industries. The updated software consists of the addition of two test type modules, the Generic and Generic Cyclic modules, with both representing a generalization of the original software. The Generic module supports the import and analysis of multiaxial data from any sequence of tensile, compression, relaxation, and/or creep test stages; and the Generic Cyclic module expands the functionality to include repeated sequences during cyclic testing. During processing, all imported data and analysis results are formatted by the software so as to be ready for immediate automated upload to the MI database, ensuring minimal overhead on the part of the user and access to persistent and reliable data for all relevant personnel.

Quick Start Guide↗

Integrated, Interoperable Software Environment for Fusion Simulation and Data Analysis Tools SBIR Phase II

This SBIR effort was focused on developing a production ready system to address the integration and interoperability challenges with analysis and visualization in fusion simulations. Our overarching technical objective was to minimize the code development simulation scientists incur when coupling their simulation codes with different analysis frameworks. To this end, we developed an open-source software library to make data exchange between application easier and a Web application to manage and display analysis extracts from simulations. We have also augmented existing libraries funded by DOE such as ADIOS and VTK-m. When used together, these make it significantly easier to integrate simulation and analysis capability. We demonstrated the flexibility of our approach using two common simulation codes in the fusion community, XGC1 and GTC.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Overview of Nuclear Data Measurement and Analysis at RPI [Slides]

This presentation discusses the experimental, simulation, and nuclear data methods that were validated for the RPI γ-Multiplicity Detector. When the neutron capture γ-cascade data is well-known, the γ-emission spectra can be accurately calculated using the modified simulation tools. The RPI γ-Multiplicity Detector system is now ready for analysis and recommendations for isotopes with deficiencies in γ-ray data. The presentation also discusses future work which includes developing a method for analyzing and adjusting nuclear data for 59 Co, 55 Mn and other measured isotopes including 181 Ta. Additionally, future work includes comparing experimental γ-emission spectra with MCNP-6.2/DICEBOX simulations for 238 U and 235 U. In summation, new capture and transmission measurements for 54 Fe will help improve resonance parameter evaluation. Neutron capture gamma cascade spectra and yields were measured in the resolved resonance region and compared to evaluations. In addition, the pulsed neutron die-away method was developed as a tool to provide data for validation of TSLs.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Towards Lightweight Data Integration Using Multi-Workflow Provenance and Data Observability

Modern large-scale scientific discovery requires multidisciplinary collaboration across diverse computing facilities, including High Performance Computing (HPC) machines and the Edge-to-Cloud continuum. Integrated data analysis plays a crucial role in scientific discovery, especially in the current AI era, by enabling Responsible AI development, FAIR, Reproducibility, and User Steering. However, the heterogeneous nature of science poses challenges such as dealing with multiple supporting tools, cross-facility environments, and efficient HPC execution. Building on data observability, adapter system design, and provenance, we propose MIDA: an approach for lightweight runtime Multi-workflow Integrated Data Analysis. MIDA defines data observability strategies and adaptability methods for various parallel systems and machine learning tools. With observability, it intercepts the dataflows in the background without requiring instrumentation while integrating domain, provenance, and telemetry data at runtime into a unified database ready for user steering queries. We conduct experiments showing end-to-end multi-workflow analysis integrating data from Dask and MLFlow in a real distributed deep learning use case for materials science that runs on multiple environments with up to 276 GPUs in parallel. We show near-zero overhead running up to 100,000 tasks on 1,680 CPU cores on the Summit supercomputer.

Santos Souza, Renan↗

Data handling and visualization for NASA's science programs

Advanced information systems capabilities are essential to conducting NASA's scientific research mission. Access to these capabilities is no longer a luxury for a select few within the science community, but rather an absolute necessity for carrying out scientific investigations. The dependence on high performance computing and networking, as well as ready and expedient access to science data, metadata, and analysis tools is the fundamental underpinning for the entire research endeavor. At the same time, advances in the whole range of information technologies continues on an almost explosive growth path, reaching beyond the research community to affect the population as a whole. Capitalizing on and exploiting these advances are critical to the continued success of space science investigations. NASA must remain abreast of developments in the field and strike an appropriate balance between being a smart buyer and a direct investor in the technology which serves its unique requirements. Another key theme deals with the need for the space and computer science communities to collaborate as partners to more fully realize the potential of information technology in the space science research environment.

Bredekamp, Joseph H.↗

DAWN Coherent Wind Profiling Lidar Flights on NASA's DC-8 During GRIP

Almost from their invention, lasers have been used to measure the velocity of wind and objects; over distances of cm to 10s of km. Long distance (remote) sensing of wind has been accomplished with continuous-wave (CW), focused pulsed, and collimated pulsed lasers; with direct and coherent (heterodyne) optical detection; and with a multitude of laser wavelengths. Airborne measurement of wind with pulsed, coherent-detection lidar was first performed in 1971 with a CW CO2 laser1, in 1972 with a pulsed CO2 laser2, in 1993 with a pulsed 2-micron laser3, and in 1999 with a pulsed CO2 laser and nadir-centered conical scanning4. Of course there were many other firsts and many other groups doing lidar wind remote sensing with coherent and direct detection. A very large FOM coherent wind lidar has been built by LaRC and flown on a DC-8. However a burn on the telescope secondary mirror prevented the full demonstration of high FOM. Both the GRIP science product and the technology and technique demonstration from aircraft are important to NASA. The technology and technique demonstrations contribute to our readiness for the 3D Winds space mission. The data analysis is beginning and we hope to present results at the conference.

Kavaya, Michael J.↗

Dynamics Explorer Science Data Processing System

The Dynamics Explorer project has acquired the ground data processing system from the Atmosphere Explorer project to provide a central computer facility for the data processing, data management and data analysis activities of the investigators. Access to this system is via remote terminals at the investigators' facilities, which provide ready access to the data sets derived from groups of instruments on both spacecraft. The original system has been upgraded with both new hardware and enhanced software systems. These new systems include color and grey scale graphics terminals, an augmentation computer, micrographics facility, a versatile data base with a directory and data management system, and graphics display software packages.

Smith, P. H.↗