A Deep Learning Approach for Surface PM2.5 Estimations from Geostationary Satellite and Numerical Model Data
Explore the source record for details and available documents.
Manisha Khatri↗
SEARCH · Search NASA
Results for “deep data”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Unexpected Phase-Shifts of Acoustic Waves Observed in Real Data as Well as in Simulations and Their Impact on Estimations of the Deep Meridional Flow
Explore the source record for details and available documents.
SMD↗
Using deep transfer learning and satellite imagery to estimate urban air quality in data-poor regions
Explore the source record for details and available documents.
Nishant Yadav↗
Enabling Space Biological Knowledge Discovery Through Image and Video Data Sharing
Increased biomedical risks and challenges associated with deep space missions and experiments (cis-Lunar, Mars transit/surface) require new knowledge discovery and development of novel ecosystems. Supporting distant and long-duration missions and experiments requires biological data (from yeast, microbes, fruit flies, C. elegans, plants, crops, rodents, humans) be findable, accessible, interoperable, reusable (FAIR), and maximally open-access. As data-intensive, bioinformatic, meta-analytical, and computer-assisted approaches continue to be a centerpiece of modern research, the NASA Biological and Physical Sciences division is expanding its Open Science capabilities beyond NASA GeneLab. The NASA Ames Life Sciences Data Archive (ALSDA) is a repository which is responsible for collecting and access to space biological imagery and video, alongside tabular and environmental data. In this presentation, we will discuss strategies dealing with archiving, curating, and accessibility of images from very distinct imaging modalities (e.g., micro-computed tomography, magnetic resonance imaging, photographic images of plants, fluorescence microscopy, behavioral videos, etc.). There are two main challenges: 1. Open-source data storage and 2. Metadata related to the imagery-video. Both have been solved by leveraging two existing open-source systems. For data storage, ALSDA is utilizing components through the Open Microscopy Environment (OME), which can read most imaging proprietary formats and display on a web interface complex multidimensional images (Z stack, multi-channel, temporal, spectral). Most technical metadata from imaging modalities are captured seamlessly. For metadata capturing experimental details, ALSDA (like GeneLab) uses the ISA-Tab specification which relies on the ISA data model to order and classify metadata. The ISA data model uses a tree structure with three files to capture the metadata: The top layer is the Investigations file, the second layer is the Study file(s), and the last layer is the Assay file(s). We believe such an approach may be useful for other types of image research data from other investigators in the AGU community.
imaging↗
Steps Toward Improved Integration, Search, and Analysis of Heterogeneous Data in the Astrobiology Habitable Environments Database
The Astrobiology Habitable Environments Database (AHED) is a new data system being developed as a long-term, open-access repository for astrobiology data. AHED is intended to store user-contributed results from NASA or externally-funded research in astrobiology, and to encourage sharing and synergy within the astrobiology community. However, the interdisciplinary nature of astrobiology presents some specific challenges to data management, integration, and analysis within AHED. In some disciplines (e.g., genomics), open databases thrive because the contributed products are fairly uniform and standardized (e.g., sequence data). In astrobiology, each investigation produces a unique set of data products; this makes it difficult to search across different datasets to find similar data, or to combine results from separate investigations. With AHED, we are taking steps to ensure there is adequate metadata - both at the dataset and record levels - to facilitate search, integration, and analysis. At the dataset level, we are developing a new metadata standard for describing astrobiology datasets, with detailed information about content, funding source, and scientific relevance, along with a set of topical keywords for characterizing datasets. At the record level, we are encouraging users to provide more structured content and finer-grained metadata. In many user-contributed science data repositories, few restrictions are placed on the uploaded data format, and minimal or no record-level metadata is required; thus users are unburdened when it comes to data preparation. The tradeoff is that deep integration and search across datasets is almost impossible without standardized structures and metadata. Although AHED users are free to upload minimally-described datasets, they will be encouraged to use database authoring tools (supplied by the underlying platform - Open Data Repository's Data Publisher) plus a set of customizable astrobiology-specific templates to help structure their data and provide standardized metadata. In reward for their extra effort, AHED will be able to deliver enhanced search, discovery, and analysis capabilities.
astrobiology↗
NASA Open Science Data Repository: Open Science for Life in Space
Space biology and health data are critical for the success of deep space missions and sustainable human presence off-world. At the core of effectively managing biomedical risks is the commitment to open science principles, which ensure that data are findable, accessible, interoperable, reusable, reproducible and maximally open. The 2021 integration of the Ames Life Sciences Data Archive with GeneLab to establish the NASA Open Science Data Repository significantly enhanced access to a wide range of life sciences, biomedical-clinical, and mission telemetry data alongside existing ‘omics data from GeneLab. This paper describes the new database, its architecture, and new data streams supporting diverse data types and enhancing data submission, retrieval, and analysis. Features include the Biological Data Management Environment for improved data submission, a new user interface, controlled data access, an enhanced API, and comprehensive public visualization tools for environmental telemetry, radiation dosimetry data, and ‘omics analyses. By fostering global collaboration through its Analysis Working Groups and training programs, the Open Science Data Repository promotes widespread engagement in space biology, ensuring transparency and inclusivity in research. It supports the global scientific community in advancing our understanding of spaceflight's impact on biological systems, ensuring humans will thrive in future deep space missions.
OSDR↗
DeepAnnotator
Given a collection of pairs ((A) written material such as reports, phrases, bulleted lists, etc., (B) structured labels called annotations) called the training data, we want to create software to take future written material (A) and correctly predict (B). In this case we have used medical reports and medical coder's structured labels as our training data. The software builds a machine learning model of type deep learning, specifically a convolutional neural network (CNN) using the training data so that when new data arrives in the future the annotations can be predicted.
Christian, JamesB.↗
Ephemeral Learning - Augmenting Triggers with Online-Trained Normalizing Flows
The large data rates at the LHC require an online trigger system to select relevant collisions. Rather than compressing individual events, we propose to compress an entire data set at once. We use a normalizing flow as a deep generative model to learn the probability density of the data online. The events are then represented by the generative neural network and can be inspected offline for anomalies or used for other analysis purposes. We demonstrate our new approach for a toy model and a correlation-enhanced bump hunt.
97 MATHEMATICS AND COMPUTING↗
Asynchronous Message Service for Deep Space Mission Operations
While the CCSDS (Consultative Committee for Space Data Systems) File Delivery Protocol (CFDP) provides internationally standardized file transfer functionality that can offer significant benefits for deep space mission operations, not all spacecraft communication requirements are necessarily best met by file transfer. In particular, continuous event-driven asynchronous message exchange may also be useful for communications with, among, and aboard spacecraft. CCSDS has therefore undertaken the development of a new Asynchronous Message Service (AMS) standard, designed to provide common functionality over a wide variety of underlying transport services, ranging from shared memory message queues to CCSDS telemetry systems. The present paper discusses the design concepts of AMS, their applicability to deep space mission operations problems, and the results of preliminary performance testing obtained from exercise of a prototype implementation.
asynchronous message exchanges↗
Coincident learning for beam-based rf station fault identification using phase information at the SLAC linac coherent light source
Anomalies in radio-frequency (rf) stations can result in unplanned downtime and performance degradation in linear accelerators such as SLAC’s Linac Coherent Light Source (LCLS). Detecting these anomalies is challenging due to the complexity of accelerator systems, high data volume, and scarcity of labeled fault data. Prior work identified faults using beam-based detection, combining rf amplitude and beam position monitor data. Due to the simplicity of the rf amplitude data, classical methods are sufficient to identify faults, but the recall is constrained by the low-frequency and asynchronous characteristics of the data. In this work, we leverage high-frequency, time-synchronous rf phase data to enhance anomaly detection in the LCLS accelerator. Due to the complexity of phase data, classical methods fail, and we instead train deep neural networks within the Coincident Anomaly Detection (CoAD) framework. We find that applying CoAD to phase data detects nearly 3 times as many anomalies as when applied to amplitude data, while achieving broader coverage across rf stations. Furthermore, the rich structure of phase data enables us to cluster anomalies into distinct physical categories. Through the integration of auxiliary system status bits, we link clusters to specific fault signatures, providing additional granularity for uncovering the root cause of faults. We also investigate interpretability via Shapley values, confirming that the learned models focus on the most informative regions of the data and providing insight for cases where the model makes mistakes. This work demonstrates that phase-based anomaly detection for rf stations improves both diagnostic coverage and root cause analysis in accelerator systems and that deep neural networks are essential for effective analysis.
Accelerator Physics (physics.acc-ph)↗
Deep sea tides determination from GEOS-3
GEOS 3 altimeter data in a 5 degree X 5 degree square centered at 30 deg N, 70 deg W were analyzed to evaluate deep sea tide determination from a spacecraft. The signal to noise ratio of known tidal variability to altimeter measurement of sea level above the ellipsoid was 0.1. A sample was obtained in a 5 deg x 5 deg area approximately once every four days. The randomly spaced time series was analyzed using two independent least squares techniques.
Maul, G. A.↗
The Telecommunications and Data Acquisition Report
This publication, one of a series formerly titled The Deep Space Network Progress Report, documents DSN progress in flight project support, tracking and data acquisition research and technology, network engineering, hardware and software implementation, and operations. In addition, developments in Earth-based radio technology as applied to geodynamics, astrophysics and the radio search for extraterrestrial intelligence are reported.
Posner, E. C.↗
The Telecommunications and Data Acquisition Report
This publication, one of a series formerly titled The Deep Space Network Progress Report, documents DSN progress in flight project support, tracking and data acquisition research and technology, network engineering, hardware and software implementation, and operations. In addition, developments in earth-based radio technology as applied to geodynamics, astrophysics, and the radio search for extraterrestrial intelligence are reported.
Posner, E. C.↗
Calibration of the DSCOVR EPIC Visible and NIR Channels using MODIS Terra and Aqua Data and EPIC Lunar Observations
The unique position of the Deep Space Climate Observatory (DSCOVR) Earth Polychromatic Imaging Camera (EPIC) at the Lagrange 1 point makes an important addition to the data from currently operating low Earth orbit observing instruments. EPIC instrument does not have an onboard calibration facility. One approach to its calibration is to compare EPIC observations to the measurements from polar-orbiting radiometers. Moderate Resolution Imaging Spectroradiometer (MODIS) is a natural choice for such comparison due to its well-established calibration record and wide use in remote sensing. We use MODIS Aqua and Terra L1B 1km reflectances to infer calibration coefficients for four EPIC visible and NIR channels: 443, 551, 680 and 780 nm. MODIS and EPIC measurements made between June 2015 and 2016 are employed for comparison. We first identify favorable MODIS pixels with scattering angle matching temporarily collocated EPIC observations. Each EPIC pixel is then spatially collocated to a subset of the favorable MODIS pixels within 25 km radius. Standard deviation of the selected MODIS pixels as well as of the adjacent EPIC pixels is used to find the most homogeneous scenes. These scenes are then used to determine calibration coefficients using a linear regression between EPIC counts/sec and reflectances in the close MODIS spectral channels. We present thus inferred EPIC calibration coefficients and discuss sources of uncertainties. The lunar EPIC observations are used to calibrate EPIC O2 absorbing channels (688 and 764 nm), assuming that there is a small difference between moon reflectances separated by approx.10 nm in wavelength provided the calibration factors of the red (680 nm) and near-IR (780 nm) are known from comparison between EPIC and MODIS.
Geogdzhayev, Igor V.↗
Constraining solar electron number density via neutrino flavor data at Borexino
Understanding the physics of the deep solar interior, and the more exotic environs of core-collapse supernovae (CCSN) and binary neutron-star (NS) mergers, is of keen interest in many avenues of research. To date, this physics is based largely on simulations via forward integration. While these simulations provide valuable constraints, it could be insightful to adopt the "inverse approach" as a point of comparison. Within this paradigm, parameters of the solar interior are not output based on an assumed model, but rather are inferred based on real data. Here, we take the specific case of solar electron number density, which historically is taken as output from the standard solar model. We show how one may arrive at an independent constraint on that density profile based on available neutrino flavor data from the Earth-based Borexino experiment. The inference technique's ability to offer a unique lens on physics can be extended to other datasets, and to analogous questions for CCSN and NS mergers, albeit with simulated data.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
Earth Science Deep Learning: Applications and Lessons Learned
Deep Learning: A subfield of machine learning; Algorithms inspired by function of the brain; Scales with amount of training data; Powerful tool without the need for feature engineering; Suitable for Earth Science applications. Deep Learning for Earth science at MSFC (Marshall Space Flight Center): Phenomena identification; Hurricane intensity (wind speed) estimation; Severe storm (hailstorm) detection; Transverse bands detection; Entity extraction for knowledge graph creation; Ephemeral water detection.
Labeled Data↗
Machine Learning for Real-time Fusion Plasma Behavior Prediction and Manipulation (Final Report)
The goal of this project is to implement real-time analysis of 2D Beam Emission Spectroscopy (BES) data to predict and control transient and high-bandwidth events at DIII-D. In essence, we wish to bring high-bandwidth fluctuation diagnostics into the realm of real-time measurements and control. The BES ML models will necessarily be deep neural networks (DNN) with a “data flow” architecture for compatibility with high-throughput, low-latency evaluation on a field-programmable gate array (FPGA) or other emerging processor technologies. The real-time output will be fed to the plasma control system (PCS) for real-time control tasks, specifically for ELM control and avoidance and for QH-mode access and sustainment. We anticipate that the real-time analysis of fluctuation diagnostics will create new enabling technologies to predict and control transient events such as confinement mode transitions, edge-localized modes, Alfven eigenmode events, and disruptions. The proposed research is aligned with ITER research needs and DIII-D programmatic goals. For instance, the prediction and avoidance of ELM events is critical for ITER machine safety. Also, H-mode access with RMP ELM suppression in ITER is an active research area due to high separatrix density, narrow SOL width, and elevated LH transition power threshold.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Variations of Cloud and Radiative Properties of Boundary-layer and Deep Convective Systems with Sea Surface Temperature Anomalies
Gridded monthly-mean satellite data contain compositing information from different cloud system types and clear-sky environments. To isolate the variations of cloud physical properties of an individual cloud system type with its environment, orbital data are needed. In this study, we will analyze the variations of cloud and radiative properties of boundary-layer clouds and deep convective cloud systems with sea surface temperature (SST) anomalies. We use Terra-CERES (Clouds and the Earth s Radiant Energy System) Level 2 data to classify distinct cloud objects defined by cloud-system types (deep convection, boundary-layer cumulus, stratocumulus and overcast clouds), sizes, geographic locations, and matched large-scale environments. This analysis method identifies a cloud object as a contiguous region of the Earth with a single dominant cloud-system type. It determines the shape and size of the cloud object from the satellite data and the cloud-system selection criteria. The statistical properties of the identified cloud objects are analyzed in terms of probability density functions (PDFs) of a single property or joint PDFs between two properties. The SST anomalies are defined as the differences from five-year annual-cycle means. Individual cloud objects are sorted into one of five equal size subsets, with the matched SST anomalies ranging from the most negative to the most positive values, for a given size category of deep convective cloud objects, boundary-layer cumulus, stratocumulus and overcast cloud objects. The PDFs of cloud and radiative properties for deep convective cloud objects (between 30 S and 30 N) are found to largely similar among the five SST anomaly subsets except for the lowest SST anomaly subset. The different characteristics from this SST anomaly subset may be related to some cloud objects resulting from equatorward movement of extratropical cloud systems. This result holds true for all three different size categories (measured by equivalent diameters of 100-150, 150-300 and > 300 km). The PDFs of cloud and radiative properties for boundary-layer cloud types are more variable for macrophysical properties than for microphysical properties, due probably to different large-scale environments where these cloud objects are formed. Implications of these results for the cloud feedback will also be discussed.
Xu, Kuan-Man↗