Convolutional dictionary learning for blind deconvolution of optical coherence tomography images
Explore the source record for details and available documents.
Information Science↗
SEARCH · Search NASA
Results for “data masking”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Ka-Band ARM Zenith Radar Corrections (KAZRCOR, KAZRCFRCOR) Value-Added Products
The KAZRCOR Value -added Product (VAP) performs several corrections to the ingested KAZR moments and also creates a significant detection mask for each radar mode. The VAP computes gaseous attenuation as a function of time and radial distance from the radar antenna, based on ambient meteorological observations, and corrects observed reflectivities for that effect. KAZRCOR also dealiases mean Doppler velocities to correct velocities whose magnitudes exceed the radar’s Nyquist velocity. Input KAZR data fields are passed through into the KAZRCOR output files, in their native time and range coordinates. Complementary corrected reflectivity and velocity fields are provided, along with a mask of significant detections and a number of data quality flags. This report covers the KAZRCOR VAP as applied to the original KAZR radars and the upgraded KAZR2 radars. Currently there are two separate code bases for the different radar versions, but once KAZR and KAZR2 data formats are harmonized, only a single code base will be required.
54 ENVIRONMENTAL SCIENCES↗
Particle trajectory representation learning with masked point modeling
Liquid argon time projection chambers (LArTPCs) offer millimeter-scale 3D images of particle trajectories, enabling precision studies of neutrino oscillation, detection of supernova and solar neutrinos, searches for exotic dark matter, and proton decay. Current approaches utilize supervised machine learning models, requiring extensive simulations of particle physics and detector response that can introduce bias. Self-supervised learning (SSL), a machine learning approach that learns useful representations of unlabeled data from the data itself, has significantly advanced how large datasets are utilized for representation learning; however, its potential for applications to sensory data in high precision particle physics experiments remains largely unexplored. We introduce the Point-based liquid argon masked autoencoder (PoLAr-MAE), a self-supervised framework that learns physically meaningful representations directly from unlabeled LArTPC images. PoLAr-MAE achieves remarkable data efficiency for a point-level segmentation task, outperforming fully supervised methods in low data regimes. Linear classifiers on model outputs demonstrate robust performance across multiple downstream tasks. Our results position sensor-level SSL as a practical foundation model strategy for LArTPCs.
Young, Samuel [Stanford Univ., CA (United States)]↗
A global gridded dataset for cloud vertical structure from combined CloudSat and CALIPSO observations
Abstract. The vertical structure of clouds has a profound effect on the global energy budget, the global circulation, and the atmospheric hydrological cycle. The CloudSat and Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) missions have taken complementary, colocated observations of cloud vertical structure for over a decade. However, no globally gridded dataset is available to the public for the full length of this unique combined data record. Here we present the 3S-GEOPROF-COMB product (Bertrand et al. 2023, https://doi.org/10.5281/zenodo.8057791), a globally gridded (level 3S) community data product summarizing geometrical profiles (GEOPROF) of hydrometeor occurrence from combined (COMB) CloudSat and CALIPSO data. Our product is calculated from the latest release (R05) of per-orbit (level-2) combined cloud mask profiles. We process a set of cloud cover, vertical cloud fraction, and sampling variables at 2.5, 5, and 10° spatial resolutions and monthly and seasonal temporal resolutions. We address the 2011 reduction in CloudSat data collection with Daylight-Only Operations (DO-Op) mode by subsampling pre-2011 data to mimic DO-Op collection patterns, thereby allowing users to evaluate the impact of the reduced sampling on their analyses. We evaluate our data product against CloudSat-only and CALIPSO-only global-gridded data products as well as four comparable surface-based sites, underscoring the added value of the combined product. Interest in the product is anticipated for the study of cloud processes, cloud–climate interactions, and as a candidate baseline climate data record for comparison to follow-up satellite missions, among other uses.
Bertrand, Leah (ORCID:0009000001607558)↗
BeyondPlanck: VII. Bayesian estimation of gain and absolute calibration for cosmic microwave background experiments
We present a Bayesian calibration algorithm for cosmic microwave background (CMB) observations as implemented within the global end-to-end BEYONDPLANCK framework and applied to the Planck Low Frequency Instrument (LFI) data. Following the most recent Planck analysis, we decomposed the full time-dependent gain into a sum of three nearly orthogonal components: one absolute calibration term, common to all detectors, one time-independent term that can vary between detectors, and one time-dependent component that was allowed to vary between one-hour pointing periods. Each term was then sampled conditionally on all other parameters in the global signal model through Gibbs sampling. The absolute calibration is sampled using only the orbital dipole as a reference source, while the two relative gain components were sampled using the full sky signal, including the orbital and Solar CMB dipoles, CMB fluctuations, and foreground contributions. We discuss various aspects of the data that influence gain estimation, including the dipole-polarization quadrupole degeneracy and processing masks. Comparing our solution to previous pipelines, we find good agreement in general, with relative deviations of -0.67% (-0.84%) for 30 GHz, 0.12% (-0.04%) for 44 GHz and -0.03% (-0.64%) for 70 GHz, compared to Planck PR4 and Planck 2018, respectively. We note that the BEYONDPLANCK calibration was performed globally, which results in better inter-frequency consistency than previous estimates. Additionally, WMAP observations were used actively in the BEYONDPLANCK analysis, which both breaks internal degeneracies in the Planck data set and results in an overall better agreement with WMAP. Finally, we used a Wiener filtering approach to smoothing the gain estimates. We show that this method avoids artifacts in the correlated noise maps as a result of oversmoothing the gain solution, which is difficult to avoid with methods like boxcar smoothing, as Wiener filtering by construction maintains a balance between data fidelity and prior knowledge. Although our presentation and algorithm are currently oriented toward LFI processing, the general procedure is fully generalizable to other experiments, as long as the Solar dipole signal is available to be used for calibration.
79 ASTRONOMY AND ASTROPHYSICS↗
Aspen ploidy level and cover maps from NEON Airborne Observation Platform, 2018, Crested Butte, Colorado
Maps of aspen ploidy (diploid or triploid) developed using National Ecological Observatory Network (NEON) Airborne Observation Platform visible to shortwave infrared imaging spectroscopy data that were acquired over the Upper East River, Colorado in 2018. Maps of aspen distribution, as well as other land cover types that were used to screen for aspen dominated pixels are also provided. Each image file is provided as a GeoTiff with internal tiling and LZW compression. For use of ploidy maps, it is necessary to mask out non-aspen and shaded pixels. Shaded pixels are provided in an associated data package. For full details, please see associated publication and methods description. This dataset is associated with the publication in Ecological Applications.
54 ENVIRONMENTAL SCIENCES↗
Ka-Band ARM Zenith Radar Corrections (KAZRCOR, KAZRCFRCOR) Value-Added Products
The Ka-Band Atmospheric Radiation Measurement (ARM) Zenith Radar Corrections (KAZRCOR) and KA-Band ARM Zenith Radar CF-Radial, Corrected (KAZRCFRCOR) value-added products (VAPs) perform several corrections to the ingested KAZR moments and also create a significant detection mask for each radar mode. The VAPs compute gaseous attenuation as a function of time and radial distance from the radar antenna, based on ambient meteorological observations, and correct observed reflectivities for that effect. Mean Doppler velocities are dealiased to correct velocities whose magnitudes exceed the radar’s Nyquist velocity. Input KAZR data fields are passed through to the KAZRCOR or KAZRCFRCOR output files, in their native time and range coordinates. Complementary corrected reflectivity and velocity fields are provided, along with a mask of significant detections and a number of data quality flags. This report covers the KAZRCOR VAP as applied to the original KAZR radars and the upgraded KAZR2 radars. Originally, prior to late 2019, there were two separate code bases for the different radar versions. Following the harmonization of KAZR and KAZR2 data formats in 2019, only a single code base is required. The new combined KAZR and KAZR2 code base is called the KAZRCFRCOR VAP. The ‘cfr portion of the VAP name refers to the Radial Climate and Forecasting data format standards that are used in these data sets. Throughout this report, references to ‘KAZRCOR’ should be taken to apply to the ‘KAZRCFRCOR’ VAP as well unless there is an explicit statement to the contrary.
54 ENVIRONMENTAL SCIENCES↗
Deep-learning-based workflow for boundary and small target segmentation in digital rock images using UNet++ and IK-EBM
We report three-dimensional (3D) X-ray micro-computed tomography (μCT) has been widely used in petroleum engineering because it can provide detailed pore structural information for a reservoir rock, which can be imported into a pore-scale numerical model to simulate the transport and distribution of multiple fluids in the pore space. The partial volume blurring (PVB) problem is a major challenge in segmenting raw μCT images of rock samples, which impacts boundaries and small targets near the resolution limit. We developed a deep-learning (DL)-based workflow for accurate and fast partial volume segmentation. The DL model's performance depends primarily on the training data quality and model architecture. This study employed the entropy-based-masking indicator kriging (IK-EBM) to segment 3D Berea sandstone images as training datasets. The comparison between IK-EBM and manual segmentation using a 3D synthetic sphere pack, which had a known ground truth, showed that IK-EBM had higher accuracy on partial volume segmentation. We then trained and tested the UNet++ model, a state-of-the-art supervised encoder-decoder model, for binary (i.e., void and solid) and four-class segmentation. We compared the UNet++ with the commonly used U-Net and wide U-Net models and showed that the UNet++ had the best performance in terms of pixel-wise and physics-based evaluation metrics. Specifically, boundary-scaled accuracy demonstrated that the UNet++ architecture outperformed the regular U-Net architecture in the segmentation of pixels near boundaries and small targets, which were subjected to the PVB effect. Feature map visualization illustrated that the UNet++ bridged the semantic gaps between the feature maps extracted at different depths of the network, thereby enabling faster convergence and more accurate extraction of fine-scale features. The developed workflow significantly enhances the performance of supervised encoder-decoder models in partial volume segmentation, which has extensive applications in fundamental studies of subsurface energy, water, and environmental systems.
02 PETROLEUM↗
Gwaihir: Jupyter Notebook graphical user interface for Bragg Coherent Diffraction Imaging
Room temperature data, under Argon atmosphere, on particle B-7. Comes with a hotpixels mask.
Bragg Coherent Diffraction Imaging↗
Observation of a single protein by ultrafast X-ray diffraction
* runs/ - The raw data from the pnCCD * histograms.h5 - Histogram of the values recorded by each pixel throughout the data acquisition. Used for pedestal and gain correction. * mask.h5 - Detector bad pixels mask. * 1SS8.pdb - PDB model of the assembled GroEL 14-mer, from the 1SS8 PDB entry. * 1SS8_H2O.pdb - Hydrated version of the assembled GroEL 14-mer, derived from the 1SS8 PDB entry. The various hydrated models are created by removing water from this initial model. * rotations.h5 - Quaternions of the rotations used for template matching
European XFEL↗
The Atacama Cosmology Telescope: DR6 Sunyaev-Zel'dovich Selected Galaxy Clusters Catalog
We present the results of a search for galaxy clusters in the Atacama Cosmology Telescope (ACT) Data Release 6 (DR6) microwave sky maps covering 16293 square degrees in three frequency bands, using data obtained over the lifetime of the project (2008-2022). We report redshifts and mass estimates for 10040 clusters detected via their Sunyaev-Zel'dovich (SZ) effect with signal-to-noise greater than 4 at a 2.4 arcminute filter scale. The catalog includes 1180 clusters at redshifts greater than 1, and 124 clusters at redshifts greater than 1.5. Using a relation between cluster SZ signal and mass that is consistent with recent weak-lensing measurements, we estimate that clusters detected with signal-to-noise greater than 5 form a sample which is 90% complete for clusters with masses greater than $5 \times 10^{14}$ MSun (measured within a spherical volume with mean density 500 times the critical density). El Gordo, a cluster found in an initial ACT survey of 755 square degrees, remains the most extreme cluster in mass and redshift; we find no cluster with a mass and redshift combination high enough to falsify the standard LCDM cosmology with Gaussian initial perturbations. We make public a variety of data products, including the full cluster candidate list, noise maps, and sky masks, along with our software for cluster detection and instructions for reproducing our cluster catalogs from the public ACT maps.
Aguena, M. [Trieste Observ.; LIneA, Rio de Janeiro↗
Octofitter: Fast, Flexible, and Accurate Orbit Modeling to Detect Exoplanets
As next-generation imaging instruments and interferometers search for planets closer to their stars, they must contend with increasing orbital motion and longer integration times. These compounding effects make it difficult to detect faint planets but also present an opportunity. Increased orbital motion makes it possible to move the search for planets into the orbital domain, where direct images can be freely combined with the radial velocity and proper motion anomaly, even without a confirmed detection in any single epoch. In this paper, we present a fast and differentiable multimethod orbit-modeling and planet detection code called Octofitter. This code is designed to be highly modular and allows users to easily adjust priors, change parameterizations, and specify arbitrary function relations between the parameters of one or more planets. Octofitter further supplies tools for examining model outputs including prior and posterior predictive checks and simulation-based calibration. We demonstrate the capabilities of Octofitter on real and simulated data from different instruments and methods, including HD 91312, simulated JWST/NIRISS aperture masking interferometry observations, radial velocity curves, and grids of images from the Gemini Planet Imager. We show that Octofitter can reliably recover faint planets in long sequences of images with arbitrary orbital motion. This publicly available tool will enable the broad application of multiepoch and multimethod exoplanet detection, which could improve how future targeted ground- and space-based surveys are performed. Finally, its rapid convergence makes it a useful addition to the existing ecosystem of tools for modeling the orbits of directly imaged planets.
79 ASTRONOMY AND ASTROPHYSICS↗
A Spectroscopic Study of Mars-analog Materials with Amorphous Sulfate and Chloride Phases: Implications for Detecting Amorphous Materials on the Martian Surface
The Chemistry and Mineralogy X-ray diffraction (XRD) instrument aboard the Curiosity rover consistently identifies amorphous material at Gale Crater, which is compositionally variable, but often includes elevated sulfur and iron, suggesting that amorphous ferric sulfate (AFS) may be present. Understanding how desiccating ferric sulfate brines affect the spectra of Martian material analogs is necessary for interpreting complex/realistic reaction assemblages. Visible and near-infrared reflectance (VNIR), mid-infrared attenuated total reflectance (MIR, FTIR-ATR), and Raman spectra, along with XRD data are presented for basaltic glass, hematite, gypsum, nontronite, and magnesite, each at three grain sizes (<25, 25–63, and 63–180 μm), mixed with ferric sulfate (+/–NaCl), deliquesced, then rapidly desiccated in 11% relative humidity or via vacuum. All desiccated products are partially or completely XRD amorphous; crystalline phases include starting materials and trace precipitates, leaving the bulk of the ferric sulfate in the amorphous fraction. Due to considerable spectral masking, AFS detectability is highly dependent on spectroscopic technique and minerals present. This has strong implications for remote and in situ observations of Martian samples that include an amorphous component. AFS is only identifiable in VNIR spectra for magnesite, nontronite, and gypsum samples; hematite and basaltic glass samples appear similar to pure materials. Sulfate features dominate Raman spectra for nontronite and basaltic glass samples; the analog material dominates Raman spectra of hematite and gypsum samples. MIR data are least affected by masking, but basaltic glass is almost undetectable in MIR spectra of those mixtures. NaCl produces similar FTIR-ATR and Raman features, regardless of analog material.
58 GEOSCIENCES↗
Neutron Imaging dataset for SMC 2021 data challenge
The neutron radiography (nR) dataset provides information of the neutron events as measured for the Siemens star mask using the Timepix3 detector. The recordings consist of the position (x and y axes), the time-stamp, and time-over-threshold (TOT) values of each neutron event.
Peterson, Peter↗
DESI DR1 Ly α 1D power spectrum: Validation of estimators
The Data Release 1 (DR1) of the Dark Energy Spectroscopic Instrument (DESI) is the largest sample to date for small-scale Lyα forest cosmology, accessed through its one-dimensional power spectrum (P 1D ). The Lyα forest P 1D is extracted from quasar spectra that are highly inhomogeneous (both in wavelength and between quasars) in noise properties due to intrinsic properties of the quasar, atmospheric and astrophysical contamination, and also sensitive to low-level details of the spectral extraction pipeline. We employ two estimators in DR1 analysis to measure P 1D : the optimal estimator and the fast Fourier transform (FFT) estimator. To ensure robustness of our DR1 measurements, we validate these two power spectrum and covariance matrix estimation methodologies against the challenging aspects of the data. First, using a set of 20 synthetic 1D realizations of DR1, we derive the masking bias corrections needed for the FFT estimator and the continuum fitting bias needed for both estimators. We demonstrate that both estimators, including their covariances, are unbiased with these corrections using the Kolmogorov-Smirnov test. Second, we substantially extend our previous suite of CCD image simulations to include 675,000 quasars, allowing us to accurately quantify the pipeline's performance. This set of simulations reveals biases at the highest k values, corresponding to a resolution error of a few percent. We base the resolution systematics error budget of DR1 P 1D on these values, but do not derive corrections from them since the simulation fidelity is insufficient for precise corrections.
Lyman alpha forest↗
Version 4 CALIPSO Imaging Infrared Radiometer ice and liquid water cloud microphysical properties – Part I: The retrieval algorithms
Following the release of the version 4 Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) data products from the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) mission, a new version (version 4; V4) of the CALIPSO Imaging Infrared Radiometer (IIR) Level 2 data products has been developed. The IIR Level 2 data products include cloud effective emissivities and cloud microphysical properties such as effective diameter and ice or liquid water path estimates. Dedicated retrievals for water clouds were added in V4, taking advantage of the high sensitivity of the IIR retrieval technique to small particle sizes. This paper (Part I) describes the improvements in the V4 algorithms compared to those used in the version 3 (V3) release, while results will be presented in a companion (Part II) paper. The IIR Level 2 algorithm has been modified in the V4 data release to improve the accuracy of the retrievals in clouds of very small (close to 0) and very large (close to 1) effective emissivities. To reduce biases at very small emissivities that were made evident in V3, the radiative transfer model used to compute clear-sky brightness temperatures over oceans has been updated and tuned for the simulations using Modern-Era Retrospective analysis for Research and Applications version 2 (MERRA-2) data to match IIR observations in clear-sky conditions. Furthermore, the clear-sky mask has been refined compared to V3 by taking advantage of additional information now available in the V4 CALIOP 5 km layer products used as an input to the IIR algorithm. After sea surface emissivity adjustments, observed and computed brightness temperatures differ by less than ±0.2 K at night for the three IIR channels centered at 08.65, 10.6, and 12.05 µm, and inter-channel biases are reduced from several tens of Kelvin in V3 to less than 0.1 K in V4. We have also improved retrievals in ice clouds having large emissivity by refining the determination of the radiative temperature needed for emissivity computation. The initial V3 estimate, namely the cloud centroid temperature derived from CALIOP, is corrected using a parameterized function of temperature difference between cloud base and top altitudes, cloud absorption optical depth, and CALIOP multiple scattering correction factor. As shown in Part II, this improvement reduces the low biases at large optical depths that were seen in V3 and increases the number of retrievals. As in V3, the IIR microphysical retrievals use the concept of microphysical indices applied to the pairs of IIR channels at 12.05 and 10.6 µm and at 12.05 and 08.65 µm. The V4 algorithm uses ice look-up tables (LUTs) built using two ice habit models from the recent “TAMUice2016” database, namely the single-hexagonal-column model and the eight-element column aggregate model, from which bulk properties are synthesized using a gamma size distribution. Four sets of effective diameters derived from a second approach are also reported in V4. Here, the LUTs are analytical functions relating microphysical index applied to IIR channels 12.05 and 10.6 µm and effective diameter as derived from in situ measurements at tropical and midlatitudes during the Tropical Composition, Cloud, and Climate Coupling (TC4) and Small Particles in Cirrus Science and Operations Plan (SPARTICUS) field experiments.
54 ENVIRONMENTAL SCIENCES↗
Active Learning Meets Foundation Models: Fast Remote Sensing Data Annotation for Object Detection
Object detection in remote sensing demands extensive, high-quality annotations—a process that is both labor-intensive and time-consuming. In this work, we introduce a real-time active learning and semi-automated labeling framework that leverages foundation models to streamline dataset annotation for object detection in remote sensing imagery. For example, by integrating a Segment Anything Model (SAM), our approach generates mask-based bounding boxes that serve as the basis for dual sampling: (a) uncertainty estimation to pinpoint challenging samples, and (b) diversity assessment to ensure broad data coverage. Furthermore, our Dynamic Box Switching Module (DBS) addresses the well-known cold start problem for object detection models by replacing its suboptimal initial predictions with SAM-derived masks, thereby enhancing early-stage localization accuracy. Extensive evaluations on multiple remote sensing datasets plus a real-world user study, demonstrate that our framework not only reduces annotation effort, but also significantly boosts detection performance compared to traditional active learning sampling methods. The code for training and the user interface will be made available.
Burges, Marvin [ORNL] (ORCID:0000000312690769)↗
Regression Analysis with the Directed Infusion of Data
Integrating artificial intelligence and machine learning tools into industry necessitates large-scale collaborative efforts that ensure the robust and accurate execution of downstream analytics such as time series prediction, uncertainty quantification, grid optimization, and condition monitoring. However, concerns related to data privacy pervade the nuclear industry due to the proprietary nature of its data and the possibility of data leakage. Legacy techniques such as encryption often require the explicit transmission of data to trustworthy parties, thereby inviting data leakage concerns. The ideal collaboration scenario avoids the explicit dissemination of data/code while maintaining experimental fidelity, which is currently accomplished using various techniques such as trusted execution environments, homomorphic encryption, differential privacy, and multimatrix masking. These techniques, however, often necessitate a trade-off between trust, efficiency, and utility. This article extends a previously proposed technique called the directed infusion of data (DIOD) that ensures data privacy, allows for scalable obfuscation, and combats the risk of data leakage without compromising utility. The experiments discussed in this article examine a regression-type scenario using DIOD with the goal of preserving the inferential link between two variables. Using the point-kinetics equations, regression experiments compare the performance of a model trained using the original data to that of a model trained using the obfuscated data, which produced identical results. Our claim is further strengthened by an information theoretic proof and experiment, which showed that the inferential content between variables remains the same after obfuscation, thereby avoiding the required communication of the proprietary data.
47 - OTHER INSTRUMENTATION↗