Data Availability and Integrity Attacks for Power Systems via Nonlinear Spatial-Temporal Modeling
Not Available
24 POWER TRANSMISSION AND DISTRIBUTION↗
SEARCH · Search NASA
Results for “multiple inputs resolution”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
United States multi-sector land use and land cover base maps to support human and Earth system models
Abstract Earth System Models (ESMs) require current and future projections of land use and landcover change (LULC) to simulate land-atmospheric interactions and global biogeochemical cycles. Among the most utilized land systems in ESMs are the Community Land Model (CLM) and the Land-Use Harmonization 2 (LUH2) products. Regional studies also use these products by extending coarse projections to finer resolutions via downscaling or by using multisector dynamic (MSD) models. One such MSD model is the Global Change Analysis Model (GCAM), which has its own independent land module, but often relies on CLM or LUH2 as spatial inputs for its base years. However, this requires harmonization of thematically incongruent land systems at multiple spatial resolutions, leading to uncertainty and error propagation. To resolve these issues, we develop a thematically consistent LULC system for the conterminous United States adaptable to multiple MSD frameworks to support research at a regional level. Using empirically derived spatial products, we developed a series of base maps for multiple contemporary years of observation at a 30-m resolution that support flexibility and interchangeability amongst LUH2, CLM, and GCAM classification systems.
Oliver, Jay↗
High Resolution Fission Fragment Spectroscopy with Superconducting Microcalorimeters
Sub-1 AMU mass determination is important for determining fission yields and neutron multiplicity, which are necessary inputs for fission models. Fission models can improve spent nuclear waste stream analysis and nuclear fuel burnup determination. Here, to achieve this goal, we have used superconducting microcalorimeter detectors to directly measure the energy of fission fragments from the spontaneous fission of 252 Cf. With a fiber coupled LED pulser setup we demonstrate that we can reach a relative energy resolution of 0.1% and better for photon pulses with energies above 60 MeV. This instrument, in conjunction with time-of-flight (TOF) measurement, would allow for sub-1 atomic mass unit (AMU) mass determination of fission fragments in a future beamline application.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
Automated Fire Detection for Industrial Settings with Pretrained Convolutional Networks
Early fire detection in industrial environments is critical to preventing equipment damage, personal injury, and operational disruptions. Traditional smoke detectors, while effective, often experience delays due to the time required for smoke to reach sensors, allowing fires to spread. Manual fire watch operations and human surveillance of camera feeds are resource-intensive and prone to human error. To address these challenges, this paper explores the application of convolutional neural networks for automated fire detection, specifically in industrial settings. By leveraging 11 different pre-trained machine vision models from TensorFlow and enhancing them with transfer learning on a custom-built industrial fire dataset, we optimized fire detection performance. Here, we analyzed each machine vision model architecture in terms of its depth, width, and input image resolution, considering both resource requirements and detection accuracy. We further explored the option of combining multiple models into an ensemble classifier to evaluate whether the performance improvements could justify the much greater computational complexity and other practical impacts. A cost-benefit analysis is presented to evaluate the trade-offs between performance and computational expense. Our findings identify that EfficientNetV2L, specifically tailored for industrial applications, provides the optimal balance between costs involved in training and using the model versus the overall fire detection performance. Additionally, we present a qualitative analysis of model performance using the technique of gradient-based class activation mapping to provide explainability by visualizing model decisions.
artificial intelligence↗
Crowdsourcing the Frontier: Advancing Hybrid Physics‐ML Climate Simulation via a $\$$50,000 Kaggle Competition
Subgrid machine-learning (machine learning [ML]) parameterizations have the potential to introduce a new generation of climate models that incorporate the effects of higher-resolution physics without incurring the prohibitive computational cost associated with more explicit physics-based simulations. However, important issues, ranging from online instability to inconsistent online performance, have limited their operational use for long-term climate projections. To more rapidly drive progress in solving these issues, domain scientists and ML researchers opened up the offline aspect of this problem to the broader ML and data science community with the release of ClimSim, a NeurIPS Data sets and Benchmarks publication, and an associated Kaggle competition. This paper reports on the downstream results of the Kaggle competition by coupling emulators inspired by the winning teams' architectures to an interactive climate model (including full cloud microphysics, a regime historically prone to online instability) and systematically evaluating their online performance. Our results demonstrate that online stability in the low-resolution real-geography setting is reproducible across multiple diverse architectures, which we consider a key milestone. All tested architectures exhibit strikingly similar offline and online biases, though their responses to architecture-agnostic design choices (e.g., expanding the list of input variables) can differ significantly. Multiple Kaggle-inspired architectures achieve state-of-the-art results on certain metrics such as zonal mean bias patterns and global Root Mean Squared Error, indicating that crowdsourcing the essence of the offline problem is one path to improving online performance in hybrid physics-AI climate simulation.
Environmental sciences↗
Bayesian inference of nuclear-matter density from proton scattering
Background: Proton elastic scattering at intermediate energy is widely employed as a tool for determining the matter radius of atomic nuclei. Here, the sensitivity of the approach relies on high-resolution measurements at small scattering angles and low-momentum transfer. Under these conditions, the Glauber multiple scattering theory accurately describes the proton-nucleus elastic cross section. Purpose: Investigate the sensitivity of the Glauber multiple scattering theory to uncertainties associated with input parameters such as the nuclear-matter density distribution and nucleon-nucleon data. Method: A joint Bayesian inference was performed using 12 angular distributions of elastic scattering at different energies on 58 Ni, 90 Zr, and 208 Pb targets. A Metropolis-Hastings algorithm was implemented to make an uncertainty quantification analysis for the input parameters used in the Glauber multiple scattering theory. Results: The experimental cross sections were fitted simultaneously using a joint Bayesian inference approach. Posterior probability density distributions of 42 input parameters were obtained from the analysis. A moderate correlation between the nuclear density parameters and the nucleon-nucleon cross sections was found. This correlation impacts the extraction of the nuclear-matter radius. Conclusions: The present analysis provided a consistent method for extracting the nuclear-matter density distribution of 58 Ni, 90 Zr, and 208 Pb from data across different incident energies. Due to the correlation of the nucleon-nucleon cross sections with the other input parameters, a constrained Bayesian inference using free nucleon-nucleon cross section data was performed. The nuclear-matter radii obtained from the analysis are in good agreement with multiple results reported in the literature.
190 ≤ A ≤ 219↗
Data for A Hybrid Biophysical-Machine Learning Framework for Diurnal Surface Energy Flux Estimation Using Proximal Sensing
Thermal infrared-based remote sensing of surface energy fluxes has traditionally relied on high spatial resolution satellite data with revisit frequencies on the order of weeks. In this study, we evaluate a biophysics-based analytical surface energy balance model for predicting latent energy (LE) and sensible heat (H) fluxes using proximal sensing observations. The Surface Temperature Initiated Closure (STIC1.2) model has been extensively validated across a wide range of spatial and temporal scales using various satellite-derived thermal infrared data sets. Here we extend this validation by applying STIC at sub-hourly temporal resolution over multiple growing seasons for four distinct agricultural systems. We further develop and evaluate novel STIC variants that incorporate machine learning (ML) techniques to eliminate the need for surface energy balance observations, specifically net radiation and soil heat flux, thereby enhancing model applicability in data-sparse settings. The integration of a ML component to estimate surface available energy is shown to have strong predictive performance for both LE (R2 = 0.81–0.94) and H (R2 = 0.46–0.72) across all agricultural systems examined here, demonstrating the potential of hybrid biophysical-machine learning approaches for surface energy balance modeling with minimal data requirements. This study concludes with a novel application of explainable machine learning (exML) to diagnose sources of model error. This exML framework attributes residual prediction errors to both model input variables and environmental drivers not explicitly included in the simulation experiments. This approach provides a new pathway for improving model design and integrating previously overlooked yet influential variables into future model iterations.
AI/ML↗
Populus VariantDB v3.2 facilitates CRISPR and functional genomics research
The success of CRISPR genome editing studies depends critically on the precision of guide RNA (gRNA) design. Sequence polymorphisms in outcrossing tree species pose design hazards that can render CRISPR genome editing ineffective. Despite recent advances in tree genome sequencing with haplotype resolution, sequence polymorphism information remains largely inaccessible to various functional genomics research efforts. The Populus VariantDB v3.2 addresses these challenges by providing a user-friendly search engine to query sequence polymorphisms of heterozygous genomes. The database accepts short sequences, such as gRNAs and primers, as input for searching against multiple poplar genomes, including hybrids, with customizable parameters. We provide examples to showcase the utilities of VariantDB in improving the precision of gRNA or primer design. The platform-agnostic nature of the probe search design makes Populus VariantDB v3.2 a versatile tool for the rapidly evolving CRISPR field and other sequence-sensitive functional genomics applications. The database schema is expandable and can accommodate additional tree genomes to broaden its user base.
59 BASIC BIOLOGICAL SCIENCES↗
Feature-Based PMU Event Classification under Variable PMU Participation and Overlapping Events
Danovo Energy Solution's presented its paper named: Feature-Based PMU Event Classification under Variable PMU Participation and Overlapping Events at the 2026 Georgia Tech Fault & Disturbance Analysis Conference. The full paper can be found at OSTI ID# 3169150 Paper Abstract—Phasor Measurement Units (PMUs) stream time synchronized, high-resolution measurements from the grid, enabling data-driven techniques for event detection and classification. Accurate event classification improves grid reliability and stability. Events can be detected by varying numbers of PMUs and exhibit different durations depending on the event type. This variability challenges standard classifiers that require uniform input sizes. Moreover, multiple events may coincide, which increases classification complexity. Standard classifiers assign each instance to the class with the highest predicted probability, whereas overlapping events may exhibit comparable probabilities across multiple classes. In this study, to handle data size variability, we extract a wide range of time–frequency domain features from all available PMUs for each event into a fixed-length vector, facilitating the application of standard machine learning classifiers, including Random Forest, XGBoost, LightGBM, Support Vector Machine, and Multilayer Perceptron. To account for overlapping events, a probabilistic post-processing step is applied. For a given data instance, if multiple predicted class probabilities exceed 30% and the differences between them are less than 10%, the event is assigned to multiple classes. Experiments using real-world PMU data demonstrate that the Random Forest and XGBoost models achieve the highest accuracy, while the proposed post-processing method yields perfect classification performance on external unseen test sets.
Nematirad, Reza [Danova Energy Solutions]↗
Feature-Based PMU Event Classification under Variable PMU Participation and Overlapping Events
This paper is the basis for a presentation help at the 2026 Georgia Tech Fault & Disturbance Analysis Conference, which can be found at OSTI # 3168287 Paper Abstract—Phasor Measurement Units (PMUs) stream time synchronized, high-resolution measurements from the grid, enabling data-driven techniques for event detection and classification. Accurate event classification improves grid reliability and stability. Events can be detected by varying numbers of PMUs and exhibit different durations depending on the event type. This variability challenges standard classifiers that require uniform input sizes. Moreover, multiple events may coincide, which increases classification complexity. Standard classifiers assign each instance to the class with the highest predicted probability, whereas overlapping events may exhibit comparable probabilities across multiple classes. In this study, to handle data size variability, we extract a wide range of time–frequency domain features from all available PMUs for each event into a fixed-length vector, facilitating the application of standard machine learning classifiers, including Random Forest, XGBoost, LightGBM, Support Vector Machine, and Multilayer Perceptron. To account for overlapping events, a probabilistic post-processing step is applied. For a given data instance, if multiple predicted class probabilities exceed 30% and the differences between them are less than 10%, the event is assigned to multiple classes. Experiments using real-world PMU data demonstrate that the Random Forest and XGBoost models achieve the highest accuracy, while the proposed post-processing method yields perfect classification performance on external unseen test sets.
Nematirad, Reza [Danovo Energy Solutions]↗
FY24 Report on Water NSTF Testing: Lower Tank Inlet Piping Configuration
The following report serves as a summary of the accomplishments and testing results by the Natural convection Shutdown heat removal Test Facility (NSTF) experimental program over the past 12-month period. A major activity included reconfiguration of the chimney piping geometry which altered the discharge position into the tank from the original 50% tank height to a new lower position at a 10.9% tank height. This modification increases the volume of available coolant thus extending long-term operating capacity, however also results in a decreased liquid driving head which has the potential to reduce the natural circulation efficiency and decrease overall performance. Examination of the tradeoffs for these two configurations is an area of interest for RCCS designers and drove planned test activities this year. Ten matrix tests were performed in FY24, totaling 195 hours of heated operations and 12.9 MWh of electrical heating throughout the year, with eight classified as Accepted per NQA-1 and two classified as Trending. Testing prior to the facility reconfiguration examined the facility response to throttling at the tank inlet, demonstrating increased sensitivity of flow instabilities to throttling within the two-phase region when compared to throttling at the lower sensitivity single-phase inlet region. Testing after the chimney reconfiguration began with a baseline test at conditions of 80% inventory fill and prototypic thermal power input of 2.1 MW t . In addition to establishing a reference for nominal system behavior and performance, repeat testing with multiple subsequent runs demonstrated strong repeatability between tests performed at the same conditions in this new configuration. These tests also examined if influences would occur to system behavior with installation of new higher-resolution instrumentation within the upper chimney. In this critical region where boiling and flashing phenomena dominate, the test results provided confidence that the new instrumentation does not uncharacteristically influence the observed behaviors. An inventory parametric series was then initiated to examine system behavior with the new chimney configuration at six varying initial inventory levels, ranging from high 80% to low 20% fill. Two-phase oscillations with similar peak and mean flow rates were observed when comparing to the previous mid tank configuration. Generally, similar system response trends with inventory were also observed, such as two phase oscillations suppressing as inventories were lowered. However, in one absolute inventory comparison at the highest fill of 80%, flow oscillations saw a gradual growth over the 4 hours of two-phase operation in the new lower tank inlet configuration, opposite of the gradual dampening observed in the mid tank inlet configuration. This can be attributed to, in part, a greater hydrostatic head pressure above the two-phase discharge region where the boiling front is developed. Furthermore, the change resulted in greater sensitivity to liquid degassing phenomena during single-phase heating, causing loop instabilities to form which trigged moderate flow degradation during the period approaching saturation and boiling conditions. This behavior had been observed previously under some conditions but was a common occurrence in recent testing with the newer lower tank inlet configuration. Initial observations from these first data sets suggest an overall larger window of stability for the mid tank inlet configuration compared to the lower tank inlet. Lastly, the lowest inventory fill test was repeated over an extended testing window to examine depletion behavior. Natural circulation flow and effective heat removal performance were observed during most of the testing period; only after the tank became fully drained (0% fill) did flow stagnate and violent geysering events occur. This early observation confirms one relative advantage over the mid tank inlet configuration, which stagnated under comparable conditions at ~20% inventory remaining in the tank.
42 ENGINEERING↗
Atomic view of photosynthetic metabolite permeability pathways and confinement in synthetic carboxysome shells
Carboxysomes are protein microcompartments found in cyanobacteria, whose shell encapsulates rubisco at the heart of carbon fixation in the Calvin cycle. Carboxysomes are thought to locally concentrate CO 2 in the shell interior to improve rubisco efficiency through selective metabolite permeability, creating a concentrated catalytic center. However, permeability coefficients have not previously been determined for these gases, or for Calvin-cycle intermediates such as bicarbonate (HCO$^–_3$), 3-phosphoglycerate, or ribulose-1,5-bisphosphate. Starting from a high-resolution cryogenic electron microscopy structure of a synthetic β-carboxysome shell, we perform unbiased all-atom molecular dynamics to track metabolite permeability across the shell. The synthetic carboxysome shell structure, lacking the bacterial microcompartment trimer proteins and encapsulation peptides, is found to have similar permeability coefficients for multiple metabolites, and is not selectively permeable to HCO$^–_3$ relative to CO 2 . To resolve how these comparable permeabilities can be reconciled with the clear role of the carboxysome in the CO 2 -concentrating mechanism in cyanobacteria, complementary atomic-resolution Brownian Dynamics simulations estimate the mean first passage time for CO 2 assimilation in a crowded model carboxysome. Despite a relatively high CO 2 permeability of approximately 10 -2 cm/s across the carboxysome shell, the shell proteins reflect enough CO 2 back toward rubisco that 2,650 CO 2 molecules can be fixed by rubisco for every 1 CO 2 molecule that escapes under typical conditions. The permeabilities determined from all-atom molecular simulation are key inputs into flux modeling, and the insight gained into carbon fixation can facilitate the engineering of carboxysomes and other bacterial microcompartments for multiple applications.
59 BASIC BIOLOGICAL SCIENCES↗
High resolution identification and quantification of diffuse deep groundwater discharge in mountain rivers using continuous boat-mounted helium measurements
Discharge of deeply sourced groundwater to streams is difficult to locate and quantify, particularly where both discrete and diffuse discharge points exist, but diffuse discharge is one of the primary controls on solute budgets in mountainous watersheds. The noble gas helium is a unique identifier of deep groundwater discharge because groundwater with long residence times is commonly enriched in helium. In this study, a portable mass spectrometer was used to measure longitudinal variation in dissolved helium concentrations in two mountainous rivers at high spatial resolution not feasible with traditional sampling techniques. Helium profiles were then simulated using a mass-balance model to quantify longitudinal variation in groundwater discharge to the receiving rivers. Results indicate helium concentrations were enriched by multiple orders of magnitude above atmospheric equilibrium in both rivers and that this persisted for up to 18 km below observed pulse inputs in the Colorado River. Helium mass-balance models match observed longitudinal patterns with the exception of sharp initial increases in helium observed in the rivers. Increased longitudinal groundwater discharge rates correspond to mapped geologic structures in both watersheds that likely transport deep geothermal water. Models show variable sensitivity to spatial assignment of input variables representing the groundwater source, illustrating the importance of collecting data from discrete groundwater discharges where possible. The methodology shows promise for field experiments designed to assess air–water exchange rates and to quantify total groundwater discharge from a combination of discrete and diffuse sources.
Deep groundwater↗
HYBRD (High Resolution HYBrid Regional Downscaling) Model: Input data and Code
The HYBRD (HYBrid Regional Downscaling) model is a high-resolution urban land downscaling model that can be used to downscale intermediate urban land use and land cover (LULC) products into a high-resolution (30-meters). HYBRD uses a sequential hybrid process, combining statistical models with cellular-automata-based spatial algorithms. This repository contains all the necessary model code and inputs needed to successfully run HYBRD for Los Angeles, California. The repo also contains example outputs of each model step, except the final simulated raster outputs. Examples of simulated raster outputs for multiple scenarios for Los Angeles are available at DOI: 10.57931/2575233. Please refer to Related Works below.
Land↗
Super-Resolution for Renewable Energy Resource Data with Wind from Reanalysis Data and Application to Ukraine
With a potentially increasing share of the electricity grid relying on wind to provide generating capacity and energy, there is an expanding global need for historically accurate, spatiotemporally continuous, high-resolution wind data. Conventional downscaling methods for generating these data based on numerical weather prediction have a high computational burden and require extensive tuning for historical accuracy. In this work, we present a novel deep learning-based spatiotemporal downscaling method using generative adversarial networks (GANs) for generating historically accurate high-resolution wind resource data from the European Centre for Medium-Range Weather Forecasting Reanalysis version 5 data (ERA5). In contrast to previous approaches, which used coarsened high-resolution data as low-resolution training data, we use true low-resolution simulation outputs. We show that by training a GAN model with ERA5 as the low-resolution input and Wind Integration National Dataset Toolkit (WTK) data as the high-resolution target, we achieved results comparable in historical accuracy and spatiotemporal variability to conventional dynamical downscaling. This GAN-based downscaling method additionally reduces computational costs over dynamical downscaling by two orders of magnitude. We applied this approach to downscale 30 km, hourly ERA5 data to 2 km, 5 min wind data for January 2000 through December 2023 at multiple hub heights over Ukraine, Moldova, and part of Romania. With WTK coverage limited to North America from 2007–2013, this is a significant spatiotemporal generalization. The geographic extent centered on Ukraine was motivated by stakeholders and energy-planning needs to rebuild the Ukrainian power grid in a decentralized manner. This 24-year data record is the first member of the super-resolution for renewable energy resource data with wind from the reanalysis data dataset (Sup3rWind).
17 WIND ENERGY↗
Comparative Assessment of U-Net-Based Deep Learning Models for Segmenting Microfractures and Pore Spaces in Digital Rocks
Segmentation of high-resolution X-ray microcomputed tomography (µCT) images is crucial in digital rock physics (DRP), affecting the characterization and analysis of microscale phenomena in the porous media. The complexity of geological structures and nonideal scanning conditions pose significant challenges to conventional image segmentation approaches. Motivated by the recent increasing popularity of deep learning (DL) techniques in image processing, this work undertakes a comparative study of DL models, specifically U-Net and its variants, for segmenting multiple targets with distinguished features in digital rocks, including discrete fracture networks (DFNs), pore spaces, and solid rock. Particularly, DFNs have a smaller volumetric fraction over others, bringing in a substantial challenge of imbalanced segmentation. The primary focus is to evaluate the architecture and feature enhancement strategies of various DL models, including U-Net, attention U-Net, residual U-Net, U-Net++, and residual U-Net++. The models were designed as 2.5D, utilizing a central 2D image and its two adjacent upper and lower 2D images as input to provide a pseudo-3D context. In addition, because the ground truth of segmentation was unknown for real-world digital rocks, we created a benchmark data set following the inverse operations of segmentation. The data synthesis started from the label images (i.e., solid rock, pore spaces, and DFNs), followed by simulating partial volume blurring, adding random background noise, and introducing ring artifacts to mimic real raw X-ray µCT images. The data set, which included various rock types (i.e., sandstone and artificial data), scanning resolution, and magnitudes of noise and artifacts, was divided into training and testing data sets with a 90% and 10% ratio, respectively. Moreover, in addition to the conventional pixel-wise evaluation metrics, the physics-based metric of the lattice-Boltzmann method (LBM) simulated permeability provided more comprehensive assessments. The results demonstrated that the residual connections, nested architectures, and redesigned skip connections contribute to the model performance and give the residual U-Net++ the highest accuracy. The improvements were mainly on the boundaries and small targets, especially the DFNs, which dominate the interconnectivity and therefore affect the permeability greatly. This study also rigorously evaluated the efficiency and generalization of each model, demonstrating that the sophisticated architectures achieved excellent practicability and maintained robust performance on completely unseen data, ensuring their suitability for diverse and challenging DRP applications.
58 GEOSCIENCES↗
Jet classification using high-level features from anatomy of top jets
Recent advancements in deep learning models have significantly enhanced jet classification performance by analyzing low-level features (LLFs). However, this approach often leads to less interpretable models, emphasizing the need to understand the decision-making process and to identify the high-level features (HLFs) crucial for explaining jet classification. To address this, we consider the top jet tagging problems and introduce an analysis model (AM) that analyzes selected HLFs designed to capture important features of top jets. Our AM mainly consists of the following three modules: a relation network analyzing two-point energy correlations, mathematical morphology and Minkowski functionals for generalizing jet constituent multiplicities, and a recursive neural network analyzing subjet constituent multiplicity to enhance sensitivity to subjet color charges. We demonstrate that our AM achieves performance comparable to the Particle Transformer (ParT) while requiring fewer computational resources in a comparison of top jet tagging using jets simulated at the hadronic calorimeter angular resolution scale. Furthermore, as a more constrained architecture than ParT, the AM exhibits smaller training uncertainties because of the bias-variance tradeoff. We also compare the information content of AM and ParT by decorrelating the features already learned by AM. Lastly, we briefly comment on the results of AM with finer angular resolution inputs.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
Super Resolving Unrolled Neural Networks for Remote Sensing
In remote sensing systems, the capabilities of the system are constrained by the complex interactions between size, weight, and power (SWAP) of potential designs. In electro-optical (EO) systems, examples of these critical parameters include the system’s sensitivity and resolution. Those parameters can be increased by ever larger optical apertures and focal planes but at the cost of more SWAP. Multi-image super resolution (MISR) techniques allow resolution to be enhanced via computation rather than more sophisticated optical hardware. These algorithms combine multiple images together into a single, higher resolution image, trading temporal resolution and computation for spatial resolution. Fielded MISR techniques, such as Drizzle, can require several hundred images to create a single super resolved image, implying reduced temporal resolution, increased data acquisition load, and limiting mission applications. Iterative techniques, such as model-based image reconstruction and compressive sensing, have been shown to create super resolved images using fewer images than Drizzle. They do this by posing an optimization problem that balances accuracy between a highly accurate physical model and an image model. In the case of super resolution, the physical model is defined by the relation between low resolution input images and the desired high resolution output image. The image model encodes some assumptions about the super resolved image. These assumptions are meant to suppress reconstruction artifacts that arise due to deterministic physical model error, stochastic measurement noise, and potential undersampling. In practice, the performance of iterative methods are limited by imaging models compatible with optimization. Deep learning-based methods can effectively learn image models of arbitrary complexity, but lack the theoretical explainability and robustness of iterative techniques. Consensus equilibrium (CE) generalizes the iterative techniques beyond optimization, enabling blackbox algorithms such as traditional and neural image denoisers to be used as the image model. CE-based approaches retain much of the explainability and robustness of iterative techniques while allowing the expressiveness of machine learning image models to be used. Additionally, by unrolling iterations of CE with an embedded image denoiser, the image denoiser can be further trained and specialized to the specific application with potentially higher quality reconstructions. Under this project, we demonstrated the feasibility of training an unrolled neural network based upon CE. While we didn’t train one, we showed that the CE process is differentiable and its gradient can be tractably computed. We also explored the usage of a variants of CE akin to generative neural works. Most importantly, we applied the CE framework to a number of problems including non-blind deconvolution, upsampling, single-image super resolution, MISR, event-based sensing, and saturated deconvolution. Our MISR prototype creates high quality reconstructions with an order of magnitude fewer images than previous approaches and, critically, produces these reconstructions fast enough for practical usage.
47 OTHER INSTRUMENTATION↗