Search NASA⌕ Search

SEARCH · Search NASA

Results for “data pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

The suitability of differentiable, physics-informed machine learning hydrologic models for ungauged regions and climate change impact assessment

As a genre of physics-informed machine learning, differentiable process-based hydrologic models (abbreviated as δ or delta models) with regionalized deep-network-based parameterization pipelines were recently shown to provide daily streamflow prediction performance closely approaching that of state-of-the-art long short-term memory (LSTM) deep networks. Meanwhile, δ models provide a full suite of diagnostic physical variables and guaranteed mass conservation. Here, we ran experiments to test (1) their ability to extrapolate to regions far from streamflow gauges and (2) their ability to make credible predictions of long-term (decadal-scale) change trends. We evaluated the models based on daily hydrograph metrics (Nash–Sutcliffe model efficiency coefficient, etc.) and predicted decadal streamflow trends. For prediction in ungauged basins (PUB; randomly sampled ungauged basins representing spatial interpolation), δ models either approached or surpassed the performance of LSTM in daily hydrograph metrics, depending on the meteorological forcing data used. They presented a comparable trend performance to LSTM for annual mean flow and high flow but worse trends for low flow. For prediction in ungauged regions (PUR; regional holdout test representing spatial extrapolation in a highly data-sparse scenario), δ models surpassed LSTM in daily hydrograph metrics, and their advantages in mean and high flow trends became prominent. In addition, an untrained variable, evapotranspiration, retained good seasonality even for extrapolated cases. The δ models' deep-network-based parameterization pipeline produced parameter fields that maintain remarkably stable spatial patterns even in highly data-scarce scenarios, which explains their robustness. Combined with their interpretability and ability to assimilate multi-source observations, the δ models are strong candidates for regional and global-scale hydrologic simulations and climate change impact assessment.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Leveraging BERT and Network-Based Attention Analysis for Identifying Treatment Milestones in EHRs

This study introduces a sophisticated data-driven framework for analyzing Electronic Health Records (EHRs) using transformer-based models to identify and disentangle overlapping treatment contexts. The framework leverages a preprocessing pipeline that transforms structured procedural codes into semantically enriched descriptive text, enabling the use of attention mechanisms to cluster medical events into treatment milestones—cohesive and distinct components of care processes. The methodology is rigorously validated using synthetic datasets derived from the MIMIC-III database, designed to simulate the heterogeneity and overlapping procedural contexts characteristic of real-world EHR scenarios. Quantitative evaluation highlights the framework’s robustness in disentangling concurrent care pathways, with attention metrics and unsupervised clustering approaches demonstrating the ability to preserve intra-context relationships while distinguishing inter-context dependencies. By addressing challenges inherent in data heterogeneity, this approach provides a foundation for uncovering complex treatment patterns, advancing clinical decision-making, and optimizing resource allocation in diverse healthcare environments.

Kim, Minsu [ORNL] (ORCID:0000000224185535)↗
DOE OSTI2023

A framework to measure the properties of intergalactic metal systems with two-point flux statistics

ABSTRACT The abundance, temperature, and clustering of metals in the intergalactic medium are important parameters for understanding their cosmic evolution and quantifying their impact on cosmological analysis with the Ly α forest. The properties of these systems are typically measured from individual quasar spectra redward of the quasar’s Ly α emission line, yet that approach may provide biased results due to selection effects. We present an alternative approach to measure these properties in an unbiased manner with the two-point statistics commonly employed to quantify large-scale structure. Our model treats the observed flux of a large sample of quasar spectra as a continuous field and describes the one-dimensional, two-point statistics of this field with three parameters per ion: the abundance (column density distribution), temperature (Doppler parameter), and clustering (cloud–cloud correlation function). We demonstrate this approach on multiple ions (e.g. ${\rm C\, \small {\rm IV}}$ , ${\rm Si\, \small {\rm IV}}$ , and ${\rm Mg\, \small {\rm II}}$ ) with early data from the Dark Energy Spectroscopic Instrument (DESI) and high-resolution spectra from the literature. Our initial results show some evidence that the ${\rm C\, \small {\rm IV}}$ abundance is higher than previous measurements and evidence for abundance evolution over time. The first full year of DESI observations will have over an order of magnitude more quasar spectra than this study. In a future paper, we will use those data to measure the growth of clustering and its impact on the Ly α forest, as well as test other DESI analysis infrastructure such as the pipeline noise estimates and the resolution matrix.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

DEEP CELLULAR RECURRENT NEURAL ARCHITECTURE FOR EFFICIENT MULTIDIMENSIONAL TIME-SERIES DATA PROCESSING

Efficient processing of time series data is a fundamental yet challenging problem in pattern recognition. Though recent developments in machine learning and deep learning have enabled remarkable improvements in processing large scale datasets in many application domains, most are designed and regulated to handle inputs that are static in time. Many real-world data, such as in biomedical, surveillance and security, financial, manufacturing and engineering applications, are rarely static in time, and demand models able to recognize patterns in both space and time. Current machine learning (ML) and deep learning (DL) models adapted for time series processing tend to grow in complexity and size to accommodate the additional dimensionality of time. Specifically, the biologically inspired learning based models known as artificial neural networks that have shown extraordinary success in pattern recognition, tend to grow prohibitively large and cumbersome in the presence of large scale multi-dimensional time series biomedical data such as EEG. Consequently, this work aims to develop representative ML and DL models for robust and efficient large scale time series processing. First, we design a novel ML pipeline with efficient feature engineering to process a large scale multi-channel scalp EEG dataset for automated detection of epileptic seizures. With the use of a sophisticated yet computationally efficient time-frequency analysis technique known as harmonic wavelet packet transform and an efficient self-similarity computation based on fractal dimension, we achieve state-of-the-art performance for automated seizure detection in EEG data. Subsequently, we investigate the development of a novel efficient deep recurrent learning model for large scale time series processing. For this, we first study the functionality and training of a biologically inspired neural network architecture known as cellular simultaneous recurrent neural network (CSRN). We obtain a generalization of this network for multiple topological image processing tasks and investigate the learning efficacy of the complex cellular architecture using several state-of-the?art training methods. Finally, we develop a novel deep cellular recurrent neural network (CDRNN) architecture based on the biologically inspired distributed processing used in CSRN for processing time series data. The proposed DCRNN leverages the cellular recurrent architecture to promote extensive weight sharing and efficient, individualized, synchronous processing of multi-source time series data. Experiments on a large scale multi-channel scalp EEG, and a machine fault detection dataset show that the proposed DCRNN offers state-of-the-art recognition performance while using substantially fewer trainable recurrent units.

Vidyaratne, Lasitha S.↗
DOE OSTI2023

A Hardware and Software Co-design Framework for Energy Efficient Neuromorphic Systems

Neuromorphic systems can be realized by a variety of algorithms and architectures. A common understanding is that spiking neuromorphic designs, which encode information into spatio-temporal spiking events, are both a biologically-accurate and efficient way of processing information. However, representing the information through timing relationships induces sophisticated circuit designs in traditional CMOS-based implementations. In recent years, high-capacity resistive memory (RRAM, aka, memristor) has demonstrated great potential in mimicking synaptic behaviors. Several RRAM-based spiking neuromorphic designs exist, most of which focus on rate coding schemes. These designs simplify circuit implementations of neuron models and explore challenges such as unsatisfactory speed, resolution, and performance. As an alternative, we will explore temporal coding spiking neuromorphic systems that encode information as the relative timing of neuron activations (spikes), which have been proven to be more adaptive and energy-efficient. Developing a neuromorphic system for spiking neural network (SNN) inference and online training, however, faces some major technical challenges: (1) It lacks circuit implementation support for temporal-coding SNN to achieve satisfying power efficiency and accuracy; (2) Although existing research works have investigated memristive synapse and neuron designs for spike-timing-dependent plasticity, the non-ideal conditions in implementation, such as device variations and signal degradation, degrade online learning accuracy of large scale systems; and (3) Non-optimized, inter-layer data traffic in SNNs, leads to unnecessary data communication costs. In this project, we plan to address these challenges by a hardware and software co-design framework that incorporates solutions at the circuit, architecture, and algorithm levels. At the circuit-level, we will elaborate on the in-situ SNN processing element designs for supporting both inference and online training modes. Variation-aware schemes will be studied to improve reliability. At the architecture level, we propose a pipelined, asynchronous architecture to retain the timing resolution of spikes. At the algorithm level, we will investigate an innovative SNN training algorithm for enabling activation sparsification and reducing unnecessary data communication costs. This neuromorphic system will provide an effective solution to real-life energy-constrained applications and significantly contribute to the exploration of next-generation high-performance computing systems under the DOE context.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Impact of Rubin Observatory Cadence Choices on Supernovae Photometric Classification

The Vera C. Rubin Observatory's Legacy Survey of Space and Time (LSST) will discover an unprecedented number of supernovae (SNe), making spectroscopic classification for all the events infeasible. LSST will thus rely on photometric classification, whose accuracy depends on the not-yet-finalized LSST observing strategy. In this work, we analyze the impact of cadence choices on classification performance using simulated multiband light curves. First, we simulate SNe with an LSST baseline cadence, a nonrolling cadence, and a presto-color cadence, which observes each sky location three times per night instead of twice. Each simulated data set includes a spectroscopically confirmed training set, which we augment to be representative of the test set as part of the classification pipeline. Then we use the photometric transient classification library snmachine to build classifiers. We find that the active region of the rolling cadence used in the baseline observing strategy yields a 25% improvement in classification performance relative to the background region. This improvement in performance in the actively rolling region is also associated with an increase of up to a factor of 2.7 in the number of cosmologically useful Type Ia SNe relative to the background region. However, adding a third visit per night as implemented in presto-color degrades classification performance due to more irregularly sampled light curves. Overall, our results establish desiderata on the observing cadence related to classification of full SNe light curves, which in turn impacts photometric SNe cosmology with LSST.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Heavy Oil Polymer EOR in the Challenging Alaskan Arctic - It Works!

Under co-sponsorship of the US Department of Energy and Hilcorp Alaska LLC the first ever polymer field pilot commenced on 8/28/2018 in the Schrader Bluff heavy oil reservoir at the Milne Point Field on Alaska North Slope (ANS). The primary objective of the pilot is to prove the efficacy of polymer Enhanced Oil Recovery (EOR) to unlock the vast heavy oil resources on ANS. More than two and half years after startup, the polymer injection, supporting laboratory experiments and simulation studies steadily continue. The pilot started injecting hydrolyzed polyacrylamide (HPAM), at a concentration of 1,750 ppm to achieve a target viscosity of 45 cP, into the two horizontal injectors in the flood pattern. Production is monitored in the two horizontal producers. Based on laboratory measurements of polymer viscosity at reservoir conditions, the team decided to reduce polymer concentration to 1,200 ppm since July 2020 in an effort to control injection pressure and optimize polymer utilization. Quality control (QC) on the field ensures uniform polymer solution properties. Representative rock and fluid systems and test conditions are utilized in the corefloods on polymer retention, effect of injection water salinity, polymer loading, and their combinations on oil recovery. A history matched reservoir simulation model for forecasting oil recovery was developed on the basis of all the available field data. Field concerns related to the post-polymer breakthrough impact on flow assurance is addressed via specialized laboratory tests. Notwithstanding early operational disruptions and hydration issues, continuous polymer injection in both injectors has been achieved. To date, 950,000 lbs of polymer or 2 million barrels of polymer solution, equating to 13% of total pore volume (PV), has been placed in the flood pattern, serving as an effective indicator of adequate polymer injectivity. So far, the success of polymer EOR is evident from drastically reduced water cut in the producers, an estimated incremental 1,000 bopd over waterflood, and a favorable polymer utilization of 1.7 lbs/barrel of incremental oil. Polymer breakthrough was observed 26 months after the start of polymer injection. Main observations from corefloods are unusually high polymer retention values in some cases and a positive response to low salinity water. Although the heterogeneity in the flood pattern and exceptionally low water cut pose some challenges, persistent novel and justifiable simulation approaches have resulted in a robust history matched model. Experimental results on produced fluids treatment provide operational guidance for improved oil-water separation and mitigation of heater tube fouling. The ongoing success of the pilot is a key indicator of bringing the team ever closer to meeting the project’s primary objective. The encouraging results of the pilot is one of the drivers that has provided the impetus to apply polymer EOR throughout the Milne Point Field, which would increase oil recovery and extend the economic life of the Trans Alaska Pipeline System. The scientific knowledge, including the many lessons learned from this pilot also has referential value for other potential heavy oil EOR projects throughout the world. The collected data, operational lessons learned, and the overall success of the pilot are summarized in the paper.

None, None↗
DOE OSTI2024

The Atacama Cosmology Telescope: A Measurement of the DR6 CMB Lensing Power Spectrum and Its Implications for Structure Growth

We present new measurements of cosmic microwave background (CMB) lensing over 9400 deg of the sky. These lensing measurements are derived from the Atacama Cosmology Telescope (ACT) Data Release 6 (DR6) CMB data set, which consists of five seasons of ACT CMB temperature and polarization observations. We determine the amplitude of the CMB lensing power spectrum at 2.3% precision (43σ significance) using a novel pipeline that minimizes sensitivity to foregrounds and to noise properties. To ensure that our results are robust, we analyze an extensive set of null tests, consistency tests, and systematic error estimates and employ a blinded analysis framework. Our CMB lensing power spectrum measurement provides constraints on the amplitude of cosmic structure that do not depend on Planck or galaxy survey data, thus giving independent information about large-scale structure growth and potential tensions in structure measurements. The baseline spectrum is well fit by a lensing amplitude of A lens = 1.013 ± 0.023 relative to the Planck 2018 CMB power spectra best-fit ΛCDM model and A lens = 1.005 ± 0.023 relative to the ACT DR4 + WMAP best-fit model. From our lensing power spectrum measurement, we derive constraints on the parameter combination $S^{CMBL}_{8}$ ≡ σ 8 (Ω m /0.3) 0.25 of $S^{CMBL}_{8}$ = 0.818 ± 0.022 from ACT DR6 CMB lensing alone and $S^{CMBL}_{8}$ = 0.813 ± 0.018 when combining ACT DR6 and Planck NPIPE CMB lensing power spectra. These results are in excellent agreement with ΛCDM model constraints from Planck or ACT DR4 + WMAP CMB power spectrum measurements. Our lensing measurements from redshifts z ~ 0.5–5 are thus fully consistent with ΛCDM structure growth predictions based on CMB anisotropies probing primarily z ~ 1100. We find no evidence for a suppression of the amplitude of cosmic structure at low redshifts.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Towards AI-assisted neutrino flavor theory design

Particle physics theories, such as those which explain neutrino flavor mixing, arise from a vast landscape of model-building possibilities. A model’s construction typically relies on the intuition of theorists. It also requires considerable effort to identify appropriate symmetry groups, assign field representations, and extract predictions for comparison with experimental data. We develop Autonomous Model Builder (AMBer), a framework in which a reinforcement learning agent interacts with a streamlined physics software pipeline to search these spaces efficiently. AMBer selects symmetry groups, particle content, and group representation assignments to construct models while minimizing the number of free parameters introduced. We validate our approach in well-studied regions of theory space and extend the exploration to a previously unexamined symmetry group. While demonstrated in the context of neutrino flavor theories, this approach of reinforcement learning with physics software feedback may be extended to other theoretical model-building problems in the future.

Baretz, Jason Benjamin↗
DOE OSTI2023

Joint velocity and density reconstruction of the Universe with nonlinear differentiable forward modeling

Reconstructing the initial conditions of the Universe from late-time observations has the potential to optimally extract cosmological information. Due to the high dimensionality of the parameter space, a differentiable forward model is needed for convergence, and recent advances have made it possible to perform reconstruction with nonlinear models based on galaxy (or halo) positions. In addition to positions, future surveys will provide measurements of galaxies' peculiar velocities through the kinematic Sunyaev-Zel'dovich effect (kSZ), type Ia supernovae, the fundamental plane relation, and the Tully-Fisher relation. Here we develop the formalism for including halo velocities, in addition to halo positions, to enhance the reconstruction of the initial conditions. We show that using velocity information can significantly improve the reconstruction accuracy compared to using only the halo density field. Here we study this improvement as a function of shot noise, velocity measurement noise, and angle to the line of sight. We also show how halo velocity data can be used to improve the reconstruction of the final nonlinear matter overdensity and velocity fields. We have built our pipeline into the differentiable Particle-Mesh FlowPM package, paving the way to perform field-level cosmological inference with joint velocity and density reconstruction. This is especially useful given the increased ability to measure peculiar velocities in the near future.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

The impact of the fiducial cosmology assumption on BAO distance scale measurements

ABSTRACT Standard analysis pipelines for measurements of Baryon Acoustic Oscillations (BAO) in galaxy surveys make use of a fiducial cosmological model to guide the data compression required to transform from observed redshifts and angles to the measured angular and radial BAO peak positions. In order to remove any dependence on the fiducial cosmology from the results, all models compared to the data should mimic the compression and its dependence on the fiducial model. In practice, approximations are made when testing models: (1) There is assumed to be no residual dependence on the fiducial cosmology after reconstruction, (2) differences in the distance–redshift relationship are assumed to match a linear scaling, and (3) differences in clustering between true and fiducial models are assumed to be removed by the free parameters used to null the non-BAO signal. We test these approximations using the current standard measurement procedure with a set of halo catalogues from the aemulus suite of N-body simulations, which span a range of wCDM cosmological models. We focus on reconstruction of the primordial BAO and locating the BAO. For the range of wCDM cosmologies covered by the aemulus suite, we find no evidence for systematic errors in the measured BAO shift parameters α∥ and α⊥ to $\lt 0.1\%$. However, the measured errors $\sigma _{\alpha _{\parallel }}$ and $\sigma _{\alpha _{\bot }}$ show a notable absolute increase by up to +0.001 and +0.002, respectively, in the case that the fiducial cosmology does not match the truth. These effects on the inferred BAO scale will be important, given the precision of measurements expected from future surveys including DESI, Euclid, and WFIRST.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Atacama Cosmology Telescope: High-resolution component-separated maps across one third of the sky

Observations of the millimeter sky contain valuable information on a number of signals, including the blackbody cosmic microwave background (CMB), Galactic emissions, and the Compton-y distortion due to the thermal Sunyaev-Zel’dovich (tSZ) effect. Extracting new insight into cosmological and astrophysical questions often requires combining multiwavelength observations to spectrally isolate one component. Here, in this work, we present a new arc-minute-resolution Compton-y map, which traces out the line-of-sight-integrated electron pressure, as well as maps of the CMB in intensity and E-mode polarization, across a third of the sky (around 13,000 deg 2 ). We produce these through a joint analysis of data from the Atacama Cosmology Telescope (ACT) data release 4 and 6 at frequencies of roughly 93, 148, and 225 GHz, together with data from the Planck satellite at frequencies between 30 and 545 GHz. We present detailed verification of an internal linear combination pipeline implemented in a needlet frame that allows us to efficiently suppress Galactic contamination and account for spatial variations in the ACT instrument noise. These maps provide a significant advance, in noise levels and resolution, over the existing Planck component-separated maps and will enable a host of science goals including studies of cluster and galaxy astrophysics, inferences of the cosmic velocity field, primordial non-Gaussianity searches, and gravitational lensing reconstruction of the CMB.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Augmented Human Analysis (AHA)

Radio frequency (RF) signal monitoring generally emphasizes intentionally generated signals, such as WiFi, Bluetooth, or cellular transmissions. However, electronic devices also produce unintended radiated emissions (UREs), which could also be useful in RF spectrum analysis. In either case, deriving intelligence from RF signals is typically a human-intensive process requiring significant domain knowledge. In the Augmented Human Analysis (AHA) project, we investigate the utility of dimensionally aligned signal projection (DASP) and machine learning (ML) algorithms for accelerating RF analysis workflows. We find that while DASP algorithms can indeed highlight signal characteristics relevant for classification tasks, the choice of algorithmic hyperparameters greatly affects performance. To address this challenge, we evaluate the quality of DASP outputs using the silhouette score, which measures how well data points cluster; high silhouette scores indicate good clustering, and thus good hyperparameter values. This approach is critical for machine learning pipelines as the DASP parameters cannot be directly optimized during model training. By identifying good DASP parameters, and thus good DASP outputs, as a preprocessing step, we can decrease the amount of effort required for downstream ML model training. We demonstrate our workflow using a dataset of UREs from common household devices, showing that even without the aid of ML, proper selection of DASP parameters enables clustering by device type.

42 ENGINEERING↗
DOE OSTI2021

Detecting Low Surface Brightness Galaxies with Mask R-CNN

Low surface brightness galaxies (LSBGs), galaxies that are fainter than the dark night sky, are famously difficult to detect. However, studies of these galaxies are essential to improve our understanding of the formation and evolution of low-mass galaxies. In this work, we train a deep learning model using the Mask R-CNN framework on a set of simulated LSBGs inserted into images from the Dark Energy Survey (DES) Data Release 2 (DR2). This deep learning model is combined with several conventional image pre-processing steps to develop a pipeline for the detection of LSBGs. We apply this pipeline to the full DES DR2 coadd image dataset, and preliminary results show the detection of 22 large, high-quality LSBG candidates that went undetected by conventional algorithms. Furthermore, we find that Galactic cirrus represents the largest contaminant in our resulting candidate list.

Levy, Caleb↗
DOE OSTI2020

The Industry's Long-term, Yet Near-term, Issues: The Need for Radiation Protection Professionals - 20266

Since the discovery of X-rays and radioactivity, the civil use of nuclear technology has thrived for more than a half-century and has matured to be safer and more efficient. A balance between radiological and non-radiological knowledge and experience has been gained and retained through the years. Nevertheless, a slowly developing gap has emerged during the past few decades. National and international data show a declining trend in the existence of health physics programs and associated enrollment and training, which directly impact the pipeline of radiation professionals (i.e., educators, technicians, researchers, and regulators). This decline can be attributed to many factors including the lack of scientific inquisitiveness, declining funding, and the perception that existing protocols are sufficiently protective. Decommissioning and dismantling nuclear facilities is a complex and multi-phased process that generates large quantities of various levels of radioactive waste. With multiple facilities reaching their end of operational life, and others being shut down for economic reasons, the current pipeline for radiation protection experts cannot meet the future's need. Consequently, international organizations have proposed a variety of programs to transfer knowledge of radiation protection principles and practice. While many change-oriented efforts to counteract this trend aim to promote mobility, this is just a partial solution to the actual issue: long-term availability. This paper suggests multidisciplinary collaborations with modern technologies as a path forward to educate young professionals on the diversity of radiation-related applications to retain expert knowledge and foster innovation. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2024

Constrained curve fitting for semi-parametric models with radial basis function networks

Common to many analysis pipelines in lattice gauge theory and the broader scientific discipline is the need to fit a semi-parametric model to data. We propose a fit method that utilizes a radial basis function network to approximate the non-parametric component of such models. The approximate parametric model is fit to data using the basin hopping global optimization algorithm. Parameter constraints are enforced through Gaussian priors. The viability of our method is tested by examining its use in a finite-size scaling analysis of the $q$-state Potts model and $p$-state clock model with $q=2,3$ and $p=4,\infty$.

Peterson, Curtis T.↗
DOE OSTI2026

Latent Twins

Over the past decade, scientific machine learning has transformed the development of mathematical and computational frameworks for analyzing, modeling, and predicting complex systems. From inverse problems to numerical partial differential equations (PDEs), dynamical systems, and model reduction, these advances have pushed the boundaries of what can be simulated. Yet they have often progressed in parallel, with representation learning and algorithmic solution methods evolving largely as separate pipelines. With Latent Twins, we propose a unifying mathematical framework that creates a hidden surrogate in latent space for the underlying equations. Whereas digital twins mirror physical systems in the digital world, Latent Twins mirror mathematical systems in a learned latent space governed by operators. Through this lens, classical modeling, inversion, model reduction, and operator approximation all emerge as special cases of a single principle. We establish the fundamental approximation properties of Latent Twins for both ordinary differential equations (ODEs) and PDEs and demonstrate the framework across three representative settings: (i) canonical ODEs, capturing diverse dynamical regimes; (ii) a PDE benchmark using the shallow-water equations, contrasting Latent Twin simulations with deep operator network and forecasts with a four-dimensional variational method baseline; and (iii) a challenging real-data geopotential reanalysis dataset, reconstructing and forecasting from sparse, noisy observations. Latent Twins provide a compact, interpretable surrogate for solution operators that evaluate across arbitrary time gaps in a single-shot, while remaining compatible with scientific pipelines such as assimilation, control, and uncertainty quantification. Looking forward, this framework offers scalable, theory-grounded surrogates that bridge data-driven representation learning and classical scientific modeling across disciplines.

Latent Twins↗
DOE OSTI2023

VDiSC: An Open Source Framework for Distributed Smart City Vision and Biometric Surveillance Networks

Recent global growth in the interest of smart cities has led to trillions of dollars of investment toward research and development. These connected cities have the potential to create a symbiosis of technology and society and revolutionize the cost of living, safety, ecological sustainability, and quality of life of societies on a world-wide scale. Some key components of the smart city construct are connected smart grids, self-driving cars, federated learning systems, smart utilities, large-scale public transit, and proactive surveillance systems. While exciting in prospect, these technologies and their subsequent integration cannot be attempted without addressing the potential societal impacts of such a high degree of automation and data sharing. Additionally, the feasibility of coordinating so many disparate tasks will require a fast, extensible, unifying framework. To that end, we propose the Distributed Smart City framework for Vision, or VDiSC. VDiSC serves as a unified biometric API harness that allows for seamless evaluation, deployment, and simple pipeline creation for heterogeneous biometric software. VDiSC additionally provides a fully declarative capability for defining and coordinating custom machine learning and sensor pipelines, allowing the distribution of processes across otherwise incompatible hardware and networks. VDiSC ultimately provides a way to quickly configure, hot-swap, and expand large coordinated or federated systems online without interruptions for maintenance. Because much of the data collected in a smart city contains Personally Identifying Information (PII), VDiSC also provides built-in tools and layers to ensure secure and encrypted streaming, storage, and access of PII data across distributed systems.

Brogan, Joel↗