Search NASA⌕ Search

SEARCH · Search NASA

Results for “data challenges”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

A Three-Tier Incremental Approach for Development of Smart Corridor Digital Twins

Development of an arterial smart corridor digital twin requires the integration of real time data streams, data storage, and simulation model execution. This process may be complex, time consuming, and susceptible to errors. To aid in smart corridor digital twin development this paper seeks to provide a framework, best practices, and development guidance. As such, a three-tier incremental approach to smart corridor digital twin development is presented. The paper highlights practical issues in digital twin construction along with key data challenges based on experiences from the development of digital twins for two large Smart Corridor deployments, one in Chattanooga, Tennessee, and the other in Atlanta, Georgia. The presented three-tier incremental approach includes: 1) development of a prepopulated offline simulation, 2) development of a pseudo digital twin that is driven by archived data, and 3) integration of real time data streams to create the online digital twin model. The three-tiered approach facilitates conducting multiple trials and scenarios with increasing complexity, allowing for incremental error processing and updating of the digital twin.

Saroj, Abhilasha↗

Data-driven materials research enabled by natural language processing and information extraction

Given the emergence of data science and machine learning throughout all aspects of society, but particularly in the scientific domain, there is increased importance placed on obtaining data. Data in materials science are particularly heterogeneous, based on the significant range in materials classes that are explored and the variety of materials properties that are of interest. This leads to data that range many orders of magnitude, and these data may manifest as numerical text or image-based information, which requires quantitative interpretation. The ability to automatically consume and codify the scientific literature across domains - enabled by techniques adapted from the field of natural language processing - therefore has immense potential to unlock and generate the rich datasets necessary for data science and machine learning. This review focuses on the progress and practices of natural language processing and text mining of materials science literature and highlights opportunities for extracting additional information beyond text contained in figures and tables in articles. Here, we discuss and provide examples for several reasons for the pursuit of natural language processing for materials, including data compilation, hypothesis development, and understanding the trends within and across fields. Current and emerging natural language processing methods along with their applications to materials science are detailed. We, then, discuss natural language processing and data challenges within the materials science domain where future directions may prove valuable.

36 MATERIALS SCIENCE↗

Identifying geological structures through microseismic cluster and burst analyses complementing active seismic interpretation

At the Decatur carbon capture and storage site (IL, USA) CO 2 has been injected from 2011–2014 and from 2017 to present near the base of the Lower Mt. Simon Sandstone saline reservoir, resulting in microseismicity. Microseismicity is mainly located in the basement and distributed in distinct spatial clusters. The lack of significant impedance contrasts within the basement makes the interpretation of active-source seismic reflection data challenging, however, recent reprocessing allowed to resolve faults above and at the top of the basement. These faults generally do not coincide with the location of microseismic events and their continuation to the general depth of the seismic events cannot be assumed. This paper shows how the interpretation of the microseismicity can complement structural interpretations of active-source seismic reflection data. In particular, we analyze clusters and bursts (abrupt increases) of microseismicity, identify unresolved, smaller-scale weaknesses and extract statistical parameters. These parameters allow comparisons with the interpreted faults, and with fracture sets intercepted by boreholes. During injection at the Decatur site, the injection pressure was kept far below fracture pressure, nevertheless, seismic events were induced and spread far beyond the expected extent of the CO 2 plume. We argue that local stress transfers related to the CO 2 injection reactivated pre-existing fractures within the critically stressed basement. Finally, we conducted a slip tendency analysis for faults interpreted from active seismic, selected cluster, bursts and nodal planes from focal mechanisms to determine if the interpreted structures are optimally oriented with respect to the stress regime. Our results suggest that the orientation of fractures close to the injection well, generally shows slight deviations from the optimal orientation for slip. This might indicate either slight local deviations of the maximum horizontal stress azimuth from the average direction used in the analysis, or the lack of optimally oriented fractures at this location.

58 GEOSCIENCES↗

Predictions for the Detectability of Milky Way Satellite Galaxies and Outer-Halo Star Clusters with the Vera C. Rubin Observatory

We predict the sensitivity of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) to faint, resolved Milky Way satellite galaxies and outer-halo star clusters. We characterize the expected sensitivity using simulated LSST data from the LSST Dark Energy Science Collaboration (DESC) Data Challenge 2 (DC2) accessed and analyzed with the Rubin Science Platform as part of the Rubin Early Science Program. We simulate resolved stellar populations of Milky Way satellite galaxies and outer-halo star clusters over a wide range of sizes, luminosities, and heliocentric distances, which are broadly consistent with expectations for the Milky Way satellite system. We inject simulated stars into the DC2 catalog with realistic photometric uncertainties and star/galaxy separation derived from the DC2 data itself. We assess the probability that each simulated system would be detected by LSST using a conventional isochrone matched-filter technique. We find that assuming perfect star/galaxy separation enables the detection of resolved stellar systems with $M_V$ = 0 mag and $r_{1/2}$ = 10 pc with >50% efficiency out to a heliocentric distance of ~250 kpc. Similar detection efficiency is possible with a simple star/galaxy separation criterion based on measured quantities, although the false positive rate is higher due to leakage of background galaxies into the stellar sample. When assuming perfect star/galaxy classification and a model for the galaxy-halo connection fit to current data, we predict that 89 +/- 20 Milky Way satellite galaxies will be detectable with a simple matched-filter algorithm applied to the LSST wide-fast-deep data set. Different assumptions about the performance of star/galaxy classification efficiency can decrease this estimate by ~7%-25%, which emphasizes the importance of high-quality star/galaxy separation for studies of the Milky Way satellite population with LSST.

79 ASTRONOMY AND ASTROPHYSICS↗

Open Radiation Monitoring: Conceptual System Design

The Open Radiation Monitoring (ORM) Project seeks to develop and demonstrate a modular radiation detection architecture designed specifically for use in arms control treaty verification (ACTV) applications that will facilitate rapid development of trusted systems to meet the needs of potential future treaties. Development of trusted systems to support potential future treaties is a complex and costly endeavor that typically results in a purpose-built system designed to perform one specific task. The majority of prior trusted system development efforts have relied on the use of commercial embedded computers or microprocessors to control the system and process the acquired data. These processors are complex, making authentication and certification of measurement systems and collected data challenging and time consuming. We believe that a modular architecture can be used to reduce more complex systems to a series of single-purpose building blocks that could be used to implement a variety of detection modalities with shared functionalities. With proper design, the functionality of individual modules can be confirmed through simple input/output testing, thereby facilitating equipment inspection and in turn building trust in the equipment by all treaty parties. Furthermore, a modular architecture can be used to control data flow within the measurement system, reducing the risk of "hidden switches" and constraining the amount of sensitive information that could potentially be inadvertently leaked. This report documents a conceptual modular system architecture that is designed to facilitate inspection in an effort to reduce overall authentication and certification burden. As of publication, this architecture remains in a conceptual phase and additional funding is required to prove out the utility of a modular architecture and test the assumptions used to rationalize the design.

61 RADIATION PROTECTION AND DOSIMETRY↗

The LSST DESC DC2 Simulated Sky Survey

Here, we describe the simulated sky survey underlying the second data challenge (DC2) carried out in preparation for analysis of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) by the LSST Dark Energy Science Collaboration (LSST DESC). Significant connections across multiple science domains will be a hallmark of LSST; the DC2 program represents a unique modeling effort that stresses this interconnectivity in a way that has not been attempted before. This effort encompasses a full end-to-end approach: starting from a large N-body simulation, through setting up LSST-like observations including realistic cadences, through image simulations, and finally processing with Rubin's LSST Science Pipelines. This last step ensures that we generate data products resembling those to be delivered by the Rubin Observatory as closely as is currently possible. The simulated DC2 sky survey covers six optical bands in a wide-fast-deep area of approximately 300 deg 2 , as well as a deep drilling field of approximately 1 deg 2 . We simulate 5 yr of the planned 10 yr survey. The DC2 sky survey has multiple purposes. First, the LSST DESC working groups can use the data set to develop a range of DESC analysis pipelines to prepare for the advent of actual data. Second, it serves as a realistic test bed for the image processing software under development for LSST by the Rubin Observatory. In particular, simulated data provide a controlled way to investigate certain image-level systematic effects. Finally, the DC2 sky survey enables the exploration of new scientific ideas in both static and time domain cosmology.

79 ASTRONOMY AND ASTROPHYSICS↗

Deep Learning Scene Classification Experiments in Automatic Detection of Slums on Planetscope Imagery

Population growth is increasingly happening in slum settlements of the large urban centers in the Global South. The term "slum" encompasses a wide range of communities, located mostly in underserved areas, and often exhibiting distinct structural and functional informalities with a relatively high concentration of marginalized populations. To address the issues confronting slums for effective planning and development, including the realistic estimation of the resident population, identifying them accurately is fundamental. Given the disagreements over a universal definition, diverse characteristic features, and socio-political limitations, global detection of slums is a veritable challenge. In this paper, we present experiments in slum detection using a scene classification algorithm and 3-meter spatial resolution satellite imagery. We train and evaluate the model for slum detection in Mumbai, India for the year 2023 and test the temporal generalization of the trained model on Mumbai in 2020 and 2018. In addition, we explore the pathways toward geographic generalization to Kolkata and Delhi (India). We discuss several limitations in the workflow and model, situate our findings in the existing literature, and suggest improvements and alternatives. With this, we establish baseline methods and experiments as a first step towards developing an image-based global slum detection framework and algorithm. This work adds to the community discussion on methods, data challenges, and open questions related to the detection of slums globally. With this research, we hope to improve our understanding of human settlements, especially in critical areas, improve population estimates, and help measure progress towards the sustainable development goals.

Arndt, Jacob↗

SNIa Cosmology Analysis Results from Simulated LSST Images: From Difference Imaging to Constraints on Dark Energy

Abstract The Vera Rubin Observatory Legacy Survey of Space and Time (LSST) is expected to process ∼10 6 transient detections per night. For precision measurements of cosmological parameters and rates, it is critical to understand the detection efficiency, magnitude limits, artifact contamination levels, and biases in the selection and photometry. Here we rigorously test the LSST Difference Image Analysis (DIA) pipeline using simulated images from the Rubin Observatory LSST Dark Energy Science Collaboration Data Challenge (DC2) simulation for the Wide-Fast-Deep survey area. DC2 is the first large-scale (300 deg 2 ) image simulation of a transient survey that includes realistic cadence, variable observing conditions, and CCD image artifacts. We analyze ∼15 deg 2 of DC2 over a 5 yr time span in which artificial point sources from Type Ia supernova (SNIa) light curves have been overlaid onto the images. The magnitude limits per filter are u = 23.66 mag, g = 24.69 mag, r = 24.06 mag, i = 23.45 mag, z = 22.54 mag, and y = 21.62 mag. The artifact contamination levels are ∼90% of all detections, corresponding to ∼1000 artifacts deg –2 in g band, and falling to 300 deg –2 in y band. The photometry has biases <1% for magnitudes 19.5 < m < 23. Our DIA performance on simulated images is similar to that of the Dark Energy Survey difference-imaging pipeline on real images. We also characterize DC2 image properties to produce catalog-level simulations needed for distance bias corrections. We find good agreement between DC2 data and simulations for distributions of signal-to-noise ratio, redshift, and fitted light-curve properties. Applying a realistic SNIa cosmology analysis for redshifts z < 1, we recover the input cosmology parameters to within statistical uncertainties.

79 ASTRONOMY AND ASTROPHYSICS↗

Predicting lake surface water phosphorus dynamics using process-guided machine learning

Phosphorus (P) loading to lakes is degrading the quality and usability of water globally. Accurate predictions of lake P dynamics are needed to understand whole-ecosystem P budgets, as well as the consequences of changing lake P concentrations for water quality. However, complex biophysical processes within lakes, along with limited observational data, challenge our capacity to reproduce short-term lake dynamics needed for water quality predictions, as well as long-term dynamics needed to understand broad scale controls over lake P. Here we use an emerging paradigm in modeling, process-guided machine learning (PGML), to produce a phosphorus budget for Lake Mendota (Wisconsin, USA) and to accurately predict epilimnetic phosphorus over a time range of days to decades. In our implementation of PGML, which we term a Process-Guided Recurrent Neural Network (PGRNN), we combine a process-based model for lake P with a recurrent neural network, and then constrain the predictions with ecological principles. We test independently the process-based model, the recurrent neural network, and the PGRNN to evaluate the overall approach. The process-based model accounted for most of the observed pattern in lake P; however it missed the long-term trend in lake P and had the worst performance in predicting winter and summer P in surface waters. The root mean square error (RMSE) for the process-based model, the recurrent neural network, and the PGRNN was 33.0 μg P L -1 , 22.7 μg P L -1 , and 20.7 μg P L -1 , respectively. All models performed better during summer, with RMSE values for the three models (same order) equal to 14.3 μg P L -1 , 10.9 μg P L -1 , and 10.7 μg P L -1 . Although the PGRNN had only marginally better RMSE during summer, it had lower bias and reproduced long-term decreases in lake P missed by the other two models. For all seasons and all years, the recurrent neural network had better predictions than process alone, with root mean square error (RMSE) of 23.8 μg P L -1 and 28.0 μg P L -1 , respectively. The output of PGRNN indicated that new processes related to water temperature, thermal stratification, and long term changes in external loads are needed to improve the process model. By using ecological knowledge, as well as the information content of complex data, PGML shows promise as a technique for accurate prediction in messy, real-world ecological dynamics, while providing valuable information that can improve our understanding of process.

54 ENVIRONMENTAL SCIENCES↗

Cross-property deep transfer learning framework for enhanced predictive analytics on small materials data

Abstract Artificial intelligence (AI) and machine learning (ML) have been increasingly used in materials science to build predictive models and accelerate discovery. For selected properties, availability of large databases has also facilitated application of deep learning (DL) and transfer learning (TL). However, unavailability of large datasets for a majority of properties prohibits widespread application of DL/TL. We present a cross-property deep-transfer-learning framework that leverages models trained on large datasets to build models on small datasets of different properties. We test the proposed framework on 39 computational and two experimental datasets and find that the TL models with only elemental fractions as input outperform ML/DL models trained from scratch even when they are allowed to use physical attributes as input, for 27/39 (≈ 69%) computational and both the experimental datasets. We believe that the proposed framework can be widely useful to tackle the small data challenge in applying AI/ML in materials science.

36 MATERIALS SCIENCE↗

Massively scalable Kerr comb-driven silicon photonic link

Abstract The growth of computing needs for artificial intelligence and machine learning is critically challenging data communications in today’s data-centre systems. Data movement, dominated by energy costs and limited ‘chip-escape’ bandwidth densities, is perhaps the singular factor determining the scalability of future systems. Using light to send information between compute nodes in such systems can dramatically increase the available bandwidth while simultaneously decreasing energy consumption. Through wavelength-division multiplexing with chip-based microresonator Kerr frequency combs, independent information channels can be encoded onto many distinct colours of light in the same optical fibre for massively parallel data transmission with low energy. Although previous high-bandwidth demonstrations have relied on benchtop equipment for filtering and modulating Kerr comb wavelength channels, data-centre interconnects require a compact on-chip form factor for these operations. Here we demonstrate a massively scalable chip-based silicon photonic data link using a Kerr comb source enabled by a new link architecture and experimentally show aggregate single-fibre data transmission of 512 Gb s −1 across 32 independent wavelength channels. The demonstrated architecture is fundamentally scalable to hundreds of wavelength channels, enabling massively parallel terabit-scale optical interconnects for future green hyperscale data centres.

Rizzo, Anthony (ORCID:000000034752797X)↗

Diet outperforms microbial transplant to drive microbiome recovery in mice

A high-fat, low-fibre Western-style diet (WD) induces microbiome dysbiosis characterized by reduced taxonomic diversity and metabolic breadth, which in turn increases risk for a wide array of metabolic, immune and systemic pathologies. Recent work has established that WD can impair microbiome resilience to acute perturbations such as antibiotic treatment, although little is known about the mechanism of impairment and the specific consequences for the host of prolonged post-antibiotic dysbiosis. Here, in this study, we characterize the trajectory by which the gut microbiome recovers its taxonomic and functional profile after antibiotic treatment in mice on regular chow (RC) or WD, and find that only mice on RC undergo a rapid successional process of recovery. Metabolic modelling indicates that a RC diet promotes the development of syntrophic cross-feeding interactions, whereas in mice on WD, a dominant taxon monopolizes readily available resources without releasing syntrophic byproducts. Intervention experiments reveal that an appropriate dietary resource environment is both necessary and sufficient for rapid and robust microbiome recovery, whereas microbial transplant is neither. Furthermore, prolonged post-antibiotic dysbiosis in mice on WD renders them susceptible to infection by the intestinal pathogen Salmonella enterica serovar Typhimurium. Our data challenge widespread enthusiasm for faecal microbiota transplant (FMT) as a strategy to address dysbiosis, and demonstrate that specific dietary interventions are, at a minimum, an essential prerequisite for effective FMT, and may afford a safer, more natural and less invasive alternative.

Kennedy, M. S. [University of Chicago, IL (United ↗

WLCG transition from X.509 to Tokens: Progress and Outlook

Since 2017, the Worldwide LHC Computing Grid (WLCG) has been working towards enabling token-based authentication and authorization throughout its entire middleware stack.Taking guidance from the WLCG Token Transition Timeline, published in 2022, substantial progress has been achieved not only in making middleware compatible with the use of tokens, but also in understanding the limitations of the WLCG Common JWT Profiles, first published in 2019. Significant scalability experience has been gained from Data Challenge 2024, during which millions of files were transferred with tokens used as credentials - a significant percentage of the total transfers completed.Besides describing the state of affairs in the transition to tokens, revisions to the WLCG token profile, and the evolving road maps, this contribution also covers the corresponding transition from VOMS-Admin to INDIGO-IAM services, with continuing improvements in terms of functionality as well as deployment.

Dack, Thomas [Rutherford Appleton Laboratory]↗

Generating synthetic cosmological data with GalSampler

As part of the effort to meet the needs of the Large Synoptic Survey Telescope Dark Energy Science Collaboration (LSST DESC) for accurate, realistically complex mock galaxy catalogues, in this work we have developed galsampler, an open-source python package that assists in generating large volumes of synthetic cosmological data. The key idea behind galsampler is to recast hydrodynamical simulations and semi-analytic models as physically motivated galaxy libraries. galsampler populates a new, larger volume halo catalogue with galaxies drawn from the baseline library; by using weighted sampling guided by empirical modelling techniques, galsampler inherits statistical accuracy from the empirical model and physically motivated complexity from the baseline library. We have recently used galsampler to produce the cosmoDC 2 extragalactic catalogue made for the LSST DESC Data Challenge 2 . Using cosmoDC 2 as a guiding example, we outline how galsampler can continue to support ongoing and near-future galaxy surveys such as the Dark Energy Survey, the Dark Energy Spectroscopic Instrument, WFIRST, and Euclid.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Quantum critical collapse abhors a naked singularity

Classical critical collapse yields naked singularities from smooth initial data, challenging cosmic censorship, and shaping the spectrum of primordial black holes. We show that one-loop vacuum polarization near the threshold qualitatively changes this outcome by dressing the singularity with a horizon within a controlled semiclassical regime. In analytically tractable Einstein-scalar critical spacetimes, a one-loop $s$-wave treatment linearized around self-similar backgrounds shows that regularity uniquely selects an asymptotically Minkowskian, vacuum-polarization state. Its renormalized stress tensor carries a universal quantum growing mode that competes with the classical unstable mode, shifts the critical point, and generates a trapped surface along with a finite mass gap at the new threshold, thereby enforcing horizon formation even under arbitrary fine-tuning. In primordial collapse, the threshold shift enters exponentially into the formation fraction, while the mass gap truncates the low-mass tail, suggesting potentially important consequences for the predicted mass spectrum. Furthermore, these results provide a self-consistent semiclassical treatment of critical collapse and yield sharp predictions within the one-loop, near-critical, linearized regime.

Anomalies↗

Online-compatible unsupervised nonresonant anomaly detection

There is a growing need for anomaly detection methods that can broaden the search for new particles in a model-agnostic manner. Most proposals for new methods focus exclusively on signal sensitivity. However, it is not enough to select anomalous events—there must also be a strategy to provide context to the selected events. In this work, we propose the first complete strategy for unsupervised detection of nonresonant anomalies that includes both signal sensitivity and a data-driven method for background estimation. Our technique is built out of two simultaneously trained autoencoders that are forced to be decorrelated from each other. This method can be deployed off-line for nonresonant anomaly detection and is also the first complete on-line-compatible anomaly detection strategy. We show that our method achieves excellent performance on a variety of signals prepared for the ADC2021 data challenge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Hybrid Attack Graph Generation with Graph Convolutional Deep-Q Learning

Critical infrastructures such as power grids have become increasingly complex, connected, and vulnerable to adverse scenarios, including cyber and physical attacks and faults. Effective risk mitigation for such cyber-physical energy systems (CPES), requires preemptive knowledge of likely adversarial attack scenarios. Hybrid Attack Graph (HAG) is a structured way to represent an adversarial scenario as an attack sequence using a threat model. However, the scarcity of documented attack sequences hinders analysts and CPES planners’ ability to identify credible attack scenarios for a given CPES. We propose a data-driven Graph Convolutional Deep-Q Network (GCDQ) to address this data challenge through generating HAGs. By leveraging limited real-world observations from the MITRE ATT&CK knowledge base, our GCDQ model synthesizes realistic graphs with the targeted attribute of minimum detectability via reinforcement learning. This generative model is the first step in creating a tool to substantially boost the attack sequence dataset and enhance the performance of CPS defense-related tasks by providing insights into likely attack sequences with given attributes.

deep learning, artificial intelligence↗