Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Challenge”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Demonstrating the data center as a flexible grid asset using a C-HIL setup

Increasing data center demand is outpacing grid infrastructure development. Artificial intelligence workloads and hyperscale cloud growth are creating unprecedented demand for power, while traditional grid expansion faces multiyear development timelines. Verrus is developing an innovative datacenter solution for this challenge, data centers that act as active grid-supportive assets rather than passive loads. Our approach integrates a novel grid-aware power flow management system with battery energy storage systems(BESS) into a microgrid-controlled, medium-voltage power distribution architecture that delivers critical capabilities, such as: * Fast response to grid disturbances such over/ under voltage or over/ under frequency * Demand flexibility that can service requests from the utility within 10 s * Uninterrupted transition to islanded operation during grid outages * Continuous uptime assurance for compute loads while maintaining all customer service level agreements. Through Verrus' strategic partnership with the National Renewable Energy Laboratory (NREL), these capabilities were validated using NREL's Advanced Research on Integrated Energy Systems (ARIES) virtual emulation environment to model a 70-MW grid-interactive data center. This paper outlines the design, methodology, and results of this emulated deployment, demonstrating that data centers can provide both critical load resilience and ancillary grid support without compromising uptime requirements. Specifically, we present a digital real time simulation of a 70 MW data center integrated with a physical microgrid controller, and demonstrate the data center response in the event of a grid voltage and frequency event, utility demand response request and utility outage.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2016

Analysis tools for MesonEx at CLAS12

The JLAB upgrade will soon be completed and the new CLAS12 detector system will collect large volumes of data allowing detailed investigations of many aspects of hadron physics. The focus of the MesonEx experiment is on the production of mesonic states by low Q2 virtual photons, or quasi-real photons. Studying such mesonic states is a particularly challenging data analysis problem, requiring well understood detector systems, clean signal and background separation, handling of large volumes of data and crucially a close collaboration between experimentalists and theorists to ensure the most sophisticated theoretical methods are used to interrogate the data. Here we briefly outline some of the analysis and methods that are being used to prepare for the MesonEx experiment.

Glazier, Derek↗
DOE OSTI2024

CIRCLEZ : Reliable photometric redshifts for active galactic nuclei computed solely using photometry from Legacy Survey Imaging for DESI

Photometric redshifts for galaxies hosting an accreting supermassive black hole in their center, known as active galactic nuclei (AGNs), are notoriously challenging. At present, they are most optimally computed via spectral energy distribution (SED) fittings, assuming that deep photometry for many wavelengths is available. However, for AGNs detected from all-sky surveys, the photometry is limited and provided by a range of instruments and studies. This makes the task of homogenizing the data challenging, presenting a dramatic drawback for the millions of AGNs that wide surveys such as SRG/eROSITA are poised to detect. This work aims to compute reliable photometric redshifts for X-ray-detected AGNs using only one dataset that covers a large area: the tenth data release of the Imaging Legacy Survey (LS10) for DESI. LS10 provides deep grizW1-W4 forced photometry within various apertures over the footprint of the eROSITA-DE survey, which avoids issues related to the cross-calibration of surveys. We present the results from CIRCLEZ, a machine-learning algorithm based on a fully connected neural network. CIRCLEZ is built on a training sample of 14 000 X-ray-detected AGNs and utilizes multi-aperture photometry, mapping the light distribution of the sources. The accuracy (σNMAD) and the fraction of outliers (η) reached in a test sample of 2913 AGNs are equal to 0.067 and 11.6%, respectively. The results are comparable to (or even better than) what was previously obtained for the same field, but with much less effort in this instance. We further tested the stability of the results by computing the photometric redshifts for the sources detected in CSC2 and Chandra-COSMOS Legacy, reaching a comparable accuracy as in eFEDS when limiting the magnitude of the counterparts to the depth of LS10. The method can be applied to fainter samples of AGNs using deeper optical data from future surveys (for example, LSST, Euclid), granting LS10-like information on the light distribution beyond the morphological type. Along with this paper, we have released an updated version of the photometric redshifts (including errors and probability distribution functions) for eROSITA/eFEDS.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2015

Comparison of algorithms for determination of rotation measure and Faraday structure. I. 1100–1400 MHz

Faraday rotation measures (RMs) and more general Faraday structures are key parameters for studying cosmic magnetism and are also sensitive probes of faint ionized thermal gas. A definition of which derived quantities are required for various scientific studies is needed, as well as addressing the challenges in determining Faraday structures. A wide variety of algorithms has been proposed to reconstruct these structures. In preparation for the Polarization Sky Survey of the Universe's Magnetism (POSSUM) to be conducted with the Australian Square Kilometre Array Pathfinder and the ongoing Galactic Arecibo L-band Feeds Array Continuum Transit Survey (GALFACTS), we run a Faraday structure determination data challenge to benchmark the currently available algorithms, including Faraday synthesis (previously called RM synthesis in the literature), wavelet, compressive sampling, and QU-fitting. The input models include sources with one Faraday thin component, two Faraday thin components, and one Faraday thick component. The frequency set is similar to POSSUM/GALFACTS with a 300 MHz bandwidth from 1.1 to 1.4 GHz. We define three figures of merit motivated by the underlying science: (1) an average RM weighted by polarized intensity, RM{sub wtd}, (2) the separation Δϕ of two Faraday components, and (3) the reduced chi-squared χ{sub r}{sup 2}. Based on the current test data with a signal-to-noise ratio of about 32, we find the following. (1) When only one Faraday thin component is present, most methods perform as expected, with occasional failures where two components are incorrectly found. (2) For two Faraday thin components, QU-fitting routines perform the best, with errors close to the theoretical ones for RM{sub wtd} but with significantly higher errors for Δϕ. All other methods, including standard Faraday synthesis, frequently identify only one component when Δϕ is below or near the width of the Faraday point-spread function. (3) No methods as currently implemented work well for Faraday thick components due to the narrow bandwidth. (4) There exist combinations of two Faraday components that produce a large range of acceptable fits and hence large uncertainties in the derived single RMs; in these cases, different RMs lead to the same Q, U behavior, so no method can recover a unique input model. Further exploration of all these issues is required before upcoming surveys will be able to provide reliable results on Faraday structures.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

A Three-Tier Incremental Approach for Development of Smart Corridor Digital Twins

Development of an arterial smart corridor digital twin requires the integration of real time data streams, data storage, and simulation model execution. This process may be complex, time consuming, and susceptible to errors. To aid in smart corridor digital twin development this paper seeks to provide a framework, best practices, and development guidance. As such, a three-tier incremental approach to smart corridor digital twin development is presented. The paper highlights practical issues in digital twin construction along with key data challenges based on experiences from the development of digital twins for two large Smart Corridor deployments, one in Chattanooga, Tennessee, and the other in Atlanta, Georgia. The presented three-tier incremental approach includes: 1) development of a prepopulated offline simulation, 2) development of a pseudo digital twin that is driven by archived data, and 3) integration of real time data streams to create the online digital twin model. The three-tiered approach facilitates conducting multiple trials and scenarios with increasing complexity, allowing for incremental error processing and updating of the digital twin.

Saroj, Abhilasha↗
DOE OSTI2014

Frequent Subgraph Discovery in Large Attributed Streaming Graphs

The problem of finding frequent subgraphs in large dynamic graphs has so far only consid- ered a dynamic graph as being represented by a series of static snapshots taken at various points in time. This representation of a dynamic graph does not lend itself well to real time processing of real world graphs like social networks or internet traffic which consist of a stream of nodes and edges. In this paper we propose an algorithm that discovers the frequent subgraphs present in a graph represented by a stream of labeled nodes and edges. Our algorithm is efficient and consists of tunable parameters that can be tuned by the user to get interesting patterns from various kinds of graph data. In our model updates to the graph arrive in the form of batches which contain new nodes and edges. Our algorithm con- tinuously reports the frequent subgraphs that are estimated to be found in the entire graph as each batch arrives. We evaluate our system using 5 large dynamic graph datasets: the Hetrec 2011 challenge data, Twitter, DBLP and two synthetic. We evaluate our approach against two popular large graph miners, i.e., SUBDUE and GERM. Our experimental re- sults show that we can find the same frequent subgraphs as a non-incremental approach applied to snapshot graphs, and in less time.

Dynamic graph, Frequent subgraph mining, pattern d↗
DOE OSTI2020

Data-driven materials research enabled by natural language processing and information extraction

Given the emergence of data science and machine learning throughout all aspects of society, but particularly in the scientific domain, there is increased importance placed on obtaining data. Data in materials science are particularly heterogeneous, based on the significant range in materials classes that are explored and the variety of materials properties that are of interest. This leads to data that range many orders of magnitude, and these data may manifest as numerical text or image-based information, which requires quantitative interpretation. The ability to automatically consume and codify the scientific literature across domains - enabled by techniques adapted from the field of natural language processing - therefore has immense potential to unlock and generate the rich datasets necessary for data science and machine learning. This review focuses on the progress and practices of natural language processing and text mining of materials science literature and highlights opportunities for extracting additional information beyond text contained in figures and tables in articles. Here, we discuss and provide examples for several reasons for the pursuit of natural language processing for materials, including data compilation, hypothesis development, and understanding the trends within and across fields. Current and emerging natural language processing methods along with their applications to materials science are detailed. We, then, discuss natural language processing and data challenges within the materials science domain where future directions may prove valuable.

36 MATERIALS SCIENCE↗
DOE OSTI2012

AERIAL MEASURING SYSTEM IN JAPAN

The U.S. Department of Energy National Nuclear Security Agency’s Aerial Measuring System deployed personnel and equipment to partner with the U.S. Air Force in Japan to conduct multiple aerial radiological surveys. These were the first and most comprehensive sources of actionable information for U.S. interests in Japan and provided early confirmation to the government of Japan as to the extent of the release from the Fukushima Daiichi Nuclear Power Generation Station. Many challenges were overcome quickly during the first 48 hours; including installation and operation of Aerial Measuring System equipment on multiple U.S. Air Force Japan aircraft, flying over difficult terrain, and flying with talented pilots who were unfamiliar with the Aerial Measuring System flight patterns. These all combined to make for a dynamic and non-textbook situation. In addition, the data challenges of the multiple and on-going releases, and integration with the Japanese government to provide valid aerial radiological survey products that both military and civilian customers could use to make informed decisions, was extremely complicated. The Aerial Measuring System Fukushima response provided insight in addressing these challenges and gave way to an opportunity for the expansion of the Aerial Measuring System’s mission beyond the borders of the US.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2021

Identifying geological structures through microseismic cluster and burst analyses complementing active seismic interpretation

At the Decatur carbon capture and storage site (IL, USA) CO 2 has been injected from 2011–2014 and from 2017 to present near the base of the Lower Mt. Simon Sandstone saline reservoir, resulting in microseismicity. Microseismicity is mainly located in the basement and distributed in distinct spatial clusters. The lack of significant impedance contrasts within the basement makes the interpretation of active-source seismic reflection data challenging, however, recent reprocessing allowed to resolve faults above and at the top of the basement. These faults generally do not coincide with the location of microseismic events and their continuation to the general depth of the seismic events cannot be assumed. This paper shows how the interpretation of the microseismicity can complement structural interpretations of active-source seismic reflection data. In particular, we analyze clusters and bursts (abrupt increases) of microseismicity, identify unresolved, smaller-scale weaknesses and extract statistical parameters. These parameters allow comparisons with the interpreted faults, and with fracture sets intercepted by boreholes. During injection at the Decatur site, the injection pressure was kept far below fracture pressure, nevertheless, seismic events were induced and spread far beyond the expected extent of the CO 2 plume. We argue that local stress transfers related to the CO 2 injection reactivated pre-existing fractures within the critically stressed basement. Finally, we conducted a slip tendency analysis for faults interpreted from active seismic, selected cluster, bursts and nodal planes from focal mechanisms to determine if the interpreted structures are optimally oriented with respect to the stress regime. Our results suggest that the orientation of fractures close to the injection well, generally shows slight deviations from the optimal orientation for slip. This might indicate either slight local deviations of the maximum horizontal stress azimuth from the average direction used in the analysis, or the lack of optimally oriented fractures at this location.

58 GEOSCIENCES↗
DOE OSTI2015

Prediction Models for Dynamic Demand Response

As Smart Grids move closer to dynamic curtailment programs, Demand Response (DR) events will become necessary not only on fixed time intervals and weekdays predetermined by static policies, but also during changing decision periods and weekends to react to real-time demand signals. Unique challenges arise in this context vis-a-vis demand prediction and curtailment estimation and the transformation of such tasks into an automated, efficient dynamic demand response (D 2 R) process. While existing work has concentrated on increasing the accuracy of prediction models for DR, there is a lack of studies for prediction models for D 2 R, which we address in this paper. Our first contribution is the formal definition of D 2 R, and the description of its challenges and requirements. Our second contribution is a feasibility analysis of very-short-term prediction of electricity consumption for D 2 R over a diverse, large-scale dataset that includes both small residential customers and large buildings. Our third, and major contribution is a set of insights into the predictability of electricity consumption in the context of D 2 R. Specifically, we focus on prediction models that can operate at a very small data granularity (here 15-min intervals), for both weekdays and weekends - all conditions that characterize scenarios for D 2 R. We find that short-term time series and simple averaging models used by Independent Service Operators and utilities achieve superior prediction accuracy. We also observe that workdays are more predictable than weekends and holiday. Also, smaller customers have large variation in consumption and are less predictable than larger buildings. Key implications of our findings are that better models are required for small customers and for non-workdays, both of which are critical for D 2 R. Also, prediction models require just few days’ worth of data indicating that small amounts of historical training data can be used to make reliable predictions, simplifying the complexity of big data challenge associated with D 2 R.

Aman, Saima↗
DOE OSTI2025

Predictions for the Detectability of Milky Way Satellite Galaxies and Outer-Halo Star Clusters with the Vera C. Rubin Observatory

We predict the sensitivity of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) to faint, resolved Milky Way satellite galaxies and outer-halo star clusters. We characterize the expected sensitivity using simulated LSST data from the LSST Dark Energy Science Collaboration (DESC) Data Challenge 2 (DC2) accessed and analyzed with the Rubin Science Platform as part of the Rubin Early Science Program. We simulate resolved stellar populations of Milky Way satellite galaxies and outer-halo star clusters over a wide range of sizes, luminosities, and heliocentric distances, which are broadly consistent with expectations for the Milky Way satellite system. We inject simulated stars into the DC2 catalog with realistic photometric uncertainties and star/galaxy separation derived from the DC2 data itself. We assess the probability that each simulated system would be detected by LSST using a conventional isochrone matched-filter technique. We find that assuming perfect star/galaxy separation enables the detection of resolved stellar systems with $M_V$ = 0 mag and $r_{1/2}$ = 10 pc with >50% efficiency out to a heliocentric distance of ~250 kpc. Similar detection efficiency is possible with a simple star/galaxy separation criterion based on measured quantities, although the false positive rate is higher due to leakage of background galaxies into the stellar sample. When assuming perfect star/galaxy classification and a model for the galaxy-halo connection fit to current data, we predict that 89 +/- 20 Milky Way satellite galaxies will be detectable with a simple matched-filter algorithm applied to the LSST wide-fast-deep data set. Different assumptions about the performance of star/galaxy classification efficiency can decrease this estimate by ~7%-25%, which emphasizes the importance of high-quality star/galaxy separation for studies of the Milky Way satellite population with LSST.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2016

REPEATABILITY OF SPITZER/IRAC EXOPLANETARY ECLIPSES WITH INDEPENDENT COMPONENT ANALYSIS

The research of effective and reliable detrending methods for Spitzer data is of paramount importance for the characterization of exoplanetary atmospheres. To date, the totality of exoplanetary observations in the mid- and far-infrared, at wavelengths >3 μm, have been taken with Spitzer. In some cases, in past years, repeated observations and multiple reanalyses of the same data sets led to discrepant results, raising questions about the accuracy and reproducibility of such measurements. Morello et al. (2014, 2015) proposed a blind-source separation method based on the Independent Component Analysis of pixel time series (pixel-ICA) to analyze InfraRed Array Camera (IRAC) data, obtaining coherent results when applied to repeated transit observations previously debated in the literature. Here we introduce a variant to the pixel-ICA through the use of wavelet transform, wavelet pixel-ICA, which extends its applicability to low-signal-to-noise-ratio cases. We describe the method and discuss the results obtained over 12 eclipses of the exoplanet XO3b observed during the “Warm Spitzer” era in the 4.5 μm band. The final results are reported, in part, also in Ingalls et al. (2016), together with results obtained with other detrending methods, and over 10 synthetic eclipses that were analyzed for the “IRAC Data Challenge 2015.” Our results are consistent within 1σ with the ones reported in Wong et al. (2014) and with most of the results reported in Ingalls et al. (2016), which appeared on arXiv while this paper was under review. Based on many statistical tests discussed in Ingalls et al. (2016), the wavelet pixel-ICA method performs as well as or better than other state-of-art methods recently developed by other teams to analyze Spitzer/IRAC data, and, in particular, it appears to be the most repeatable and the most reliable, while reaching the photon noise limit, at least for the particular data set analyzed. Another strength of the ICA approach is its highest objectivity, as it does not use prior information about the instrument systematics, making it a promising method to analyze data from other observatories. The self-consistency of individual measurements of eclipse depth and phase curve slope over a span of more than three years proves the stability of Warm Spitzer/IRAC photometry within the error bars, at the level of 1 part in 10{sup 4} in stellar flux.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Open Radiation Monitoring: Conceptual System Design

The Open Radiation Monitoring (ORM) Project seeks to develop and demonstrate a modular radiation detection architecture designed specifically for use in arms control treaty verification (ACTV) applications that will facilitate rapid development of trusted systems to meet the needs of potential future treaties. Development of trusted systems to support potential future treaties is a complex and costly endeavor that typically results in a purpose-built system designed to perform one specific task. The majority of prior trusted system development efforts have relied on the use of commercial embedded computers or microprocessors to control the system and process the acquired data. These processors are complex, making authentication and certification of measurement systems and collected data challenging and time consuming. We believe that a modular architecture can be used to reduce more complex systems to a series of single-purpose building blocks that could be used to implement a variety of detection modalities with shared functionalities. With proper design, the functionality of individual modules can be confirmed through simple input/output testing, thereby facilitating equipment inspection and in turn building trust in the equipment by all treaty parties. Furthermore, a modular architecture can be used to control data flow within the measurement system, reducing the risk of "hidden switches" and constraining the amount of sensitive information that could potentially be inadvertently leaked. This report documents a conceptual modular system architecture that is designed to facilitate inspection in an effort to reduce overall authentication and certification burden. As of publication, this architecture remains in a conceptual phase and additional funding is required to prove out the utility of a modular architecture and test the assumptions used to rationalize the design.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2017

Anomalous Galactic Cosmic Rays in the Framework of AMS-02

The cosmic-ray (CR) energy spectra of protons and helium nuclei, which are the most abundant components of cosmic radiation, exhibit a remarkable hardening at energies above 100 GeV/nucleon. Recent data from AMS-02 confirm this feature with a higher significance. These data challenge the current models of CR acceleration in Galactic sources and propagation in the Galaxy. Here, we explain the observed break in the spectra of protons and helium nuclei in light of recent advances in CR diffusion theories in turbulent astrophysical sources as being a result of a transition between different CR diffusion regimes. We reconstruct the observed CR spectra using the fact that a transition from normal diffusion to superdiffusion changes the efficiency of particle acceleration and causes the change in the spectral index. We find that calculated proton and helium spectra match the data very well.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

The LSST DESC DC2 Simulated Sky Survey

Here, we describe the simulated sky survey underlying the second data challenge (DC2) carried out in preparation for analysis of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) by the LSST Dark Energy Science Collaboration (LSST DESC). Significant connections across multiple science domains will be a hallmark of LSST; the DC2 program represents a unique modeling effort that stresses this interconnectivity in a way that has not been attempted before. This effort encompasses a full end-to-end approach: starting from a large N-body simulation, through setting up LSST-like observations including realistic cadences, through image simulations, and finally processing with Rubin's LSST Science Pipelines. This last step ensures that we generate data products resembling those to be delivered by the Rubin Observatory as closely as is currently possible. The simulated DC2 sky survey covers six optical bands in a wide-fast-deep area of approximately 300 deg 2 , as well as a deep drilling field of approximately 1 deg 2 . We simulate 5 yr of the planned 10 yr survey. The DC2 sky survey has multiple purposes. First, the LSST DESC working groups can use the data set to develop a range of DESC analysis pipelines to prepare for the advent of actual data. Second, it serves as a realistic test bed for the image processing software under development for LSST by the Rubin Observatory. In particular, simulated data provide a controlled way to investigate certain image-level systematic effects. Finally, the DC2 sky survey enables the exploration of new scientific ideas in both static and time domain cosmology.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2019

CosmoDC2: A Synthetic Sky Catalog for Dark Energy Science with LSST

This paper introduces cosmoDC2, a large synthetic galaxy catalog designed to support precision dark energy science with the Large Synoptic Survey Telescope (LSST). CosmoDC2 is the starting point for the second data challenge (DC2) carried out by the LSST Dark Energy Science Collaboration (LSST DESC). The catalog is based on a trillion-particle, (4.225 Gpc) 3 box cosmological N-body simulation, the Outer Rim run. It covers 440 deg 2 of sky area to a redshift of z = 3 and matches expected number densities from contemporary surveys to a magnitude depth of 28 in the r band. Each galaxy is characterized by a multitude of galaxy properties including stellar mass, morphology, spectral energy distributions, broadband filter magnitudes, host halo information, and weak lensing shear. The size and complexity of cosmoDC2 requires an efficient catalog generation methodology; our approach is based on a new hybrid technique that combines data-based empirical approaches with semianalytic galaxy modeling. A wide range of observation-based validation tests has been implemented to ensure that cosmoDC2 enables the science goals of the planned LSST DESC DC2 analyses. This paper also represents the official release of the cosmoDC2 data set, including an efficient reader that facilitates interaction with the data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Deep Learning Scene Classification Experiments in Automatic Detection of Slums on Planetscope Imagery

Population growth is increasingly happening in slum settlements of the large urban centers in the Global South. The term "slum" encompasses a wide range of communities, located mostly in underserved areas, and often exhibiting distinct structural and functional informalities with a relatively high concentration of marginalized populations. To address the issues confronting slums for effective planning and development, including the realistic estimation of the resident population, identifying them accurately is fundamental. Given the disagreements over a universal definition, diverse characteristic features, and socio-political limitations, global detection of slums is a veritable challenge. In this paper, we present experiments in slum detection using a scene classification algorithm and 3-meter spatial resolution satellite imagery. We train and evaluate the model for slum detection in Mumbai, India for the year 2023 and test the temporal generalization of the trained model on Mumbai in 2020 and 2018. In addition, we explore the pathways toward geographic generalization to Kolkata and Delhi (India). We discuss several limitations in the workflow and model, situate our findings in the existing literature, and suggest improvements and alternatives. With this, we establish baseline methods and experiments as a first step towards developing an image-based global slum detection framework and algorithm. This work adds to the community discussion on methods, data challenges, and open questions related to the detection of slums globally. With this research, we hope to improve our understanding of human settlements, especially in critical areas, improve population estimates, and help measure progress towards the sustainable development goals.

Arndt, Jacob↗