Search NASA⌕ Search

SEARCH · Search NASA

Results for “data challenges”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2009

The Status of the Mock LISA Data Challenges

For the last four years, many gravitational-wave researchers around the world have participated in the Mock LISA Data Challenges (MLDCs), a program to demonstrate and encourage the development of LISA data-analysis capabilities, tools and techniques. In this poster, we present a summary of the results of MLDC 3, which was completed in 2009. During MLDC 3, 27 participants from 15 institutions successfully analyzed data sets that included Galactic binaries, coalescing spinning massive black holes, extreme-mass-ratio inspirals, cosmic-string cusp bursts and a stochastic gravitational-wave background. We also describe the technical and scientific challenges that will be addressed by future MLI)Cs, starting with MLDC 4, which is currently in progress.

Baker, John↗
NASA NTRS2021

GLOBE Observer and the GO on a Trail Data Challenge: A Citizen Science Approach to Generating a Global Land Cover Land Use Reference Dataset

Land cover and land use are highly visible indicators of climate change and human disruption to natural processes. While land cover is frequently monitored over a large area using satellite data, ground-based reference data is valuable as a comparison point. The NASA-funded GLOBE Observer (GO) program provides volunteer-collected land cover photos tagged with location, date and time, and, in some cases, land cover type. When making a full land cover observation, volunteers take six photos of the site, one facing north, south, east, and west (N-S-E-W) respectively, one pointing straight up to capture canopy and sky, and one pointing down to document ground cover. Together, the photos document a 100 meter square of land. Volunteers may then optionally tag each N-S-E-W photo with the land cover types present. Volunteers collect the data through a smartphone app, also called GLOBE Observer, resulting in consistent data. While land cover data collected through GLOBE Observer is ongoing, this paper presents the results of a data challenge held between June 1 and October 15, 2019. Called “GO on a Trail,” the challenge focused on collecting land cover observations at designated points along the Lewis and Clark National Historic Trail (LCNHT), which transects approximately 7,900 kilometers of the United States from east to west. The challenge also included an international component in which participants were recognized for the most data collected during the period. The challenge resulted in more than 2800 land cover data points from around the world with just over 900 land cover data points along the LCNHT. The data collected along the LCNHT is largely within a 1,000 meter corridor along the historic route. Internationally, the challenge had strong participation in Australia, where a Scouts Australia team-based competition was conducted in conjunction with GO on a Trail. GLOBE Observer collections can serve as reference data, ground truthing satellite imagery for the improvement and verification of broad land cover maps. Continued collection using this protocol will build a database documenting climate-related land cover and land use change into the future.

land cover↗
DOE OSTI2024

MSD CoP Webinar: AI and Extreme Events - Overcoming Data Challenges for Improved Characterization of Climate Extremes

Context: This webinar was hosted by the MultiSector Dynamics Community of Practice (MSD CoP; https://multisectordynamics.org). Abstract: Artificial Intelligence (AI) models require large volumes of data for training and testing. Data requirements present challenges for using AI to explore extreme events with limited observational data. This webinar will showcase two innovative methods developed by part of the European Climate Intelligence (CLINT) project to overcome data challenges and harness AI to improve our understanding of climate extremes. Dr. Ascenso will present his research on data augmentation methods to improve estimates of tropical cyclones using satellite data. His presentation will review established methods for data augmentation and explore opportunities and challenges for using generative AI to generate images of extreme, life-threatening tropical cyclones. Next, Dr. Plesiat will present his research on deep learning techniques to overcome limited observational data sets. His presentation will illustrate deep learning methods to develop AI reconstructions of four climate indices across Europe. Presenters : Dr. Guido Ascenso (post-doctoral researcher, Politecnico di Milano); Dr. Étienne Plésiat (German Climate Computing Centre - DKRZ) Moderator(s): Stefano Galelli (MSD CoP WG Co-Lead), David Gold (MSD CoP WG Co-Lead), Jillian Sturtevant (MSD CoP WG Communications Officer), Matteo Giuliani (Politecnico di Milano, MSD CoP WG Member, Moderator and Organizer) This webinar was held on: October 11, 2024 from 11AM - 1PM ET

AI↗
NASA NTRS2015

High Resolution Nature Runs and the Big Data Challenge

NASA's Global Modeling and Assimilation Office at Goddard Space Flight Center is undertaking a series of very computationally intensive Nature Runs and a downscaled reanalysis. The nature runs use the GEOS-5 as an Atmospheric General Circulation Model (AGCM) while the reanalysis uses the GEOS-5 in Data Assimilation mode. This paper will present computational challenges from three runs, two of which are AGCM and one is downscaled reanalysis using the full DAS. The nature runs will be completed at two surface grid resolutions, 7 and 3 kilometers and 72 vertical levels. The 7 km run spanned 2 years (2005-2006) and produced 4 PB of data while the 3 km run will span one year and generate 4 BP of data. The downscaled reanalysis (MERRA-II Modern-Era Reanalysis for Research and Applications) will cover 15 years and generate 1 PB of data. Our efforts to address the big data challenges of climate science, we are moving toward a notion of Climate Analytics-as-a-Service (CAaaS), a specialization of the concept of business process-as-a-service that is an evolving extension of IaaS, PaaS, and SaaS enabled by cloud computing. In this presentation, we will describe two projects that demonstrate this shift. MERRA Analytic Services (MERRA/AS) is an example of cloud-enabled CAaaS. MERRA/AS enables MapReduce analytics over MERRA reanalysis data collection by bringing together the high-performance computing, scalable data management, and a domain-specific climate data services API. NASA's High-Performance Science Cloud (HPSC) is an example of the type of compute-storage fabric required to support CAaaS. The HPSC comprises a high speed Infinib and network, high performance file systems and object storage, and a virtual system environments specific for data intensive, science applications. These technologies are providing a new tier in the data and analytic services stack that helps connect earthbound, enterprise-level data and computational resources to new customers and new mobility-driven applications and modes of work. In our experience, CAaaS lowers the barriers and risk to organizational change, fosters innovation and experimentation, and provides the agility required to meet our customers' increasing and changing needs

big data analysis↗
NASA NTRS2012

Big Data Challenges for Large Radio Arrays

Future large radio astronomy arrays, particularly the Square Kilometre Array (SKA), will be able to generate data at rates far higher than can be analyzed or stored affordably with current practices. This is, by definition, a "big data" problem, and requires an end-to-end solution if future radio arrays are to reach their full scientific potential. Similar data processing, transport, storage, and management challenges face next-generation facilities in many other fields.

Combining↗
NASA NTRS2020

Clouds Around the World: How a Simple Citizen Science Data Challenge Became a Worldwide Success

Citizen science is often recognized for its potential to directly engage the public in science, and is uniquely positioned to support and extend participants’ learning in science. In March 2018, the Global Learning and Observations to Benefit the Environment (GLOBE) Program, NASA’s largest and longest-lasting citizen science program about Earth, organized a month-long event that asked people around the world to contribute daily cloud observations and photographs of the sky (15 March–15 April 2018). What was considered a simple engagement activity turned into an unprecedented worldwide event that garnered major public interest and media recognition, collecting over 55,000 observations from 99 different countries, in more than 15,000 locations, on every continent including Antarctica. The event was called the “Spring Cloud Challenge” and was created to 1) engage the general public in the scientific process and promote the use of the GLOBE Observer app, 2) collect ground-based visual observations of varying cloud types during boreal spring, and 3) increase the number and locations of ground-based visual cloud observations collocated with cloud-observing satellites. The event resulted in roughly 3 times more observations than during the historic and highly publicized 2017 North American total solar eclipse. The dataset also includes observations over the Drake Passage in Antarctica and reports from intense Saharan dust events. This article describes how the challenge was crafted, outreach to volunteer scientists around the world, details of the data collected, and impact of the data.

GLOBE↗
DOE OSTI2020

VULCAN temperature dataset for SMC data challenge

The VULCAN Beamline dataset provides the sample measurement, where temperatures is recorded in two physically different places on the sample. These are held in two different hdf5 groups in the data file.

42 ENGINEERING↗
DOE OSTI2021

2021 Smoky Mountains Conference Data Challenge Synthetic-to-Real Domain Adaptation for Autonomous Driving Dataset

The dataset is comprised of both real and synthetic images from a vehicle's forward-facing camera. Each camera image is accompanied by a corresponding pixel-level semantic segmentation image (all files are .png files). In total, the dataset contains 5600 images in the training/validation set and 1400 images in the testing set. The training dataset contains mostly synthetic RGB images collected with a wide range of weather and lighting conditions using the CARLA simulator [1]. In addition, the training data also includes a small pre-selected subset of data from the Cityscapes training dataset – which is comprised of RGB-segmentation image pairs from driving scenarios in various European cities [2]. The testing data is split into three sets. The first set contains synthetic CARLA images with weather/lighting conditions that were not present in the training set. The second set is a subset of the Cityscapes testing dataset. Finally, the third set is an unknown testing set which will not be revealed to the participants until after the submission deadline. [1] Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, October). CARLA: An open urban driving simulator. In Conference on robot learning (pp. 1-16). PMLR. [2] Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., ... and Schiele, B. (2016). The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3213-3223).

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2021

Deep glassy state dynamic data challenge glass models: Elastic models

The idea of an “ideal” glass transition temperature has persisted at least since the work from Kauzmann when it was observed that the entropy of glass-forming liquids extrapolated to below that of the crystal, thus suggesting the need for a phase transition at a finite or non-zero absolute temperature. This thermodynamic paradox was also found to be related to the observations of a diverging of the extrapolated viscosity or relaxation times at a temperature near to this ideal glass transition. Recently, however, we have carried out experiments using both an ancient amber material and an ultra-stable amorphous fluoropolymer that challenge the ideas of the divergence of the viscosity or relaxation times at this ideal glass transition. In the present manuscript we have evaluated two theories of the glass transition that are based on ideas related to elasticity of the amorphous glass-forming material. We find that the models from both J. Dyre and his group and of K.S. Schweizer and his group not only show non-diverging behavior but are also in some agreement with the new data, though still showing somewhat slower dynamics than those observed in the experiments. The work shows that the data are good enough to distinguish between theories and it is suggested that other mechanisms may be needed to fully describe the non-diverging responses of the ultra-stable glasses.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Data Challenges for Structural Health Monitoring of Electrical Machines [Slides]

Induction motors (ID) consist of a stator and a rotor. Induction motor slip decreases with decreasing load. Induction motor slip decreases with decreasing load. This talk will focus on removing closely coupled power grid frequencies in an effort to observe the true dynamic response signature of induction motors. Using the time difference between the zeros, we found the frequency as it changes over time for each of the 3 phases. The frequencies of the 3 phases were averaged to reduce the effects of sidebands on the instantaneous frequency. The frequencies of the 3 phases were averaged to reduce the effects of sidebands on the instantaneous frequency. The Kalman filter rapidly synchronizes with the expected output. The sidebands were recreated using spectral subtraction. The frequency domain shows the reduction in magnitude of the power grid frequency the Kalman output was subtracted out. Kalman filtering also reveals important data about the sidebands in healthy motors. The Continuous Wavelet Transform is an alternative to a Bandpass IIR filter used in the analysis of the acceleration data. Analysis of the acceleration data reveals 4 sidebands around the slip frequency.

42 ENGINEERING↗
DOE OSTI2021

SMC 2021 Data Challenge: Analyzing Resource Utilization and User Behavior on Titan Supercomputer

Resource utilization statistics of submitted jobs on a supercomputer can help us understand how users from various scientific domains use HPC platforms and better design a job scheduler. We explore to generate insight regarding workload distribution and usage pattern domains from job scheduler trace, GPU failure information, and project-specific information collected from Titan supercomputer. Furthermore, we want to know how the scheduler performance varies over time and how the users' scheduling behavior changes following a system failure. These observations have the potential to provide valuable insight, which is helpful to prepare for system failures. These practices will help us develop and apply novel machine learning algorithms in understanding system behavior, requirement, and better scheduling of HPC systems. There are two datasets, RUR and GPU: RUR dataset is the job scheduler traces collected from the Titan supercomputer from 01/01/2015 to 07/31/2019 (2015.csv - 2019.csv). These were collected using resource Utilization Report (RUR), a Cray-developed resource-usage data collection and reporting system. It contains the usage information of its critical resources (CPU, Memory, GPU, and I/O) of each running job on Titan during that period (https://ieeexplore.ieee.org/abstract/document/8891001). It includes ProjectAreas as additional information, every job is associated with a project ID. TheProjectAreas.csv dataset provides a mapping of the project ID to its domain science. GPU dataset has information regarding GPU failure on Titan. There have been some hardware-related issues in the GPUs in Titan that caused some GPUs to fail, sometimes irrecoverably during some job runs. This dataset provides information regarding these failures during the execution of the submitted jobs. GPUs on Titan are uniquely identified by a serial number (SN), and they are installed in a location. A GPU can be installed in a location, then removed from that location following a failure, and then re-installed in a different location after fixing the problem. If the failure can't be recovered, the GPU might be removed entirely from Titan. There are two prominent types of failures that resulted in the removal of GPUs from Titan: Double Bit Error (DBE) and Out of the Bus (OTB). The dataset (gc_full.csv) has seven attributes, we provided a short description of these attributes in the ReadMe file. To learn more about this dataset, please refer to the git repository https://github.com/olcf/TitanGPULife and the related publication (https://ieeexplore.ieee.org/abstract/document/9355319).

42 ENGINEERING↗
DOE OSTI2021

Deep glassy state dynamic data challenge glass models: Configurational entropy models

This work was the result of the student's (Dongjie Chen) qualifying exam response. In it the experimental observation from our labs that the relaxation times of ultrastable glasses (a 20 million year old amber and a vapor deposited amorphous teflon) do not follow the expected Vogel-Fulcher or WLF types of divergence as the temperatures fall below the glass transition temperature. In the work we evaluated several theories by considering them in different ways. We used them as generally used and show that they do not capture the non-diverging behavior. Because the models were all entropy-based models, we also considered the possibility that the temperature dependence of the entropy used in the models is incorrect and chose two ways of estimating the entropy that do not follow the classical theoretical models. First we used the actually measured calorimetric response for the two ultrastable glasses, and we used the Milchev model, which does not go to zero above 0 Kelvin. Both of these approaches led to improved agreement between the theories and the experimental data. Finally, we also asked what the temperature dependence of the entropy would have to be so that the models agreed with the observed data. In some cases the entropy so-calculated was reasonable, while in others it was unphysical. In sum, the important point of the work is that the observation of non-diverging time-scales that suggest that there is no ideal glass transition can be reasonably captured by current theories if they take more realistic models of the entropy than is usually the case.

36 MATERIALS SCIENCE↗
DOE OSTI2022

The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider

We describe the outcome of a data challenge conducted as part of the Dark Machines (https://www.darkmachines.org) initiative and the Les Houches 2019 workshop on Physics at TeV colliders. The challenged aims to detect signals of new physics at the Large Hadron Collider (LHC) using unsupervised machine learning algorithms. First, we propose how an anomaly score could be implemented to define model-independent signal regions in LHC searches. We define and describe a large benchmark dataset, consisting of >1 billion simulated LHC events corresponding to 10\, fb^{-1} 10 f b − 1 of proton-proton collisions at a center-of-mass energy of 13 TeV. We then review a wide range of anomaly detection and density estimation algorithms, developed in the context of the data challenge, and we measure their performance in a set of realistic analysis environments. We draw a number of useful conclusions that will aid the development of unsupervised new physics searches during the third run of the LHC, and provide our benchmark dataset for future studies at https://www.phenoMLdata.org. Code to reproduce the analysis is provided at https://github.com/bostdiek/DarkMachines-UnsupervisedChallenge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Neutron Imaging dataset for SMC 2021 data challenge

The neutron radiography (nR) dataset provides information of the neutron events as measured for the Siemens star mask using the Timepix3 detector. The recordings consist of the position (x and y axes), the time-stamp, and time-over-threshold (TOT) values of each neutron event.

Peterson, Peter↗