Search NASA⌕ Search

SEARCH · Search NASA

Results for “Crowdsourced Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Machine learning model inputs, outputs, and scripts associated with “Artificial intelligence-guided iterations between observations and modeling significantly improve environmental predictions”

NOTE: The manuscript associated with this data package is currently in review. The data may be revised based on reviewer feedback. Upon manuscript acceptance, this data package will be updated with the final dataset and additional metadata. This data package is associated with the manuscript “Artificial intelligence-guided iterations between observations and modeling significantly improve environmental predictions” (Malhotra et al., in prep). This effort was designed following ICON (integrated, coordinated, open, and networked) principles to facilitate a model-experiment (ModEx) iteration approach, leveraging crowdsourced sampling across the contiguous United States (CONUS). New machine learning models were created every month to guide sampling locations. Data from the resulting samples were used to test and rebuild the machine learning models for the next round of sampling guidance. Associated sediment and water geochemistry and in situ sensor data can be found at https://data.ess-dive.lbl.gov/datasets/doi:10.15485/1923689, https://data.ess-dive.lbl.gov/datasets/doi:10.15485/1729719, and https://data.ess-dive.lbl.gov/datasets/doi:10.15485/1603775. This data package is associated with two GitHub repositories found at https://github.com/parallelworks/dynamic-learning-rivers and https://github.com/WHONDRS-Hub/ICON-ModEx_Open_Manuscript. In addition to this readme, this data package also includes two file-level metadata (FLMD) files that describes each file and two data dictionaries (DD) that describe all column/row headers and variable definitions. This data package consists of two main folders (1) dynamic-learning-rivers and (2) ICON-ModEx_Open_Manuscript which contain snapshots of the associated GitHub repositories. The input data, output data, and machine learning models used to guide sampling locations are within dynamic-learning-rivers. The folder is organized into five top-level directories: (1) “input_data” holds the training data for the ML models; (2) “ml_models” holds machine learning (ML) models trained on the data in “input_data”; (3) “examples” contains files for direct experimentation with the machine learning model, including scripts for setting up “hindcast” run; (4) “scripts” contains data preprocessing and postprocessing scripts and intermediate results specific to this data set that bookend the ML workflow; and (5) “output_data” holds the overall results of the ML model on that branch. Each trained ML model resides on its own branch in the repository; this means that inputs and outputs can be different branch-to-branch. There is also one hidden directory “.github/workflows”. This hidden directory contains information for how to run the ML workflow as an end-to-end automated GitHub Action but it is not needed for reusing the ML models archived here. Please see the top-level README.md in the GitHub repository for more details on the automation. The scripts and data used to create figures in the manuscript are within ICON-ModEx_Open_Manuscript. The folder is organized into four folders which contain the scripts, data, and pdf for each figure. Within the “fig-model-score-evolution” folder, there is a folder called “intermediate_branch_data” which contains some intermediate files pulled from dynamic-learning-rivers and reorganized to easily integrate into the workflows. NOTE: THIS FOLDER INCLUDES THE FILES AT THE POINT OF PAPER SUBMISSION. IT WILL BE UPDATED ONCE THE PAPER IS ACCEPTED WITH ANY REVISIONS AND WILL INCLUDE A DD/FLMD AT THAT POINT. We thank the United States Forest Service, Washington Department of Fish and Wildlife, Washington Department of Natural Resources, Cowiche Canyon Conservatory, Washington State Parks and Recreation Commission (Scientific Research Permit #210901), and the Confederated Tribes and Bands of the Yakama Nation for access to field locations where the samples labeled “SSS” were collected. We also thank the Yakama Nation Tribal Council and Yakama Nation Fisheries for working with us to facilitate sample collection and optimization of data usage according to their values and worldview. WHONDRS consortium members were asked to provide any acknowledgments for the collection of samples labeled “CM” and the following is a list of acknowledgments that were submitted with their corresponding Site IDs: (MART) Research activities were conducted in part on the Wind River Experimental Forest within the Gifford Pinchot National Forest; (MP- 100379) Philadelphia is part of Lenapehoking, the ancestral homelands of the Lenape peoples; (MP-102398) Land surveyed is the ancestral homelands of the Nookhose'iinenno (Arapaho), Tsis tsis'tas (Cheyenne), and Nuuchu (Ute); (MP-100749 and MP- 100747) Georgia Coastal Ecosystem LTER, OCE-1832178; (SP-70 and SP-72) Eastern Shoshone, Shoshone-Bannock; (MP- 102944) Funded by Oregon Watershed Enhancement Board. On the traditional lands of the Confederated Tribes of the Siletz, Confederated Tribes of the Grand Rhonde, and the Clatsop-Nehalem Confederated Tribe; (MP- 100607) Holiday Creek is located on the traditional territory of the Monacan Indian Nation; (SP-45) Lafayette Blue Springs State Park; (MP-102420) NSF DEB-2016749; (MP-100019) New Hampshire Agriculture Experiment Station; (SP-35) Rayonier (land owner; https://www.rayonier.com/); (MP- 101276) US Department of Energy, Office of Science, Biological and Environmental Research, Subsurface Biogeochemical Research, Watershed Dynamics and Evolution SFA at ORNL; (MP- 103224) Watershed Dynamics and Evolution SFA at ORNL; (MP- 101584) Traditional lands of the Oceti Sakowin (Dakota, Lakota, Nakoda) and Anishinaabe Peoples.

54 ENVIRONMENTAL SCIENCES↗

Camera settings and biome influence the accuracy of citizen science approaches to camera trap image classification

Scientists are increasingly using volunteer efforts of citizen scientists to classify images captured by motion-activated trail cameras. The rising popularity of citizen science reflects its potential to engage the public in conservation science and accelerate processing of the large volume of images generated by trail cameras. While image classification accuracy by citizen scientists can vary across species, the influence of other factors on accuracy is poorly understood. Inaccuracy diminishes the value of citizen science derived data and prompts the need for specific best-practice protocols to decrease error. We compare the accuracy between three programs that use crowdsourced citizen scientists to process images online: Snapshot Serengeti, Wildwatch Kenya, and AmazonCam Tambopata. We hypothesized that habitat type and camera settings would influence accuracy. To evaluate these factors, each photograph was circulated to multiple volunteers. All volunteer classifications were aggregated to a single best answer for each photograph using a plurality algorithm. Subsequently, a subset of these images underwent expert review and were compared to the citizen scientist results. Classification errors were categorized by the nature of the error (e.g., false species or false empty), and reason for the false classification (e.g., misidentification). Our results show that Snapshot Serengeti had the highest accuracy (97.9%), followed by AmazonCam Tambopata (93.5%), then Wildwatch Kenya (83.4%). Error type was influenced by habitat, with false empty images more prevalent in open-grassy habitat (27%) compared to woodlands (10%). For medium to large animal surveys across all habitat types, our results suggest that to significantly improve accuracy in crowdsourced projects, researchers should use a trail camera set up protocol with a burst of three consecutive photographs, a short field of view, and determine camera sensitivity settings based on in situ testing. Accuracy level comparisons such as this study can improve reliability of future citizen science projects, and subsequently encourage the increased use of such data.

54 ENVIRONMENTAL SCIENCES↗

Decision Making Under Uncertainty Human Subjects Data - Fire Evacuation Task

This dataset contains de-identified data from human subjects experiments, along with the images and code that were used to run the experiments (as a crowdsourced online study). In this study, participants were shown the probability of a house being in the burn zone of a wildfire. They were asked if they would stay in the house or evacuate in that scenario. The probability information was presented in different ways, including text and maps. The studies tested the impact of different visual cues on the participants' patterns of decisions.

Matzen, Laura E. [Sandia National Laboratories (SN↗

The Case for the Use of Active Social Media in Nonproliferation and Nuclear Security

A great amount of attention and consideration is being directed at possible applications of social media in many challenging areas. The use of social media has already shown its importance in the area of disaster response, where, each citizen is essentially acting as a sensor in reporting local conditions. In the aggregate, valuable information is obtained to enable a more effective response as well as provide timely information to those in the disaster area. No one needs to be trained to understand what constitutes a disaster, so a social media data stream from the public is literally always active and ready to engage. A similar but more focused approach is the use of crowdsourcing for science, where specific challenges in areas such as mathematics, astronomy, and biology are posted to social media and solved by the crowd.

Schanfein, Mark J.↗

Assessing the Reliability of Relevant Tweets and Validation Using Manual and Automatic Approaches for Flood Risk Communication

While Twitter has been touted as a preeminent source of up-to-date information on hazard events, the reliability of tweets is still a concern. Our previous publication extracted relevant tweets containing information about the 2013 Colorado flood event and its impacts. Using the relevant tweets, this research further examined the reliability (accuracy and trueness) of the tweets by examining the text and image content and comparing them to other publicly available data sources. Both manual identification of text information and automated (Google Cloud Vision, application programming interface (API)) extraction of images were implemented to balance accurate information verification and efficient processing time. The results showed that both the text and images contained useful information about damaged/flooded roads/streets. This information will help emergency response coordination efforts and informed allocation of resources when enough tweets contain geocoordinates or location/venue names. This research will identify reliable crowdsourced risk information to facilitate near real-time emergency response through better use of crowdsourced risk communication platforms.

54 ENVIRONMENTAL SCIENCES↗

Galaxy Zoo: 3D – crowdsourced bar, spiral, and foreground star masks for MaNGA target galaxies

ABSTRACT The challenge of consistent identification of internal structure in galaxies – in particular disc galaxy components like spiral arms, bars, and bulges – has hindered our ability to study the physical impact of such structure across large samples. In this paper we present Galaxy Zoo: 3D (GZ:3D) a crowdsourcing project built on the Zooniverse platform that we used to create spatial pixel (spaxel) maps that identify galaxy centres, foreground stars, galactic bars, and spiral arms for 29 831 galaxies that were potential targets of the MaNGA survey (Mapping Nearby Galaxies at Apache Point Observatory, part of the fourth phase of the Sloan Digital Sky Surveys or SDSS-IV), including nearly all of the 10 010 galaxies ultimately observed. Our crowdsourced visual identification of asymmetric internal structures provides valuable insight on the evolutionary role of non-axisymmetric processes that is otherwise lost when MaNGA data cubes are azimuthally averaged. We present the publicly available GZ:3D catalogue alongside validation tests and example use cases. These data may in the future provide a useful training set for automated identification of spiral arm features. As an illustration, we use the spiral masks in a sample of 825 galaxies to measure the enhancement of star formation spatially linked to spiral arms, which we measure to be a factor of three over the background disc, and how this enhancement increases with radius.

Masters, Karen L. (ORCID:0000000308469578)↗

Mining Twitter Data to Augment NASA GPM Validation

The Twitter data stream is an important new source of real-time and historical global information for potentially augmenting the validation program of NASA's Global Precipitation Measurement (GPM) mission. There have been other similar uses of Twitter, though mostly related to natural hazards monitoring and management. The validation of satellite precipitation estimates is challenging, because many regions lack data or access to data, especially outside of the U.S. and in remote and developing areas. The time-varying set of "precipitation" tweets can be thought of as an organic network of rain gauges, potentially providing a widespread view of precipitation occurrence. Twitter provides a large source of crowd for crowdsourcing. During a 24-hour period in the middle of the snow storm this past March in the U.S. Northeast, we collected more than 13,000 relevant precipitation tweets with exact geolocation. The overall objective of our project is to determine the extent to which processed tweets can provide additional information that improves the validation of GPM data. Though our current effort focuses on tweets and precipitation, our approach is general and applicable to other social media and other geophysical measurements. Specifically, we have developed an operational infrastructure for processing tweets, in a format suitable for analysis with GPM data; engaged with potential participants, both passive and active, to "enrich" the Twitter stream; and inter-compared "precipitation" tweet data, ground station data, and GPM retrievals. In this presentation, we detail the technical capabilities of our tweet processing infrastructure, including data abstraction, feature extraction, search engine, context-awareness, real-time processing, and high volume (big) data processing; various means for "enriching" the Twitter stream; and results of inter-comparisons. Our project should bring a new kind of visibility to Twitter and engender a new kind of appreciation of the value of Twitter by the science research communities.

validatio↗

Using Social Media and Mobile Devices to Discover and Share Disaster Data Products Derived From Satellites

Data products derived from Earth observing satellites are difficult to find and share without specialized software and often times a highly paid and specialized staff. For our research effort, we endeavored to prototype a distributed architecture that depends on a standardized communication protocol and applications program interface (API) that makes it easy for anyone to discover and access disaster related data. Providers can easily supply the public with their disaster related products by building an adapter for our API. Users can use the API to browse and find products that relate to the disaster at hand, without a centralized catalogue, for example floods, and then are able to share that data via social media. Furthermore, a longerterm goal for this architecture is to enable other users who see the shared disaster product to be able to generate the same product for other areas of interest via simple point and click actions on the API on their mobile device. Furthermore, the user will be able to edit the data with on the ground local observations and return the updated information to the original repository of this information if configured for this function. This architecture leverages SensorWeb functionality [1] presented at previous IGARSS conferences. The architecture is divided into two pieces, the frontend, which is the GeoSocial API, and the backend, which is a standardized disaster node that knows how to talk to other disaster nodes, and also can communicate with the GeoSocial API. The GeoSocial API, along with the disaster node basic functionality enables crowdsourcing and thus can leverage insitu observations by people external to a group to perform tasks such as improving water reference maps, which are maps of existing water before floods. This can lower the cost of generating precision water maps. Keywords-Data Discovery, Disaster Decision Support, Disaster Management, Interoperability, CEOS WGISS Disaster Architecture

Dust Mitigation Technology to Enable Survive the Night Capabilities

Introduction: As we return to the Moon, the lunar regolith (i.e. lunar dust) covering the surface will be an obstacle to nominal operations. Accounts from Apollo astronauts and analysis of hardware returned from the surface illustrate just how deleterious the dust can be [1]. During Apollo missions, the lunar dust adhered to hardware mechanically and electrostatically [2]. Surviving the Night: Mitigating the lunar dust will be critical to surviving the night. Going hand-in-hand with other extreme environment considerations, dust mitigation is critical to mission success. Dust Impacts on Other Systems: The lunar dust can have negative implications for power, thermal, mechanisms, and several other systems or sub-systems. For example, Apollo encountered marked degradation of performance in heat rejection systems for the lunar roving vehicle, science packages, and other components because of the lunar dust [1]. For power alone, dust can cause internal clogging for power connectors, heat rejection issues, excessive dust on reflective surfaces, reduced power output for solar arrays, and so on. Dust Mitigation Strategy: In addition to considering technology solutions, it is important for hardware, systems, and or components to have a dust mitigation strategy. At a high level, hardware that will encounter the lunar dust should consider these things when defining a dust mitigation strategy: • Understand Natural Environment • Understand Induced Environment • Understand Tolerance to Dust • Write Dust Requirements • Select Dust Mitigation Solutions • Test Hardware in Dusty Environment More information on each of these can be provided to hardware owners. Dust Mitigation Technology Development: NASA has a series of technologies that may be available for hardware that needs to survive the lunar night. Many of these solutions are leveraging dust mitigation technology development efforts from NASA’s Space Technology Mission Directorate (STMD), as well as efforts from ESDMD programs, industry, and academia. Through a series of STMD programs (both internal to NASA and through partnerships), there are several technologies in development as considerations as dust mitigation solutions for hardware. Within STMD, the Game Changing Development Program (GCD) has funded several internal dust mitigation projects including low to mid TRL development, demonstrations on CLPS landers of high TRL solutions, and creating standards and best practices for dust mitigation. STMD dust mitigation efforts also include a series of partnerships for developing technologies and advancing the state of dust mitigation at NASA. This includes the Lunar Surface Innovation Consortium (LSIC), Small Business Innovation Research, Early Stage Innovations (ESI), Space Technology Research Grants (STRG), Announcement of Collaboration Opportunities (ACOs) and Tipping Points (TPs), and Challenges and Crowdsourcing, among others. There are also a series of dust mitigation solutions that have been widely used terrestrially, or during Apollo. In recent years, several studies have produced more data on the efficacy of these potential solutions in the lunar environment. Dust Mitigation Solutions: Dust mitigation solutions generally fall into four categories: • Dust Tolerant Mechanisms • Passive Dust Mitigation Capabilities • Active Dust Mitigation Capabilities • Dust Measurement Capabilities There are a series of solutions that may prove beneficial for hardware that needs to survive the lunar night, including new technology development as well as proven, terrestrial solutions. This presentation will discuss in more detail what some of these solutions are for payloads going to the surface. References: [1] J. R. Gaier, NASA/TM—2005-213610, The Effects of Lunar Dust on EVA Systems During the Apollo Missions [2] T. J. Stubbs, et al. Impact of Dust on Lunar Exploration, 2005

dust mitigation↗

Survive the Dust: Dust Mitigation Technology to Enable Survive the Night Capabilities

Introduction: As we return to the Moon, the lunar regolith (i.e. lunar dust) covering the surface will be an obstacle to nominal operations. Accounts from Apollo astronauts and analysis of hardware returned from the surface illustrate just how deleterious the dust can be [1]. During Apollo missions, the lunar dust adhered to hardware mechanically and electrostatically [2]. Surviving the Night: Mitigating the lunar dust will be critical to surviving the night. Going hand-in-hand with other extreme environment considerations, dust mitigation is critical to mission success. Dust Impacts on Other Systems: The lunar dust can have negative implications for power, thermal, mechanisms, and several other systems or sub-systems. For example, Apollo encountered marked degradation of performance in heat rejection systems for the lunar roving vehicle, science packages, and other components because of the lunar dust [1]. For power alone, dust can cause internal clogging for power connectors, heat rejection issues, excessive dust on reflective surfaces, reduced power output for solar arrays, and so on. Dust Mitigation Strategy: In addition to considering technology solutions, it is important for hardware, systems, and or components to have a dust mitigation strategy. At a high level, hardware that will encounter the lunar dust should consider these things when defining a dust mitigation strategy: • Understand Natural Environment • Understand Induced Environment • Understand Tolerance to Dust • Write Dust Requirements • Select Dust Mitigation Solutions • Test Hardware in Dusty Environment More information on each of these can be provided to hardware owners. Dust Mitigation Technology Development: NASA has a series of technologies that may be available for hardware that needs to survive the lunar night. Many of these solutions are leveraging dust mitigation technology development efforts from NASA’s Space Technology Mission Directorate (STMD), as well as efforts from ESDMD programs, industry, and academia. Through a series of STMD programs (both internal to NASA and through partnerships), there are several technologies in development as considerations as dust mitigation solutions for hardware. Within STMD, the Game Changing Development Program (GCD) has funded several internal dust mitigation projects including low to mid TRL development, demonstrations on CLPS landers of high TRL solutions, and creating standards and best practices for dust mitigation. STMD dust mitigation efforts also include a series of partnerships for developing technologies and advancing the state of dust mitigation at NASA. This includes the Lunar Surface Innovation Consortium (LSIC), Small Business Innovation Research, Early Stage Innovations (ESI), Space Technology Research Grants (STRG), Announcement of Collaboration Opportunities (ACOs) and Tipping Points (TPs), and Challenges and Crowdsourcing, among others. There are also a series of dust mitigation solutions that have been widely used terrestrially, or during Apollo. In recent years, several studies have produced more data on the efficacy of these potential solutions in the lunar environment. Dust Mitigation Solutions: Dust mitigation solutions generally fall into four categories: • Dust Tolerant Mechanisms • Passive Dust Mitigation Capabilities • Active Dust Mitigation Capabilities • Dust Measurement Capabilities There are a series of solutions that may prove beneficial for hardware that needs to survive the lunar night, including new technology development as well as proven, terrestrial solutions. This presentation will discuss in more detail what some of these solutions are for payloads going to the surface. References: [1] J. R. Gaier, NASA/TM—2005-213610, The Effects of Lunar Dust on EVA Systems During the Apollo Missions [2] T. J. Stubbs, et al. Impact of Dust on Lunar Exploration, 2005

dust mitigation↗

Developing a System for Testing Computational Social Models using Amazon Mechanical Turk

The US faces persistent, distributed threats from malevolent individuals, groups and organizations around the world. Computational Social Models (CSMs) help anticipate the dynamics and behaviors of these actors by modeling the behavior and interactions of individuals, groups and organizations. For strategic planners to trust the results of CSMs, they must have confidence in the validity of the models. Establishing validity before model use will enhance confidence and reduce the risk of error. One problem with validation is designing an appropriate controlled test of the model, similar to the testing of physical models. Lab experiments can do this, but are often limited to small numbers of subjects, with low subject diversity and are often in a contrived environment. Natural studies attempt to test models by gathering large-scale observational data (e.g., social media) however this loses the controlled aspect. We propose a new approach to run large-scale, controlled online experiments on diverse populations. Using Amazon Mechanical Turk, a crowdsourcing tool, we will draw large populations into controlled experiments in a manner that was not possible just a few years ago.

42 ENGINEERING↗

Laboratory earthquake forecasting: A machine learning competition

Earthquake prediction, the long-sought holy grail of earthquake science, continues to confound Earth scientists. Could we make advances by crowdsourcing, drawing from the vast knowledge and creativity of the machine learning (ML) community? We used Google’s ML competition platform, Kaggle, to engage the worldwide ML community with a competition to develop and improve data analysis approaches on a forecasting problem that uses laboratory earthquake data. The competitors were tasked with predicting the time remaining before the next earthquake of successive laboratory quake events, based on only a small portion of the laboratory seismic data. The more than 4,500 participating teams created and shared more than 400 computer programs in openly accessible notebooks. Complementing the now well-known features of seismic data that map to fault criticality in the laboratory, the winning teams employed unexpected strategies based on rescaling failure times as a fraction of the seismic cycle and comparing input distribution of training and testing data. In addition to yielding scientific insights into fault processes in the laboratory and their relation with the evolution of the statistical properties of the associated seismic data, the competition serves as a pedagogical tool for teaching ML in geophysics. The approach may provide a model for other competitions in geosciences or other domains of study to help engage the ML community on problems of significance.

58 GEOSCIENCES↗

Assessing residential PM 2.5 concentrations and infiltration factors with high spatiotemporal resolution using crowdsourced sensors

Building conditions, outdoor climate, and human behavior influence residential concentrations of fine particulate matter (PM 2.5 ). To study PM 2.5 spatiotemporal variability in residences, we acquired paired indoor and outdoor PM 2.5 measurements at 3,977 residences across the United States totaling >10,000 monitor-years of time-resolved data (10-min resolution) from the PurpleAir network. Time-series analysis and statistical modeling apportioned residential PM 2.5 concentrations to outdoor sources (median residential contribution = 52% of total, coefficient of variation = 69%), episodic indoor emission events such as cooking (28%, CV = 210%) and persistent indoor sources (20%, CV = 112%). Residences in the temperate marine climate zone experienced higher infiltration factors, consistent with expectations for more time with open windows in milder climates. Likewise, for all climate zones, infiltration factors were highest in summer and lowest in winter, decreasing by approximately half in most climate zones. Large outdoor–indoor temperature differences were associated with lower infiltration factors, suggesting particle losses from active filtration occurred during heating and cooling. Absolute contributions from both outdoor and indoor sources increased during wildfire events. Infiltration factors decreased during periods of high outdoor PM 2.5 , such as during wildfires, reducing potential exposures from outdoor-origin particles but increasing potential exposures to indoor-origin particles. Time-of-day analysis reveals that episodic emission events are most frequent during mealtimes as well as on holidays (Thanksgiving and Christmas), indicating that cooking-related activities are a strong episodic emission source of indoor PM 2.5 in monitored residences.

54 ENVIRONMENTAL SCIENCES↗

Scalable and efficient learning from crowds with Gaussian processes

Over the last few years, multiply-annotated data has become a very popular source of information. Online platforms such as Amazon Mechanical Turk have revolutionized the labelling process needed for any classification task, sharing the effort between a number of annotators (instead of the classical single expert). This crowdsourcing approach has introduced new challenging problems, such as handling disagreements on the annotated samples or combining the unknown expertise of the annotators. Probabilistic methods, such as Gaussian Processes (GP), have proven successful to model this new crowdsourcing scenario. However, GPs do not scale up well with the training set size, which makes them prohibitive for medium-to-large datasets (beyond 10K training instances). This constitutes a serious limitation for current real-world applications. In this work, we introduce two scalable and efficient GP-based crowdsourcing methods that allow for processing previously-prohibitive datasets. The first one is an efficient and fast approximation to GP with squared exponential (SE) kernel. The second allows for learning a more flexible kernel at the expense of a heavier training (but still scalable to large datasets). Since the latter is not a GP-SE approximation, it can be also considered as a whole new scalable and efficient crowdsourcing method, useful for any dataset size. Both methods use Fourier features and variational inference, can predict the class of new samples, and estimate the expertise of the involved annotators. A complete experimentation compares them with state-of-the-art probabilistic approaches in synthetic and real crowdsourcing datasets of different sizes. Finally, they stand out as the best performing approach for large scale problems. Moreover, the second method is competitive with the current state-of-the-art for small datasets.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

A Comprehensive List of Geothermal Reservoir Properties for the Development of Geothermal Occurrence Models

Despite the fact that geothermal occurrence models (GOMs) are essential components of early stage exploration, few attempts have been made to compile comprehensive lists of data on world - wide geothermal resources in order to develop such models. The properties of these geothermal resources - including the structural setting, geothermal features, tectonic setting, temperature of the reservoir, and host/cap rock characteristics - are being catalogued on OpenEI, a semantic wiki platform for crowdsourced knowledge-sharing. This additional in - formation added to the wiki platform will provide the public with access to an enormous amount of geothermal information, the goal of which is to stimulate the development of GOMs and aid in exploration. This paper reviews the significance of the project, the methodologies involved in the classification of resources, and the intended future work.

15 GEOTHERMAL ENERGY↗

Crowdsourcing quality control for Dark Energy Survey images

We have developed a crowdsourcing web application for image quality control employed by the Dark Energy Survey. Dubbed the "DES exposure checker", it renders science-grade images directly to a web browser and allows users to mark problematic features from a set of predefined classes. Users can also generate custom labels and thus help identify previously unknown problem classes. User reports are fed back to hardware and software experts to help mitigate and eliminate recognized issues. We report on the implementation of the application and our experience with its over 100 users, the majority of which are professional or prospective astronomers but not data management experts. We discuss aspects of user training and engagement, and demonstrate how problem reports have been pivotal to rapidly correct artifacts which would likely have been too subtle or infrequent to be recognized otherwise. We conclude with a number of important lessons learned, suggest possible improvements, and recommend this collective exploratory approach for future astronomical surveys or other extensive data sets with a sufficiently large user base. We also release open-source code of the web application and host an online demo versionat http://des-exp-checker.pmelchior.net

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Crowdsourcing the Frontier: Advancing Hybrid Physics‐ML Climate Simulation via a $\$$50,000 Kaggle Competition

Subgrid machine-learning (machine learning [ML]) parameterizations have the potential to introduce a new generation of climate models that incorporate the effects of higher-resolution physics without incurring the prohibitive computational cost associated with more explicit physics-based simulations. However, important issues, ranging from online instability to inconsistent online performance, have limited their operational use for long-term climate projections. To more rapidly drive progress in solving these issues, domain scientists and ML researchers opened up the offline aspect of this problem to the broader ML and data science community with the release of ClimSim, a NeurIPS Data sets and Benchmarks publication, and an associated Kaggle competition. This paper reports on the downstream results of the Kaggle competition by coupling emulators inspired by the winning teams' architectures to an interactive climate model (including full cloud microphysics, a regime historically prone to online instability) and systematically evaluating their online performance. Our results demonstrate that online stability in the low-resolution real-geography setting is reproducible across multiple diverse architectures, which we consider a key milestone. All tested architectures exhibit strikingly similar offline and online biases, though their responses to architecture-agnostic design choices (e.g., expanding the list of input variables) can differ significantly. Multiple Kaggle-inspired architectures achieve state-of-the-art results on certain metrics such as zonal mean bias patterns and global Root Mean Squared Error, indicating that crowdsourcing the essence of the offline problem is one path to improving online performance in hybrid physics-AI climate simulation.

Environmental sciences↗

Citizen Science

Scientists and engineers constantly face new challenges, despite myriad advances in computing. More sets of data are collected today from earth and sky than there is time or resources available to carefully analyze them. Some problems either don't have fast algorithms to solve them or have solutions that must be found among millions of options, a situation akin to finding a needle in a haystack. But all hope is not lost: advances in technology and the Internet have empowered the general public to participate in the scientific process via individual computational resources and brain cognition, which isn't matched by any machine. Citizen scientists are volunteers who perform scientific work by making observations, collecting and disseminating data, making measurements, and analyzing or interpreting data without necessarily having any scientific training. In so doing, individuals from all over the world can contribute to science in ways that wouldn't have been otherwise possible.

distributed computing↗