SEARCH · Search NASA
Results for “Crowdsourced Data”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Ultrahigh-resolution mass spectrometry data associated with the manuscript “A functional microbiome catalog crowdsourced from North American rivers"
This data package is associated with the publication “A functional microbiome catalog crowdsourced from North American rivers” submitted to Nature (Borton et al., 2024); (https://www.biorxiv.org/content/10.1101/2023.07.22.550117v1). Predicting elemental cycles and maintaining water quality under increasing anthropogenic influence requires understanding the spatial drivers of river microbiomes. However, the unifying microbial determinants governing river biogeochemistry are hindered by a lack of genome-resolved functional insights and sampling across multiple rivers. Here we employed a community science effort to accelerate the sampling of river microbiomes to create the Genome Resolved Open Watersheds database (GROWdb). GROWdb is a publicly available resource that paves the way for watershed predictive modeling and microbiome-based management practices. This resource profiled the identity, distribution, function, and expression of thousands of microbial genomes across rivers covering 90% of United States watersheds. We identified the most cosmopolitan microbiome members, while also revealing local drivers of strain endemism across ecological dimensions. We provide the first evidence that microbial functional trait expression followed the tenets of the River Continuum Concept, suggesting the structure and function of river microbiomes is predictable. The Fourier-transform ion cyclotron resonance mass spectrometry (FTICR-MS) data were one of many different data types used in establishing the ecological dimensions along which different microbes were detected .This data package only contains the processed FTICR-MS data associated with this manuscript; all other data is accessible via Zenodo (https://zenodo.org/records/8173287), GitHub (https://github.com/jmikayla1991/Genome-Resolved-Open-Watersheds-database-GROWdb), KBase (https://doi.org/10.25982/109073.30/1895615), and NCBI via Bioproject PRJNA946291.This dataset consists of (1) a file-level metadata (flmd) file; (2) a data dictionary (dd) file; (3) a readme; (4) three Fourier-transform ion cyclotron resonance mass spectrometry (FTICR-MS) processed data files (a ‘data’ file containing peak-by-sample observations, a ‘mol’ file containing peak metadata, and a transformation profile containing transformation-by-sample observations). All files are .csv or .pdf.
Data object classification using feature generation through crowdsourcing
In a computing device that implements a data object classification tool, a method for classifying data may include detecting change in spatial coordinates for each of at least two of a set of data objects within a canvas space. Each of the data objects may be associated with a vector of features. A rule set may be generated based on the vector of features associated with each of the at least two data objects. The rule set may use feature(s) that explain the changed spatial coordinates. The data objects may be selectively rearranged within the canvas space by applying the generated rule set to any remaining data objects among the set of data objects so as to assign spatial coordinates to the remaining objects. For each of the data objects, the spatial coordinates may be stored as new semantic feature(s) within the vector of features for that data object.
Scalable and Actionable Performance Measures for Traffic Signal Systems using Probe Vehicle Trajectory Data
Scalable and actionable performance measures for traffic signal systems provide opportunities for practitioners to measure and improve the transportation network. Historically, traffic signal improvements have relied on scheduled signal retiming based on limited data collection, or on the public to call and alert engineers of an issue. This inefficient method of improving signal timing led to the creation of automated traffic signal performance measures (ATSPMs). These metrics rely on expensive infrastructure, including detection and communications, which has produced barriers for numerous agencies to fully adopt. Recently, third-party data providers have begun to release vehicle trajectory data, which allows for enhanced signal metrics with no investment in physical equipment. The purpose of this study is to demonstrate the use of these data and summarize the scalability of the created metrics. This work builds on previous efforts to quantify signal performance on nine intersections in Michigan, U.S. Ten signalized corridors in Columbus, Ohio, were chosen to scale a performance assessment using crowdsourced trajectory data. A total of 136 intersections were assessed in 2-h intervals using data from all weekdays in 2017. High-level corridor summary metrics including average percent of vehicles stopping (18%–32%), average delay (9.4–20.5 s), and level of travel time reliability (1.23–2.73) were calculated for each corridor direction. Intersection-level metrics were also introduced, which can be used by practitioners to identify problems, improve signal timings, and prioritize future infrastructure investments.
Crowd-based spatial risk assessment of urban flooding: Results from a municipal flood hotline in Detroit, MI
Climate change is increasing the frequency and intensity of extreme precipitation events, raising the risk of urban flood disasters. This study uses a crowd-sourced municipal call database to characterize the spatial distribution of flood risk in Detroit, MI. Call data including dates and addresses were obtained from the City of Detroit Department of Public Works for 2021. Calls were mapped and aggregated to census tract counts and merged with neighborhood-level data. Associations of predictors with flood calls were tested using spatial regression models. Flooding calls were located throughout the city but were concentrated in specific areas. Multivariate models of census tract level call counts indicated that increased poverty and Black, immigrant, and older residents were positively associated with flood calls, while increased elevation was associated with protective effects. Longer distances from waste water interceptors were associated with higher risk for calls. Crowd-sourced flood hotline call data can be used for effective spatial flood risk assessment. Though flooding occurs throughout the city of Detroit, infrastructural, neighborhood, and household factors influence flooding extent. Limitations included the self-reported nature of calls. Future modeling efforts might include input from local stakeholders to improve spatial risk assessment.
Urban Versus Lake Impacts on Heat Stress and Its Disparities in a Shoreline City
Abstract Shoreline cities are influenced by both urban‐scale processes and land‐water interactions, with consequences on heat exposure and its disparities. Heat exposure studies over these cities have focused on air and skin temperature, even though moisture advection from water bodies can also modulate heat stress. Here, using an ensemble of model simulations covering Chicago, we find that Lake Michigan strongly reduces heat exposure (2.75°C reduction in maximum average air temperature in Chicago) and heat stress (maximum average wet bulb globe temperature reduced by 0.86°C) during the day, while urbanization enhances them at night (2.75 and 1.57°C increases in minimum average air and wet bulb globe temperature, respectively). We also demonstrate that urban and lake impacts on temperature (particularly skin temperature), including their extremes, and lake‐to‐land gradients, are stronger than the corresponding impacts on heat stress, partly due to humidity‐related feedback. Likewise, environmental disparities across community areas in Chicago seen for skin temperature are much higher (1.29°C increase for maximum average values per $10,000 higher median income per capita) than disparities in air temperature (0.50°C increase) and wet bulb globe temperature (0.23°C increase). The results call for consistent use of physiologically relevant heat exposure metrics to accurately capture the public health implications of urbanization.
Linkages Between Mineral Element Composition of Soils and Sediments With Hyporheic Zone Dissolved Organic Matter Chemistry Across the Contiguous United States
The hyporheic zone is a hotspot for biogeochemical cycling where interactions with mineral metals preserve the release and biodegradation of organic matter (OM). A small fraction of OM can still be exchanged between localized sediments and the overlying water column, and recent evidence suggests there exists a longitudinal structuring in sediment dissolved OM (DOM) chemistry across the continental United States (CONUS). In this study, we tested a hypothesis that water extractable sediment DOM chemistry could be explained by sediment metal contents and integrative watershed scale features at the CONUS scale. Crowdsourced samples were characterized for high resolution mass spectrometry and coupled with sediment metals determined via x-ray fluorescence as well as with land cover and soil elemental information obtained from national databases. Our results highlight weak relationships between DOM chemistry and elemental composition at the CONUS scale indicating limited transferability of organo-metal linkages into multi-scale hydrobiogeochemical models.
THE MILKY WAY PROJECT: LEVERAGING CITIZEN SCIENCE AND MACHINE LEARNING TO DETECT INTERSTELLAR BUBBLES
We present Brut, an algorithm to identify bubbles in infrared images of the Galactic midplane. Brut is based on the Random Forest algorithm, and uses bubbles identified by >35,000 citizen scientists from the Milky Way Project to discover the identifying characteristics of bubbles in images from the Spitzer Space Telescope. We demonstrate that Brut's ability to identify bubbles is comparable to expert astronomers. We use Brut to re-assess the bubbles in the Milky Way Project catalog, and find that 10%-30% of the objects in this catalog are non-bubble interlopers. Relative to these interlopers, high-reliability bubbles are more confined to the mid-plane, and display a stronger excess of young stellar objects along and within bubble rims. Furthermore, Brut is able to discover bubbles missed by previous searches—particularly bubbles near bright sources which have low contrast relative to their surroundings. Brut demonstrates the synergies that exist between citizen scientists, professional scientists, and machine learning techniques. In cases where ''untrained' citizens can identify patterns that machines cannot detect without training, machine learning algorithms like Brut can use the output of citizen science projects as input training sets, offering tremendous opportunities to speed the pace of scientific discovery. A hybrid model of machine learning combined with crowdsourced training data from citizen scientists can not only classify large quantities of data, but also address the weakness of each approach if deployed alone.
Secondary Crash Identification using Crowdsourced Waze User Reports
Secondary crashes are crashes that occur as a result of the nonrecurrent congestion originating from primary crashes, and always have a greater impact on safety and traffic than a single crash. A better understanding of secondary crashes would benefit traffic incident management, and this requires accurate identification of secondary crashes. This study explores using crowdsourced Waze user reports to identify secondary crashes. Here, a network-based clustering algorithm is proposed to extract the primary crash cluster, including all user reports originating from the primary crash, and any crash that occurred within the cluster would be a secondary crash. This method works as a filter to select accurate primary–secondary relationships, thus precisely identifying secondary crashes. A case study is performed with crashes occurring from June to December 2019 on a 30-mi stretch of I-40 in Knoxville, TN. A static threshold method (crash duration and 10 mi) was used to preselect the potential primary–secondary crash pairs, and 75 out of 708 crashes were identified as potential secondary crashes. Based on the preselected primary–secondary crash pairs, 17 secondary crashes were obtained with the proposed method and the results were compared with one of the commonly used methods, the speed contour plot method. Though the proposed method captured fewer secondary crashes, it did identify several secondary crashes that could not be observed with the speed contour plot method. The results showed the applicability of the method and the potential of crowdsourced Waze user reports in secondary crash identification.
Labeling sequential data from noisy annotations
Crowdsourcing algorithms often work under the assumption that the data samples are independent. Recent work has shown that data dependence, such as temporal correlations in sequential data, can be leveraged to improve the label quality. Existing methods that exploit this special structure rely on third-order statistics of the annotator outputs to ensure the identifiability of key latent parameters, which are costly to acquire. This work proposes an approach for integrating crowdsourced annotations under the Dawid-Skene/Hidden Markov Model (DS-HMM) for sequential data based on second-order statistics, which naturally enjoys a lower sample complexity. An effective algorithm is proposed to tackle the challenging optimization problem associated with the proposed estimator. Numerical experiments showcase the effectiveness of the data labeling paradigm.
Enriching the Twitter Stream Increasing Data Mining Yield and Quality Using Machine Learning
Social media data streams are important sources of real-time and historical global information for science applications. At the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), we are exploring the Twitter data stream for its potential in augmenting the validation program of NASA Earth science missions, specifically the Global Precipitation Measurement (GPM) mission. We have implemented a tweet processing infrastructure that outputs classified precipitation tweets. Inputs are "passive" tweets, along with a smaller number of tweets from "active" participants, i.e., those knowingly contributing to our effort. The "active" tweets, presumably of higher quality, enrich the Twitter stream. "Active" sources include data scraped from other social media (e.g., public Facebook posts) and data from existing crowdsourcing programs (e.g., mPING reports). In addition, there is likely relevant precipitation information in images and documents that are the end points of links often included in tweets. Information derived from these "active" sources could then be tweeted into the Twitter stream, thus enriching its quality. The objective of our current work is to mine these tweet linked images and documents, using neural networks, to increase the information content and quality related to precipitation. For images, we classified them as either precipitation-related or not. For training and validation, we used images obtained via the Google custom search API. We created two models: (1) by training a simple Convolutional Neural Network and (2) by using transfer learning principles to adapt a pre-trained object recognition model. For documents, both those linked to tweets and the tweet contents, we trained Hierarchical Attention Networks to determine precipitation occurrence, type, and intensity. For training and validation, we used a keyword-filtered tweet data set labelled with ground truth data from Dark Sky (an API to retrieve weather-related labels) and the National Severe Storms Laboratory's Multi Radar/Multi-Sensor (MRMS) system. Our results demonstrated the efficacy of our machine learning approaches for enriching the Twitter stream, to derive information potentially useful for validation of earth science satellite data.
Estimating building occupancy: a machine learning system for day, night, and episodic events
Building occupancy research increasingly emphasizes understanding the social and physical dynamics of how people occupy space. Opportunities in the open source domain including social media, Volunteered Geographic Information, crowdsourcing, and sensor data have proliferated, resulting in the exploration of building occupancy dynamics at varying spatiotemporal scales. At Oak Ridge National Laboratory, research into building occupancies through the development of a global learning framework that accommodates exploitation of open source authoritative sources, including governmental census and surveys, journal articles, real estate databases, and more, to report national and subnational building occupancies across the world continues through the Population Density Tables (PDT) project. This probabilistic learning system accommodates expert knowledge, experience, and open-source data to capture local, socioeconomic, and cultural information about human activity. It does so through a systematic process of data harmonization techniques in the development of observation models for over 50 building types to dynamically update baseline estimates and report probabilistic diurnal and episodic building occupancy estimates. This discussion will explore how PDT is implemented at scale and expanded based on the development of observation model classes and will explain how to interpret and spatially apply the reported probability occupancy estimates and uncertainty.
Determining the biogeochemical transformations of organic matter composition in rivers using molecular signatures
Inland waters are hotspots for biogeochemical activity, but the environmental and biological factors that govern the transformation of organic matter (OM) flowing through them are still poorly constrained. Here we evaluate data from a crowdsourced sampling campaign led by the Worldwide Hydrobiogeochemistry Observation Network for Dynamic River Systems (WHONDRS) consortium to investigate broad continental-scale trends in OM composition compared to localized events that influence biogeochemical transformations. Samples from two different OM compartments, sediments and surface water, were collected from 97 streams throughout the Northern Hemisphere and analyzed to identify differences in biogeochemical processes involved in OM transformations. By using dimensional reduction techniques, we identified that putative biogeochemical transformations and microbial respiration rates vary across sediment and surface water along river continua independent of latitude (18°N–68°N). In contrast, we reveal small- and large-scale patterns in OM composition related to local (sediment vs. water column) and reach (stream order, latitude) characteristics. These patterns lay the foundation to modeling the linkage between ecological processes and biogeochemical signals. We further showed how spatial, physical, and biogeochemical factors influence the reactivity of the two OM pools in local reaches yet find emergent broad-scale patterns between OM concentrations and stream order. OM processing will likely change as hydrologic flow regimes shift and vertical mixing occurs on different spatial and temporal scales. As our planet continues to warm and the timing and magnitude of surface and subsurface flows shift, understanding changes in OM cycling across hydrologic systems is critical, given the unknown broad-scale responses and consequences for riverine OM.
It takes a village: using a crowdsourced approach to investigate organic matter composition in global rivers through the lens of ecological theory
Though community-based scientific approaches are becoming more common, many scientific efforts are conducted by small groups of researchers that together develop a concept, analyze data, and interpret results that ultimately translate into a publication. Here, we present a community effort that breaks these traditional boundaries of the publication process by engaging the scientific community from initial hypothesis generation to final publication. We leverage community-generated data from the Worldwide Hydrobiogeochemistry Observation Network for Dynamic River Systems (WHONDRS) consortium to study organic matter composition through the lens of ecological theory. This community endeavor will use a suite of paired physical and chemical datasets collected from 97 river corridors across the globe. With our first step aimed at ideation, we engaged a community of scientists from 20 countries and 60 institutions, spanning disciplines and career stages by holding a virtual workshop (April 2021). In the workshop, participants generated content for questions, hypotheses, and proposed analyses based on the WHONDRS dataset. These ideation efforts resulted in several narratives investigating different questions led by different teams, which will be the basis for research articles in a Frontiers in Water collection. Currently, the community is collectively analyzing, interpreting, and synthesizing these data that will result in seven crowdsourced articles using a single, existing WHONDRS dataset. The use of a shared dataset across articles not only lowers barriers for broad participation by not requiring generation of new data, but also provides unique opportunities for emergent learning by connecting outcomes across studies. Here we will explain methods used to enable this community endeavor aimed to promote a greater diversity of thinking on river corridor biogeochemistry through community science.
Reward-Based Participant Management for Crowdsourcing Rainfall Monitoring: An Agent-Based Model Simulation
Crowdsourcing incorporates common citizens as rich sources of data and is promising for environmental monitoring. Here, we propose and test the idea of incorporating incentives to crowdsourcing management for rainfall monitoring. In particular, we model the allocation of incentives (quantitatively measurable and limited rewards) among crowdsourcing participants for a theoretical rainfall monitoring case. For this purpose, we develop an integrated model comprising a reward allocation component to represent the decision-making process of a central manager, an agent-based model to simulate the interactions between the manager and participants, and a rainfall simulation model to evaluate the effectiveness of various reward allocation policies. We simulate six reward allocation policies of varying levels of administrative cost, and consideration of participant and rainfall spatial heterogeneities. The findings indicate the performance of each policy to improve with the reward budget and their spatial uniformity. Among the six policies tested, we discover that the participant density weighted maximum participation policy yields the most accurate estimation of rainfall intensity due to its more explicit consideration of the spatial distribution of participants; however, this policy associates with a high administrative cost. This highlights the trade-off between performance and cost in designing effective reward allocation policies. This paper provides a physical and behavior simulation modeling tool to study the feasibility and complexity of reward-based participant management for crowdsourcing rainfall monitoring. The proposed crowdsourcing method is beneficial for a wide range of applications that require rainfall data with fine resolution, such as storm water management and water availability and biomass assessment for food and energy crops.
Integrating Machine Learning into a Crowdsourced Model for Earthquake-Induced Damage Assessment
On January 12th, 2010, a catastrophic 7.0M earthquake devastated the country of Haiti. In the aftermath of an earthquake, it is important to rapidly assess damaged areas in order to mobilize the appropriate resources. The Haiti damage assessment effort introduced a promising model that uses crowdsourcing to map damaged areas in freely available remotely-sensed data. This paper proposes the application of machine learning methods to improve this model. Specifically, we apply work on learning from multiple, imperfect experts to the assessment of volunteer reliability, and propose the use of image segmentation to automate the detection of damaged areas. We wrap both tasks in an active learning framework in order to shift volunteer effort from mapping a full catalog of images to the generation of high-quality training data. We hypothesize that the integration of machine learning into this model improves its reliability, maintains the speed of damage assessment, and allows the model to scale to higher data volumes.
Improved learning from data competitions through strategic design of training and test data sets
Leveraging the depth and breadth of solutions generated through crowdsourcing can be a powerful accelerator to method development for high consequence problems. While data science competitions have become quite popular and prevalent, particularly in supervised learning formats, their implementations by the host are highly variable. Without careful planning, a supervised learning competition is vulnerable to overfitting, where the winning solutions are so closely tuned to the particular set of provided data that they cannot generalize to the general underlying problem of interest to the host. This paper outlines important considerations for strategically designing relevant and informative data sets to maximize the learning outcome from hosting a competition. These include: precisely defining the scope of the problem, encouraging participation by competitors from diverse technical backgrounds, specifying the most interesting solution space in order to encourage improvement and distinguish between competitors, strategically generating data sets that enable testing for interpolation and extrapolation to new scenarios of interest, leveraging design of experiment principles for strategic data design while preventing unintentional artifacts in the competition data set that an algorithm could exploit without addressing the real problem of interest, and carefully designing the leaderboard scoring metric to select top solutions that exactly match the overall competition goals. The methods are illustrated with a recently completed competition in the context of urban radiological search to evaluate algorithms capable of detecting, identifying, and locating radioactive materials. Simulated data were used in the urban search competition. Ideas for using measured real data in competitions are also suggested
Laboratory time series moisture manipulative experiment from sediment across the contiguous US: time series aerobic respiration and geochemistry (v2)
This dataset supports a broader study examining the effects of wetting and drying on hyporheic zone respiration across the contiguous United States (CONUS). The dataset provides data generated from a laboratory moisture manipulation experiment. The contents include time series aerobic respiration and moisture; dissolved oxygen; sediment geochemistry data; and field metadata (including qualitative information on instream and river corridor characteristics). Samples were collected as part of the WHONDRS CONUS-Scale Model-Sample Study (CM). This study was designed following ICON (integrated, coordinated, open, and networked) principles to facilitate a model-experiment (ModEx) iteration approach, leveraging crowdsourced sampling across the CONUS. The data package associated with the CM study is available at https://data.ess-dive.lbl.gov/view/doi:10.15485/1923689. CM sampling began in April 2022 and ended in October 2023. This study uses subsamples from a subset of CM samples collected between June 2022 and June 2023. The original field samples were labeled as CM_###. Subsequent subsamples for this study were labeled as EC_###. The labels from the field samples and the EC subsamples can be mapped directly based on the digits following the prefix and underscore (i.e., EC_001 is a subsample from CM_001). See the critical details section below for more details on sample naming. This data package was originally published in August 2024. It was updated in February 2026 (v2; new and modified files). See the change history section in the readme for more details. For details on how to navigate this data package, see this infographic from the River Corridor SFA https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. This dataset is comprised of one folder of raw Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) data and one main data folder containing (1) file-level metadata; (2) data dictionary; (3) field metadata; (4) readme; (5) field protocol; and a (6) a subfolder with sediment sample data from the incubation experiment. The sample data subfolder contains (1) dissolved organic carbon (DOC, measured as non-purgeable organic carbon, NPOC); (2) total nitrogen (TN); (3) adenosine triphosphate (ATP); (4) percent carbon and nitrogen; (5) effect size; (6) iron (II); (7) gravimetric moisture; (8) respiration rates and raw dissolved oxygen values; (9) specific conductance; (10) pH; (11) temperature; (12) a summary containing median values of each data type for each treatment (wet and dry); (13) methods codes; (14) FTICR-MS methods; and (15) a subfolder of 9.4 Tesla FTICR-MS data. This folder contains three subfolders, one containing the sediment .xml data files, one containing the sediment CoreMS output files, the other containing instructions and scripts for processing the files in CoreMS (https://github.com/EMSL-Computing/CoreMS). All files are .csv, .pdf, .R, .ref, or .xml.