Search NASA⌕ Search

SEARCH · Search NASA

Results for “crowdsourcing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Toward the validation of crowdsourced experiments for lightness perception

Crowdsource platforms have been used to study a range of perceptual stimuli such as the graphical perception of scatterplots and various aspects of human color perception. Given the lack of control over a crowdsourced participant’s experimental setup, there are valid concerns on the use of crowdsourcing for color studies as the perception of the stimuli is highly dependent on the stimulus presentation. Here, we propose that the error due to a crowdsourced experimental design can be effectively averaged out because the crowdsourced experiment can be accommodated by the Thurstonian model as the convolution of two normal distributions, one that is perceptual in nature and one that captures the error due to variability in stimulus presentation. Based on this, we provide a mathematical estimate for the sample size needed to produce a crowdsourced experiment with the same power as the corresponding in-person study. We tested this claim by replicating a large-scale, crowdsourced study of human lightness perception with a diverse sample with a highly controlled, in-person study with a sample taken from psychology undergraduates. Our claim was supported by the replication of the results from the latter. These findings suggest that, with sufficient sample size, color vision studies may be completed online, giving access to a larger and more representative sample. With this framework at hand, experimentalists have the validation that choosing either many online participants or few in person participants will not sacrifice the impact of their results.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2011

Crowdsourcing for Challenging Technical Problems - It Works!

The NASA Johnson Space Center Space Life Sciences Directorate (SLSD) and Wyle Integrated Science and Engineering (Wyle) will conduct a one-day business cluster at the 62nd IAC so that IAC attendees will understand the benefits of open innovation (crowdsourcing), review successful results of conducting technical challenges in various open innovation projects, and learn how an organization can effectively deploy these new problem solving tools to innovate more efficiently and effectively. Results from both the SLSD open innovation pilot program and the open innovation workshop conducted by the NASA Human Health and Performance Center will be discussed. NHHPC members will be recruited to participate in the business cluster (see membership http://nhhpc.nasa.gov) and as IAF members. Crowdsourcing may be defined as the act of outsourcing tasks that are traditionally performed by an employee or contractor to an undefined, generally large group of people or community (a crowd) in the form of an open call. The open call may be issued by the organization wishing to find a solution to a particular problem or complete a task, or by an open innovation service provider on behalf of that organization. In 2008, the SLSD, with the support of Wyle, established and implemented pilot projects in open innovation (crowdsourcing) to determine if these new internet-based platforms could indeed find solutions to difficult technical challenges. These unsolved technical problems were converted to problem statements, called Challenges by some open innovation service providers, and were then posted externally to seek solutions to these problems. In addition, an open call was issued internally to NASA employees Agency wide (11 Field Centers and NASA HQ) using an open innovation service provider crowdsourcing platform to post NASA challenges from each Center for the others to propose solutions). From 2008 to 2010, the SLSD issued 34 challenges, 14 externally and 20 internally. The 14 external problems or challenges were posted through three different vendors: InnoCentive, yet2.com and TopCoder. The 20 internal challenges were conducted using the InnoCentive crowdsourcing platform designed for use internal to an organization and customized for NASA use, and promoted as NASA@Work. The results were significant. Of the seven InnoCentive external challenges, two full and five partial awards were made in complex technical areas such as predicting solar flares and long-duration food packaging.

Davis, Jeffrey R.↗
NASA NTRS2019

Developing a Machine-Learning-Based Processing Framework for Twitter and Other Crowdsourced Data

Crowdsourced data streams such as Twitter and other social media are important sources of real-time and historical global information for Earth science applications. At the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), we have been exploring the Twitter data stream for its potential in augmenting the validation program of NASA's Global Precipitation Measurement (GPM) mission. To realize this potential, we need to increase the information density and enhance the quality of filtered precipitation tweets. We have implemented various components of a machine learning (ML)-based processing infrastructure for crowdsourced data that outputs, in this instance, useful and usable information derived from precipitation tweets. We have test enriched the Twitter stream with higher quality active tweets from those knowingly contributing to our effort and from existing crowdsourced programs (e.g., mPING, CoCoRaHS). We have experimented with various algorithms for processing tweets, including Naà ve Bayes, Convolutional Neural Network (CNN), Hierarchical Attention Network (HAN), and semi-supervised learning (with tri-training). Our current work focuses on (1) automated review of Earth science-related publications to determine relationships between discipline research needs and ML algorithms; (2) investigating Sequential Generative Adversarial Network (SeqGAN) for processing precipitation tweets for anomaly detection; and (3) managing crowdsourced data in a way that is compatible with existing NASA satellite data archives and using the data for ML applications. Key results include (1) network visualization of NLP-processed publications in various Earth science disciplines; (2) difference between GPM-linked, generated tweets and collected actual tweets that is small for GPM-determined light to moderate rain cases and high for GPM-determined heavy rain cases; and (3) identification of MongoDB for storing raw tweets and Zarr format for gridded tweets (compatible with GPM data). Our results have taken us a step closer to an operational ML-based tweet processing infrastructure and have already demonstrated that tweet-derived precipitation information is potentially useful for validation of Earth science satellite data.

Teng, William↗
NASA NTRSDate not supplied

Crowdsourcing for Innovative Personal Protection Technology

Personal protective equipment (PPE) plays a vital role in safeguarding workers from workplace hazards and minimizing exposure to harmful substances or conditions. To meet the changing needs and challenges faced by workers in various industries, PPE is continuously evolving. As new technologies emerge, advances in materials, design, and functionality contribute to PPE improvements. By incorporating personal protective technology (PPT) into traditional PPE, worker safety and well-being can be further enhanced. Crowdsourcing can harness a wide range of perspectives and expertise to identify novel technologies, materials, designs, or applications that can revolutionize PPE and improve worker protection. Partnering with the National Aeronautics and Space Administration (NASA) Center for Center of Excellence for Collaborative Innovation (CoECI), the National Institute for Occupational Safety and Health (NIOSH) National Personal Protection Technology Laboratory (NPPTL) launched three crowdsourcing challenges. These challenges aimed to promote innovation in several PPE areas: protective clothing, counterfeit respirators, and respirator fit. Preliminary results produced promising technologies and approaches promoting improved awareness, comfort, adaptability, and effectiveness against workplace hazards.

crowdsourcing↗
NASA NTRS2011

Integrating Machine Learning into a Crowdsourced Model for Earthquake-Induced Damage Assessment

On January 12th, 2010, a catastrophic 7.0M earthquake devastated the country of Haiti. In the aftermath of an earthquake, it is important to rapidly assess damaged areas in order to mobilize the appropriate resources. The Haiti damage assessment effort introduced a promising model that uses crowdsourcing to map damaged areas in freely available remotely-sensed data. This paper proposes the application of machine learning methods to improve this model. Specifically, we apply work on learning from multiple, imperfect experts to the assessment of volunteer reliability, and propose the use of image segmentation to automate the detection of damaged areas. We wrap both tasks in an active learning framework in order to shift volunteer effort from mapping a full catalog of images to the generation of high-quality training data. We hypothesize that the integration of machine learning into this model improves its reliability, maintains the speed of damage assessment, and allows the model to scale to higher data volumes.

crowdsourcing↗
DOE OSTI2026

Crowdsourcing the Frontier: Advancing Hybrid Physics‐ML Climate Simulation via a $\$$50,000 Kaggle Competition

Subgrid machine-learning (machine learning [ML]) parameterizations have the potential to introduce a new generation of climate models that incorporate the effects of higher-resolution physics without incurring the prohibitive computational cost associated with more explicit physics-based simulations. However, important issues, ranging from online instability to inconsistent online performance, have limited their operational use for long-term climate projections. To more rapidly drive progress in solving these issues, domain scientists and ML researchers opened up the offline aspect of this problem to the broader ML and data science community with the release of ClimSim, a NeurIPS Data sets and Benchmarks publication, and an associated Kaggle competition. This paper reports on the downstream results of the Kaggle competition by coupling emulators inspired by the winning teams' architectures to an interactive climate model (including full cloud microphysics, a regime historically prone to online instability) and systematically evaluating their online performance. Our results demonstrate that online stability in the low-resolution real-geography setting is reproducible across multiple diverse architectures, which we consider a key milestone. All tested architectures exhibit strikingly similar offline and online biases, though their responses to architecture-agnostic design choices (e.g., expanding the list of input variables) can differ significantly. Multiple Kaggle-inspired architectures achieve state-of-the-art results on certain metrics such as zonal mean bias patterns and global Root Mean Squared Error, indicating that crowdsourcing the essence of the offline problem is one path to improving online performance in hybrid physics-AI climate simulation.

Environmental sciences↗
NASA NTRS2019

NASA Centennial Challenges Program: A Crowdsourcing Tool to Advance Life Support Technologies for Future NASA Missions

Historically, competitions and prizes such as those executed by the NASA Centennial Challenges (CC) program have created broader avenues through which to spur innovation from unlikely sources. In 2005, Congress amended the National Aeronautics and Space Act of 1958 to authorize NASA to create challenges through which prizes could be awarded to United States citizens or entities that succeeded in meeting the challenge objectives. Over the past 13 years, the CC program has initiated more than 19 challenges in a variety of technology areas, including propulsion, robotics, communications and navigation, human health, science instrumentation, nanotech, materials/structures and aerodynamics. This paper will discuss the status and the accomplishments of the CC program and discuss results of an ideation process designed to identify and formulate topics for a potential Centennial Challenge competition targeting a life support technology gap for future long-term exploration missions. Status of this challenge formulation process with information on how to use crowdsourcing tools will be discussed. An overview of the CC Program’s accomplishments, including strategic objectives, past challenges, and current challenge development and execution. This program exemplifies the values that have formed the bedrock of the culture at NASA since the beginning: innovation, imagination, and a passion for exploration.

innovation↗
DOE OSTI2024

A functional microbiome catalogue crowdsourced from North American rivers

Predicting elemental cycles and maintaining water quality under increasing anthropogenic influence requires knowledge of the spatial drivers of river microbiomes. However, understanding of the core microbial processes governing river biogeochemistry is hindered by a lack of genome-resolved functional insights and sampling across multiple rivers. Here we used a community science effort to accelerate the sampling, sequencing and genome-resolved analyses of river microbiomes to create the Genome Resolved Open Watersheds database (GROWdb). GROWdb profiles the identity, distribution, function and expression of microbial genomes across river surface waters covering 90% of United States watersheds. Specifically, GROWdb encompasses microbial lineages from 27 phyla, including novel members from 10 families and 128 genera, and defines the core river microbiome at the genome level. GROWdb analyses coupled to extensive geospatial information reveals local and regional drivers of microbial community structuring, while also presenting foundational hypotheses about ecosystem function. Building on the previously conceived River Continuum Concept, we layer on microbial functional trait expression, which suggests that the structure and function of river microbiomes is predictable. We make GROWdb available through various collaborative cyberinfrastructures, so that it can be widely accessed across disciplines for watershed predictive modelling and microbiome-based management practices.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2017

Framework for Processing Citizens Science Data for Applications to NASA Earth Science Missions

Citizen science (or crowdsourcing) has drawn much high-level recent and ongoing interest and support. It is poised to be applied, beyond the by-now fairly familiar use of, e.g., Twitter for natural hazards monitoring, to science research, such as augmenting the validation of NASA earth science mission data. This interest and support is seen in the 2014 National Plan for Civil Earth Observations, the 2015 White House forum on citizen science and crowdsourcing, the ongoing Senate Bill 2013 (Crowdsourcing and Citizen Science Act of 2015), the recent (August 2016) Open Geospatial Consortium (OGC) call for public participation in its newly-established Citizen Science Domain Working Group, and NASA's initiation of a new Citizen Science for Earth Systems Program (along with its first citizen science-focused solicitation for proposals). Over the past several years, we have been exploring the feasibility of extracting from the Twitter data stream useful information for application to NASA precipitation research, with both "passive" and "active" participation by the twitterers. The Twitter database, which recently passed its tenth anniversary, is potentially a rich source of real-time and historical global information for science applications. The time-varying set of "precipitation" tweets can be thought of as an organic network of rain gauges, potentially providing a widespread view of precipitation occurrence. The validation of satellite precipitation estimates is challenging, because many regions lack data or access to data, especially outside of the U.S. and in remote and developing areas. Mining the Twitter stream could augment these validation programs and, potentially, help tune existing algorithms. Our ongoing work, though exploratory, has resulted in key components for processing and managing tweets, including the capabilities to filter the Twitter stream in real time, to extract location information, to filter for exact phrases, and to plot tweet distributions. The key step is to process the "precipitation" tweets to be compatible with satellite-retrieved precipitation data. These key components for processing and managing "precipitation" tweets (and additional ones to be developed) are not limited to precipitation, nor are they limited to the Twitter social medium. Indeed, to maximize the value of our work for NASA earth science programs, these components should be generalized and be part of an overall framework for processing citizen science data for science research. In this paper, we outline such a framework.

earth science satellite data↗
NASA NTRSDate not supplied

Machine-learning Solution for Automatic Spacesuit Motion Recognition and Measurement from Conventional Video

Extravehicular Activity (EVA) spacesuits exhibit unique movement patterns due to their design characteristics. Mobility assessments using traditional motion capture systems are cost prohibitive and not feasible for some training conditions (e.g., simulated lunar outdoor terrain). This paper aims to present the ongoing development of machine learning solutions to quantify suit motions from conventional videos without special sensors or hardware. Given the fast growth in deep/machine learning technologies, external expertise was sought from open-source communities. This was expected to accelerate development and provide more cost-effective, time-saving solutions. This work was selected for a NASA Crowdsourcing project through an agency-wide solicitation. Partnerships were formed with the NASA JSC Center of Excellence for Collaborative Innovation and an execution crowdsourcing platform partner to solicit framework developments from external contenders. NASA provided contenders with video clips of spacesuits and simultaneously measured motion capture data during EVA simulation tasks. The contenders used this data to train and develop generalized algorithms to predict motions. At the end of the crowdsourcing event, five solutions were selected from 250 submissions. Each submission was tested and scored using video clips not previously disclosed to the contenders. The scoring metrics measured how well the algorithm detected the suit shape, the 2D suit joint detection accuracy, and 3D joint detection accuracy. The winning solution was able to achieve roughly 85% prediction accuracy (weighted combination of scoring metrics). Overall, the algorithms could efficiently detect various types of spacesuits and motions across different EVA simulation environments such as the Neutral Buoyancy Lab (NBL). However, 3D joint identification is less reliable when parts of the suit were obstructed in the image. After continued improvements and validation, the fully developed system will enable EVA stakeholders to quantify suit kinematic patterns, which can help optimize suit, hardware, and task designs.

Linh Vu↗
NASA NTRSDate not supplied

Keeping Up, with the Crowd

High-level presentation explaining the innovation imperative (reason for crowdsourcing) in addition to NASA CoECI's crowdsourcing program and how it's been successful.

Crowdsourcing↗
DOE OSTI2024

Labeling sequential data from noisy annotations

Crowdsourcing algorithms often work under the assumption that the data samples are independent. Recent work has shown that data dependence, such as temporal correlations in sequential data, can be leveraged to improve the label quality. Existing methods that exploit this special structure rely on third-order statistics of the annotator outputs to ensure the identifiability of key latent parameters, which are costly to acquire. This work proposes an approach for integrating crowdsourced annotations under the Dawid-Skene/Hidden Markov Model (DS-HMM) for sequential data based on second-order statistics, which naturally enjoys a lower sample complexity. An effective algorithm is proposed to tackle the challenging optimization problem associated with the proposed estimator. Numerical experiments showcase the effectiveness of the data labeling paradigm.

Marrinan, Timothy P.↗