Search NASA⌕ Search

SEARCH · Search NASA

Results for “crowdsourcing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

It takes a village: using a crowdsourced approach to investigate organic matter composition in global rivers through the lens of ecological theory

Though community-based scientific approaches are becoming more common, many scientific efforts are conducted by small groups of researchers that together develop a concept, analyze data, and interpret results that ultimately translate into a publication. Here, we present a community effort that breaks these traditional boundaries of the publication process by engaging the scientific community from initial hypothesis generation to final publication. We leverage community-generated data from the Worldwide Hydrobiogeochemistry Observation Network for Dynamic River Systems (WHONDRS) consortium to study organic matter composition through the lens of ecological theory. This community endeavor will use a suite of paired physical and chemical datasets collected from 97 river corridors across the globe. With our first step aimed at ideation, we engaged a community of scientists from 20 countries and 60 institutions, spanning disciplines and career stages by holding a virtual workshop (April 2021). In the workshop, participants generated content for questions, hypotheses, and proposed analyses based on the WHONDRS dataset. These ideation efforts resulted in several narratives investigating different questions led by different teams, which will be the basis for research articles in a Frontiers in Water collection. Currently, the community is collectively analyzing, interpreting, and synthesizing these data that will result in seven crowdsourced articles using a single, existing WHONDRS dataset. The use of a shared dataset across articles not only lowers barriers for broad participation by not requiring generation of new data, but also provides unique opportunities for emergent learning by connecting outcomes across studies. Here we will explain methods used to enable this community endeavor aimed to promote a greater diversity of thinking on river corridor biogeochemistry through community science.

Borton, Mikayla A.↗
DOE OSTI2024

Toward the validation of crowdsourced experiments for lightness perception

Crowdsource platforms have been used to study a range of perceptual stimuli such as the graphical perception of scatterplots and various aspects of human color perception. Given the lack of control over a crowdsourced participant’s experimental setup, there are valid concerns on the use of crowdsourcing for color studies as the perception of the stimuli is highly dependent on the stimulus presentation. Here, we propose that the error due to a crowdsourced experimental design can be effectively averaged out because the crowdsourced experiment can be accommodated by the Thurstonian model as the convolution of two normal distributions, one that is perceptual in nature and one that captures the error due to variability in stimulus presentation. Based on this, we provide a mathematical estimate for the sample size needed to produce a crowdsourced experiment with the same power as the corresponding in-person study. We tested this claim by replicating a large-scale, crowdsourced study of human lightness perception with a diverse sample with a highly controlled, in-person study with a sample taken from psychology undergraduates. Our claim was supported by the replication of the results from the latter. These findings suggest that, with sufficient sample size, color vision studies may be completed online, giving access to a larger and more representative sample. With this framework at hand, experimentalists have the validation that choosing either many online participants or few in person participants will not sacrifice the impact of their results.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Reward-Based Participant Management for Crowdsourcing Rainfall Monitoring: An Agent-Based Model Simulation

Crowdsourcing incorporates common citizens as rich sources of data and is promising for environmental monitoring. Here, we propose and test the idea of incorporating incentives to crowdsourcing management for rainfall monitoring. In particular, we model the allocation of incentives (quantitatively measurable and limited rewards) among crowdsourcing participants for a theoretical rainfall monitoring case. For this purpose, we develop an integrated model comprising a reward allocation component to represent the decision-making process of a central manager, an agent-based model to simulate the interactions between the manager and participants, and a rainfall simulation model to evaluate the effectiveness of various reward allocation policies. We simulate six reward allocation policies of varying levels of administrative cost, and consideration of participant and rainfall spatial heterogeneities. The findings indicate the performance of each policy to improve with the reward budget and their spatial uniformity. Among the six policies tested, we discover that the participant density weighted maximum participation policy yields the most accurate estimation of rainfall intensity due to its more explicit consideration of the spatial distribution of participants; however, this policy associates with a high administrative cost. This highlights the trade-off between performance and cost in designing effective reward allocation policies. This paper provides a physical and behavior simulation modeling tool to study the feasibility and complexity of reward-based participant management for crowdsourcing rainfall monitoring. The proposed crowdsourcing method is beneficial for a wide range of applications that require rainfall data with fine resolution, such as storm water management and water availability and biomass assessment for food and energy crops.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

Crowdsourcing and curation: perspectives from biology and natural language processing

Crowdsourcing is increasingly utilized for performing tasks in both natural language processing and biocuration. Although there have been many applications of crowdsourcing in these fields, there have been fewer high-level discussions of the methodology and its applicability to biocuration. This paper explores crowdsourcing for biocuration through several case studies that highlight different ways of leveraging ‘the crowd’; these raise issues about the kind(s) of expertise needed, the motivations of participants, and questions related to feasibility, cost and quality. The paper is an outgrowth of a panel session held at BioCreative V (Seville, September 9–11, 2015). The session consisted of four short talks, followed by a discussion. In their talks, the panelists explored the role of expertise and the potential to improve crowd performance by training; the challenge of decomposing tasks to make them amenable to crowdsourcing; and the capture of biological data and metadata through community editing.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2017

FY2017 Final Report: Power of the People: A technical ethical and experimental examination of the use of crowdsourcing to support international nuclear safeguards verification.

Recent advances in information technology have led to an expansion of crowdsourcing activities that utilize the “power of the people” harnessed via online games, communities of interest, and other platforms to collect, analyze, verify, and provide technological solutions for challenges from a multitude of domains. To related this surge in popularity, the research team developed a taxonomy of crowdsourcing activities as they relate to international nuclear safeguards, evaluated the potential legal and ethical issues surrounding the use of crowdsourcing to support safeguards, and proposed experimental designs to test the capabilities and prospect for the use of crowdsourcing to support nuclear safeguards verification.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2023

A general spatial-temporal framework for short-term building temperature forecasting at arbitrary locations with crowdsourcing weather data

Weather forecasting has been a critical component to predict and control building energy consumption for better building energy management. Without accessibility to other data sources, the onsite observed temperatures or the airport temperatures are used in forecast models. In this paper, we present a novel approach by utilizing the crowdsourcing weather data from neighboring personal weather stations (PWS) to improve the weather forecast accuracy around buildings using a general spatial-temporal modeling framework. The final forecast is based on the ensemble of local forecasts for the target location using neighboring PWSs. Our approach is distinguished from existing literature in various aspects. First, we leverage the crowdsourcing weather data from PWS in addition to public data sources. In this way, the data is at much finer time resolution (e.g., at 5-minute frequency) and spatial resolution (e.g., arbitrary location vs grid). Second, our proposed model incorporates spatial-temporal correlation information of weather variables between the target building and a set of neighboring PWSs so that underlying correlations can be effectively captured to improve forecasting performance. Here, we demonstrate the performance of the proposed framework by comparing to the benchmark models on temperature forecasting for a building located at an arbitrary location at San Antonio, Texas, USA. In general, the proposed model framework equipped with machine learning technique such as Random Forest can improve forecasting by 50% compares with persistent model and has 90% chance to outperform airport forecast in short-term forecasting. In a real-time setting, the proposed model framework can provide more accurate temperature forecasting results compared with using airport temperature forecast for most forecast horizon. Moreover, we analyze the sensitivity of model parameters to gain insights on how crowdsourcing data from the neighboring personal weather stations impacts forecasting performance. Finally, we implement our model in other cities such as Syracuse and Chicago to test the model's performance in different landforms and climate types.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

Crowdsourced Microfinance for Energy Efficiency in Underserved Communities

BlocPower’s mission is to provide access to energy efficiency financing for underserved communities across the United States. This project, “Crowdsourced Microfinance for Energy Efficiency in Underserved Communities,” is an extension of that goal and is grounded in the principles of providing engineering and financing services to those in need. The project is based on the creation of a BlocPower Marketplace as a central hub for connecting shovel-ready green buildings to institutional investors. This ‘connection’ entails using online crowdfunding to aggregate debt and equity capital from institutional investors to connect to customers (building owners) across various financial portfolios. BlocPower Marketplace is intended to bring social, environmental, and financial returns to investors while also decreasing investor risk by loaning out funds for energy installations in individual buildings. In detail, the intended benefits of crowdsourcing are two-sided. Firstly, for building owners, clean energy retrofit installations improve building operations, reduce utility costs, and reduce harmful impacts to their surrounding environment. Secondly, for institutional investors, they gain access to a new market of energy efficiency and are able to provide debt or equity capital with high financial returns. This gives investors the opportunity to create social and environmental impact in communities around the country as well. With this in mind, BlocPower designed the marketplace to specifically answer exploratory research questions with respect to the pricing of energy financing. Institutional investors typically charge high rates on project financing solutions in the energy space, particularly in low and middle-income communities, because of fears that required debt service will not be made. This makes access to energy capital exorbitantly difficult for those that need it the most. Through this project, BlocPower tested investor appetite to determine if crowdsourcing would lower prices and subsequently lower barriers to entry for underserved communities’ access to energy capital. BlocPower’s results in this project were extremely informative for the industry. The project demonstrates that the marketplace is a scalable tool to help overcome barriers to entry for small building owners in underserved communities to access energy efficiency financing, but that crowdfunding by itself does not necessarily lower interest rates and make energy efficiency projects feasible. For that, we need a repayment mechanism that lowers perceived risk. That mechanism is on bill repayment.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Crowdsourcing Global Perspectives in Ecology Using Social Media

Transparent, open, and reproducible research is still far from routine, and the full potential of open science has not yet been realized. Crowdsourcing–defined as the usage of a flexible open call to a heterogeneous group of individuals to recruit volunteers for a task –is an emerging scientific model that encourages larger and more outwardly transparent collaborations. While crowdsourcing, particularly through citizen- or community-based science, has been increasing over the last decade in ecological research, it remains infrequently used as a means of generating scientific knowledge in comparison to more traditional approaches. We explored a new implementation of crowdsourcing by using an open call on social media to assess its utility to address fundamental ecological questions. We specifically focused on pervasive challenges in predicting, mitigating, and understanding the consequences of disturbances. In this paper, we briefly review open science concepts and their benefits, and then focus on the new methods we used to generate a scientific publication. We share our approach, lessons learned, and potential pathways forward for expanding open science. Our model is based on the beliefs that social media can be a powerful tool for idea generation and that open collaborative writing processes can enhance scientific outcomes. We structured the project in five phases: (1) draft idea generation, (2) leadership team recruitment and project development, (3) open collaborator recruitment via social media, (4) iterative paper development, and (5) final editing, authorship assignment, and submission by the leadership team. We observed benefits including: facilitating connections between unusual networks of scientists, providing opportunities for early career and underrepresented groups of scientists, and rapid knowledge exchange that generated multidisciplinary ideas. We also identified areas for improvement, highlighting biases in the individuals that self-selected participation and acknowledging remaining barriers to contributing new or incompletely formed ideas into a public document. While shifting scientific paradigms to completely open science is a long-term process, our hope in publishing this work is to encourage others to build upon and improve our efforts in new and creative ways.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Secondary Crash Identification using Crowdsourced Waze User Reports

Secondary crashes are crashes that occur as a result of the nonrecurrent congestion originating from primary crashes, and always have a greater impact on safety and traffic than a single crash. A better understanding of secondary crashes would benefit traffic incident management, and this requires accurate identification of secondary crashes. This study explores using crowdsourced Waze user reports to identify secondary crashes. Here, a network-based clustering algorithm is proposed to extract the primary crash cluster, including all user reports originating from the primary crash, and any crash that occurred within the cluster would be a secondary crash. This method works as a filter to select accurate primary–secondary relationships, thus precisely identifying secondary crashes. A case study is performed with crashes occurring from June to December 2019 on a 30-mi stretch of I-40 in Knoxville, TN. A static threshold method (crash duration and 10 mi) was used to preselect the potential primary–secondary crash pairs, and 75 out of 708 crashes were identified as potential secondary crashes. Based on the preselected primary–secondary crash pairs, 17 secondary crashes were obtained with the proposed method and the results were compared with one of the commonly used methods, the speed contour plot method. Though the proposed method captured fewer secondary crashes, it did identify several secondary crashes that could not be observed with the speed contour plot method. The results showed the applicability of the method and the potential of crowdsourced Waze user reports in secondary crash identification.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2019

LLOCUS: Learning-based Localization Using crowdSourcing

We propose a novel learning-based system, LLOCUS, enabled by crowdsourcing on mobile devices, for localizing mobile transmitters with unknown transmit power. Our approach carefully tackles several challenges in learning and localizing using a crowdsourced system with both ends’ mobility, the ‘ends’ being the transmitters and the receivers. We decouple the problem of localizing a transmitter with unknown transmit power into two problems, 1) predicting the transmit power of an unknown transmitter, and 2) localizing a transmitter with known transmit power. LLOCUS first estimates the transmit power of the unknown transmitter and then scales the reported RSS values such that the unknown transmit power problem is transparent to the method of localization. We evaluate LLOCUS using four experiments, in different indoor and outdoor environments, with the size of the areas ranging from 225 to 5500 square meters. We find that LLOCUS reduces the localization error by 17-47%, over a number of non-learning methods. LLOCUS also estimates the transmit power of an unknown transmitter with average error in the range of 3-5 dB, across our experiments.

42 ENGINEERING↗
DOE OSTI2021

Galaxy Zoo: 3D – crowdsourced bar, spiral, and foreground star masks for MaNGA target galaxies

ABSTRACT The challenge of consistent identification of internal structure in galaxies – in particular disc galaxy components like spiral arms, bars, and bulges – has hindered our ability to study the physical impact of such structure across large samples. In this paper we present Galaxy Zoo: 3D (GZ:3D) a crowdsourcing project built on the Zooniverse platform that we used to create spatial pixel (spaxel) maps that identify galaxy centres, foreground stars, galactic bars, and spiral arms for 29 831 galaxies that were potential targets of the MaNGA survey (Mapping Nearby Galaxies at Apache Point Observatory, part of the fourth phase of the Sloan Digital Sky Surveys or SDSS-IV), including nearly all of the 10 010 galaxies ultimately observed. Our crowdsourced visual identification of asymmetric internal structures provides valuable insight on the evolutionary role of non-axisymmetric processes that is otherwise lost when MaNGA data cubes are azimuthally averaged. We present the publicly available GZ:3D catalogue alongside validation tests and example use cases. These data may in the future provide a useful training set for automated identification of spiral arm features. As an illustration, we use the spiral masks in a sample of 825 galaxies to measure the enhancement of star formation spatially linked to spiral arms, which we measure to be a factor of three over the background disc, and how this enhancement increases with radius.

Masters, Karen L. (ORCID:0000000308469578)↗
DOE OSTI2016

Predicting plant attractiveness to pollinators with passive crowdsourcing

Global concern regarding pollinator decline has intensified interest in enhancing pollinator resources in managed landscapes. These efforts frequently emphasize restoration or planting of flowering plants to provide pollen and nectar resources that are highly attractive to the desired pollinators. However, determining exactly which plant species should be used to enhance a landscape is difficult. Empirical screening of plants for such purposes is logistically daunting, but could be streamlined by crowdsourcing data to create lists of plants most probable to attract the desired pollinator taxa. People frequently photograph plants in bloom and the Internet has become a vast repository of such images. A proportion of these images also capture floral visitation by arthropods. Here, we test the hypothesis that the abundance of floral images containing identifiable pollinator and other beneficial insects is positively associated with the observed attractiveness of the same species in controlled field trials from previously published studies. We used Google Image searches to determine the correlation of pollinator visitation captured by photographs on the Internet relative to the attractiveness of the same species in common-garden field trials for 43 plant species. From the first 30 photographs, which successfully identified the plant, we recorded the number of Apis (managed honeybees), non-Apis (exclusively wild bees) and the number of bee-mimicking syrphid flies. We used these observations from search hits as well as bloom period (BP) as predictor variables in Generalized Linear Models (GLMs) for field-observed abundances of each of these groups. We found that non-Apis bees observed in controlled field trials were positively associated with observations of these taxa in Google Image searches (pseudo-R 2 of 0.668). Syrphid fly observations in the field were also associated with the frequency they were observed in images, but this relationship was weak. Apis bee observations were not associated with Internet images, but were slightly associated with BP. Our results suggest that passively crowdsourced image data can potentially be a useful screening tool to identify candidate plants for pollinator habitat restoration efforts directed at wild bee conservation. Increasing our understanding of the attractiveness of a greater diversity of plants increases the potential for more rapid and efficient research in creating pollinator-supportive landscapes.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2016

Crowdsourcing quality control for Dark Energy Survey images

We have developed a crowdsourcing web application for image quality control employed by the Dark Energy Survey. Dubbed the "DES exposure checker", it renders science-grade images directly to a web browser and allows users to mark problematic features from a set of predefined classes. Users can also generate custom labels and thus help identify previously unknown problem classes. User reports are fed back to hardware and software experts to help mitigate and eliminate recognized issues. We report on the implementation of the application and our experience with its over 100 users, the majority of which are professional or prospective astronomers but not data management experts. We discuss aspects of user training and engagement, and demonstrate how problem reports have been pivotal to rapidly correct artifacts which would likely have been too subtle or infrequent to be recognized otherwise. We conclude with a number of important lessons learned, suggest possible improvements, and recommend this collective exploratory approach for future astronomical surveys or other extensive data sets with a sufficiently large user base. We also release open-source code of the web application and host an online demo versionat http://des-exp-checker.pmelchior.net

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

Crowdsourcing the Frontier: Advancing Hybrid Physics‐ML Climate Simulation via a $\$$50,000 Kaggle Competition

Subgrid machine-learning (machine learning [ML]) parameterizations have the potential to introduce a new generation of climate models that incorporate the effects of higher-resolution physics without incurring the prohibitive computational cost associated with more explicit physics-based simulations. However, important issues, ranging from online instability to inconsistent online performance, have limited their operational use for long-term climate projections. To more rapidly drive progress in solving these issues, domain scientists and ML researchers opened up the offline aspect of this problem to the broader ML and data science community with the release of ClimSim, a NeurIPS Data sets and Benchmarks publication, and an associated Kaggle competition. This paper reports on the downstream results of the Kaggle competition by coupling emulators inspired by the winning teams' architectures to an interactive climate model (including full cloud microphysics, a regime historically prone to online instability) and systematically evaluating their online performance. Our results demonstrate that online stability in the low-resolution real-geography setting is reproducible across multiple diverse architectures, which we consider a key milestone. All tested architectures exhibit strikingly similar offline and online biases, though their responses to architecture-agnostic design choices (e.g., expanding the list of input variables) can differ significantly. Multiple Kaggle-inspired architectures achieve state-of-the-art results on certain metrics such as zonal mean bias patterns and global Root Mean Squared Error, indicating that crowdsourcing the essence of the offline problem is one path to improving online performance in hybrid physics-AI climate simulation.

Environmental sciences↗
DOE OSTI2016

Determining crystal structures through crowdsourcing and coursework

We show here that computer game players can build high-quality crystal structures. Introduction of a new feature into the computer game Foldit allows players to build and real-space refine structures into electron density maps. To assess the usefulness of this feature, we held a crystallographic model-building competition between trained crystallographers, undergraduate students, Foldit players and automatic model-building algorithms. After removal of disordered residues, a team of Foldit players achieved the most accurate structure. Analysing the target protein of the competition, YPL067C, uncovered a new family of histidine triad proteins apparently involved in the prevention of amyloid toxicity. From this study, we conclude that crystallographers can utilize crowdsourcing to interpret electron density information and to produce structure solutions of the highest quality.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Crowdsourcing biocuration: The Community Assessment of Community Annotation with Ontologies (CACAO)

Experimental data about gene functions curated from the primary literature have enormous value for research scientists in understanding biology. Using the Gene Ontology (GO), manual curation by experts has provided an important resource for studying gene function, especially within model organisms. Unprecedented expansion of the scientific literature and validation of the predicted proteins have increased both data value and the challenges of keeping pace. Capturing literature-based functional annotations is limited by the ability of biocurators to handle the massive and rapidly growing scientific literature. Within the community-oriented wiki framework for GO annotation called the Gene Ontology Normal Usage Tracking System (GONUTS), we describe an approach to expand biocuration through crowdsourcing with undergraduates. This multiplies the number of high-quality annotations in international databases, enriches our coverage of the literature on normal gene function, and pushes the field in new directions. From an intercollegiate competition judged by experienced biocurators, Community Assessment of Community Annotation with Ontologies (CACAO), we have contributed nearly 5,000 literature-based annotations. Many of those annotations are to organisms not currently well-represented within GO. Over a 10-year history, our community contributors have spurred changes to the ontology not traditionally covered by professional biocurators. The CACAO principle of relying on community members to participate in and shape the future of biocuration in GO is a powerful and scalable model used to promote the scientific enterprise. It also provides undergraduate students with a unique and enriching introduction to critical reading of primary literature and acquisition of marketable skills.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Ultrahigh-resolution mass spectrometry data associated with the manuscript “A functional microbiome catalog crowdsourced from North American rivers"

This data package is associated with the publication “A functional microbiome catalog crowdsourced from North American rivers” submitted to Nature (Borton et al., 2024); (https://www.biorxiv.org/content/10.1101/2023.07.22.550117v1). Predicting elemental cycles and maintaining water quality under increasing anthropogenic influence requires understanding the spatial drivers of river microbiomes. However, the unifying microbial determinants governing river biogeochemistry are hindered by a lack of genome-resolved functional insights and sampling across multiple rivers. Here we employed a community science effort to accelerate the sampling of river microbiomes to create the Genome Resolved Open Watersheds database (GROWdb). GROWdb is a publicly available resource that paves the way for watershed predictive modeling and microbiome-based management practices. This resource profiled the identity, distribution, function, and expression of thousands of microbial genomes across rivers covering 90% of United States watersheds. We identified the most cosmopolitan microbiome members, while also revealing local drivers of strain endemism across ecological dimensions. We provide the first evidence that microbial functional trait expression followed the tenets of the River Continuum Concept, suggesting the structure and function of river microbiomes is predictable. The Fourier-transform ion cyclotron resonance mass spectrometry (FTICR-MS) data were one of many different data types used in establishing the ecological dimensions along which different microbes were detected .This data package only contains the processed FTICR-MS data associated with this manuscript; all other data is accessible via Zenodo (https://zenodo.org/records/8173287), GitHub (https://github.com/jmikayla1991/Genome-Resolved-Open-Watersheds-database-GROWdb), KBase (https://doi.org/10.25982/109073.30/1895615), and NCBI via Bioproject PRJNA946291.This dataset consists of (1) a file-level metadata (flmd) file; (2) a data dictionary (dd) file; (3) a readme; (4) three Fourier-transform ion cyclotron resonance mass spectrometry (FTICR-MS) processed data files (a ‘data’ file containing peak-by-sample observations, a ‘mol’ file containing peak metadata, and a transformation profile containing transformation-by-sample observations). All files are .csv or .pdf.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

Crowdsourced reliable labeling of safety-rule violations on images of complex construction scenes for advanced vision-based workplace safety

Construction workplace hazard detection requires engineers to analyze scenes manually against many safety rules, which is time-consuming, labor-intensive, and error-prone. Computer vision algorithms are yet to achieve reliable discrimination of anomalous and benign object relations underpinning safety violation detections. Recently developed deep learning-based computer vision algorithms need tens of thousands of images, including labels of the safety rules violated, in order to train deep-learning networks for acquiring spatiotemporal reasoning capacity in complex workplaces. Such training processes need human experts to label images and indicate whether the relationship between the worker, resource, and equipment in the scenes violate spatiotemporal arrangement rules for safe and productive operations. False alarms in those manual labels (labeling no-violation images as having violations) can significantly mislead the machine learning process and result in computer vision models that produce inaccurate hazard detections. Compared with false alarms, another type of mislabels, false negatives (labeling images having violations as “no violations”), seem to have fewer impacts on the reliability of the trained computer vision models.

97 MATHEMATICS AND COMPUTING↗