Search NASA⌕ Search

SEARCH · Search NASA

Results for “data sources”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

Aurorasaurus Database of Real-Time, Crowd-Sourced Aurora Data for Space Weather Research

This technical report documents the details of Aurorasaurus citizen science data for the period spanning 2015 and 2016 as well as its routine data filtering protocols. Aurorasaurus citizen science data is a collection of auroral sightings submitted to the project via its website or apps and mined from social media. It is a robust data set and particularly abundant during strong geomagnetic storms when auroral precipitation models have the highest uncertainty. These data are offered to the scientific community for use through an openaccess database in its raw and scientific formats, each of which is described in detail in this technical report. Furthermore, by demonstrating its scientific utility, we aim to encourage its integration into auroral research.

Citizen science↗

Application of Low-Cost Fine Particulate Mass Monitors to Convert Satellite Aerosol Optical Depth Measurements to Surface Concentrations in North America and Africa

Low-cost particulate mass sensors provide opportunities to assess air quality at unprecedented spatial and temporal resolutions. Established traditional monitoring networks have limited spatial resolution and are simply absent in many major cities across sub-Saharan Africa (SSA). Satellites provide snapshots of regional air pollution but require ground-truthing. Low-cost monitors can supplement and extend data coverage from these sources worldwide, providing a better overall air quality picture. We investigate the utility of such a multi-source data integration approach using two case studies. First, in Pittsburgh, Pennsylvania, both traditional monitoring and dense low-cost sensor networks are compared with satellite aerosol optical depth (AOD) data from NASA's MODIS system, and a linear conversion factor is developed to convert AOD to surface fine particulate matter mass concentration (as PM2.5). With 10 or more ground monitors in Pittsburgh, there is a 2-fold reduction in surface PM2.5 estimation mean absolute error compared to using only a single ground monitor. Second, we assess the ability of combined regional-scale satellite retrievals and local-scale low-cost sensor measurements to improve surface PM2.5 estimation at several urban sites in SSA. In Rwanda, we find that combining local ground monitoring information with satellite data provides a 40 % improvement in surface PM2.5 estimation accuracy with respect to using low-cost ground monitoring data alone. A linear AOD-to-surface-PM2.5 conversion factor developed in Kigali, Rwanda, did not generalize well to other parts of SSA and varied seasonally for the same location, emphasizing the need for ongoing and localized ground-based monitoring, which can be facilitated by low-cost sensors. Overall, we find that combining ground-based low-cost sensor and satellite data, even without including additional meteorological or land use information, can improve and expand spatiotemporal air quality data coverage, especially in data-sparse regions.

AOD↗

The Impact of All-Sky Hyperspectral Infrared Radiance Assimilation on the Simulation and Forecast of Hurricane Sally in GEOS

Hyperspectral infrared (IR) radiance observations have been one of the major data sources assimilated in the data assimilation system over the last 20 years. However, observations peaking in the lower and mid-troposphere are underutilized in clear-sky radiance data assimilation as the quality control procedure removes a significant portion of cloud-affected observations from the assimilation. To include these cloud-affected observations, one approach is to assimilate them directly under all-sky conditions. The framework for all-sky assimilation of hyperspectral IR radiance observations has been developed in GEOS. The preliminary evaluation of simulated cloud-affected IR observations has been conducted, along with the corresponding sensitivities with respect to all hydrometeors. The symmetric cloud effect has been used as a cloud proxy in the observation error modeling, providing a balanced representation that mitigates the discrepancies between observations and model simulations for IR all-sky assimilation. The observation errors vary for different cloud conditions and have been modeled as cloud amount dependent and inter-channel correlated. Preliminary evaluation of model simulations for IR all-sky assimilation indicates that the simulated brightness temperature tends to have broader structures and lacks small-scale details. Excessive clouds generated by the model are also observed. To address this issue, Hurricane Sally (2020), which formed near the Bahamas, is used as a case study. Several cloud overlap schemes and cloud lookup tables (LUTs), along with other important aspects of all-sky IR radiance assimilation, will be evaluated and tested, aiming to improve the impact on Hurricane Sally's simulation and forecast.

Wei Gu↗

Open Research Challenges with Big Data - A Data-Scientist s Perspective

In this paper, we discuss data-driven discovery challenges of the Big Data era. We observe that recent innovations in being able to collect, access, organize, integrate, and query massive amounts of data from a wide variety of data sources have brought statistical data mining and machine learning under more scrutiny and evaluation for gleaning insights from the data than ever before. In that context, we pose and debate the question - Are data mining algorithms scaling with the ability to store and compute? If yes, how? If not, why not? We survey recent developments in the state-of-the-art to discuss emerging and outstanding challenges in the design and implementation of machine learning algorithms at scale. We leverage experience from real-world Big Data knowledge discovery projects across domains of national security, healthcare and manufacturing to suggest our efforts be focused along the following axes: (i) the data science challenge - designing scalable and flexible computational architectures for machine learning (beyond just data-retrieval); (ii) the science of data challenge the ability to understand characteristics of data before applying machine learning algorithms and tools; and (iii) the scalable predictive functions challenge the ability to construct, learn and infer with increasing sample size, dimensionality, and categories of labels. We conclude with a discussion of opportunities and directions for future research.

Big data↗

Secure optionally passive RFID tag or sensor with external power source and data logging

A secure optionally passive RFID tag or sensor system comprises a passive RFID tag having means for receiving radio signals from at least one base station and for transmitting radio signals to at least one base station, where the tag is capable of being powered exclusively by received radio energy, and an external power and data logging device having at least one battery and electronic circuitry including a digital memory configured for storing and recalling data. The external power and data logging device has a means for powering the tag, and also has a means.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Reconciling Scratch Space Consumption, Exposure, and Volatility to Achieve Timely Staging of Job Input Data

Innovative scientific applications and emerging dense data sources are creating a data deluge for high-end computing systems. Processing such large input data typically involves copying (or staging) onto the supercomputer's specialized high-speed storage, scratch space, for sustained high I/O throughput. The current practice of conservatively staging data as early as possible makes the data vulnerable to storage failures, which may entail re-staging and consequently reduced job throughput. To address this, we present a timely staging framework that uses a combination of job startup time predictions, user-specified intermediate nodes, and decentralized data delivery to coincide input data staging with job start-up. By delaying staging to when it is necessary, the exposure to failures and its effects can be reduced. Evaluation using both PlanetLab and simulations based on three years of Jaguar (No. 1 in Top500) job logs show as much as 85.9% reduction in staging times compared to direct transfers, 75.2% reduction in wait time on scratch, and 2.4% reduction in usage/hour.

99 GENERAL AND MISCELLANEOUS↗

COVID-19, An Exercise in Data Governance at Sandia National Laboratories

In April of 2020, Sandia National Laboratories had an urgent need to identify and manage the data that could be used to create mobile applications, models, reports, and visualizations to assist management in safely bringing the workforce onsite during the COVID-19 pandemic. Multiple divisions volunteered to design and build software solutions; meanwhile, requests for new data sources, including duplicate requests, were inundating Information Technology (IT) and data owners. The Enterprise Data Governance Team was assigned to resolve obtaining and accessing new sources of data in an accelerated timeframe. Through successful collaboration with multiple stakeholders and domain owners across Sandia, the Enterprise Data Governance Team rapidly developed a centralized data strategy and solution for use in safeguarding the Sandia workforce during the COVID-19 pandemic. This foundation enabled teams to successfully develop solutions, including reports for executives and management as well as the data for modeling and scientific analysis.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Provenance infrastructure for multimodal x-ray experiments and reproducible analysis

Exploring relationships between material structures and desired properties for material discovery and synthesis requires mining machine-readable databases and linking experimental data with sufficiently rich annotations. There are vast gaps both in these data sources and in the connections between them. We present a provenance-based data management and analysis framework that enables capturing, persisting, and reanalyzing experimental data from the BNL National Synchrotron Light Source II (NSLS-II). Our system leverages NSLS-II Bluesky for experimental data acquisition, captures analysis parameters, and enriches the search space with results from the scientific literature. We describe in detail the challenges and benefits of data-driven approaches when comparing computational and experimental data, such as the multiple possible scenarios of interest in often ill-posed inverse problems and missing data. Our infrastructure utilizes the NSLS-II event model and includes discovery mechanisms across multiple beamlines. A text and data mining module using natural language processing classification techniques extracts relevant information from the literature that is added to data sources. We describe a provenance acquisition and analysis rerun workflow based on a directed acyclic graph that keeps track of process and data provenance and allows to replay analysis.

97 MATHEMATICS AND COMPUTING↗

Chimera D-Series Gravitational Wave Emission Sourced from Neutrino Anisotropy

Gravitational wave data sourced from the time-dependent anisotropic neutrino emission, as well as the time-dependent fluid quadrupole motion, in the Chimera D-Series three-dimensional core collapse supernova simulations. Data from three models initiated from three different progenitors are presented: D9.6-3D, D15-3D, and D25-3D. Please see the README for more information about the data structure and progenitors.

79 ASTRONOMY AND ASTROPHYSICS↗

A means for utilizing ancillary information in multispectral classification

Attention is given to a method which makes it possible to incorporate information from ancillary data sources into the results of an existing classification of remotely sensed data. The method is based upon probabilistic label relaxation procedures which are used for imbedding spatial context data in image-labeling problems. The source of ancillary information is utilized in the form of a set of probabilities. These are introduced into a modified relaxation method called supervised relaxation labeling which, on application, develops a labeling for remotely sensed data that strikes a balance in consistency between spectral, spatial, and ancillary data sources of information. A description is given of results of a forestry classification in which accuracy is improved from 68% to 81% by incorporating topographic elevation in the outlined manner.

Richards, J. A.↗

Revamping Spacecraft Operational Intelligence

The EPOXI flight mission has been testing a new commercial system, Splunk, which employs data mining techniques to organize and present spacecraft telemetry data in a high-level manner. By abstracting away data-source specific details, Splunk unifies arbitrary data formats into one uniform system. This not only reduces the time and effort for retrieving relevant data, but it also increases operational visibility by allowing a spacecraft team to correlate data across many different sources. Splunk's scalable architecture coupled with its graphing modules also provide a solid toolset for generating data visualizations and building real-time applications such as browser-based telemetry displays.

Hwang, Victor↗

Leveraging Large Language Models for Real-World Data Evidence: A Framework for Automated Treatment Extraction and Data Harmonization

Background: The ability to comprehensively collect treatment information from cancer patient medical records would enable studies to evaluate real-world benefits and risks tied to specific treatments. Currently, it is difficult to system- atically collect high-quality treatment information because it is often stored in unstructured text. Manually extracting and standardizing drug and regimen data is time-intensive. Recent advances in large language models (LLMs) offer a potential solution for automated extraction of structured treatment information from clinical text. Objective: This study systematically evaluates the utility of four LLMs from the Llama family for automated extraction of oncology treatment information from clinical text. This information can guide researchers using cancer registry data to provide insights into cancer care and outcomes beyond clinical trials. Methods: Four instruction-tuned Llama models with varying parameter counts (1B, 3B, 8B, and 70B) were evaluated for their ability to extract treatment information from clinical documents. A unified oncology knowledge base integrating seven major public data sources was developed to standardize and normalize extracted entities—a critical step for harmonizing data from diverse sources. Extracted treatment data were compared against expert-annotated ground truth. Model performance was assessed using accuracy metrics (Precision, Recall, F1-Score) and opera- tional feasibility metrics, including processing speed and structural compliance of the output. Results: A strong positive correlation was observed between model size and extraction accuracy. F1-score improved from 0.609 for the 1B model to 0.710 (3B), 0.807 (8B), and 0.828 (70B). While larger models demonstrated superior accuracy and compliance, they incurred higher computational costs. The modest performance difference between 8B and 70B suggests diminishing returns with increasing model size. Conclusions: LLMs represent a viable technology for automating oncology treatment extraction. The 8B-parameter model emerged as a highly effective option, balancing high accuracy and computational efficiency. Selecting an appropriate LLM for deployment in cancer registries involves a trade-off between desired accuracy and available operational resources. Harmonizing extracted entities with the oncology knowledge base facilitates standardized integration into common data models, enhancing data quality for real-world evidence analyses.

artificial intelligence↗

Event Definition for the Automated Detection of Nuclear Proliferation Activities

In FY2020, Savannah River National Laboratory (SRNL) in collaboration with the Discovery Analytics Center (DAC) at Virginia Polytechnic Institute and State University (VT) began developing a demonstration prototype system that uses multiple machine learning and data analytic methods on largescale open data sources to identify new, developing, or undeclared nuclear programs. One of the most challenging aspects of applying machine learning techniques to such a problem is the high likelihood of extremely sparse data from disparate sources. To overcome this challenge, the current work will use a strategic combination of supervised, semi-supervised, and unsupervised learning techniques to ingest and fuse data streams to make a forecast of nuclear activities in a targeted geospatial location. Identifying potential data sources and training supervised learning algorithms is dependent upon the development of a robust foundation of targeted event domains that fundamentally define the nuclear activities of interest. This report documents the definition of a hierarchical structure for both nuclear activity and event domains that will be used to guide the research team in development or use of existing semantic dictionaries that are instrumental to searching, parsing, and categorizing events for the forecasting system’s use.

97 MATHEMATICS AND COMPUTING↗