Eigenanalysis of Space Weather Climate Data: Discovering Structure within Large Multivariate Data Sets
No abstract available
Krause, Linda Habash↗
SEARCH · Search NASA
Results for “data discover”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
EDXplorer: A Utility for APA Analysis
The automated particle analysis (APA) method of scanning electron microscopy (SEM) energy dispersive X-ray spectroscopy (EDS/EDX) is a useful tool for analyzing the elemental and morphological data of particulate samples. Often, such datasets have many thousands of particles, and it can be difficult to sift through the data to find meaningful trends. EDXplorer is a software utility for processing APA data. They enable the user to easily load data and determine the important components and aspects of the datasets using a powerful and versatile library of data plotting functions, mainly centered around scatter plots and histograms. These programs are intended to fill a void in the data processing of data from certain instruments, where often the user must rely on their own code or other software that is not user-friendly. EDXplorer is intended as a general plotting utility for browsing through data and discovering data trends.
Moseley, Duncan [ORNL] (ORCID:0000000343518347)↗
Exploiting Commonly-Reported Age-Hardening Data and Discovering Systematics Across Metallic Alloys and Alloy Systems to Identify Corrosion’s Most Influential Factors
This work explored ways to data mine legacy literature and predict solid-solid precipitation in support of simpler assessment of corrosion propensity in metallic alloys. Of interest was locating the peak age watershed (maximum hardness or strength), beyond which lies the regime termed “overaging.” A diligent search of literature and reference books, discussions with SMEs, and application of various algorithms showed that none of the premises going in held up to scrutiny.
36 MATERIALS SCIENCE↗
Using ADOPT Algorithm and Operational Data to Discover Precursors to Aviation Adverse Events
The US National Airspace System (NAS) is making its transition to the NextGen system and assuring safety is one of the top priorities in NextGen. At present, safety is managed reactively (correct after occurrence of an unsafe event). While this strategy works for current operations, it may soon become ineffective for future airspace designs and high density operations. There is a need for proactive management of safety risks by identifying hidden and "unknown" risks and evaluating the impacts on future operations. To this end, NASA Ames has developed data mining algorithms that finds anomalies and precursors (high-risk states) to safety issues in the NAS. In this paper, we describe a recently developed algorithm called ADOPT that analyzes large volumes of data and automatically identifies precursors from real world data. Precursors help in detecting safety risks early so that the operator can mitigate the risk in time. In addition, precursors also help identify causal factors and help predict the safety incident. The ADOPT algorithm scales well to large data sets and to multidimensional time series, reduce analyst time significantly, quantify multiple safety risks giving a holistic view of safety among other benefits. This paper details the algorithm and includes several case studies to demonstrate its application to discover the "known" and "unknown" safety precursors in aviation operation.
aviation safet↗
An ultra-faint galaxy candidate discovered in early data from the magellanic satellites survey
We report a new ultra-faint stellar system found in Dark Energy Camera data from the first observing run of the Magellanic Satellites Survey (MagLiteS). MagLiteS J0644−5953 (Pictor II or Pic II) is a low surface brightness (μ=28.5{sub −1}{sup +1} mag arcsec{sup −2} within its half-light radius) resolved overdensity of old and metal-poor stars located at a heliocentric distance of 45{sub −4}{sup +5} kpc. The physical size (r{sub 1/2}=46{sub −11}{sup +15} pc ) and low luminosity (M{sub V}=−3.2{sub −0.5}{sup +0.4} mag ) of this satellite are consistent with the locus of spectroscopically confirmed ultra-faint galaxies. MagLiteS J0644−5953 (Pic II) is located 11.3{sub −0.9}{sup +3.1} kpc from the Large Magellanic Cloud (LMC), and comparisons with simulation results in the literature suggest that this satellite was likely accreted with the LMC. The close proximity of MagLiteS J0644−5953 (Pic II) to the LMC also makes it the most likely ultra-faint galaxy candidate to still be gravitationally bound to the LMC.
79 ASTRONOMY AND ASTROPHYSICS↗
Metadata Authoring with Versatility and Extensibility
NASA's Global Change Master Directory (GCMD) assists the scientific community in the discovery of and linkage to Earth science data sets and related services. The GCMD holds over 13,800 data set descriptions in Directory Interchange Format (DIF) and 700 data service descriptions in Service Entry Resource Format (SERF), encompassing the disciplines of geology, hydrology, oceanography, meteorology, and ecology. Data descriptions also contain geographic coverage information and direct links to the data, thus allowing researchers to discover data pertaining to a geographic location of interest, then quickly acquire those data. The GCMD strives to be the preferred data locator for world-wide directory-level metadata. In this vein, scientists and data providers must have access to intuitive and efficient metadata authoring tools. Existing GCMD tools are attracting widespread usage; however, a need for tools that are portable, customizable and versatile still exists. With tool usage directly influencing metadata population, it has become apparent that new tools are needed to fill these voids. As a result, the GCMD has released a new authoring tool allowing for both web-based and stand-alone authoring of descriptions. Furthermore, this tool incorporates the ability to plug-and-play the metadata format of choice, offering users options of DIF, SERF, FGDC, ISO or any other defined standard. Allowing data holders to work with their preferred format, as well as an option of a stand-alone application or web-based environment, docBUlLDER will assist the scientific community in efficiently creating quality data and services metadata.
Pollack, Janine↗
Leveraging Application Data Constraints to Optimize Database-Backed Web Applications
Exploiting the relationships among data is a classical query optimization technique. As persistent data is increasingly being created and maintained programmatically, prior work that infers data relationships from data statistics misses an important opportunity. We present Coco, the first tool that identifies data relationships by analyzing database-backed applications. Once identified, Coco leverages the constraints to optimize the application's physical design and query execution. Instead of developing a fixed set of predefined rewriting rules, Coco employs an enumerate-test-verify technique to automatically exploit the discovered data constraints to improve query execution. Each resulting rewrite is provably equivalent to the original query. Using 14 real-world web applications, our experiments show that Coco can discover numerous data constraints from code analysis and improve real-world application performance significantly.
Computer Science↗
Data Transferability and Collection Consistency in Marine Renewable Energy
Concerns about the potential effects of tidal turbines and wave energy devices on the marine environment continue to slow siting and permitting/consenting of single devices and arrays worldwide. While research studies and early results from post-installation monitoring over the past decade have informed interactions between marine renewable energy (MRE) devices, marine animals, and habitats, regulators still demonstrate considerable reluctance to accelerate the permitting/consenting process for devices and arrays. Furthermore, the MRE industry is struggling with the high costs of baseline assessments and post-installation monitoring, as well as long timelines for obtaining permits, which leads to uncertainty and risk related to project financing. Regulators require assessment and monitoring information to allow them to carry out the necessary analyses to describe, permit/consent, and manage the environmental risks associated with new MRE technologies and new uses of ocean space. One way to reduce risks to the industry and the environment and to allow for acceleration of the permitting/consenting process could be to transfer learning, analyses, and data sets from one country to another, among projects, and across jurisdictional boundaries. In addition, data are collected around early-stage MRE devices using many different methods, instruments, and measurement scales. If similar parameters and accessible methods of data collection were used for baseline assessments and post-installation monitoring around all early-stage devices and MRE developments, the results would be more readily comparable. This comparability would lead to a decrease in scientific uncertainty and support a common understanding of the risk of MRE devices to the marine environment. This in turn would facilitate more efficient and shorter permitting/consenting processes, which would decrease the financial risk for MRE project development. As a means of addressing the concept of transferring data (information, learning, analyses, and data sets) among projects and collecting data consistently, Annex IV has developed a data transferability process that has been socialized with the MRE community, which includes regulators, industry, developers, consultants, and researchers. The data transferability process consists of five components: 1. A Data Transferability Framework brings together data sets in an organized fashion, compares the applicability of each data set for use on other projects, and guides the process of data transfer 2. A Data Collection Consistency Table provides preferred measurement methods or processes, reporting units, and the most common methods of analysis or interpretation and use of data 3. A Monitoring Data Sets Discoverability Matrix allows a practitioner to discover data sets based on the approach presented in the Framework 4. Best Management Practices (BMPs) include five BMPs related to data transferability and collection consistency 5. An Implementation Plan presents an approach for implementing and applying the data transferability process. This report documents the background and development of the data transferability process and associated components and summarizes the next steps needed to successfully implement and apply the data transferability process.
16 TIDAL AND WAVE POWER↗
Empowering Geothermal Research: The Geothermal Data Repository's New AI Research Assistant: Preprint
The Department of Energy's (DOE) Geothermal Data Repository (GDR) team has integrated a Large Language Model (LLM) with the metadata and supporting documents associated with GDR datasets to create an Artificially Intelligent (AI) research assistant. By leveraging work done to make GDR metadata machine-readable and an open-source LLM integration model called the Energy Language Model, developed by the National Renewable Energy Laboratory, AskGDR serves as a virtual research assistant to GDR users. It provides answers to a variety of user-provided questions using natural language processing and generative machine learning. Users can get answers to questions about specific datasets, including inquiries about the equipment, assumptions and methodologies used in the origination of the data; or more abstract questions, such as the applicability of data to specific research fields. AskGDR improves the discoverability of geothermal data by helping guide users to datasets beyond simple keyword searches. It enables users to find data based on properties of the data, discover information contained within supporting documents, and explore data from projects related to their research objectives.
access↗
Empowering Geothermal Research: The Geothermal Data Repository's New AI Research Assistant
The Department of Energy's (DOE) Geothermal Data Repository (GDR) team has integrated a Large Language Model (LLM) with the metadata and supporting documents associated with GDR datasets to create an Artificially Intelligent (AI) research assistant. By leveraging work done to make GDR metadata machine-readable and an open-source LLM integration model called the Energy Language Model, developed by the National Renewable Energy Laboratory, AskGDR serves as a virtual research assistant to GDR users. It provides answers to a variety of user-provided questions using natural language processing and generative machine learning. Users can get answers to questions about specific datasets, including inquiries about the equipment, assumptions and methodologies used in the origination of the data; or more abstract questions, such as the applicability of data to specific research fields. AskGDR improves the discoverability of geothermal data by helping guide users to datasets beyond simple keyword searches. It enables users to find data based on properties of the data, discover information contained within supporting documents, and explore data from projects related to their research objectives. This paper will outline the development, integration, output, and efficacy of the AskGDR LLM, including adherence to scientific rigor through improvements designed to increase the accuracy of generated answers, avoid speculation, and provide proper references for all resources used.
access↗
In Interactive, Web-Based Approach to Metadata Authoring
NASA's Global Change Master Directory (GCMD) serves a growing number of users by assisting the scientific community in the discovery of and linkage to Earth science data sets and related services. The GCMD holds over 8000 data set descriptions in Directory Interchange Format (DIF) and 200 data service descriptions in Service Entry Resource Format (SERF), encompassing the disciplines of geology, hydrology, oceanography, meteorology, and ecology. Data descriptions also contain geographic coverage information, thus allowing researchers to discover data pertaining to a particular geographic location, as well as subject of interest. The GCMD strives to be the preeminent data locator for world-wide directory level metadata. In this vein, scientists and data providers must have access to intuitive and efficient metadata authoring tools. Existing GCMD tools are not currently attracting. widespread usage. With usage being the prime indicator of utility, it has become apparent that current tools must be improved. As a result, the GCMD has released a new suite of web-based authoring tools that enable a user to create new data and service entries, as well as modify existing data entries. With these tools, a more interactive approach to metadata authoring is taken, as they feature a visual "checklist" of data/service fields that automatically update when a field is completed. In this way, the user can quickly gauge which of the required and optional fields have not been populated. With the release of these tools, the Earth science community will be further assisted in efficiently creating quality data and services metadata. Keywords: metadata, Earth science, metadata authoring tools
Pollack, Janine↗
Data and Reasoning Fabric (DRF)
There are multiple opportunities to utilize Data & Reasoning Fabric technologies in wildfire response: (1) Supporting autonomous agents to discover data sources and transact with systems, (2) Fusing data in real-time, and (3) Supporting complex operations in a dynamic real-time environment .
Aeronautics↗
Semi-Analytical Hierarchical Bayesian Inference of Nonlinear Model Structure in Stochastic Dynamics: Applied to Compartmental Models of Infectious Diseases
A Bayesian computational framework for parsimonious inference in stochastic nonlinear dynamical systems is presented. This framework enables the concurrent estimation of system states, time-varying parameters, time-invariant parameters, and the optimal sparsity structure of the model parameters. Because differential equation-based models are often simplified mechanistic or phenomenological representations, robust inference from noisy measurement data requires explicit treatment of model error and uncertainty. Model error and time-varying parameters can be represented as random processes, enabling inference while making minimal assumptions about the underlying sources of discrepancy and variability. Adopting stochastic differential equation representations affords the model significant flexibility, but can also render it susceptible to overfitting during statistical inversion, where the inferred model may track noise rather than the underlying signal. To alleviate the effects of overfitting and to enable the discovery of the optimal sparse representation of the time-invariant parameters, a Bayesian sparse learning algorithm is embedded within the framework. This sparse learning framework adopts an approximate hierarchical Bayesian setting defined by a series of semi-analytical expressions. The model structure inference framework is validated using a stochastic compartmental model for tracking and forecasting active cases of an infectious disease. Compartmental models describe population-level infectious disease dynamics through interactions among population fractions grouped by disease state. Mathematically, such models consist of a system of coupled ordinary differential equations. This example adopts an expressive compartmental model that includes multiple possible interactions between disease states, motivated by early uncertainty surrounding COVID-19 reinfection dynamics and their implications for long-term epidemic forecasting. The sparse learning exercise permits the inference of a priori unknown epidemiological dynamics from simulated public health data, discovering the nested compartmental model that optimizes the trade-off between average data-fit and model complexity. It is shown that inducing sparsity among the model parameters eliminates redundant interactions between compartments, equivalently revealing the optimal coupling structure between differential equations.
97 MATHEMATICS AND COMPUTING↗
Exploring and Visualizing A-Train Instrument Data
The succession of US and international satellites that follow each other in close succession, known as the A-Train, affords an opportunity to atmospheric researchers that no single platform could provide: Increasing the number of observations at any given geographic location.. . a more complete "virtual science platform". However, vertically and horizontally, co-registering and regridding datasets from independently developed missions, Aqua, Calipso, Cloudsat, Parasol, and Aura, so that they can be inter-compared can be daunting to some, and may be repeated by many. Scientists will individually spend much of their time and resources acquiring A-Train datasets of interest residing at various locations, developing algorithms to match up and graph datasets along the A-Train track, and search through large amounts of data for areas and/or phenomena of interest. The aggregate amount of effort that can be expended on repeating pre-science tasks could climb into the tens of millions of dollars. The goal of the A-Train Data Depot (ATDD) is to enable free movement of remotely located A-Train data so that they are combined to create a consolidated vertical view of the Earth's Atmosphere along the A-Train tracks. The innovative approach of analyzing and visualizing atmospheric profiles along the platforms track (i.e., time) is accomplished by through the ATDDs Giovanni data analysis and visualization tool. Giovanni brings together data from Aqua (MODIS, AIRS, AMSR-E), Cloudsat (cloud profiling radar) and Calipso (CALIOP, IIR), as well as the Aura (OMI, MLS, HIRDLS, TES) to create a consolidated vertical view of the Earth's Atmosphere along the A-Train tracks. This easy to learn and use exploration tool will allow users to create vertical profiles of any desired A-Train dataset, for any given time of choice. This presentation shows the power of Giovanni by describing and illustrating how this tool facilitates and aids A-Train science and research. A web based display system Giovanni provides users with the capability of creating co-located profile images of temperature and humidity data from the MODIS, MLS and AIRS instruments for a user specified time and spatial area. In addition, Cloud and Aerosol profiles may also be displayed for the Cloudsat and Caliop instruments. The ability to modify horizontal and vertical axis range, data range and dynamic color range is also provided. Two dimensional strip plots of MODIS, AIRS, OM1 and POLDER parameters, co-located along the Cloudsat reference track, can also be plotted along with the Cloudsat cloud profiling data. Center swath pixels for the same parameters can also be shown as line plots overlaying the Cloudsat or Calipso profile images. Images and subsetted data produced in each analysis run may be downloaded. Users truly can explore and discover data specific to their needs prior to ever transferring data to their analysis tools.
Kempler, S.↗
Restructuring Big Data to Improve Data Access and Performance in Analytic Services Making Research More Efficient for the Study of Extreme Weather Events and Application User Communities
By developing and enhancing various services and tools, the GES DISC provides users with the capability to access and visualize data, and to make comparisons of data from multiple sensor and models via a number of cross-discipline projects. Discovering Data via Faceted Web Interface Web interface to data products and services Search and Download mechanisms Dataset Landing Pages Accessing Data through Interoperable Services: GDS – GrADS Data Server OPeNDAP - Open-source Project for a Network Data Access Protocol WMS – OGC service GIS connector – allowing IS tools to access data easier (coming soon) HTTPS -- direct online access Downloading Data Basics: Subset and egridding Service – Parameter, Spatial, Time, Vertical, Mean averaging, format conversion, and regridding for L3/L4 gridded data Swath Data Subsetter – Parameter, spatial subset of L2 /L1 data. Visualizing Data Online: Giovanni –Visualization and Analysis L3/L4 gridded data AIRS NRT Viewer – AIRS near-real-time DQVis – L2 data quality visualization
data cube↗
A Knowledge Graph Approach for the Secondary Use of Cancer Registry Data
Population-based central cancer registries collect valuable structured and unstructured data primarily for cancer surveillance and research, enhancing insights into clinical features associated with cancer occurrence, cancer treatment, and cancer outcomes to guide interventions which reduce the cancer burden. Cancer registries primarily collect data on (1) cancer type (case or tumor); (2) patient demographics such as age, gender, and residential address at time of diagnosis; (3) planned first course of treatment; and (4) date of last contact, vital status, and cause of death. Cancer registry data is dynamic, structured data, which is extracted from many unstructured sources such as electronic healthcare records, and consolidated for reporting and other purposes. While available advanced analytic tools such as SEER*Stat have the ability to build SAS queries, we, however, explore an innovative knowledge graph approach to organizing cancer registry data for advanced analytics and visualization, which has unique advantages over approaches of existing tools. This innovative knowledge graph approach semantically enriches the data and easily enables linkage with third-party data, which can better explain variation in outcomes. We have developed a prototype knowledge graph based on data from the Louisiana Tumor Registry and other publicly available datasets including Behavioral Risk Factor Surveillance System, Clinical Trials, DBpedia, GeoNames, Rural-Urban Continuum Codes, and Semantic MEDLINE. The resource description framework (RDF) data model was selected to represent our knowledge graph, which contains more than 25 billion triples and is ~4TB in storage size. To exhibit the benefits of the knowledge graph approach, we used scenario specific queries, which find the relationships between cancer treatment sequences and outcomes. To illustrate its ease of use in iterative analysis, the knowledge graph was linked to external datasets for performing complex queries across multiple datasets. In addition, we used knowledge graphs to identify data discrepancies and to handle schema changes. Finally, we visualized the knowledge graph to discover data patterns. Our results demonstrate this graph-based solution enables cancer researchers to execute complex queries and more easily perform iterative analyses to improve understanding of cancer registry data. In the future, we would like to use high-performance computing (HPC) resources for faster-generating hypotheses with clinical potential from our knowledge graph.
Hasan, S M Shamimul↗
Utilization of Geographic Information System in Lunar Mapping
Substantial digital remote sensing, lunar orbital photography, Earth-based remote sensing, and mapping of a variety of surficial lunar phenomena have occurred since the advent of the Space Age. This has led to a bewildering and quite disparate collection of archival sources insofar as this digital data and its cartographic representation can be found within many countries of the world. The importance of this mapping program in support of human expansion onto our nearest planetary neighbor has been recognized. A series of small scale maps of the Moon at 1 km to 1 cm, done with the support of Geographic Information System (GIS), would serve decision makers well in the process of accessing the development of manned occupance of the Moon. Maps and the data that they are derived from are the primary way in which people explore new environments and use previously discovered data to increase the bounties of any exploration. The inherent advantage of GIS is that it would allow immediate online access on the Moon of topographically represented data with analysis either on site or from Earth.
Mardon, A. A.↗
Resource Discovery for Extreme Scale Collaboration (RDESC) Final Report - RPI/TWC - Year 3
The amount of data produced in the practice of science is growing rapidly. Despite the accumulation and demand for scientific data, relatively little is actually made available for the broader scientific community. We surmise that the root of the problem is the perceived difficulty to electronically publish scientific data and associated metadata in a way that makes it discoverable. We propose to exploit Semantic Web technologies and practices to make (meta)data discoverable and easy to publish. We share our experiences in curating metadata to illustrate both the flexibility of our approach and the pain of discovering data in the current research environment. We also make recommendations by concrete example of how data publishers can provide their (meta)data by adding some limited, additional markup to HTML pages on the Web. With little additional effort from data publishers, the difficulty of data discovery/access/sharing can be greatly reduced and the impact of research data greatly enhanced.
96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗