Search NASA⌕ Search

SEARCH · Search NASA

Results for “query processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Machine Learning and Social Media to Mine and Disseminate Big Scientific Data

One of the challenges in supplying the communities with wider access to scientific databases is the need for knowledge of database languages like Structured Query Language (SQL). Although the SQL language has been published in many forms, not everybody is able to write SQL queries. Another challenge is that it might not be practical to make the public aware of the structure of databases. There is a need for novice users to query relational databases using their natural language. To solve this problem, many natural language interfaces to structured databases have been developed. The goal is to provide a more intuitive method for generating database queries and delivering responses. Through social media, which makes it possible to interact with a wide section of the population, and with the help of natural language processing, researchers at the Atmospheric Radiation Measurement (ARM) Data Center at Oak Ridge National Laboratory (ORNL) have developed a concept to enable easy search and retrieval of data from several environmental data centers for the scientific community through social media.Using a machine learning framework that maps natural language text to thousands of datasets, instruments, variables, and data streams, the prototype system would allow users to request data through Twitter and receive a link (via tweet) to applicable data results on the project's search catalog tailored to their key words. This automated identification of relevant data from various petascale archives at ORNL could increase convenience, access, and use of the project's data by the broader community. In this paper we discuss how some data-intensive projects at ORNL are using innovative ways to help in data discovery.

Devarakonda, Ranjeet↗
DOE OSTI2024

DQL (Django Query Logger) [SWR-24-96]

The Django Query Logger (DQL) is a tool that allows developers of Django web applications to stream the raw database queries that are being executed in real-time from any Django application for review, profiling, filtering, formatting, and analysis. Reference herein to any specific commercial products, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or Alliance for Sustainable Energy, LLC. The views and opinions of authors expressed in the available or referenced documents do not necessarily state or reflect those of the United States Government or Alliance.

Swindler, Alexander↗
DOE OSTI2024

The Integration and Mapping of an Open-Source National Well Resource to Inform Geologic Carbon Storage Site Selection and Risk Prevention: The CO2-Locate Database

Geologic carbon storage (GCS) offers a way to capture and permanently store CO₂ from fossil fuel operations in underground geologic structures, aiding in the transition to a carbon-neutral energy economy. However, CO₂ injection sites can experience gas leakage through existing wells that penetrate storage reservoirs, making knowledge of well locations and characteristics crucial for permitting, infrastructure reusability, and risk assessment in GCS. Currently, public wellbore data from state, federal, and tribal entities are inconsistent and fragmented, with gaps and redundancies. To address this, the National Energy Technology Laboratory (NETL) developed CO2-Locate, an open-source, geospatial database and online application. CO2-Locate integrates over 50 data sources from federal, state, and tribal entities, creating a standardized national well database. Funded by the Bipartisan Infrastructure Law, the database is publicly available through the Energy Data eXchange (EDX) and viewable via the CO2-Locate web mapping application. This tool allows users to query, filter, and visualize well data to support GCS planning, permitting, and risk assessments. This presentation covers the methods used to create CO2-Locate, including data acquisition, processing, attribute mapping, and integration, much of which is automated for future updates. The web mapping application and its role in GCS site selection will also be discussed.

Tetteh, Daniel A.↗
NASA NTRS2004

System, method and apparatus for generating phrases from a database

A phrase generation is a method of generating sequences of terms, such as phrases, that may occur within a database of subsets containing sequences of terms, such as text. A database is provided and a relational model of the database is created. A query is then input. The query includes a term or a sequence of terms or multiple individual terms or multiple sequences of terms or combinations thereof. Next, several sequences of terms that are contextually related to the query are assembled from contextual relations in the model of the database. The sequences of terms are then sorted and output. Phrase generation can also be an iterative process used to produce sequences of terms from a relational model of a database.

McGreevy, Michael W.↗
DOE OSTI2020

Building Interfaces with Echo Using R

The following report details the functionality that was developed to directly query the MongoDB and read in echo records from within R, and the ability to create a cinema database directly from echo records. Both of these efforts used the data collected under the MINOS project for development. The nuclear facility under consideration is the High-Flux Isotope Reactor (HFIR) and co-located Radiochemical Engineering Development Center (REDC) at Oak Ridge National Laboratory in Oak Ridge, TN. HFIR is an 85 MW research reactor and is used primarily for production of a medical radioisotopes, material irradiation experiments, neutron activation, and neutron scattering. Targets for the reactor are fabricated and processed and dissolved at REDC, in addition to other glove-box and hot-cell type activities. An overhead view of the facility is provided. The rest of the report is organized as follows: Section 2 describes how to query the MongoDB and load in echo records from within R, with background information on the database type, MongoDB, and the file format of echo records, hdf5, as well as lessons learned during the development phase. Section 3 details the functionality and usage of creating a cinema database directly from echo records within R. A series of appendixes display all the R code (Section 5), some example scripts where the code is used (Section 6), and useful links for further exploration (Section 7).

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2019

A SQL to C compiler in 500 lines of code

Abstract We present the design and implementation of a SQL query processor that outperforms existing database systems and is written in just about 500 lines of Scala code – a convincing case study that high-level functional programming can handily beat C for systems-level programming where the last drop of performance matters. The key enabler is a shift in perspective toward generative programming. The core of the query engine is an interpreter for relational-algebra operations, written in Scala. Using the open-source lightweight modular staging framework, we turn this interpreter into a query compiler with very low effort. To do so, we capitalize on an old and widely known result from partial evaluation: the first Futamura projection, which states that a process that can specialize an interpreter to any given input program is equivalent to a compiler. In this context, we discuss lightweight modular staging programming patterns such as mixed-stage data structures (e.g., data records with static schema and dynamic field components) and techniques to generate low-level C code, including specialized data structures and data loading primitives.

Computer Science↗
DOE OSTI2023

A General Framework for Progressive Data Compression and Retrieval

In scientific simulations, observations, and experiments, the transfer of data to and from disk and across networks has become a major bottleneck for data analysis and visualization. Compression techniques have been employed to tackle this challenge, but traditional lossy methods often demand conservative error tolerances to meet the numerical accuracy requirements of both anticipated and unknown data analysis tasks. Progressive data compression and retrieval has emerged as a promising solution, where each analysis task dictates its own accuracy needs. However, few analysis algorithms inherently support progressive data processing, and adapting compression techniques, file formats, client/server frameworks, and APIs to support progressivity can be challenging. Here, this paper presents a framework that enables progressive-precision data queries for any data compressor or numerical representation. Our strategy hinges on a multi-component representation that successively reduces the error between the original and compressed field, allowing each field in the progressive sequence to be expressed as a partial sum of components. We have implemented this approach with four established scientific data compressors and assessed its effectiveness using real-world data sets from the SDRBench collection. The results show that our framework competes in accuracy with the standalone compressors it is based upon. Additionally, (de)compression time is proportional to the number of components requested by the user. Finally, our framework allows for fully lossless compression using lossy compressors when a sufficient number of components are employed.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Redox Biochemistry of Energy Conservation in Methanogens and their Syntrophic Partners

The primary objective of this project was to uncover previously unknown modes by which living organisms harvested energy. Specifically, it queried microbial systems, which were once thought to not be able to utilize oxygen. Through the use of cutting-edge approaches, we were able to identify novel mechanistic processes that allow organisms to thrive under varying environmental conditions. This constitutes a paradigm shift in the field. Our work has opened new vistas for expanding the horizons of energy conservation in biological systems, as well as shed light on green energy innovations.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2005

PlanWorks: A Debugging Environment for Constraint Based Planning Systems

Numerous planning and scheduling systems employ underlying constraint reasoning systems. Debugging such systems involves the search for errors in model rules, constraint reasoning algorithms, search heuristics, and the problem instance (initial state and goals). In order to effectively find such problems, users must see why each state or action is in a plan by tracking causal chains back to part of the initial problem instance. They must be able to visualize complex relationships among many different entities and distinguish between those entities easily. For example, a variable can be in the scope of several constraints, as well as part of a state or activity in a plan; the activity can arise as a consequence of another activity and a model rule. Finally, they must be able to track each logical inference made during planning. We have developed PlanWorks, a comprehensive system for debugging constraint-based planning and scheduling systems. PlanWorks assumes a strong transaction model of the entire planning process, including adding and removing parts of the constraint network, variable assignment, and constraint propagation. A planner logs all transactions to a relational database that is tailored to support queries for of specialized views to display different forms of data (e.g. constraints, activities, resources, and causal links). PlanWorks was specifically developed for the Extensible Universal Remote Operations Planning Architecture (EUROPA(sub 2)) developed at NASA, but the underlying principles behind PlanWorks make it useful for many constraint-based planning systems. The paper is organized as follows. We first describe some fundamentals of EUROPA(sub 2). We then describe PlanWorks' principal components. We then discuss each component in detail, and then describe inter-component navigation features. We close with a discussion of how PlanWorks is used to find model flaws.

Daley, Patrick↗
NASA NTRS2006

Precipitation links (PrecipLinks) - a prototype directory for precipitation information

This poster describes a web directory of research oriented precipitation links. In this era of sophisticated search engines and web agents, it might seem counterproductive to establish such a directory of links. However, entering precipitation into a search engine like google will yield over one million hits. To further exacerbate this situation many of the returned links are dead, duplicates of other links, incomplete, or only marginally related to research precipitation or even the broader precipitation area. Sometimes connecting the linked URL causes the browser to lose context and not be able to get back to the original page. Even using more sophisticated search engines query parameters or agents while reducing the overall return doesn't eliminate all of the other issues listed. As part of the development of the measurement-based Precipitation Processing System (PPS) that will support Tropical Rainfall Measuring Mission (TRMM) version 7 reprocessing and the Global Precipitation Measurement (GPM) mission a precipitation links (PrecipLinks) facility is being developed. PrecipLinks is intended to share locations of other sites that contain information or data pertaining to precipitation research. Potential contributors can log-on to the PrecipLinks website and register their site for inclusion in the directory. The price for inclusion is the requirement to place a link back to PrecipLinks on the webpage that is registered. This ensures that users will be able to easily get back to PrecipLinks regardless of any context issues that browsers might have. Perhaps more importantly users while visiting one site that they know can be referred to a location that has many others sites with which they might not be familiar. PrecipLinks is designed to have a very flat structure. This poster summarizes these categories (information, data, services) and the reasons for their selection. Providers may register multiple pages to which they wish to direct users. However, each page may be attached to only one of these categories. Each page to which they refer users will also have a return link to PrecipLinks. The poster describes the operation of the system both the automated and the human processes. It also provides images for the various steps in the registration and use.

Velanthapillia, Balendran↗
NASA NTRS1992

Integrating a local database into the StarView distributed user interface

A distributed user interface to the Space Telescope Data Archive and Distribution Service (DADS) known as StarView is being developed. The DADS architecture consists of the data archive as well as a relational database catalog describing the archive. StarView is a client/server system in which the user interface is the front-end client to the DADS catalog and archive servers. Users query the DADS catalog from the StarView interface. Query commands are transmitted via a network and evaluated by the database. The results are returned via the network and are displayed on StarView forms. Based on the results, users decide which data sets to retrieve from the DADS archive. Archive requests are packaged by StarView and sent to DADS, which returns the requested data sets to the users. The advantages of distributed client/server user interfaces over traditional one-machine systems are well known. Since users run software on machines separate from the database, the overall client response time is much faster. Also, since the server is free to process only database requests, the database response time is much faster. Disadvantages inherent in this architecture are slow overall database access time due to the network delays, lack of a 'get previous row' command, and that refinements of a previously issued query must be submitted to the database server, even though the domain of values have already been returned by the previous query. This architecture also does not allow users to cross correlate DADS catalog data with other catalogs. Clearly, a distributed user interface would be more powerful if it overcame these disadvantages. A local database is being integrated into StarView to overcome these disadvantages. When a query is made through a StarView form, which is often composed of fields from multiple tables, it is translated to an SQL query and issued to the DADS catalog. At the same time, a local database table is created to contain the resulting rows of the query. The returned rows are displayed on the form as well as inserted into the local database table. Identical results are produced by reissuing the query to either the DADS catalog or to the local table. Relational databases do not provide a 'get previous row' function because of the inherent complexity of retrieving previous rows of multiple-table joins. However, since this function is easily implemented on a single table, StarView uses the local table to retrieve the previous row. Also, StarView issues subsequent query refinements to the local table instead of the DADS catalog, eliminating the network transmission overhead. Finally, other catalogs can be imported into the local database for cross correlation with local tables. Overall, it is believe that this is a more powerful architecture for distributed, database user interfaces.

Silberberg, D. P.↗
NASA NTRS2006

Active Learning with Irrelevant Examples

Active learning algorithms attempt to accelerate the learning process by requesting labels for the most informative items first. In real-world problems, however, there may exist unlabeled items that are irrelevant to the user's classification goals. Queries about these points slow down learning because they provide no information about the problem of interest. We have observed that when irrelevant items are present, active learning can perform worse than random selection, requiring more time (queries) to achieve the same level of accuracy. Therefore, we propose a novel approach, Relevance Bias, in which the active learner combines its default selection heuristic with the output of a simultaneously trained relevance classifier to favor items that are likely to be both informative and relevant. In our experiments on a real-world problem and two benchmark datasets, the Relevance Bias approach significantly improved the learning rate of three different active learning approaches.

machine learning↗
NASA NTRS1994

Multivariate statistical analysis software technologies for astrophysical research involving large data bases

We developed a package to process and analyze the data from the digital version of the Second Palomar Sky Survey. This system, called SKICAT, incorporates the latest in machine learning and expert systems software technology, in order to classify the detected objects objectively and uniformly, and facilitate handling of the enormous data sets from digital sky surveys and other sources. The system provides a powerful, integrated environment for the manipulation and scientific investigation of catalogs from virtually any source. It serves three principal functions: image catalog construction, catalog management, and catalog analysis. Through use of the GID3* Decision Tree artificial induction software, SKICAT automates the process of classifying objects within CCD and digitized plate images. To exploit these catalogs, the system also provides tools to merge them into a large, complete database which may be easily queried and modified when new data or better methods of calibrating or classifying become available. The most innovative feature of SKICAT is the facility it provides to experiment with and apply the latest in machine learning technology to the tasks of catalog construction and analysis. SKICAT provides a unique environment for implementing these tools for any number of future scientific purposes. Initial scientific verification and performance tests have been made using galaxy counts and measurements of galaxy clustering from small subsets of the survey data, and a search for very high redshift quasars. All of the tests were successful, and produced new and interesting scientific results. Attachments to this report give detailed accounts of the technical aspects for multivariate statistical analysis of small and moderate-size data sets, called STATPROG. The package was tested extensively on a number of real scientific applications, and has produced real, published results.

Djorgovski, S. George↗
NASA NTRS1994

Multivariate Statistical Analysis Software Technologies for Astrophysical Research Involving Large Data Bases

We developed a package to process and analyze the data from the digital version of the Second Palomar Sky Survey. This system, called SKICAT, incorporates the latest in machine learning and expert systems software technology, in order to classify the detected objects objectively and uniformly, and facilitate handling of the enormous data sets from digital sky surveys and other sources. The system provides a powerful, integrated environment for the manipulation and scientific investigation of catalogs from virtually any source. It serves three principal functions: image catalog construction, catalog management, and catalog analysis. Through use of the GID3* Decision Tree artificial induction software, SKICAT automates the process of classifying objects within CCD and digitized plate images. To exploit these catalogs, the system also provides tools to merge them into a large, complex database which may be easily queried and modified when new data or better methods of calibrating or classifying become available. The most innovative feature of SKICAT is the facility it provides to experiment with and apply the latest in machine learning technology to the tasks of catalog construction and analysis. SKICAT provides a unique environment for implementing these tools for any number of future scientific purposes. Initial scientific verification and performance tests have been made using galaxy counts and measurements of galaxy clustering from small subsets of the survey data, and a search for very high redshift quasars. All of the tests were successful and produced new and interesting scientific results. Attachments to this report give detailed accounts of the technical aspects of the SKICAT system, and of some of the scientific results achieved to date. We also developed a user-friendly package for multivariate statistical analysis of small and moderate-size data sets, called STATPROG. The package was tested extensively on a number of real scientific applications and has produced real, published results.

Djorgovski, S. G.↗
NASA NTRS1997

Database Development for Electrical, Electronic, and Electromechanical (EEE) Parts for the International Space Station Alpha

More effective electronic communication processes are needed to transfer contractor and international partner data into NASA and prime contractor baseline database systems. It is estimated that the International Space Station Alpha (ISSA) parts database will contain up to one million parts each of which may require database capabilities for approximately one thousand bytes of data for each part. The resulting gigabyte database must provide easy access to users who will be preparing multiple analyses and reports in order to verify as-designed, as-built, launch, on-orbit, and return configurations for up to 45 missions associated with the construction of the ISSA. Additionally, Internet access to this data base is strongly indicated to allow multiple user access from clients located in many foreign countries. This summer's project involved familiarization and evaluation of the ISSA Electrical, Electronic, and Electromechanical (EEE) Parts data and the process of electronically managing these data. Particular attention was devoted to improving the interfaces among the many elements of the ISSA information system and its global customers and suppliers. Additionally, prototype queries were developed to facilitate the identification of data changes in the data base, verifications that the designs used only approved parts, and certifications that the flight hardware containing EEE parts was ready for flight. This project also resulted in specific recommendations to NASA for further development in the area of EEE parts database development and usage.

Wassil-Grimm, Andrew D.↗
NASA NTRS2018

The Planetary Materials Database

NASA provides funds for a variety of research programs whose principal focus is to collect and analyze terrestrial analog materials. These data are used to (1) understand and interpret planetary geology; (2) identify and characterize habitable environments and pre-biotic/biotic processes; (3) interpret returned data from present and past missions; and (4) evaluate future mission and instrument concepts prior to selection for flight. Data management plans are now required for these programs, but the collected data are still not generally available to the community. There is also little possibility to re-analyze the collected materials by other techniques, since there is no requirement to archive collected samples. The Planetary Materials Database (PMD) is a central, high-quality, long-term data repository, which aims to promote the field of astrobiology and increase scientific returns from NASA funded research by enabling data sharing, collaboration and exposure of non-NASA scientists to NASA research initiatives and missions. The PMD is a linked collection of databases developed using the Open Data Repository (ODR) system. The PMD will include detailed descriptions of terrestrial analog planetary materials as well as data from the instruments used in their analysis. The goal is to provide example patterns/spectra/analyses, etc. and background information suitable for use by the Space Science community. An early example showing the utility of these databases (although not in the ODR format) is the RRUFF mineral database. RRUFF, comprising 4,000+ pure mineral standards, is the most popular and widely used dataset of minerals and receives more than 180,000 queries per week from geologists and mineralogists worldwide. The PMD will be patterned after the CheMin database [3], a resource that contains all of the data collected by the MSL CheMin XRD instrument on Mars. Raw and processed CheMin data can be viewed, downloaded, reprocessed and reanalyzed using cloud-based “applications” linked to the data.

Blake, David↗
NASA NTRS1993

EOSDIS - Its role in the EOS program and its importance to the scientific community

In order to manage the global data sets required to understand the earth as a system, the EOS Data and Information System (EOSDIS) will collect and store satellite, aircraft, and in situ measurements and their resultant data products, and will distribute the data conveniently. EOSDIS will also provide product generation and science computing facilities to support the development, processing, and validation of standard EOS science data products. The overall architecture of EOSDIS, and how the Distributed Active Archive Centers fit into that structure, are shown. EOSDIS will enable users to query data bases nationally, make use of keywords and other mnemonic identifiers, and see graphic images of subsets of available data prior to ordering full (or selected pieces of) data sets for use in their 'home' environment.

Price, Robert D.↗
NASA NTRS2006

Flight Operations Analysis Tool

Flight Operations Analysis Tool (FLOAT) is a computer program that partly automates the process of assessing the benefits of planning spacecraft missions to incorporate various combinations of launch vehicles and payloads. Designed primarily for use by an experienced systems engineer, FLOAT makes it possible to perform a preliminary analysis of trade-offs and costs of a proposed mission in days, whereas previously, such an analysis typically lasted months. FLOAT surveys a variety of prior missions by querying data from authoritative NASA sources pertaining to 20 to 30 mission and interface parameters that define space missions. FLOAT provides automated, flexible means for comparing the parameters to determine compatibility or the lack thereof among payloads, spacecraft, and launch vehicles, and for displaying the results of such comparisons. Sparseness, typical of the data available for analysis, does not confound this software. FLOAT effects an iterative process that identifies modifications of parameters that could render compatible an otherwise incompatible mission set.

Easter, Robert↗