Search NASA⌕ Search

SEARCH · Search NASA

Results for “Query log”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

The Bird project: Using Big Data tools to support Search Analytics

The Bird project explored the use of big data analytics tool to improve the findability of information within the Sandia internal network. We were able to perform query classification utilizing the supervised learning algorithms in the Apache Spark library. By relying on the distributed processing capabilities provided by the Apache Hadoop framework, we successfully processed the large query log files needed to train the models in this effort. The capabilities developed in this project are being used to enhance the effectiveness of the enterprise search engine.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

SAPLE: Sandia Advanced Personnel Locator Engine.

We present the Sandia Advanced Personnel Locator Engine (SAPLE) web application, a directory search application for use by Sandia National Laboratories personnel. SAPLE's purpose is to return Sandia personnel 'results' as a function of user search queries, with its mission to make it easier and faster to find people at Sandia. To accomplish this, SAPLE breaks from more traditional directory application approaches by aiming to return the correct set of results while placing minimal constraints on the user's query. Two key features form the core of SAPLE: advanced search query interpretation and inexact string matching. SAPLE's query interpretation permits the user to perform compound queries when typing into a single search field; where able, SAPLE infers the type of field that the user intends to search on based on the value of the search term. SAPLE's inexact string matching feature yields a high-quality ranking of personnel search results even when there are no exact matches to the user's query. This paper explores these two key features, describing in detail the architecture and operation of SAPLE. Finally, an extensive analysis on logged search query data taken from an 11-week sample period is presented.

97 MATHEMATICS AND COMPUTING↗

iDASH 2018: Task 1

The goal of this track is to develop blockchain-based ledgering solutions to log and query the user activities of accessing genomic datasets (e.g., GTEx) across multiple sites.

Pattengale, Nick↗

Semantic Features for Classifying Referring Search Terms

When an internet user clicks on a result in a search engine, a request is submitted to the destination web server that includes a referrer field containing the search terms given by the user. Using this information, website owners can analyze the search terms leading to their websites to better understand their visitors needs. This work explores some of the features that can be used for classification-based analysis of such referring search terms. We present initial results for the example task of classifying HTTP requests countries of origin. A system that can accurately predict the country of origin from query text may be a valuable complement to IP lookup methods which are susceptible to the obfuscation of dereferrers or proxies. We suggest that the addition of semantic features improves classifier performance in this example application. We begin by looking at related work and presenting our approach. After describing initial experiments and results, we discuss paths forward for this work.

99 GENERAL AND MISCELLANEOUS↗

Generalized Quantum Signal Processing

Quantum signal processing (QSP) and quantum singular value transformation (QSVT) currently stand as the most efficient techniques for implementing functions of block-encoded matrices, a central task that lies at the heart of most prominent quantum algorithms. However, current QSP approaches face several challenges, such as the restrictions imposed on the family of achievable polynomials and the difficulty of calculating the required phase angles for specific transformations. In this paper, we present a generalized quantum signal processing (GQSP) approach, employing general SU(2) rotations as our signal-processing operators, rather than relying solely on rotations in a single basis. Our approach lifts all practical restrictions on the family of achievable transformations, with the sole remaining condition being that | P | ≤ 1 , a restriction necessary due to the unitary nature of quantum computation. Furthermore, GQSP provides a straightforward recursive formula for determining the rotation angles needed to construct the polynomials in cases where P and Q are known. In cases where only P is known, we provide an efficient optimization algorithm capable of identifying in under a minute of GPU time, a corresponding Q for polynomials of degree on the order of 10 7 . We further illustrate GQSP simplifies QSP-based strategies for Hamiltonian simulation, offer an optimal solution to the ϵ -approximate fractional query problem that requires O ( ( 1 / δ ) + log ( 1 / ϵ ) ) queries to perform where O ( 1 / δ ) is a proved lower bound, and introduces novel approaches for implementing bosonic operators. Moreover, we propose a novel framework for the implementation of normal matrices, demonstrating its applicability through synthesis of diagonal matrices, as well as the development of a new algorithm for convolution through synthesis of circulant matrices using only O ( d log N + log 2 N ) 1 and 2-qubit gates for a filter of lengths d . Published by the American Physical Society 2024

Motlagh, Danial↗

Translation-Invariant Quantum Algorithms for Ordered Search are Optimal

Ordered search is the task of finding an item in an ordered list using comparison queries. The best exact classical algorithm for this fundamental problem uses [log 2 n] queries for a list of length n. Quantum computers can achieve a constant-factor speedup, but the best possible coefficient of log 2 n for exact quantum algorithms is only known to lie between (ln2)/π ≈ 0.221 and 4/log 2 605 ≈ 0.4333. We consider a special class of translation-invariant algorithms with no workspace, introduced by Farhi, Goldstone, Gutmann, and Sipser, that has been used to find the best known upper bounds. First, we show that any bounded-error, k-query quantum algorithm for ordered search can be implemented by a k-query algorithm in this special class. Second, we use linear programming to show that the best exact 5-query quantum algorithm can search a list of length 7265, giving an ordered search algorithm that asymptotically uses 5 log 7265 n ≈ 0.390 log 2 n quantum queries.

Translation-invariant quantum algorithms↗

Creative Analytics of Mission Ops Event Messages

Historically, tremendous effort has been put into processing and displaying mission health and safety telemetry data; and relatively little attention has been paid to extracting information from missions time-tagged event log messages. Todays missions may log tens of thousands of messages per day and the numbers are expected to dramatically increase as satellite fleets and constellations are launched, as security monitoring continues to evolve, and as the overall complexity of ground system operations increases. The logs may contain information about orbital events, scheduled and actual observations, device status and anomalies, when operators were logged on, when commands were resent, when there were data drop outs or system failures, and much much more. When dealing with distributed space missions or operational fleets, it becomes even more important to systematically analyze this data. Several advanced information systems technologies make it appropriate to now develop analytic capabilities which can increase mission situational awareness, reduce mission risk, enable better event-driven automation and cross-mission collaborations, and lead to improved operations strategies: Industry Standard for Log Messages. The Object Management Group (OMG) Space Domain Task Force (SDTF) standards organization is in the process of creating a formal standard for industry for event log messages. The format is based on work at NASA GSFC. Open System Architectures. The DoD, NASA, and others are moving towards common open system architectures for mission ground data systems based on work at NASA GSFC with the full support of the commercial product industry and major integration contractors. Text Analytics. A specific area of data analytics which applies statistical, linguistic, and structural techniques to extract and classify information from textual sources. This presentation describes work now underway at NASA to increase situational awareness through the collection of non-telemetry mission operations information into a common log format and then providing display and analytics tools to provide in-depth assessment of the log contents. The work includes: Common interface formats for acquiring time-tagged text messages Conversion of common files for schedules, orbital events, and stored commands to the common log format Innovative displays to depict thousands of messages on a single display Structured English text queries against the log message data store, extensible to a more mature natural language query capability Goal of speech-to-text and text-to-speech additions to create a personal mission operations assistant to aid on-console operations. A wide variety of planned uses identified by the mission operations teams will be discussed.

events↗

Anno v1

The Anno log annotation system provides tools for creating and querying annotations to supercomputing log data.

Greiner, AnnetteM↗

Mass Storage Performance Information System

The purpose of this task is to develop a data warehouse to enable system administrators and their managers to gather information by querying the data logs of the MDSDS. Currently detailed logs capture the activity of the MDSDS internal to the different systems. The elements to be included in the data warehouse are requirements analysis, data cleansing, database design, database population, hardware/software acquisition, data transformation, query and report generation, and data mining.

Scheuermann, Peter↗

Ocpp 2.0.1. Interim Kpi Calculator

The project is split into four pieces. The first is a raw OCPP log parser. The second is a file splitter. The third is a message parser. The final piece is the Interim KPI calculator. The OCPP log parser was created from two different formats of raw OCPP 2.0.1 data. Its intended purpose is to extract device IDs and OCPP event messages from nontabular text logs. The parser looks for specific substrings in the logs to identify which of the two "standards" it should select from. The KPI generator does not perform any of its calculations in parallel. Instead, we opt for a naive batching approach. The splitter takes the file generated from the parser and creates many smaller files for each of the device IDs in the dataset. This allows the pandas queries in the log formatter to be iterate over a significantly smaller slice of data, increasing performance significantly. The message parser step takes messages from each of the files (containing distinct device IDs) and breaks the message out into pieces. The final result is a file with different columns specifying different attributes of the JSON message. The file is an aggregation of all different devices. This is the most complex portion of the code. The KPI calculator takes the parsed messages, as a single file, and calculates the KPI from that data. An excel file is produced with four sheets. These contain the metrics for Session Success, Charge Start Success, Charge End Success, and Charge Start Time. It includes the metrics for the different equations in the Interim KPI Implementation Guide as well as a weighted sum of the different equations for each KPI (excluding Charge End Success and Charge Start Time).

Quinn, Casey↗

Properly Learning Decision Trees in almost Polynomial Time

We give an n O (log log n ) -time membership query algorithm for properly and agnostically learning decision trees under the uniform distribution over { ± 1} n . Even in the realizable setting, the previous fastest runtime was n O (log n ) , a consequence of a classic algorithm of Ehrenfeucht and Haussler. Our algorithm shares similarities with practical heuristics for learning decision trees, which we augment with additional ideas to circumvent known lower bounds against these heuristics. To analyze our algorithm, we prove a new structural result for decision trees that strengthens a theorem of O’Donnell, Saks, Schramm, and Servedio. While the OSSS theorem says that every decision tree has an influential variable, we show how every decision tree can be “pruned” so that every variable in the resulting tree is influential.

Computer Science↗

SCDU Testbed Automated In-Situ Alignment, Data Acquisition and Analysis

In the course of fulfilling its mandate, the Spectral Calibration Development Unit (SCDU) testbed for SIM-Lite produces copious amounts of raw data. To effectively spend time attempting to understand the science driving the data, the team devised computerized automations to limit the time spent bringing the testbed to a healthy state and commanding it, and instead focus on analyzing the processed results. We developed a multi-layered scripting language that emphasized the scientific experiments we conducted, which drastically shortened our experiment scripts, improved their readability, and all-but-eliminated testbed operator errors. In addition to scientific experiment functions, we also developed a set of automated alignments that bring the testbed up to a well-aligned state with little more than the push of a button. These scripts were written in the scripting language, and in Matlab via an interface library, allowing all members of the team to augment the existing scripting language with complex analysis scripts. To keep track of these results, we created an easily-parseable state log in which we logged both the state of the testbed and relevant metadata. Finally, we designed a distributed processing system that allowed us to farm lengthy analyses to a collection of client computers which reported their results in a central log. Since these logs were parseable, we wrote query scripts that gave us an effortless way to compare results collected under different conditions. This paper serves as a case-study, detailing the motivating requirements for the decisions we made and explaining the implementation process.

Automation↗

Knowledge management information tool analytics with distributed database engine - 15737

The D and D KM-IT is a web-based knowledge management information tool custom built for the deactivation and decommissioning (D and D) user community. The system allows interested users to post information which goes through a custom work flow. To better improve the performance and user experience, the analysis of server logs is being conducted using MongoDB. MongoDB is a document database that provides high performance, high availability, and easy scalability. Objectives: - Feasibility study of Distributed Database Engine, MongoDB for web analytics; - Analyze server logs to evaluate D and D KM-IT module usage; - Generate useful information from the analysis. Method: - The server logs were loaded and stored in MongoDB for analysis; - Analysis and visualization was performed using NodeJS and Java Script library. Results: - It was possible to store log files in MongoDB; - Log files with different time stamps and groups from different servers can be aggregated; - Queries can be written to extract meaningful information from aggregated log files (e.g. users from different countries/organizations). Conclusions: - MongoDB is useful for server log analytics as the analyst can immediately access, analyze, and visualize MongoDB data based on the requirements; - The information can be extracted to develop custom reports; - Optimized queries can be run for fast data access. Path forward: - Custom queries need to be developed to extract information leading to improved user information.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Testing SOAR tools in use

Investigations within Security Operation Centers (SOCs) are tedious as they rely on manual efforts to query diverse data sources, overlay related logs, correlate the data into information, and then document results in a ticketing system. Security Orchestration, Automation, and Response (SOAR) tools are a relatively new technology that promise, with appropriate configuration, to collect, filter, and display needed diverse information; automate many of the common tasks that unnecessarily require SOC analysts’ time; facilitate SOC collaboration; and, in doing so, improve both efficiency and consistency of SOCs. There has been no prior research to test SOAR tools in practice; hence, understanding and evaluation of their effect is nascent and needed. Here, in this paper, we design and administer the first hands-on user study of SOAR tools, involving 24 participants and six commercial SOAR tools. Our contributions include the experimental design, itemizing six characteristics of SOAR tools, and a methodology for testing them. We describe configuration of a cyber range test environment, including network, user, and threat emulation; a full SOC tool suite; and creation of artifacts allowing multiple representative investigation scenarios to permit testing. We present the first research results on SOAR tools. Concisely, our findings are that: per-SOC SOAR configuration is extremely important; SOAR tools increase efficiency and reduce context switching, although with potentially decreased ticketing accuracy/completeness; user preference is slightly negatively correlated with their performance with the tool; internet dependence varies widely among SOAR tools; and balance of automation with assisting decision making is preferred by senior participants. We deliver a public user- and tool-anonymized and -obfuscated version of the data.

97 MATHEMATICS AND COMPUTING↗

Training quantum neural networks using the quantum information bottleneck method

Abstract We provide in this paper a concrete method for training a quantum neural network to maximize the relevant information about a property that is transmitted through the network. This is significant because it gives an operationally well founded quantity to optimize when training autoencoders for problems where the inputs and outputs are fully quantum. We provide a rigorous algorithm for computing the value of the quantum information bottleneck quantity within error ε that requires O ( log 2 ⁡ ( 1 / ϵ ) + 1 / δ 2 ) queries to a purification of the input density operator if its spectrum is supported on { 0 } ⋃ [ δ , 1 − δ ] for δ > 0 and the kernels of the relevant density matrices are disjoint. We further provide algorithms for estimating the derivatives of the QIB function, showing that quantum neural networks can be trained efficiently using the QIB quantity given that the number of gradient steps required is polynomial.

Çatlı, Ahmet Burak (ORCID:0000000152294141)↗

Fermion determinants on a quantum computer

We present a quantum algorithm to compute the logarithm of the determinant of the fermion matrix, assuming access to a classical lattice gauge field configuration. The algorithm uses the quantum eigenvalue transform, and quantum mean estimation, giving a query complexity that scales like O ( V log ( V ) ) in the matrix dimension V . Published by the American Physical Society 2025

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

GUIDE: A Scalable Information Directory Service to Collect, Federate, and Analyze Logs for Operational Insights into a Leadership HPC Facility

In this paper, we describe the GUIDE framework to collect, federate, and analyze log data from the Oak Ridge Leadership Computing Facility's (OLCF), and how we use it derive insights into center operations. We collectsystem logs and extract monitoring data at every level of the various OLCF subsystems, and have developed a suite of pre-processing tools to make the raw data consumable. The cleansed logs are then ingested and federated into a central, scalable data warehouse, Splunk, that offers storage, indexing, querying, and visualization capabilities. We have further developed and deployed a set of analytics tools to analyze these multiple disparate log streams in concert, and derive operational insights. We describe our experience from developing and deploying the GUIDE infrastructure, and deriving valuable insights on the various subsystems, based on two years of operations in the production OLCF environment.

Vazhkudai, Sudharshan↗