SEARCH · Search NASA

Results for “Hashing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Modern Approaches to Exact Diagonalization and Selected Configuration Interaction with the Adaptive Sampling CI Method

Recent advances in selected configuration interaction methods have made them competitive with the most accurate techniques available and, hence, creating an increasingly powerful tool for solving quantum Hamiltonians. In this work, we build on recent advances from the adaptive sampling configuration interaction (ASCI) algorithm. We show that a useful paradigm for generating efficient selected CI/exact diagonalization algorithms is driven by fast sorting algorithms, much in the same way iterative diagonalization is based on the paradigm of matrix vector multiplication. We present several new algorithms for all parts of performing a selected CI, which includes new ASCI search, dynamic bit masking, fast orbital rotations, fast diagonal matrix elements, and residue arrays. The ASCI search algorithm can be used in several different modes, which includes an integral driven search and a coefficient driven search. The algorithms presented here are fast and scalable, and we find that because they are built on fast sorting algorithms they are more efficient than all other approaches we considered. After introducing these techniques, we present ASCI results applied to a large range of systems and basis sets to demonstrate the types of simulations that can be practically treated at the full-CI level with modern methods and hardware, presenting double- and triple-ζ benchmark data for the G1 data set. The largest of these calculations is Si$_2$H$_6$ which is a simulation of 34 electrons in 152 orbitals. We also present some preliminary results for fast deterministic perturbation theory simulations that use hash functions to maintain high efficiency for treating large basis sets.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2025

Traceable random numbers from a non-local quantum advantage

The unpredictability of random numbers is fundamental to both digital security and applications that fairly distribute resources. However, existing random number generators have limitations—the generation processes cannot be fully traced, audited and certified to be unpredictable. The algorithmic steps used in pseudorandom number generators are auditable, but they cannot guarantee that their outputs were a priori unpredictable given knowledge of the initial seed. Device-independent quantum random number generators can ensure that the source of randomness was unknown beforehand, but the steps used to extract the randomness are vulnerable to tampering. Here we demonstrate a fully traceable random number generation protocol based on device-independent techniques. Our protocol extracts randomness from unpredictable non-local quantum correlations, and uses distributed intertwined hash chains to cryptographically trace and verify the extraction process. This protocol forms the basis for a public traceable and certifiable quantum randomness beacon that we have launched. Over the first 40 days of operation, we completed the protocol 7,434 out of 7,454 attempts—a success rate of 99.7%. Each time the protocol succeeded, the beacon emitted a pulse of 512 bits of traceable randomness. The bits are certified to be uniform with error multiplied by actual success probability bounded by 2−64. Further, the generation of certifiable and traceable randomness represents a public service that operates with an entanglement-derived advantage over comparable classical approaches.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

True randomness from an incoherent source

Quantum random number generators (QRNGs) harness the intrinsic randomness in measurement processes: the measurement outputs are truly random, given the input state is a superposition of the eigenstates of the measurement operators. In the case of trusted devices, true randomness could be generated from a mixed state ρ so long as the system entangled with ρ is well protected. We propose a random number generation scheme based on measuring the quadrature fluctuations of a single mode thermal state using an optical homodyne detector. By mixing the output of a broadband amplified spontaneous emission (ASE) source with a single mode local oscillator (LO) at a beam splitter and performing differential photo-detection, we can selectively detect the quadrature fluctuation of a single mode output of the ASE source, thanks to the filtering function of the LO. Experimentally, a quadrature variance about three orders of magnitude larger than the vacuum noise has been observed, suggesting this scheme can tolerate much higher detector noise in comparison with QRNGs based on measuring the vacuum noise. The high quality of this entropy source is evidenced by the small correlation coefficients of the acquired data. A Toeplitz-hashing extractor is applied to generate unbiased random bits from the Gaussian distributed raw data, achieving an efficiency of 5.12 bits per sample. Furthermore, the output of the Toeplitz extractor successfully passes all the NIST statistical tests for random numbers.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2018

On efficient Hessian computation using the edge pushing algorithm in Julia

Evaluating the Hessian matrix of second-order derivatives at a sequence of points can be costly when applying second-order methods for nonlinear optimization. In this work, we discuss our experiences implementing the recently proposed Edge Pushing (EP) method in Julia as an experimental replacement for the current colouring-based methods used by JuMP, an open-source algebraic modelling language. We propose an alternative data structure for sparse Hessians to avoid the use of hash tables and analyse the space and time complexity of EP method. In our benchmarks, we find that EP is very competitive in terms of both preprocessing time and Hessian evaluation time. As a result, we identify cases where EP closes the performance gap between JuMP's previous implementation and the implementation in AMPL, a commercial software package with similar functionality.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

The parallelism motifs of genomic data analysis

Genomic datasets are growing dramatically as the cost of sequencing continues to decline and small sequencing devices become available. Enormous community databases store and share these data with the research community, but some of these genomic data analysis problems require large-scale computational platforms to meet both the memory and computational requirements. These applications differ from scientific simulations that dominate the workload on high-end parallel systems today and place different requirements on programming support, software libraries and parallel architectural design. For example, they involve irregular communication patterns such as asynchronous updates to shared data structures. We consider several problems in high-performance genomics analysis, including alignment, profiling, clustering and assembly for both single genomes and metagenomes. We identify some of the common computational patterns or ‘motifs’ that help inform parallelization strategies and compare our motifs to some of the established lists, arguing that at least two key patterns, sorting and hashing, are missing. This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2020

Quantum Overlapping Tomography

It is now experimentally possible to entangle thousands of qubits, and efficiently measure each qubit in parallel in a distinct basis. To fully characterize an unknown entangled state of $\textit{n}$ qubits, one requires an exponential number of measurements in $\textit{n}$, which is experimentally unfeasible even for modest system sizes. By leveraging (i) that single-qubit measurements can be made in parallel, and (ii) the theory of perfect hash families, we show that all $\textit{k}$-qubit reduced density matrices of an $\textit{n}$ qubit state can be determined with at most $e^{\mathcal{O}}(k) \text{log}^2(n)$ rounds of parallel measurements. In this work, we provide concrete measurement protocols which realize this bound. As an example, we argue that with near-term experiments, every two-point correlator in a system of 1024 qubits could be measured and completely characterized in a few days. This corresponds to determining nearly 4.5 million correlators.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2021

Blockchain-Based Man-in-the-Middle (MITM) Attack Detection for Photovoltaic Systems

Cybersecurity of photovoltaic (PV) systems entails a much larger scope than just encryption and firewall of communications. For instance, integrity of data in transit between inverters and a cloud server can be compromised by authorized third-party, devices, and internal network within security perimeter (i.e., man-in-the-middle (MITM) attack). To address this challenge, this paper proposes a blockchain-based MITM attack detection method for a PV system. A breakthrough method includes screening network data, network intrusion detection, and hash comparison of in-transit data using distributed ledgers. Furthermore, the proposed method is implemented in Internet-of-Thing (IoT) security modules as clients of a blockchain network and validated by experiments.

blockchain↗

DOE OSTI2018

Domain-Adapted Convolutional Networks for Satellite Image Classification: A Large-Scale Interactive Learning Workflow

Satellite imagery often exhibits large spatial extent areas that encompass object classes with considerable variability. This often limits large-scale model generalization with machine learning algorithms. Notably, acquisition conditions, including dates, sensor position, lighting condition, and sensor types, often translate into class distribution shifts introducing complex nonlinear factors and hamper the potential impact of machine learning classifiers. Here, this article investigates the challenge of exploiting satellite images using convolutional neural networks (CNN) for settlement classification where the class distribution shifts are significant. We present a large-scale human settlement mapping workflow based-off multiple modules to adapt a pretrained CNN to address the negative impact of distribution shift on classification performance. To extend a locally trained classifier onto large spatial extents areas we introduce several submodules: First, a human-in-the-loop element for relabeling of misclassified target domain samples to generate representative examples for model adaptation; second, an efficient hashing module to minimize redundancy and noisy samples from the mass-selected examples; and third, a novel relevance ranking module to minimize the dominance of source example on the target domain. The workflow presents a novel and practical approach to achieve large-scale domain adaptation with binary classifiers that are based-off CNN features. Experimental evaluations are conducted on areas of interest that encompass various image characteristics, including multisensors, multitemporal, and multiangular conditions. Domain adaptation is assessed on source–target pairs through the transfer loss and transfer ratio metrics to illustrate the utility of the workflow.

47 OTHER INSTRUMENTATION↗

DOE OSTI2017

Efficient Management of Certificate Revocation Lists in Smart Grid Advanced Metering Infrastructure

Advanced Metering Infrastructure (AMI) forms a communication network for the collection of power data from smart meters in Smart Grid. As the communication within an AMI needs to be secure, key management becomes an issue due to overhead and limited resources. While using public-keys eliminate some of the overhead of key management, there is still challenges regarding certificates that store and certify the publickeys. In particular, distribution and storage of certificate revocation list (CRL) is major a challenge due to cost of distribution and storage in AMI networks which typically consist of wireless multi-hop networks. Motivated by the need of keeping the CRL distribution and storage cost effective and scalable, in this paper, we present a distributed CRL management model utilizing the idea of distributed hash trees (DHTs) from peer-to-peer (P2P) networks. The basic idea is to share the burden of storage of CRLs among all the smart meters by exploiting the meshing capability of the smart meters among each other. Thus, using DHTs not only reduces the space requirements for CRLs but also makes the CRL updates more convenient. We implemented this structure on ns-3 using IEEE 802.11s mesh standard as a model for AMI and demonstrated its superior performance with respect to traditional methods of CRL management through extensive simulations.

25 ENERGY STORAGE↗

DOE OSTI2021

Digital data provenance for the power grid based on a Keyless Infrastructure Security Solution

In this work a data provenance system for grid-oriented applications is presented. The proposed Keyless Infrastructure Security Solution (KISS) provides mechanisms to store and maintain digital data fingerprints that can later be used to validate and assert data provenance using a time-based, hash tree mechanism. The developed solution has been designed to satisfy the stringent requirements of the modern power grid including execution time and storage necessities. Its applicability has been tested using a lab-scale, proof-of-concept deployment that secures an energy management system against the attack sequence observed on the 2016 Ukrainian power grid cyberattack. The results demonstrate a strong potential for enabling data provenance in a wide array of applications, including speed-sensitive applications such as those found in control room environments.

Sebastian Cardenas, David J.↗

DOE OSTI2020

Evaluation and Demonstration of Blockchain Applicability Framework

Blockchain technology has been gaining great interest from a variety of industry sectors, including financial, food processing, and power and energy markets. Realizing the strength of blockchain technology beyond the successful application in the cryptocurrency arena, researchers have been evaluating and using blockchain for applications such as supply chain management, transactive industry (both financial and energy), system integrity, device cybersecurity, identity management, and much more. One of the unique elements of the blockchain technology that made it such a captivating technology to researchers is its plethora of features. Some of the features include smart contracts, cryptocurrency and tokenizing, immutable distributed ledger, cryptographic hashing, and digital signature. In addition, there are multiple types of blockchains, such as permissioned/private and permissionless/public, and various consensus models, such as proof-of-work, proof-of-authority, proof-of-burn, and proof-of-stake. Therefore, it is often non-trivial to determine if an application requires a blockchain. If so, what kind of blockchain and consensus is most appropriate? This paper discusses the blockchain applicability framework (BAF), which was specifically designed with the purpose to answer those questions. BAF is divided into five domains, 18 subdomains, and about 100 controls. It is designed to ingest detailed user requirements to perform a weighted evaluation that is built on mathematical constructs to determine the ideal combination of blockchain that is appropriate for an application. Along with the core logical formulation of BAF, this paper depicts the efficacy of BAF through two use cases

Gourisetti, Sri Nikhil G.↗

DOE OSTI2018

A Blockchain-Based Decentralized Data Storage and Access Framework for PingER

The blockchain is an innovative technology which opened doors to new applications for solving numerous problems in distributed environments. In this study, we design a blockchain-based data storage and access framework for PingER (worldwide end-to-end Internet performance measurement project) to remove its total dependence on a centralized repository. We use the permissioned blockchain and Distributed Hash Tables (DHT) for this purpose. In the proposed framework, metadata of the files are stored on the blockchain whereas the actual files are stored off-chain through DHT at multiple locations using a peer-to-peer network of PingER Monitoring Agents. Lastly, this will provide decentralized storage, distributed processing, and efficient lookup capabilities to the PingER framework.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

Structure- and Physics-Preserving Reductions of Power Grid Models

The large size of multiscale, distribution and transmission, power grids hinders fast systemwide estimation and real-time control and optimization of operations. This paper studies graph reduction methods of power grids that are favorable for fast simulations and follow-up applications. While the classical Kron reduction has been successful in reduced order modeling of power grids with traditional, hierarchical design, the selection of reference nodes for the reduced model in a multiscale, distribution and transmission, network becomes ambiguous. In this work we extend the use of the iterative Kron reduction by utilizing the electric grid's graph topology for the selection of reference nodes, consistent with the design features of multiscale networks. Additionally, we propose further reductions by aggregation of coherent subnetworks of triangular meshes, based on the graph topology and network characteristics, in order to preserve currents and build another power-flow equivalent network. Our reductions are achieved through the use of iterative aggregation of subgraphs that include general tree structures, lines, and triangles. Important features of our reduction algorithms include that (i) the reductions are either equivalent to the Kron reduction or otherwise produce a power-flow equivalent network; (ii) due to the mentioned power-flow equivalence, the reduced network can model the dynamic of the swing equations for a lossless, inductive, steady state network; (iii) the algorithms efficiently utilize hash tables to store the sequential reduction steps. The third feature allows for easy reintroduction of detailed models into the reduced, conceptual network and makes the final reduced order model backward compatible with a sequence of intermediate, partially reduced networks with varying resolution---the ordered sequence of iterative reductions corresponds to a sequence of reduced order models. The performance of our graph reduction algorithms and features of the reduced grids are discussed on a real-word transmission and distribution grid. We produce visualizations of the reduced models through open source libraries and release our reduction algorithms with example code and toy data.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2019

BCL: A Cross-Platform Distributed Data Structures Library

One-sided communication is a useful paradigm for irregular parallel applications, but most one-sided programming environments, including MPI's one-sided interface and PGAS programming languages, lack application-level libraries to support these applications. We present the Berkeley Container Library, a set of generic, cross-platform, high-performance data structures for irregular applications, including queues, hash tables, Bloom filters and more. BCL is written in C++ using an internal DSL called the BCL Core that provides one-sided communication primitives such as remote get and remote put operations. The BCL Core has backends for MPI, OpenSHMEM, GASNet-EX, and UPC++, allowing BCL data structures to be used natively in programs written using any of these programming environments. Along with our internal DSL, we present the BCL ObjectContainer abstraction, which allows BCL data structures to transparently serialize complex data types while maintaining efficiency for primitive types. We also introduce the set of BCL data structures and evaluate their performance across a number of high-performance computing systems, demonstrating that BCL programs are competitive with hand-optimized code, even while hiding many of the underlying details of message aggregation, serialization, and synchronization.

Brock, Benjamin↗

DOE OSTI2026

Integrase-On-Demand-Pipeline Data Set

Files needed to run the Integrase-On-Demand-Pipeline, a program designed to provide users with a list of putative attachment site and integrase pairs for a prokaryotic genome of interest. isles.pkl: Serialized python-object file, containing a dictionary of attachment site sequences and reference genomic island information extracted from the Genomic island database ints.gff: Gene format file containing annotations for all integrases referenced in isles.pkl. The source genome, gene coordinates, integrase name, protein IDs and amino acid sequence included. reps.msh: Binary file containing 1000 128-bit MurmurHash3 hashes for >80,000 genomes

McClain, Hannah Marie [Sandia National Laboratorie↗

DOE OSTI2019

Blockchain Based Remote Data Integrity Checking Tool v1.0

Computer System Security: The software is used to perform remote data integrity checking without relying on a single third party. In order to remove the trusted third party our tool uses distributed ledger (Blockchain) technology. The ledger is used to store evidences (in the form of hash values). These values are used as an input in the integrity checking process.

Wonjiga, Amir↗

DOE OSTI2020

eBPF Community ID Calculation

The code titled “eBPF Community ID Calculation” allows easy correlation of monitoring logs with existing network forensics logs on high-performance computing (HPC) systems. This code implements the Community ID Flow Hashing standard to calculate community ID Flow identifiers. These identifiers will be used to correlate host forensics logs with network forensics logs. The correlation provides valuable insight and the ability to easily pivot between disparate data sources. The computer code removes the libpcap dependency from the code base.

Maerz, StefanA↗

DOE OSTI2021

PMatch v.1.0

SAND2021-14236 O PMatch is a function matching plugin for Ghidra. PMatch uses hashing and p-code to create function databases and complete function matching on other binary files. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Wright, Christopher↗