Knowledge Graph Embedding using Large Language Models for COVID-19
CoVID-19 (cord19) open data source for training knowledge graph with LLM
Cord19↗
Engineering topics
Potok, Thomas E.
Publications and source records attributed to Potok, Thomas E..
Optimizing neural networks
A system and method design and optimize neural networks. The system and method include a data store that stores a plurality of gene vectors that represent diverse and distinct neural networks and an evaluation queue stored with the plurality of gene vectors. Secondary nodes construct, train, and evaluate the neural network and automatically render a plurality of fitness values asynchronously. A primary node executes a gene amplification on a select plurality of gene vectors, a crossing-over of the amplified gene vectors, and a mutation of the crossing-over gene vectors automatically and asynchronously, which are then transmitted to the evaluation queue. The process continuously repeats itself by processing the gene vectors inserted into the evaluation queue until a fitness level is reached, a network's accuracy level plateaus, a processing time period expires, or when some stopping condition or performance metric is met or exceeded.
97 MATHEMATICS AND COMPUTING↗
System and method of content based recommendation using hypernym expansion
A system and method for document recommendation combines linguistic and term frequency approaches in order to improve overall performance and recommendation. Hypernyms are added to the source document set so that related documents can be recommended from the source document set. There are a variety of different parameters to configure that change operation of the document recommendation system.
97 MATHEMATICS AND COMPUTING↗
Deep Learning for Vertex Reconstruction of Neutrino-nucleus Interaction Events with Combined Energy and Time Data
We present a deep learning approach for vertex reconstruction of neutrino-nucleus interaction events, a problem in the domain of high energy physics. In this approach, we combine both energy and timing data that are collected in the MIN-ERvA detector to perform classification and regression tasks. We show that the resulting network achieves higher accuracy than previous results while requiring a smaller model size and less training time. In particular, the proposed model outperforms the state-of-the-art by 4.00% on classification accuracy. For the regression task, our model achieves 0.9919 on the coefficient of determination, higher than the previous work (0.96).
Song, Linghao↗
A Classical-Quantum Hybrid Approach for Unsupervised Probabilistic Machine Learning
For training unsupervised probabilistic machine learning models, matrix computation and sample generation are the two key steps. While GPUs excel at matrix computation, they use pseudo-random numbers to generate samples. Contrarily, Adiabatic Quantum Processors (AQP) use quantum mechanical systems to generate samples accurately and quickly, but are not suited for matrix computation. We present a Classical-Quantum Hybrid Approach for training unsupervised probabilistic machine learning models, leveraging GPUs for matrix computations and the D-Wave quantum sampling library for sample generation. We compare this approach to classical and quantum approaches across four performance metrics. Our results indicate that while the hybrid approach–which uses one AQP and one GPU–outperforms quantum and one of the classical approaches, it performs comparably to the GPU approach, and is outperformed by the CPU approach, which uses 56 high-end CPUs. Lastly, we compare sampling on AQP versus sampling library and show that AQP performs better.
Date, Prasanna↗
167-PFlops Deep Learning for Electron Microscopy: From Learning Physics to Atomic Manipulation
An artificial intelligence system called MENNDL, which used 25,200 Nvidia Volta GPUs on Oak Ridge National Laboratory’s Summit machine, automatically designed an optimal deep learning network in order to extract structural information from raw atomic-resolution microscopy data. In a few hours, MENNDL creates and evaluates millions of networks using a scalable, parallel, asynchronous genetic algorithm augmented with a support vector machine to automatically find a superior deep learning network topology and hyper-parameter set than a human expert can find in months. For the application of electron microscopy, the system furthers the goal of improving our understanding of the electron-beam-matter interactions and real-time image-based feedback, which enables a huge step beyond human capacity toward nanofabricating materials automatically. Finally, MENNDL has been scaled to the 4,200 available nodes of Summit achieving a measured 152.5 PFlops, with an estimated sustained performance of 167 PFlops when the entire machine is available.
97 MATHEMATICS AND COMPUTING↗
A Study of Complex Deep Learning Networks on High-Performance, Neuromorphic, and Quantum Computers
Current deep learning approaches have been very successful using convolutional neural networks trained on large graphical-processing-unit-based computers. Three limitations of this approach are that (1) they are based on a simple layered network topology, i.e., highly connected layers, without intra-layer connections; (2) the networks are manually configured to achieve optimal results, and (3) the implementation of the network model is expensive in both cost and power. In this paper, we evaluate deep learning models using three different computing architectures to address these problems: quantum computing to train complex topologies, high performance computing to automatically determine network topology, and neuromorphic computing for a low-power hardware implementation. We use the MNIST dataset for our experiment, due to input size limitations of current quantum computers. Our results show the feasibility of using the three architectures in tandem to address the above deep learning limitations. Finally, we show that a quantum computer can find high quality values of intra-layer connection weights in a tractable time as the complexity of the network increases, a high performance computer can find optimal layer-based topologies, and a neuromorphic computer can represent the complex topology and weights derived from the other architectures in low power memristive hardware.
97 MATHEMATICS AND COMPUTING↗
Neuromorphic computing for temporal scientific data classification
In this work, we apply a spiking neural network model and an associated memristive neuromorphic implementation to an application in classifying temporal scientific data. We demonstrate that the spiking neural network model achieves comparable results to a previously reported convolutional neural network model, with significantly fewer neurons and synapses required.
Schuman, Catherine↗
Evolving Deep Networks Using HPC
While a large number of deep learning networks have been studiedand published that produce outstanding results on natural imagedatasets, these datasets only make up a fraction of those to whichdeep learning can be applied. These datasets include text data, audiodata, and arrays of sensors that have very different characteristicsthan natural images. As these “best” networks for natural imageshave been largely discovered through experimentation and cannotbe proven optimal on some theoretical basis, there is no reasonto believe that they are the optimal network for these drasticallydifferent datasets. Hyperparameter search is thus often a very im-portant process when applying deep learning to a new problem. Inthis work we present an evolutionary approach to searching thepossible space of network hyperparameters and construction thatcan scale to 18, 000 nodes. This approach is applied to datasets ofvarying types and characteristics where we demonstrate the abilityto rapidly find best hyperparameters in order to enable practitionersto quickly iterate between idea and result.
Young, Steven↗
Audience Based View of Publication Impact
With increasing pressure to demonstrate a return on investment, scienti c user facilities and government funded research projects must strive to demonstrate positive impact of their research on oth- ers. Historically, citation counts and journal impact factor have been widely used as a single quantitative measure of impact. However, with the expansion of various modes of communication, citation counts no longer hold as much merit as a single measure of impact. This work seeks to expand on citation counts by considering the audience to which a publication draws attention. Based on commu- nication mode, there are various audiences that may be "impacted" but those audiences are not fully captured by citation count. Conse- quently, we propose that impact measures for publications should be oriented around the audience with quanti able measures based on the various communication modes.
Patton, Robert↗
In-situ trainable intrusion detection system
A computer implemented method detects intrusions using a computer by analyzing network traffic. The method includes a semi-supervised learning module connected to a network node. The learning module uses labeled and unlabeled data to train a semi-supervised machine learning sensor. The method records events that include a feature set made up of unauthorized intrusions and benign computer requests. The method identifies at least some of the benign computer requests that occur during the recording of the events while treating the remainder of the data as unlabeled. The method trains the semi-supervised learning module at the network node in-situ, such that the semi-supervised learning modules may identify malicious traffic without relying on specific rules, signatures, or anomaly detection.
97 MATHEMATICS AND COMPUTING↗
A Study of Complex Deep Learning Networks on High Performance, Neuromorphic, and Quantum Computers
Current Deep Learning models use highly optimized convolutional neural networks (CNN) trained on large graphical processing units (GPU)-based computers with a fairly simple layered network topology, i.e., highly connected layers, without intra-layer connections. Complex topologies have been proposed, but are intractable to train on current systems. Building the topologies of the deep learning network requires hand tuning, and implementing the network in hardware is expensive in both cost and power.In this paper, we evaluate deep learning models using three different computing architectures to address these problems: quantum computing to train complex topologies, high performance computing (HPC) to automatically determine network topology, and neuromorphic computing for a low-power hardware implementation. Due to input size limitations of current quantum computers we use the MNIST dataset for our evaluation.The results show the possibility of using the three architectures in tandem to explore complex deep learning networks that are untrainable using a von Neumann architecture. We show that a quantum computer can find high quality values of intra-layer connections and weights, while yielding a tractable time result as the complexity of the network increases; a high performance computer can find optimal layer-based topologies; and a neuromorphic computer can represent the complex topology and weights derived from the other architectures in low power memristive hardware.This represents a new capability that is not feasible with current von Neumann architecture. It potentially enables the ability to solve very complicated problems unsolvable with current computing technologies.
97 MATHEMATICS AND COMPUTING↗
Method and system of filtering and recommending documents
Disclosed is a method and system for discovering documents using a computer and providing a small set of the most relevant documents to the attention of a human observer. Using the method, the computer obtains a seed document from the user and generates a seed document vector using term frequency-inverse corpus frequency weighting. A keyword index for a plurality of source documents can be compared with the weighted terms of the seed document vector. The comparison is then filtered to reduce the number of documents, which define an initial subset of the source documents. Initial subset vectors are generated and compared to the seed document vector to obtain a similarity value for each comparison. Based on the similarity value, the method then recommends one or more of the source documents.
97 MATHEMATICS AND COMPUTING↗
Machine Learning and Understanding for Intelligent Extreme Scale Scientific Computing and Discovery. DOE Workshop Report, January 7–9, 2015, Rockville, MD
Large-scale parallel simulations and data analysis drive scientific discovery across many disciplines. To drive larger and more detailed simulations, and deal with larger data volumes, exascale machines capable of 1018 operations per second are expected within the next five to ten years. However, the complexity of developing and adapting modern simulation codes for these architectures is increasing rapidly. Workloads on exascale machines will be billion-way parallel. Exploiting the full capability of the machine will require carefully assigning application tasks to cores, accelerators, deep memories, and other heterogeneous compute resources, while simultaneously optimizing for time to solution, data movement, power, and resilience. Optimizations that improve performance on one machine may slow down another. Worse, applications themselves are dauntingly complex. Production simulations comprise millions of lines of code and use sophisticated, adaptive algorithms whose performance is input-dependent. Complex workflows can couple multi-physics simulation with data preprocessing and post-processing modules.
97 MATHEMATICS AND COMPUTING↗
A Bridge for Accelerating Materials by Design
Recent technical advances in the area of nanoscale imaging, spectroscopy, and scattering/diffraction have led to unprecedented capabilities for investigating materials structural, dynamical and functional characteristics. In addition, recent advances in computational algorithms and computer capacities that are orders of magnitude larger/faster have enabled large-scale simulations of materials properties starting with nothing but the identity of the atomic species and the basic principles of quantum- and statistical-mechanics and thermodynamics. Along with these advances, an explosion of high-resolution data has emerged. This confluence of capabilities and rise of big data offer grand opportunities for advancing materials sciences but also introduce several challenges. In this editorial we identify challenges impeding progress towards advancing materials by design (e.g., the design/discovery of materials with improved properties/performance), possible solutions, and provide examples of scientific issues that can be addressed by using a tightly integrated approach where theory and experiments are linked through big-deep data.
97 MATHEMATICS AND COMPUTING↗
ASCR Cybersecurity for Scientific Computing Integrity - Research Pathways and Ideas Workshop
At the request of the U.S. Department of Energy's (DOE) Office of Science (SC) Advanced Scientific Computing Research (ASCR) program office, a workshop was held June 2-3, 2015, in Gaithersburg, MD, to identify potential long term (10 to +20 year) cybersecurity fundamental basic research and development challenges, strategies and roadmap facing future high performance computing (HPC), networks, data centers, and extreme-scale scientific user facilities. This workshop was a follow-on to the workshop held January 7-9, 2015, in Rockville, MD, that examined higher level ideas about scientific computing integrity specific to the mission of the DOE Office of Science. Issues included research computation and simulation that takes place on ASCR computing facilities and networks, as well as network-connected scientific instruments, such as those run by various DOE Office of Science programs. Workshop participants included researchers and operational staff from DOE national laboratories, as well as academic researchers and industry experts. Participants were selected based on the submission of abstracts relating to the topics discussed in the previous workshop report [1] and also from other ASCR reports, including "Abstract Machine Models and Proxy Architectures for Exascale Computing" [27], the DOE "Preliminary Conceptual Design for an Exascale Computing Initiative" [28], and the January 2015 machine learning workshop [29]. The workshop was also attended by several observers from DOE and other government agencies. The workshop was divided into three topic areas: (1) Trustworthy Supercomputing, (2) Extreme-Scale Data, Knowledge, and Analytics for Understanding and Improving Cybersecurity, and (3) Trust within High-end Networking and Data Centers. Participants were divided into three corresponding teams based on the category of their abstracts. The workshop began with a series of talks from the program manager and workshop chair, followed by the leaders for each of the three topics and a representative of each of the four major DOE Office of Science Advanced Scientific Computing Research Facilities: the Argonne Leadership Computing Facility (ALCF), the Energy Sciences Network (ESnet), the National Energy Research Scientific Computing Center (NERSC), and the Oak Ridge Leadership Computing Facility (OLCF). The rest of the workshop consisted of topical breakout discussions and focused writing periods that produced much of this report.
97 MATHEMATICS AND COMPUTING↗
Cloud computing method for dynamically scaling a process across physical machine boundaries
A cloud computing platform includes first device having a graph or tree structure with a node which receives data. The data is processed by the node or communicated to a child node for processing. A first node in the graph or tree structure determines the reconfiguration of a portion of the graph or tree structure on a second device. The reconfiguration may include moving a second node and some or all of its descendant nodes. The second and descendant nodes may be copied to the second device.
Gillen, Robert E.↗