Search NASA⌕ Search

SEARCH · Search NASA

Results for “large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46

Ephemeral Learning - Augmenting Triggers with Online-Trained Normalizing Flows

The large data rates at the LHC require an online trigger system to select relevant collisions. Rather than compressing individual events, we propose to compress an entire data set at once. We use a normalizing flow as a deep generative model to learn the probability density of the data online. The events are then represented by the generative neural network and can be inspected offline for anomalies or used for other analysis purposes. We demonstrate our new approach for a toy model and a correlation-enhanced bump hunt.

97 MATHEMATICS AND COMPUTING↗

Mass agnostic jet taggers

Searching for new physics in large data sets needs a balance between two competing effects—signal identification vs background distortion. In this work, we perform a systematic study of both single variable and multivariate jet tagging methods that aim for this balance. The methods preserve the shape of the background distribution by either augmenting the training procedure or the data itself. Multiple quantitative metrics to compare the methods are considered, for tagging 2-, 3-, or 4-prong jets from the QCD background. This is the first study to show that the data augmentation techniques of Planing and PCA based scaling deliver similar performance as the augmented training techniques of Adversarial NN and uBoost, but are both easier to implement and computationally cheaper.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Water resource recovery modelling 2021 (WRRmod2021 conference)

Our society is transitioning fast into the digital age, spurred by development of cheap and new sensing technology, breakthroughs in computing, and development of efficient algorithms for optimization. This transition is also visible in the field of wastewater treatment and is driving new model developments, especially by exploiting the large data sets available with many utilities. Not surprisingly, WRRmod2021 had featured a strong session on ‘data-driven models and digitalization’ focused on this hot topic. At the same time, engineering practice calls for more robust models for performance evaluation and optimization of both conventional facilities and innovative processes. As a result, the WRRmod2021 program also exhibited sessions on modelling of new process units (e.g., aerobic granular sludge), modelling of the nitrogen cycle, and integrated/plant-wide modelling.

54 ENVIRONMENTAL SCIENCES↗

OVIS 3.2 user's guide.

This document describes how to obtain, install, use, and enjoy a better life with OVIS version 3.2. The OVIS project targets scalable, real-time analysis of very large data sets. We characterize the behaviors of elements and aggregations of elements (e.g., across space and time) in data sets in order to detect meaningful conditions and anomalous behaviors. We are particularly interested in determining anomalous behaviors that can be used as advance indicators of significant events of which notification can be made or upon which action can be taken or invoked. The OVIS open source tool (BSD license) is available for download at ovis.ca.sandia.gov. While we intend for it to support a variety of application domains, the OVIS tool was initially developed for, and continues to be primarily tuned for, the investigation of High Performance Compute (HPC) cluster system health. In this application it is intended to be both a system administrator tool for monitoring and a system engineer tool for exploring the system state in depth. OVIS 3.2 provides a variety of statistical tools for examining the behavior of elements in a cluster (e.g., nodes, racks) and associated resources (e.g., storage appliances and network switches). It provides an interactive 3-D physical view in which the cluster elements can be colored by raw or derived element values (e.g., temperatures, memory errors). The visual display allows the user to easily determine abnormal or outlier behaviors. Additionally, it provides search capabilities for certain scheduler logs. The OVIS capabilities were designed to be highly interactive - for example, the job search may drive an analysis which in turn may drive the user generation of a derived value which would then be examined on the physical display. The OVIS project envisions the capabilities of its tools applied to compute cluster monitoring. In the future, integration with the scheduler or resource manager will be included in a release to enable intelligent resource utilization. For example, nodes that are deemed less healthy (i.e., nodes that exhibit outlier behavior with respect to some set of variables shown to be correlated with future failure) can be discovered and assigned to shorter duration or less important jobs. Further, HPC applications with fault-tolerant capabilities would respond to changes in resource health and other OVIS notifications as needed, rather than undertaking preventative measures (e.g. checkpointing) at regular intervals unnecessarily.

97 MATHEMATICS AND COMPUTING↗

Fourteenth-Sixteenth Microbial Genomics Conference-2006-2008

The concept of an annual meeting on the E. coli genome was formulated at the Banbury Center Conference on the Genome of E. coli in October, 1991. The first meeting was held on September 10-14, 1992 at the University of Wisconsin, and this was followed by a yearly series of meetings, and by an expansion to include The fourteenth meeting took place September 24-28, 2006 at Lake Arrowhead, CA, the fifteenth September 16-20, 2007 at the University of Maryland, College Park, MD, and the sixteenth September 14-18, 2008 at Lake Arrowhead. The full program for the 16th meeting is attached. There have been rapid and exciting advances in microbial genomics that now make possible comparing large data sets of sequences from a wide variety of microbial genomes, and from whole microbial communities. Examining the “microbiomes”, the living microbial communities in different host organisms opens up many possibilities for understanding the landscape presented to pathogenic microorganisms. For quite some time there has been a shifting emphasis from pure sequence data to trying to understand how to use that information to solve biological problems. Towards this end new technologies are being developed and improved. Using genetics, functional genomics, and proteomics has been the recent focus of many different laboratories. A key element is the integration of different aspects of microbiology, sequencing technology, analysis techniques, and bioinformatics. The goal of these conference is to provide a regular forum for these interactions to occur. While there have been a number of genome conferences, what distinguishes the Microbial Genomics Conference is its emphasis on bringing together biology and genetics with sequencing and bioinformatics. Also, this conference is the longest continuing meeting, now established as a major regular annual meeting. In addition to its coverage of microbial genomes and biodiversity, the meetings also highlight microbial communities and the use of genomic information to aid in the understanding of pathogens and biothreats. An additional focus cover s“bioenergetics. The meetings have a mix of invited and participant-initiated presentations and poster sessions during which investigators from different disciplines become familiar with available data bases and new tools facilitating coordination of information. The fields are moving very fast both in the acquisition of new knowledge of genome contents and also in the management and analysis of the information. The key is connecting bodies of knowledge on sequences, genetic organization and regulation to be able to relate the significance of this information to understanding cellular processes. To our knowledge, no other meeting synthesizes the biology of organisms, sequence information and database analysis, as well as the comparison with other completed genome sequences.

59 BASIC BIOLOGICAL SCIENCES↗

OVIS 2.0 user%3CU%2B2019%3Es guide.

This document describes how to obtain, install, use, and enjoy a better life with OVIS version 2.0. The OVIS project targets scalable, real-time analysis of very large data sets. We characterize the behaviors of elements and aggregations of elements (e.g., across space and time) in data sets in order to detect anomalous behaviors. We are particularly interested in determining anomalous behaviors that can be used as advance indicators of significant events of which notification can be made or upon which action can be taken or invoked. The OVIS open source tool (BSD license) is available for download at ovis.ca.sandia.gov. While we intend for it to support a variety of application domains, the OVIS tool was initially developed for, and continues to be primarily tuned for, the investigation of High Performance Compute (HPC) cluster system health. In this application it is intended to be both a system administrator tool for monitoring and a system engineer tool for exploring the system state in depth. OVIS 2.0 provides a variety of statistical tools for examining the behavior of elements in a cluster (e.g., nodes, racks) and associated resources (e.g., storage appliances and network switches). It calculates and reports model values and outliers relative to those models. Additionally, it provides an interactive 3D physical view in which the cluster elements can be colored by raw element values (e.g., temperatures, memory errors) or by the comparison of those values to a given model. The analysis tools and the visual display allow the user to easily determine abnormal or outlier behaviors. The OVIS project envisions the OVIS tool, when applied to compute cluster monitoring, to be used in conjunction with the scheduler or resource manager in order to enable intelligent resource utilization. For example, nodes that are deemed less healthy, that is, nodes that exhibit outlier behavior in some variable, or set of variables, that has shown to be correlated with future failure, can be discovered and assigned to shorter duration or less important jobs. Further, applications with fault-tolerant capabilities can invoke those mechanisms on demand, based upon notification of a node exhibiting impending failure conditions, rather than performing such mechanisms (e.g. checkpointing) at regular intervals unnecessarily.

99 GENERAL AND MISCELLANEOUS↗

Fast 3D Surface Extraction

Ocean scientists searching for isosurfaces and/or thresholds of interest in high resolution 3D datasets required a tedious and time-consuming interactive exploration experience. PISTON research and development activities are enabling ocean scientists to rapidly and interactively explore isosurfaces and thresholds in their large data sets using a simple slider with real time calculation and visualization of these features. Ocean Scientists can now visualize more features in less time, helping them gain a better understanding of the high resolution data sets they work with on a daily basis. Isosurface timings (512 3 grid): VTK 7.7 s, Parallel VTK (48-core) 1.3 s, PISTON OpenMP (48-core) 0.2 s, PISTON CUDA (Quadro 6000) 0.1 s.

97 MATHEMATICS AND COMPUTING↗

Mastering Uncertainty and Risk at Multiple Time Scales in the Future Electrical Grid

Today's electrical grids enjoy a relatively clean separation of spatio-temporal scales yielding a compartmentalization of grid design, optimization, control and risk assessment allowing for the use of conventional mathematical tools within each area. In contrast, the future grid will incorporate time-intermittent renewable generation, operate via faster electrical markets, and tap the latent control capability at finer grid modeling scales; creating a fundamentally new set of couplings across spatiotemporal scales and requiring revolutionary advances in mathematics techniques to bridge these scales. One example is found in decade-scale grid expansion planning in which today's algorithms assume accurate load forecasts and well-controlled generation. Incorporating intermittent renewable generation creates fluctuating network flows at the hourly time scale, inherently linking the ability of a transmission line to deliver electrical power to hourly operational decisions. New operations-based planning algorithms are required, creating new mathematical challenges. Spatio-temporal scales are also crossed when the future grid's minute-scale fluctuations in network flows (due to intermittent generation) create a disordered state upon which second-scale transient grid dynamics propagate effectively invalidating today's on-line dynamic stability analyses. Addressing this challenge requires new on-line algorithms that use large data streams from new grid sensing technologies to physically aggregate across many spatial scales to create responsive, data-driven dynamic models. Here, we sketch the mathematical foundations of these problems and potential solutions.

24 POWER TRANSMISSION AND DISTRIBUTION↗

The Secret Life of Quarks, Final Report for the University of North Carolina at Chapel Hill

This final report summarizes activities and results at the University of North Carolina as part of the the SciDAC-2 Project The Secret Life of Quarks: National Computational Infrastructure for Lattice Quantum Chromodynamics. The overall objective of the project is to construct the software needed to study quantum chromo- dynamics (QCD), the theory of the strong interactions of subatomic physics, and similar strongly coupled gauge theories anticipated to be of importance in the LHC era. It built upon the successful efforts of the SciDAC-1 project National Computational Infrastructure for Lattice Gauge Theory, in which a QCD Applications Programming Interface (QCD API) was developed that enables lat- tice gauge theorists to make effective use of a wide variety of massively parallel computers. In the SciDAC-2 project, optimized versions of the QCD API were being created for the IBM Blue- Gene/L (BG/L) and BlueGene/P (BG/P), the Cray XT3/XT4 and its successors, and clusters based on multi-core processors and Infiniband communications networks. The QCD API is being used to enhance the performance of the major QCD community codes and to create new applications. Software libraries of physics tools have been expanded to contain sharable building blocks for inclusion in application codes, performance analysis and visualization tools, and software for au- tomation of physics work flow. New software tools were designed for managing the large data sets generated in lattice QCD simulations, and for sharing them through the International Lattice Data Grid consortium. As part of the overall project, researchers at UNC were funded through ASCR to work in three general areas. The main thrust has been performance instrumentation and analysis in support of the SciDAC QCD code base as it evolved and as it moved to new computation platforms. In support of the performance activities, performance data was to be collected in a database for the purpose of broader analysis. Third, the UNC work was done at RENCI (Renaissance Computing Institute), which has extensive expertise and facilities for scientific data visualization, so we acted in an ongoing consulting and support role in that area.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Final Technical Report for "High Energy Physics at The University of Iowa"

Particle Physics explores the very fundamental building blocks of our universe: the nature of forces, of space and time. By exploring very energetic collisions of sub-nuclear particles with sophisticated detectors at the colliding beam accelerators (as well as others), experimental particle physicists have established the current theory known as the Standard Model (SM), one of the several theoretical postulates to explain our everyday world. It explains all phenomena known up to a very small fraction of a second after the Big Bang to a high precision; the Higgs boson, discovered recently, was the last of the particle predicted by the SM. However, many other phenomena, like existence of dark energy, dark matter, absence of anti-matter, the parameters in the SM, neutrino masses etc. are not explained by the SM. So, in order to find out what lies beyond the SM, i.e., what conditions at the earliest fractions of the first second of the universe gave rise to the SM, we constructed the Large Hadron Collider (LHC) at CERN after the Tevatron collider at Fermi National Accelerator Laboratory. Each of these projects helped us push the boundary further with new insights as we explore a yet higher energy regime. The experiments are extremely complex, and as we push the boundaries of our existing knowledge, it also requires pushing the boundaries of our technical knowhow. So, not only do we pursue humankind’s most basic intellectual pursuit of knowledge, we help develop technology that benefits today’s highly technical society. Our trained Ph.D. students become experts at fast computing, manipulation of large data volumes and databases, developing cloud computing, fast electronics, advanced detector developments, and complex interfaces in several of these areas. Many of the Particle physics Ph.D.s build their careers at various technology and computing facilities, even financial institutions use some of their skills of simulation and statistical prowess. Additionally, last but not least, today’s discoveries make for tomorrow’s practical uses of an improved life style, case in point, internet technology, fiber optics, and many such things. At The University of Iowa we are involved in the LHC experiments, ATLAS and CMS, building equipment, with calibration and maintenance, supporting the infrastructure in hardware, software and analysis as well as participating in various aspects of data analyses. Our theory group works on fundamentals of field theories and on exploration of non-accelerator high energy neutrinos and possible dark matter searches.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Center for Plasma Edge Simulation (CPES) (Rutgers University Final Report)

The CPES scientific simulations run at scale on leadership class machines, collaborate at runtime and produce and exchange large data sizes, which present multiple I/O and data management challenges. During the CPES project, the Rutgers team worked with the rest of the CPES team to address these challenges at different levels, and specifically (1) at the data transport and communication level through the DART (Decoupled and Asynchronous Remote Data Transfers) framework, and (2) at the data management and services level through the DataSpaces and ActiveSpaces frameworks. These frameworks and their impact are briefly described.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Toward Interactive Scenario Analysis and Exploration

As Modeling and Simulation (M&S) tools have matured, their applicability and importance have increased across many national security challenges. In particular, they provide a way to test how something may behave without the need to do real world testing. However, current and future changes across several factors including capabilities, policy, and funding are driving a need for rapid response or evaluation in ways that many M&S tools cannot address. Issues around large data, computational requirements, delivery mechanisms, and analyst involvement already exist and pose significant challenges. Furthermore, rising expectations, rising input complexity, and increasing depth of analysis will only increase the difficulty of these challenges. In this study we examine whether innovations in M&S software coupled with advances in ''cloud'' computing and ''big-data'' methodologies can overcome many of these challenges. In particular, we propose a simple, horizontally-scalable distributed computing environment that could provide the foundation (i.e. ''cloud'') for next-generation M&S-based applications based on the notion of ''parallel multi-simulation''. In our context, the goal of parallel multi- simulation is to consider as many simultaneous paths of execution as possible. Therefore, with sufficient resources, the complexity is dominated by the cost of single scenario runs as opposed to the number of runs required. We show the feasibility of this architecture through a stable prototype implementation coupled with the Umbra Simulation Framework [6]. Finally, we highlight the utility through multiple novel analysis tools and by showing the performance improvement compared to existing tools.

97 MATHEMATICS AND COMPUTING↗

Institute for Sustained Performance, Energy, and Resilience (SuPER)

The University of Tennessee (UTK) and University of Texas at El Paso (UTEP) partnership supported the three main thrusts of the SUPER project---performance, energy, and resilience. The UTK-UTEP effort thus helped advance the main goal of SUPER, which was to ensure that DOE's computational scientists can successfully exploit the emerging generation of high performance computing (HPC) systems. This goal is being met by providing application scientists with strategies and tools to productively maximize performance, conserve energy, and attain resilience. The primary vehicle through which UTK provided performance measurement support to SUPER and the larger HPC community is the Performance Application Programming Interface (PAPI). PAPI is an ongoing project that provides a consistent interface and methodology for collecting hardware performance information from various hardware and software components, including most major CPUs, GPUs and accelerators, interconnects, I/O systems, and power interfaces, as well as virtual cloud environments. The PAPI software is widely used for performance modeling of scientific and engineering applications---for example, the HOMME (High Order Methods Modeling Environment) climate code, and the GAMESS and NWChem computational chemistry codes---on DOE supercomputers. PAPI is widely deployed as middleware for use by higher-level profiling, tracing, and sampling tools (e.g., CrayPat, HPCToolkit, Scalasca, Score-P, TAU, Vampir, PerfExpert), making it the de facto standard for hardware counter analysis. PAPI has established itself as fundamental software infrastructure in every application domain (spanning academia, government, and industry), where improving performance can be mission critical. Ultimately, as more application scientists migrate their applications to HPC platforms, they will benefit from the extended capabilities this grant brought to PAPI to analyze and optimize performance in these environments, whether they use PAPI directly, or via third-party performance tools. Capabilities added to PAPI through this grant include support for new architectures such as the lastest GPU and Xeon Phi accelerators, and advanced power measurement and management features. Another important topic for the UTK team was providing support for a rich ecosystem of different fault management strategies in the context of parallel computing. Our long term efforts have been oriented toward proposing flexible strategies and providing building boxes that application developers can use to build the most efficient fault management technique for their application. These efforts span across the entire software spectrum, from theoretical models of existing strategies to easily assess their performance, to algorithmic modifications to take advantage of specific mathematical properties for data redundancy and to extensions to widely used programming paradigms to empower the application developers to deal with all types of faults. We have also continued our tight collaborations with users to help them adopt these technologies to ensure their application always deliver meaningful scientific data. Large supercomputer systems are becoming more and more power and energy constrained, and future systems and applications running on them will need to be optimized to run under power caps and/or minimize energy consumption. The UTEP team contributed to the SUPER energy thrust by developing power modeling methodologies and investigating power management strategies. Scalability modeling results showed that some applications can scale better with respect to an increasing power budget than with respect to only the number of processors. Power management, in particular shifting power to processors on the critical path of an application execution, can reduce perturbation due to system noise and other sources of runtime variability, which are growing problems on large-scale power-constrained computer systems.

42 ENGINEERING↗

Evaluating Near-Term Adiabatic Quantum Computing

This report summarizes the first year’s effort on the Enceladus project, under which Sandia was asked to evaluate the potential advantages of adiabatic quantum computing for analyzing large data sets in the near future, 5-to-10 years from now. We were not specifically evaluating the machine being sold by D-Wave Systems, Inc; we were asked to anticipate what future adiabatic quantum computers might be able to achieve. While realizing that the greatest potential anticipated from quantum computation is still far into the future, a special purpose quantum computing capability, Adiabatic Quantum Optimization (AQO), is under active development and is maturing relatively rapidly; indeed, D-Wave Systems Inc. already offers an AQO device based on superconducting flux qubits. The AQO architecture solves a particular class of problem, namely unconstrained quadratic Boolean optimization. Problems in this class include many interesting and important instances. Because of this, further investigation is warranted into the range of applicability of this class of problem for addressing challenges of analyzing big data sets and the effectiveness of AQO devices to perform specific analyses on big data. Further, it is of interest to also consider the potential effectiveness of anticipated special purpose adiabatic quantum computers (AQCs), in general, for accelerating the analysis of big data sets. The objective of the present investigation is an evaluation of the potential of AQC to benefit analysis of big data problems in the next five to ten years, with our main focus being on AQO because of its relative maturity. We are not specifically assessing the efficacy of the D-Wave computing systems, though we do hope to perform some experimental calculations on that device in the sequel to this project, at least to provide some data to compare with our theoretical estimates.

97 MATHEMATICS AND COMPUTING↗

An Analytical Technology Platform to Identify Symbiotic Interactions Between PGPR and Root Microbiome Members from Biofuel Feedstocks

The goal of this STTR project was to establish a commericially-viable technology platform that can be used to develop formulations of plant growth promoting rhizobacteria (PGPR) that can be used as biofertilizers for improved production of woody biomass. The platform has been designed to screen thousands of different formulations of microorganisms in search for those formulations that best support the growth and survival of PGPR species. The focus of the Phase I project was to provide the existing technology with new capabilities that define its commerical pathway. These goals were to (1) adapt the existing platform for use with Populus isolates and (2) developing a highly-automated image analysis software tool that can rapidly screen large data sets to statistically identify optimized biofertilizer formulations. The project has produced a software routine for this screening and identification need, and has generated preliminary data for use of this approach to identify symbiotic PGPR communities that may be useful for promoting Populus growth.

09 BIOMASS FUELS↗

Redox transistors based on TiO 2 for analogue neuromorphic computing

The ability to train deep neural networks on large data sets have made significant impacts onto artificial intelligence, but consume significant amounts of energy due to the need to move information from memory to logic units. In-memory "neuromorphic" computing presents an alternative framework that processes information directly on memory elements. In-memory computing has been limited by the poor performance of the analogue information storage element, often phase-change memory or memristors. To solve this problem, we developed two types of "redox transistors" using TiO 2 (anatase) which stores analogue information states through the electrochemical concentration of dopants in the crystal. The first type of redox transistor uses lithium as the electrochemical dopant ion, and its key advantage is low operating voltage. The second uses oxygen vacancies as the dopant, which is CMOS compatible and can retain state even when scaled to nanosized dimensions. Both devices offer significant advantages in terms of predictable analogue switching over conventional filamentary-based devices, and provide a significant advance in developing materials and devices for neuromorphic computing.

36 MATERIALS SCIENCE↗

Rheo-Structural Spectroscopy: Fingerprinting the In Situ Response of Fluids to Arbitrary Flow Fields

The objectives of this project were to develop new sample environments, measurement methodologies and associated modeling tools for characterizing the structural response to arbitrarily complex processing flows using small angle scattering, and to apply these new tools for understanding the fundamental physics governing the structuring of anisotropic particulate and polymeric materials under flow histories and conditions relevant to industrial processing flows. The research resulted in the development and implementation of a new sample environment, the fluidic four roll mill (FFoRM), for in situ small angle neutron and X-ray scattering (SANS/SAXS) measurements. These measurements are capable of generating large data sets that “fingerprint” how a complex fluid responds to a wide range of flow histories involving time variations in deformation type and rate. New modeling tools were developed to extract detailed microstructural information from such data sets, including orientation distribution functions and interparticle correlation functions, as well as reduced-order parametric descriptors of these high-dimensional functions that can be used to readily map, visualize and interpret a fluid’s structural response to its flow history. These new tools were applied to a range of model materials involving elongated particle suspensions in order to provide new insights into the physics of how flow couples with orientational and structural order in complex flows, particularly under non-dilute conditions for which no accurate theories currently exist. Using these investigations, we elucidated a number of new insights into the fundamental phenomena driving such process-structure-property relationships. These findings provide guidance for the further development of rheological models, and ultimately can inform the rational and model-based design of flow processes to achieve optimized orientational ordering that is key to the properties and function of a wide range of energy-relevant materials.

36 MATERIALS SCIENCE↗

Simultaneous mitigation of density and energy errors in approximate DFT for transition metal chemistry (Final Technical Report)

There were three major goals and objectives of this project: 1) Develop tools to understand density-driven errors in transition metal complexes, 2) Evaluate and minimize energy delocalization error and static correlation error through judicious functional choice, and 3) applying this workflow to machine-learning accelerated screening of redox couples. Over the reporting period, we developed a framework for eliminating flat plane errors. We introduced fully non-empirical coefficients. We demonstrated the approach on both molecules and solids. We investigated and eliminated density driven errors and demonstrated their impact on potential energy surfaces. We trained machine learning models both in a method-dependent fashion and to predict errors in method accuracy. We built large data sets of small molecule energetics and multi-reference character.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗