Search NASA⌕ Search

SEARCH · Search NASA

Results for “structured data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

ATDM/ECP Milestone Memo WBS 2.3.4.04 / SNL ATDM Data and Visualization Projects STDV04-21 - [MS1/YR2] Q3: Prototype Catalyst/ParaView in-situ viz for unsteady RV flow on ATS-1

ParaView Catalyst is an API for accessing the scalable visualization infrastructure of ParaView in an in-situ context. In-situ visualization allows simulation codes to access data post-processing operations while the simulation is running. In-situ techniques can reduce data post-processing time, allow computational steering, and increase the resolution and frequency of data output. For a simulation code to use ParaView Catalyst, adapter code needs to be created that interfaces the simulations data structures to ParaView/VTK data structures. Under ATDM, Catalyst is to be integrated with SPARC, a code used for simulation of unsteady reentry vehicle flow.

97 MATHEMATICS AND COMPUTING↗

Preventing Failures By Dataset Shift Detection in Safety-Critical Graph Applications

Dataset shift refers to the problem where the input data distribution may change over time (e.g., between training and test stages). Since this can be a critical bottleneck in several safety-critical applications such as healthcare, drug-discovery, etc., dataset shift detection has become an important research issue in machine learning. Though several existing efforts have focused on image/video data, applications with graph-structured data have not received sufficient attention. Therefore, in this paper, we investigate the problem of detecting shifts in graph structured data through the lens of statistical hypothesis testing. Specifically, we propose a practical two-sample test based approach for shift detection in large-scale graph structured data. Our approach is very flexible in that it is suitable for both undirected and directed graphs, and eliminates the need for equal sample sizes. Using empirical studies, we demonstrate the effectiveness of the proposed test in detecting dataset shifts. We also corroborate these findings using real-world datasets, characterized by directed graphs and a large number of nodes.

97 MATHEMATICS AND COMPUTING↗

Simit: A Language for Physical Simulation

With existing programming tools, writing high-performance simulation code is labor intensive and requires sacrificing readability and portability. Here, the alternative is to prototype simulations in a high-level language like Matlab, thereby sacrificing performance. The Matlab programming model naturally describes the behavior of an entire physical system using the language of linear algebra. However, simulations also manipulate individual geometric elements, which are best represented using linked data structures like meshes. Translating between the linked data structures and linear algebra comes at significant cost, both to the programmer and to the machine. High-performance implementations avoid the cost by rephrasing the computation in terms of linked or index data structures, leaving the code complicated and monolithic, often increasing its size by an order of magnitude. In this article, we present Simit, a new language for physical simulations that lets the programmer view the system both as a linked data structure in the form of a hypergraph and as a set of global vectors, matrices, and tensors depending on what is convenient at any given time. Simit provides a novel assembly construct that makes it conceptually easy and computationally efficient to move between the two abstractions. Using the information provided by the assembly construct, the compiler generates efficient in-place computation on the graph. We demonstrate that Simit is easy to use: a Simit program is typically shorter than a Matlab program; that it is high performance: a Simit program running sequentially on a CPU performs comparably to hand-optimized simulations; and that it is portable: Simit programs can be compiled for GPUs with no change to the program, delivering 4 to 20× speedups over our optimized CPU code.

97 MATHEMATICS AND COMPUTING↗

Evolving data standards for cryo-EM structures

Electron cryo-microscopy (cryo-EM) is increasingly being used to determine 3D structures of a broad spectrum of biological specimens from molecules to cells. Anticipating this progress in the early 2000s, an international collaboration of scientists with expertise in both cryo-EM and structure data archiving was established (EMDataResource, previously known as EMDataBank). The major goals of the collaboration have been twofold: to develop the necessary infrastructure for archiving cryo-EM-derived density maps and models, and to promote development of cryo-EM structure validation standards. We describe how cryo-EM data archiving and validation have been developed and jointly coordinated for the Electron Microscopy Data Bank and Protein Data Bank archives over the past two decades, as well as the impact of evolving technology on data standards. Just as for X-ray crystallography and nuclear magnetic resonance, engaging the scientific community via workshops and challenging activities has played a central role in developing recommendations and requirements for the cryo-EM structure data archives.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Denoising Autoencoder for Reconstructing Sensor Observation Data and Predicting Evapotranspiration: Noisy and Missing Values Repair and Uncertainty Quantification

Abstract Machine learning (ML) methods applied in scientific research often deal with interrelated features in high‐dimensional data. Reducing data noise and redundancy is needed to increase prediction accuracy and efficiency especially when dealing with data from field sensors. We explored an unsupervised learning method, the denoising autoencoder (DAE), to extract the underlying data structure from noisy raw data in the context of predicting hydrologic quantities from multiple field sensors. These sensors have intrinsic instrumental noise and occasional malfunctions that cause missing values. Our DAE neural network reconstructed meteorological sensor data containing noise and missing values to predict evapotranspiration in a mountainous watershed. The DAE reconstructed the sensor variables with a mean coefficient of determination value of 0.77 across 15 dimensions representing individual sensors. It reduced variance and bias uncertainties compared to a classical autoencoder model. The reconstruction quality varied across dimensions depending on their cross‐correlation and alignment with the underlying data structure. Uncertainties arising from the model structure were overall higher than those resulting from data corruption. We attached the DAE structure to a downstream ET‐prediction neural network in three formats and achieved reasonably accurate ET predictions . The use of the DAE notably reduced variance uncertainty in ET prediction. However, excessive variance reduction may be accompanied by an increase in bias due to the intrinsic bias‐variance tradeoff. Our method of evaluating and reducing uncertainties in aggregated data from different sources can be used to improve predictive models, process understanding, and uncertainty quantification for better water resource management. Plain Language Summary We present a machine learning method, namely the denoising autoencoder, which reduces the effects of data noise and missing values typically present in scientific data sets collected through sensor measurements. This method selects the most relevant information from noisy raw data collected by the instruments and fills in missing values. To demonstrate the effectiveness of our method, we applied it to predict evapotranspiration, a hydrologic variable that represents the water moved from the land surface to the atmosphere through a combination of evaporation and plant water use (transpiration). We also used a random sampling technique (the Monte Carlo method) to compare the uncertainty in the predictions when using the raw and noisy data versus the reconstructed data. The denoising process produced more accurate predictions of evapotranspiration with less uncertainty. Improved predictions of evapotranspiration can lead to a better understanding and accounting of water budgets. This ML approach is broadly suitable for a wide variety of applications that involve noisy sensor data with missing values. Key Points We used a denoising autoencoder (DAE) neural network to reduce noise in meteorological and soil sensor observations by on average We used Monte Carlo sampling to estimate the bias and variance of all model outputs, including uncertainty sources from data and the model We attached the DAE component to a downstream neural network to predict ET with the variance reduced by , compared to that without the DAE

denoising autoencoder↗

SLoG: Large-Scale Logging Middleware for HPC and Big Data Convergence

Cloud developers traditionally rely on purpose-specific services to provide the storage model they need for an application. In contrast, HPC developers have a much more limited choice, typically restricted to a centralized parallel file system for persistent storage. Unfortunately, these systems often offer low performance when subject to highly concurrent, conflicting I/O patterns. This makes difficult the implementation of inherently concurrent data structures such as distributed shared logs. Yet, this data structure is key to applications such as computational steering, data collection from physical sensor grids, or discrete event generators. In this paper we tackle this issue. We present SLoG, shared log middleware providing a shared log abstraction over a parallel file system, designed to circumvent the aforementioned limitations. We evaluate SLoG’s design on up to 100,000 cores of the Theta supercomputer: the results show high append velocity at scale while also providing substantial benefits for other persistent backend storage systems.

big data↗

Per-page control of physical address space distribution among memory modules

Systems, apparatuses, and methods for implementing per-page control of physical address space distribution among memory modules are disclosed. A computing system includes a plurality of processing units coupled to a plurality of memory modules. A determination is made as to which physical address space distribution granularity to implement for physical memory pages allocated for a first data structure. The determination can be made on a per-data-structure basis (e.g., file, page, block, etc.) or on a per-application-basis. A physical address space distribution granularity is encoded as a property of each physical memory page allocated for the first data structure, and physical memory pages of the first data structure distributed across the plurality of memory modules based on a selected physical address space distribution granularity. Page table entries (PTEs) may be annotated with the selected physical address space distribution granularity, using an addressing mapping granularity (AMG) field of a page table entry, where the granularity may be, for example, a fine-grain distribution granularity or a coarse-grain distribution granularity.

Jayasena, Nuwan S.↗

Robust structural analysis of native biological macromolecules from multi-crystal anomalous diffraction data

Anomalous diffraction signals from typical native macromolecules are very weak, frustrating their use in structure determination. Here, native SAD procedures are described for enhancing the signal to noise in anomalous diffraction by using multiple crystals are described. Five applications demonstrate that truly routine structure determination is possible without the need for heavy atoms. Structure determinations for biological macromolecules that have no known structural antecedents typically involve the incorporation of heavier atoms than those found natively in biological molecules. Currently, selenomethionyl proteins analyzed using single- or multi-wavelength anomalous diffraction (SAD or MAD) data predominate for such de novo analyses. Naturally occurring metal ions such as zinc or iron often suffice in MAD or SAD experiments, and sulfur SAD has been an option since it was first demonstrated using crambin 30 years ago; however, SAD analyses of structures containing only light atoms (Z{sub max} ≤ 20) have not been common. Here, robust procedures for enhancing the signal to noise in measurements of anomalous diffraction by combining data collected from several crystals at a lower than usual X-ray energy are described. This multi-crystal native SAD method was applied in five structure determinations, using between five and 13 crystals to determine substructures of between four and 52 anomalous scatterers (Z ≤ 20) and then the full structures ranging from 127 to 1200 ordered residues per asymmetric unit at resolutions from 2.3 to 2.8 Å. Tests were devised to assure that all of the crystals used were statistically equivalent. Elemental identities for Ca, Cl, S, P and Mg were proven by f′′ scattering-factor refinements. The procedures are robust, indicating that truly routine structure determination of typical native macromolecules is realised. Synchrotron beamlines that are optimized for low-energy X-ray diffraction measurements will facilitate such direct structural analysis.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Glprof: A Gprof inspired, Callgraph-oriented Per-Object Disseminating Memory Access Multi-Cache Profiler

Application analysis is facilitated through a number of program profiling tools. The tools vary in their complexity, ease of deployment, design, and profiling detail. Specifically, understand- ing, analyzing, and optimizing is of particular importance for scientific applications where minor changes in code paths and data-structure layout can have profound effects. Understanding how intricate data-structures are accessed and how a given memory system responds is a complex task. In this paper we describe a trace profiling tool, Glprof, specifically aimed to lessen the burden of the programmer to pin-point heavily involved data-structures during an application's run-time, and understand data-structure run-time usage. Moreover, we showcase the tool's modularity using additional cache simulation components. We elaborate on the tool's design, and features. Finally we demonstrate the application of our tool in the context of Spec bench- marks using the Glprof profiler and two concurrently running cache simulators, PPC440 and AMD Interlagos.

Janjusic, Tommy↗

Metall: A persistent memory allocator for data-centric analytics

Data analytics applications transform raw input data into analytics-specific data structures before performing analytics. Unfortunately, such data ingestion steps are often more expensive than analytics. In addition, various types of NVRAM devices are already used in many HPC systems today. Such devices will be useful for storing and reusing data structures beyond a single process life cycle. We developed Metall, a persistent memory allocator built on top of the memory-mapped file mechanism. Metall enables applications to transparently allocate custom C++ data structures into various types of persistent memories. Metall incorporates a concise and high-performance memory management algorithm inspired by Supermalloc and the rich C++ interface developed by Boost.Interprocess library. On a dynamic graph construction workload, Metall achieved up to 11.7x and 48.3x performance improvements over Boost.Interprocess and memkind (PMEM kind), respectively. We also demonstrate Metall’s high adaptability by integrating Metall into a graph processing framework, GraphBLAS Template Library. Here this study’s outcomes indicate that Metall will be a strong tool for accelerating future large-scale data analytics by allowing applications to leverage persistent memory efficiently.

97 MATHEMATICS AND COMPUTING↗

Lithium cobalt(II) pyrophosphate, Li[subscript 1.86]CoP[subscript 2]O[subscript 7], from synchrotron X-ray powder data

Structure refinement of high-resolution X-ray powder diffraction data of the title compound gave the composition Li{sub 1.865}CoP{sub 2}O{sub 7}, which is also verified by the ICP measurement. Two Co sites exist in the structure: one is a CoO{sub 5} square pyramid and the other is a CoO{sub 6} octahedron. They share edges and are further interconnected through P{sub 2}O{sub 7} groups, forming a three-dimensional framework, which exhibits different kinds of intersecting tunnels containing Li cations and could be of great interest in Li ion battery chemistry. The structure also exhibits cation disorder with 13.5% Co residing at the lithium (Li1) site. Co seems to have an average oxidation state of 2.135, as obtained from the strutural stochiometry that closely supports the magnetic susceptibility findings.

Zhou, Hui↗

System and method for memory allocation in a multiclass memory system

A system for memory allocation in a multiclass memory system includes a processor coupleable to a plurality of memories sharing a unified memory address space, and a library store to store a library of software functions. The processor identifies a type of a data structure in response to a memory allocation function call to the library for allocating memory to the data structure. Using the library, the processor allocates portions of the data structure among multiple memories of the multiclass memory system based on the type of the data structure.

97 MATHEMATICS AND COMPUTING↗

Graph-Based Analysis of Nuclear Smuggling Data

Much of the data that is collected and analyzed today is structural, consisting not only of entities but also of relationships between the entities. As a result, analysis applications rely upon automated structural data mining approaches to find patterns and concepts of interest. This ability to analyze structural data has become a particular challenge in many security-related domains. In these domains, focusing on the relationships between entities in the data is critical to detect important underlying patterns. In this study we apply structural data mining techniques to automate analysis of nuclear smuggling data. In particular, we choose to model the data as a graph and use graph-based relational learning to identify patterns and concepts of interest in the data. In this paper, we identify the analysis questions that are of importance to security analysts and describe the knowledge representation and data mining approach that we adopt for this challenge. We analyze the results using the Russian nuclear smuggling event database.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

A Mountaintop View Requires Minimal Sorting: A Faster Contour Tree Algorithm

Consider a scalar field f : M → R, where M is a triangulated simplicial mesh in R d . A level set, or contour, at value v is a connected component of f –1 (v). As v is changed, these contours change topology, merge into each other, or split. Contour trees are concise representations of f that track this contour behavior. The vertices of these trees are the critical points of f, where the gradient is zero. The edges represent changes in the topology of contours. It is a fundamental data structure in data analysis and visualization, and there is significant previous work (both theoretical and practical) on algorithms for constructing contour trees. Suppose M has n vertices, N facets, and t critical points. A classic result of Carr, Snoeyink, and Axen (2000) gives an algorithm that takes O(n log n+Nα(N)) time (where α(·) is the inverse Ackermann function). A further improvement to O(t log t + N) time was given by Chiang et al. All these algorithms involve a global sort of the critical points, a significant computational bottleneck. Unfortunately, lower bounds of Ω(t log t) also exist. We present the first algorithm that can avoid the global sort and has a refined time complexity that depends on the contour tree structure. Intuitively, if the tree is short and fat, we get significant improvements in running time. For a partition of the contour tree into a set of descending paths, P, our algorithm runs in O($\Sigma$ pϵP |p| log |p| + tα(t) + N). This is at most O(t log D + N), where D is the diameter of the contour tree. Moreover, it is O(tα(t) + N) for balanced trees, a significant improvement over the previous complexity. Our algorithm requires numerous ideas: partitioning the contour tree into join and split trees, a local growing procedure to iteratively build contour trees, and the use of heavy path decompositions for the time complexity analysis. There is a crucial use of a family of binomial heaps to maintain priorities, ensuring that any comparison made is between comparable nodes of the contour tree. We also prove lower bounds showing that the $\Sigma$ pϵP |p| log |p| complexity is inherent to computing contour trees.

97 MATHEMATICS AND COMPUTING↗

In-depth analysis on parallel processing patterns for high-performance Dataframes

The Data Science domain has expanded monumentally in both research and industry communities during the past decade, predominantly owing to the Big Data revolution. Artificial Intelligence (AI) and Machine Learning (ML) are bringing more complexities to data engineering applications, which are now integrated into data processing pipelines to process terabytes of data. Typically, a significant amount of time is spent on data preprocessing in these pipelines, and hence improving its efficiency directly impacts the overall pipeline performance. The community has recently embraced the concept of Dataframes as the de-facto data structure for data representation and manipulation. However, the most widely used serial Dataframes today (R, pandas) experience performance limitations while working on even moderately large data sets. We believe that there is plenty of room for improvement by taking a look at this problem from a high-performance computing point of view. In a prior publication, we presented a set of parallel processing patterns for distributed dataframe operators and the reference runtime implementation, Cylon. In this paper, we are expanding on the initial concept by introducing a cost model for evaluating the said patterns. Furthermore, we evaluate the performance of Cylon on the ORNL Summit supercomputer.

97 MATHEMATICS AND COMPUTING↗

Clinical Natural Language Processing for Radiation Oncology: A Review and Practical Primer

Natural language processing (NLP), which aims to convert human language into expressions that can be analyzed by computers, is one of the most rapidly developing and widely used technologies in the field of artificial intelligence. Natural language processing algorithms convert unstructured free text data into structured data that can be extracted and analyzed at scale. In medicine, this unlocking of the rich, expressive data within clinical free text in electronic medical records will help untap the full potential of big data for research and clinical purposes. Recent major NLP algorithmic advances have significantly improved the performance of these algorithms, leading to a surge in academic and industry interest in developing tools to automate information extraction and phenotyping from clinical texts. Thus, these technologies are poised to transform medical research and alter clinical practices in the future. Radiation oncology stands to benefit from NLP algorithms if they are appropriately developed and deployed, as they may enable advances such as automated inclusion of radiation therapy details into cancer registries, discovery of novel insights about cancer care, and improved patient data curation and presentation at the point of care. However, challenges remain before the full value of NLP is realized, such as the plethora of jargon specific to radiation oncology, nonstandard nomenclature, a lack of publicly available labeled data for model development, and interoperability limitations between radiation oncology data silos. Successful development and implementation of high quality and high value NLP models for radiation oncology will require close collaboration between computer scientists and the radiation oncology community. Here, we present a primer on artificial intelligence algorithms in general and NLP algorithms in particular; provide guidance on how to assess the performance of such algorithms; review prior research on NLP algorithms for oncology; and describe future avenues for NLP in radiation oncology research and clinics.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Donor behavior of Sb in ZnO

Electrical behavior of Sb in ZnO:Sb layers doped in a wide concentration range was studied using temperature dependent Hall effect measurements. The layers were grown by plasma-enhanced molecular beam epitaxy, and the Sb concentration was changed by varying the Sb flux, resulting in electron concentrations in the range of 10{sup 16} to nearly 10{sup 20} cm{sup -3}. Upon annealing, the electron concentration increased slightly and more notable was that the electron mobility significantly improved, reaching a room-temperature value of 110 cm{sup 2}/V s and a low-temperature value of 145 cm{sup 2}/V s, close to the maximum of {approx}155 cm{sup 2}/V s set by ionized impurity scattering. Hall data and structural data suggest that Sb predominantly occupies Zn sublattice positions and acts as a shallow donor in the whole concentration range studied. In the layers with high Sb content ({approx}1 at. %), acceptor-type compensating defects (possibly Sb on oxygen sites and/or point-defect complexes involving Sb{sub O}) are formed. The increase of electron concentration with increasing oxygen pressure and the increase in ZnO:Sb lattice parameter at high Sb concentrations suggest that acceptors involving Sb{sub O} rather than Sb{sub Zn}-2V{sub Zn} complexes are responsible for the compensation of the donors.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Data for Post-Access Report: Tidal Energy Resource Characterization and Model Validation via the Assessment of Cost-effective Spatially Distributed Current Measurements from microFloat Swarms

This project evaluated how high-resolution, spatially distributed field data can be used to refine and validate site-scale hydrodynamic simulations of tidal channels. Use of such spatially-distributed field observations or site-scale hydrodynamic simulations will be needed for producing accurate predictions of tidal energy production over larger arrays of tidal turbines. The data contained in this archive includes field observations of water velocity from a series of short surveys in Agate Pass taken on August 20th, 2020 using station-keeping ADCP (acoustic doppler current profiler) measurements, drifting downward-looking ADCPs, and microFloats, gathered by researchers at APL-UW under a previous project. It also includes high-resolution hydrodynamic model runs for the site during that time generated under TEAMER support. An explanatory file (.pdf format) describes the survey and field data structure. Processed data is stored in .mat format. Survey summary: start time, number of floats, water level. microFloat data: - Cleaned data for each float in survey, - Time series of position (lat., lon.), depth (relative to surface), and horizontal velocity (u, v) Drifting ADCP data: - Cleaned data for each drifting ADCP in survey - Time series of position (lat., lon.), sample depth (relative to surface), and observed water velocity (u,v,w) Stationary ADCP data: - Cleaned data for each station-keeping data set in survey - Time series of position (lat., lon.), sample depth (relative to surface), and observed water velocity (u,v,w) FVCOM (Finite Volume Coastal Ocean Model) model outputs: Water level and velocity, as well as derived variables (e.g., power density) from the final refined model simulations for the Agate Pass subdomain. Contained in NetCDF and ASCII files for the duration of August 18th and 20th 2020 field surveys. Additional scripts for processing the data and generating the figures in the report are included.

16 TIDAL AND WAVE POWER↗