Search NASA⌕ Search

SEARCH · Search NASA

Results for “VISUALIZATION LIBRARY”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
NASA NTRS2024

Creating Camera Controls for First-Person Camera in VulkanSceneGraph

The way users interact with a virtual, three-dimensional (3D) scene is heavily influenced by the way the camera used to view the scene is controlled. A first-person camera is a common form of camera control for computer programs. Its role is to create an immersive viewer experience, which allows users to traverse a scene as they might in the real world. Allowing for the implementation of the first-person camera makes for a more holistic, well-rounded way to interact with the visualization program. Accomplishing this involves mathematical calculations that define how the camera should be moved for the computer system. These movements equate to the rotation, translation, and scale of the changes. It should be noted that a computer does not inherently process what movement directions (left, right, up, or down) mean. The mathematical equations used define these principles in a way the computer can process. Further aspects to consider are detecting when the camera has moved and how far. This is most frequently accomplished through user |input through external devices. These devices, for the purpose of this report, include mouse input and keyboard input. Additionally, the in-development program this report is referencing works with the VulkanSceneGraph (VSG) library to create the scene and build the base of the camera controls. Although VSG is a powerful library with many capabilities, additional Application Programming Interfaces (APIs) might be needed during development to produce the desired results, as is the case in this program. Through combining proper mathematical calculations, utilizing additional APIs, and implementing the existing VSG library capabilities, implementing first-person camera controls is possible in a 3D scene.

Kristie O'Brien↗
DOE OSTI2017

In vitro cardiotoxicity assessment of environmental chemicals using an organotypic human induced pluripotent stem cell-derived model

An important target area for addressing data gaps through in vitro screening is the detection of potential cardiotoxicants. Despite the fact that current conservative estimates relate at least 23% of all cardiovascular disease cases to environmental exposures, the identities of the causative agents remain largely uncharacterized. Here, we evaluate the feasibility of a combinatorial in vitro/in silico screening approach for functional and mechanistic cardiotoxicity profiling of environmental hazards using a library of 69 representative environmental chemicals and drugs. Human induced pluripotent stem cell-derived cardiomyocytes were exposed in concentration-response for 30 min or 24 h and effects on cardiomyocyte beating and cellular and mitochondrial toxicity were assessed by kinetic measurements of intracellular Ca{sup 2+} flux and high-content imaging using the nuclear dye Hoechst 33342, the cell viability marker Calcein AM, and the mitochondrial depolarization probe JC-10. More than half of the tested chemicals exhibited effects on cardiomyocyte beating after 30 min of exposure. In contrast, after 24 h, effects on cell beating without concomitant cytotoxicity were observed in about one third of the compounds. Concentration-response data for in vitro bioactivity phenotypes visualized using the Toxicological Prioritization Index (ToxPi) showed chemical class-specific clustering of environmental chemicals, including pesticides, flame retardants, and polycyclic aromatic hydrocarbons. For environmental chemicals with human exposure predictions, the activity-to-exposure ratios between modeled blood concentrations and in vitro bioactivity were between one and five orders of magnitude. These findings not only demonstrate that some ubiquitous environmental pollutants might have the potential at high exposure levels to alter cardiomyocyte function, but also indicate similarities in the mechanism of these effects both within and among chemicals and classes. - Highlights: • The cardiotoxic potential of a broad range of environmental chemicals was evaluated. • Human iPSC cardiomyocytes were used for multiparametric in vitro profiling. • Dose-response relationships were established for functional and cytotoxicity assays. • Computational data integration provided chemical-specific bioactivity profiles. • An assessment of the profiles revealed chemical class-specific similarities.

60 APPLIED LIFE SCIENCES↗
NASA NTRS1989

A distributed program composition system

A graphical technique for creating distributed computer programs is investigated and a prototype implementation is described which serves as a testbed for the concepts. The type of programs under examination is restricted to those comprising relatively heavyweight parts that intercommunicate by passing messages of typed objects. Such programs are often presented visually as a directed graph with computer program parts as the nodes and communication channels as the edges. This class of programs, called parts-based programs, is not well supported by existing computer systems; much manual work is required to describe the program to the system, establish the communication paths, accommodate the heterogeneity of data types, and to locate the parts of the program on the various systems involved. The work described solves most of these problems by providing an interface for describing parts-based programs in this class in a way that closely models the way programmers think about them: using sketches of diagraphs. Program parts, the computational modes of the larger program system are categorized in libraries and are accessed with browsers. The process of programming has the programmer draw the program graph interactively. Heterogeneity is automatically accommodated by the insertion of type translators where necessary between the parts. Many decisions are necessary in the creation of a comprehensive tool for interactive creation of programs in this class. Possibilities are explored and the issues behind such decisions are presented. An approach to program composition is described, not a carefully implemented programming environment. However, a prototype implementation is described that can demonstrate the ideas presented.

Brown, Robert L.↗
DOE OSTI2020

BinaryCIF and CIFTools—Lightweight, efficient and extensible macromolecular data management

3D macromolecular structural data is growing ever more complex and plentiful in the wake of substantive advances in experimental and computational structure determination methods including macromolecular crystallography, cryo-electron microscopy, and integrative methods. Efficient means of working with 3D macromolecular structural data for archiving, analyses, and visualization are central to facilitating interoperability and reusability in compliance with the FAIR Principles. We address two challenges posed by growth in data size and complexity. First, data size is reduced by bespoke compression techniques. Second, complexity is managed through improved software tooling and fully leveraging available data dictionary schemas. To this end, we introduce BinaryCIF, a serialization of Crystallographic Information File (CIF) format files that maintains full compatibility to related data schemas, such as PDBx/mmCIF, while reducing file sizes by more than a factor of two versus gzip compressed CIF files. Moreover, for the largest structures, BinaryCIF provides even better compression—factor ten and four versus CIF files and gzipped CIF files, respectively. Herein, we describe CIFTools, a set of libraries in Java and TypeScript for generic and typed handling of CIF and BinaryCIF files. Together, BinaryCIF and CIFTools enable lightweight, efficient, and extensible handling of 3D macromolecular structural data.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2016

Validation of Criticality Analysis Capability of SuperMC with ICSBEP

SuperMC is a general purpose, intelligent and multifunctional, continuous energy Monte Carlo program for the design and safety evaluation of nuclear systems. It is designed to perform the comprehensive neutronics calculation, taking the radiation transport as the core and including the depletion, radiation source term/dose/biohazard, material activation and transmutation, etc. The latest version of SuperMC can perform the simulation of neutron, photon and coupled neutron and photon transport, and also equipped with the functions of automatic modeling and visualization. SuperMC has been verified by more than 2000 benchmark models and experiments including the handbook of International Criticality Safety Benchmark Evaluation Project (ICSBEP), the Shielding Integral Benchmark Archive Database (SINBAD), etc. The fusion reactor (ITER benchmark model, FDS Fusion Reactors), fast reactors (BN600, CLEAR-I), PWR reactors (HM, BEAVRS) and cases from the International Reactor Physics handbook Evaluation Program (IRPhEP) were employed for validating the comprehensive capability for reactor applications. In this paper, the validation of SuperMC based on the handbook of ICSBEP will be presented. ICSBEP, released by OECD/NEA, contains criticality safety benchmark specifications derived from experiments that were performed at various nuclear critical facilities around the world, which was widely used to verify neutronic codes and associated nuclear data libraries for criticality safety analysis. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2026

UCB-GLOBES: An open-access mass spectral database of identified and unidentified atmospheric organic compounds

Chemical characterization of atmospheric organic aerosols using gas chromatography with 70 eV electron ionization mass spectrometry (GC/EI-MS) has been used for decades in advancing molecular marker detection and identification, though primarily through suspect screening and/or targeted analyses. To advance non-targeted analyses of environmental samples, we have catalogued approximately 27 000 mass spectra (MS) of the trimethylsilyl derivatives of semi-volatile organic aerosol (OA) analytes in the open-access University of California Berkeley Goldstein Library of Organic Biogenic Environmental Spectra (UCB-GLOBES). Analytes were observed in ambient samples from the U.S. and the Central Amazon and/or laboratory simulations of secondary OA (SOA) formation. These samples are representative of OA under urban and biomass burning influences as well as SOA derived from biogenic precursors (e.g., isoprene, monoterpenes, sesquiterpenes) and biomass burning intermediates. MS are documented in UCB-GLOBES without regard to known chemical identity, annotated with extensive metadata such as sample source/experimental conditions, any structural information gained from MS analyses, and predicted chemical properties such as average carbon oxidation state and carbon number. UCB-GLOBES MS are compatible for importing into the NIST MS Search program, and we have also provided a Jupyter Notebook for MS visualization and comparisons. We demonstrate the utility of UCB-GLOBES through MS reanalyses of prior analytes observed in ambient data, finding a 20 % reduction in the number of analytes assigned to OA source categories reliant solely on time series correlation and an overall 11 % increase in new MS-based OA source categorization for the Southeast U.S. For 1513 analytes observed previously in the Central Amazon, we found 375 MS matches using UCB-GLOBES vs. 136 MS matches during prior analyses, representing a 14 % gain in newly confirmed or newly categorized OA species. While OA from laboratory oxidation experiments in UCB-GLOBES are highly diverse chemically, on average only 29 % of UCB-GLOBES MS have a mass spectral match to another MS entry in UCB-GLOBES and/or in databases of known compounds (i.e. NIST MS Database, Adams Essential Oil, MANE Flavor and Fragrance Company). This indicates that roughly 70 % of UCB-GLOBES MS are unique thus far, not observed more than once among the laboratory oxidation samples and ambient data in UCB-GLOBES MS. Further, only 18 % can be positively identified using these databases or known authentic standards. This points to a large gap between these laboratory simulations and ambient OA. Overall, the UCB-GLOBES database can be utilized for improving confidence in OA source categorization and/or identification, novel chemical marker discovery, tracking chemical diversity, de novo structure and properties prediction, and improving MS search and matching algorithms. This can ultimately inform future research priorities for the chemical characterization of atmospheric organic samples.

Mass spectrometry↗
NASA NTRS1979

The acquisition, storage, and dissemination of LANDSAT and other LACIE support data

Activities performed at the LACIE physical data library are described. These include the researching, acquisition, indexing, maintenance, distribution, tracking, and control of LACIE operational data and documents. Much of the data available can be incorporated into an Earth resources data base. Elements of the data collection that can support future remote sensing programs include: (1) the LANDSAT full-frame image files; (2) the microfilm file of aerial and space photographic and multispectral maps and charts that encompasses a large portion of the Earth's surface; (3) the map/chart collection that includes various scale maps and charts for a good portion of the U.S. and the LACIE area in foreign countries; (4) computer-compatible tapes of good quality LANDSAT scenes; (5) basic remote sensing data, project data, reference material, and associated publications; (6) visual aids to support presentation on remote sensing projects; and (7) research acquisition and handling procedures for managing data.

Abbotts, L. F.↗
DOE OSTI2025

Mapping Rare Earths and Toxics in E-Waste via Hyperspectral Imaging and Machine Learning

Electronic waste (e-waste) presents a mounting challenge to environmental sustainability due to its complex composition, which includes high-value rare earth elements, hazardous organic compounds, and non-recyclable plastics. Accurate and scalable material classification is essential for enabling efficient resource recovery and safe recycling practices. This study introduces a confidence-aware classification pipeline that combines mid-infrared hyperspectral imaging (HSI), spectral angle mapping (SAM), and iterative machine learning to perform pixel-level material identification across e-waste devices. A curated spectral library encompassing artificial materials (e.g., plastic iron oxide, galvanized metals), minerals (e.g., allanite, hematite), and organic compounds (e.g., benzanthracene, toluene) was used to generate pseudo-labels, each assigned a confidence score based on SAM-derived spectral similarity. High-confidence samples from seven consumer electronics—digital cameras, keyboards, laptop fans, modems, motherboards, TV remotes, and speakers—were iteratively expanded and classified using models such as Support Vector Machine (SVM), Random Forest, Gradient Boosting Classifier, Partial Least Squares Discriminant Analysis (PLSDA) and Logistic Regression. The best-performing classifiers achieved macro F1 scores approaching 1.0. Results revealed widespread plastic content (dominated by plastic iron oxide), the presence of rare earth-bearing minerals like cerium-containing allanite, and pervasive detection of hazardous organics such as benzanthracene. Principal Component Analysis (PCA) visualizations and confusion matrices confirmed high separability and robust classification performance. This methodology enables precise, non-destructive, and scalable classification of heterogeneous e-waste streams. It supports automated, hazard-aware sorting in recycling workflows, facilitating selective recovery of critical materials and compliance with circular economy goals. The confidence-aware framework provides a foundation for real-time deployment in industrial settings, offering significant implications for smart e-recycling infrastructure and policy-driven material stewardship.

Circular economy↗
DOE OSTI2021

The Exascale Framework for High Fidelity coupled Simulations (EFFIS): Enabling whole device modeling in fusion science

We present the Exascale Framework for High Fidelity coupled Simulations (EFFIS), a workflow and code coupling framework developed as part of the Whole Device Modeling Application (WDMApp) in the Exascale Computing Project. EFFIS consists of a library, command line utilities, and a collection of run-time daemons. Together, these software products enable users to easily compose and execute workflows that include: strong or weak coupling, in situ (or offline) analysis/visualization/monitoring, command-and-control actions, remote dashboard integration, and more. We describe WDMApp physics coupling cases and computer science requirements that motivate the design of the EFFIS framework. Furthermore, we explain the essential enabling technology that EFFIS leverages: ADIOS for performant data movement, PerfStubs/TAU for performance monitoring, and an advanced COUPLER for transforming coupling data from its native format to the representation needed by another application. Finally, we demonstrate EFFIS using coupled multi-simulation WDMApp workflows and exemplify how the framework supports the project’s needs. We show that EFFIS and its associated services for data movement, visualization, and performance collection does not introduce appreciable overhead to the WDMApp workflow and that the resource-dominant application’s idle time while waiting for data is minimal.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Towards a Scalable and Adaptive Application Support Platform for Large-Scale Distributed E-Sciences in High-Performance Network Environments

The advent of large-scale collaborative scientific applications has demonstrated the potential for broad scientific communities to pool globally distributed resources to produce unprecedented data acquisition, movement, and analysis. System resources including supercomputers, data repositories, computing facilities, network infrastructures, storage systems, and display devices have been increasingly deployed at national laboratories and academic institutes. These resources are typically shared by large communities of users over Internet or dedicated networks and hence exhibit an inherent dynamic nature in their availability, accessibility, capacity, and stability. Scientific applications using either experimental facilities or computation-based simulations with various physical, chemical, climatic, and biological models feature diverse scientific workflows as simple as linear pipelines or as complex as a directed acyclic graphs, which must be executed and supported over wide-area networks with massively distributed resources. Application users oftentimes need to manually configure their computing tasks over networks in an ad hoc manner, hence significantly limiting the productivity of scientists and constraining the utilization of resources. The success of these large-scale distributed applications requires a highly adaptive and massively scalable workflow platform that provides automated and optimized computing and networking services. This project is to design and develop a generic Scientific Workflow Automation and Management Platform (SWAMP), which contains a web-based user interface specially tailored for a target application, a set of user libraries, and several easy-to-use computing and networking toolkits for application scientists to conveniently assemble, execute, monitor, and control complex computing workflows in heterogeneous high-performance network environments. SWAMP will enable the automation and management of the entire process of scientific workflows with the convenience of a few mouse clicks while hiding the implementation and technical details from end users. Particularly, we will consider two types of applications with distinct performance requirements: data-centric and service-centric applications. For data-centric applications, the main workflow task involves large-volume data generation, catalog, storage, and movement typically from supercomputers or experimental facilities to a team of geographically distributed users; while for service-centric applications, the main focus of workflow is on data archiving, preprocessing, filtering, synthesis, visualization, and other application-specific analysis. We will conduct a comprehensive comparison of existing workflow systems and choose the best suited one with open-source code, a flexible system structure, and a large user base as the starting point for our development. Based on the chosen system, we will develop and integrate new components including a black box design of computing modules, performance monitoring and prediction, and workflow optimization and reconfiguration, which are missing from existing workflow systems. A modular design for separating specification, execution, and monitoring aspects will be adopted to establish a common generic infrastructure suited for a wide spectrum of science applications. We will further design and develop efficient workflow mapping and scheduling algorithms to optimize the workflow performance in terms of minimum end-to-end delay, maximum frame rate, and highest reliability. We will develop and demonstrate the SWAMP system in a local environment, the grid network, and the 100Gpbs Advanced Network Initiative (ANI) testbed. The demonstration will target scientific applications in climate modeling and high energy physics and the functions to be demonstrated include workflow deployment, execution, steering, and reconfiguration. Throughout the project period, we will work closely with the science communities in the fields of climate modeling and high energy physics including Spallation Neutron Source (SNS) and Large Hadron Collider (LHC) projects to mature the system for production use.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2014

NASA Tech Briefs, January 2014

Topics include: Multi-Source Autonomous Response for Targeting and Monitoring of Volcanic Activity; Software Suite to Support In-Flight Characterization of Remote Sensing Systems; Visual Image Sensor Organ Replacement; Ultra-Wideband, Dual-Polarized, Beam-Steering P-Band Array Antenna; Centering a DDR Strobe in the Middle of a Data Packet; Using a Commercial Ethernet PHY Device in a Radiation Environment; Submerged AUV Charging Station; Habitat Demonstration Unit (HDU) Vertical Cylinder Habitat; Origami-Inspired Folding of Thick, Rigid Panels; A Novel Protocol for Decoating and Permeabilizing Bacterial Spores for Epifluorescent Microscopy; Method and Apparatus for Automated Isolation of Nucleic Acids from Small Cell Samples; Enabling Microliquid Chromatography by Microbead Packing of Microchannels; On-Command Force and Torque Impeding Devices (OC-FTID) Using ERF; Deployable Fresnel Rings; Transition-Edge Hot-Electron Microbolometers for Millimeter and Submillimeter Astrophysics; Spacecraft Trajectory Analysis and Mission Planning Simulation (STAMPS) Software; Cross Support Transfer Service (CSTS) Framework Library; Arbitrary Shape Deformation in CFD Design; Range Safety Flight Elevation Limit Calculation Software; Frequency-Modulated, Continuous-Wave Laser Ranging Using Photon-Counting Detectors; Calculation of Operations Efficiency Factors for Mars Surface Missions; GPU Lossless Hyperspectral Data Compression System; Robust, Optimal Subsonic Airfoil Shapes; Protograph-Based Raptor-Like Codes; Fuzzy Neuron: Method and Hardware Realization; Kalman Filter Input Processor for Boresight Calibration; Organizing Compression of Hyperspectral Imagery to Allow Efficient Parallel Decompression; and Temperature Dependences of Mechanisms Responsible for the Water-Vapor Continuum Absorption.

Source record↗
NASA NTRSDate not supplied

FUN3D Mesh Adaptation Simulations of 4th AIAA High-Lift Prediction Workshop Case 1a

This paper describes in detail the FUN3D Mesh adaptation simulation results for Case 1a of the 4th AIAA High-Lift Prediction Workshop (HLPW-4). The Stabilized Finite Element library with FUN3D is used along with the Negative Spalart-Allmaras One-Equation Model to simulate fully turbulent free-air flow around the High Lift version of the Common Research Model at three flap settings. Meshes are adapted using multi-scale and goal-oriented metrics. The results are presented and the discussion focuses on the predicted differences between the three flap settings. Coefficients of lift, drag, and pitching moment are predicted within 7% of values measured in the QinetiQ 5m Wind Tunnel at each flap setting. Predicted surface pressures and skin friction distributions show inconsistent discrepancies with measurements across the flap settings. Consequently, flap increments for these figures of merit are not well predicted. These predictions are consistent with submissions to the HLPW-4 from other RANS-SA methods. Visualizations of Liutex vortex cores colored by angular velocity are presented as possible new quantitative and qualitative perspective for solution evaluation.

High-Lift Prediction Workshop↗
DOE OSTI2019

Numerical Analysis of Detonability Assessment in a Natural Gas-Air Fueled Rotating Detonation Engine

Rotating Detonation Engines (RDE) offer an alternative combustion strategy to replace conventional constant pressure combustion with a process that could produce a pressure gain without the use of a mechanical compressor. Recent numerical and experimental publications that consider air as the oxidizer have primarily focused on the ability of these annular combustors to sustain a stable continuous detonation wave when fueled by hydrogen. However, for this to be a viable consideration for the land-based power generation it is necessary to explore the ability to detonate natural gas and air within the confines of the annular geometry of an RDE. Previous studies on confined detonations have expressed the importance of permitting detonation cells to fully form within the combustor in order to achieve stability. This poses a challenge for natural gas–air fueled processes as their detonation cell size can be quite large even at moderate pressures. Despite the practical importance, only a few studies are available on natural gas detonations for air-breathing RDE applications. Moreover, the extreme thermodynamic condition (high temperature inside the combustor) allows limited accessibility inside the combustor for detailed experimental instrumentations, providing mostly single-point data. Recent experimental studies at the National Energy Technology Laboratory (NETL) have reported detonation failure at higher methane concentration in an air-breathing RDE fueled by natural gas-hydrogen fuel blends. This encourages to perform a detailed numerical investigation on the wave characteristics of detonation in a natural gas-air fueled RDE to understand the various aspects of instability associated with the natural gas-air detonation.This study is a numerical consideration of a methane-air fueled RDE with varying operating conditions to ascertain the ability to achieve a stable, continuous detonation wave. The simulations have been performed in a 2D unwrapped RDE geometry using the open-source CFD library “OpenFOAM” employing an unsteady pressure-based compressible reactive flow solver with a <em>k</em>–<em>ε</em> turbulence model in a structured rectangular grid system. Both reduced and detailed chemical kinetic models have been used to assess the effect of the chemistry on the detonation wave characteristics and the underlying flow features. A systematic grid sensitivity study has been conducted with various grid sizes to quantify the weakly stable overdriven detonation on a coarse mesh and oscillating features at fine mesh resolutions.The main focus of the current study is to investigate the effects of operating injection pressure on detonation wave characteristics of an air-breathing Rotating Detonation Engine (RDE) fueled with natural gas-hydrogen fuel blends. Wave speeds, peak pressures and temperatures, and dominant frequencies have been computed from the time histories. The flow structures were then visualized using 2D contours of temperature and species concentration.

Saha, Pankaj↗
DOE OSTI2024

Tractometry of the Human Connectome Project: resources and insights

The Human Connectome Project (HCP) has become a keystone dataset in human neuroscience, with a plethora of important applications in advancing brain imaging methods and an understanding of the human brain. We focused on tractometry of HCP diffusion-weighted MRI (dMRI) data. We used an open-source software library (pyAFQ; https://yeatmanlab.github.io/pyAFQ) to perform probabilistic tractography and delineate the major white matter pathways in the HCP subjects that have a complete dMRI acquisition (n = 1,041). We used diffusion kurtosis imaging (DKI) to model white matter microstructure in each voxel of the white matter, and extracted tract profiles of DKI-derived tissue properties along the length of the tracts. We explored the empirical properties of the data: first, we assessed the heritability of DKI tissue properties using the known genetic linkage of the large number of twin pairs sampled in HCP. Second, we tested the ability of tractometry to serve as the basis for predictive models of individual characteristics (e.g., age, crystallized/fluid intelligence, reading ability, etc.), compared to local connectome features. To facilitate the exploration of the dataset we created a new web-based visualization tool and use this tool to visualize the data in the HCP tractometry dataset. Finally, we used the HCP dataset as a test-bed for a new technological innovation: the TRX file-format for representation of dMRI-based streamlines. We released the processing outputs and tract profiles as a publicly available data resource through the AWS Open Data program's Open Neurodata repository. We found heritability as high as 0.9 for DKI-based metrics in some brain pathways. We also found that tractometry extracts as much useful information about individual differences as the local connectome method. We released a new web-based visualization tool for tractometry—“Tractoscope” (https://nrdg.github.io/tractoscope). We found that the TRX files require considerably less disk space-a crucial attribute for large datasets like HCP. In addition, TRX incorporates a specification for grouping streamlines, further simplifying tractometry analysis.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

HPC Campaign Management: Remote data access with user-defined error bound using ADIOS and ZFP

Remote access to large-scale scientific datasets, like those generated by combustion simulations or other high-performance computing (HPC) applications, presents a significant challenge. Downloading entire datasets is often impractical due to their size and the bandwidth limitations of typical networks. To address this challenge, we propose a novel approach that enables efficient remote access to large datasets distributed across multiple facilities. Our method enables technologies to download only the data values of a select variable, in a select region of interest, to a user-defined accuracy. For this purpose, we extended the ADIOS IO library to provide read functions with user-defined accuracy, a remote data server that understands multidimensional selections of specific variables, steps and accuracy from an ADIOS dataset, and which uses lossy compression on the remote site to reduce the data to be transferred back to the client. In addition, our extension of the ADIOS library collects metadata from multiple datasets in small files called Campaign Archives, which can be shared among project participants on any HPC, cloud or laptop, and which can easily facilitate the discovery of content and pointers to the data location as well as remote access to the data by local tools as if data was local. This feature called Campaign Management, enables a group of scientists to manage related datasets stored in multiple files, across multiple facilities as if it was in a single file/database. We demonstrate the effectiveness of our approach using a 1.5 TB dataset from the S3D combustion simulation on Frontier at the Oak Ridge Leadership Facility. Even a single variable from this dataset, at 64 GB, is too large to be processed on a standard laptop. We show two different reading patterns for 2D plots and 3D visualization, with careful settings that a scientist studying combustion data would do and show that running the same Python scripts on Frontier directly takes comparable time than running them on the local laptop with remote access to the data on Frontier.

Podhorszki, Norbert [ORNL] (ORCID:000000019647542X↗
DOE OSTI2022

The DSA Toolkit Shines Light Into Dark and Stormy Archives

Web archive collections are created with a particular purpose in mind. A curator selects seeds, or original resources, which are then captured by an archiving system and stored as archived web pages, or mementos. The systems that build web archive collections are often configured to revisit the same original resource multiple times. This is incredibly useful for understanding an unfolding news story or the evolution of an organization. Unfortunately, over time, some of these original resources can go off-topic and no longer suit the purpose for which the collection was originally created. They can go off-topic due to web site redesigns, changes in domain ownership, financial issues, hacking, technical problems, or because their content has moved on from the original topic. Even though they are off-topic, the archiving system will still capture them, thus it becomes imperative to anyone performing research on these collections to identify these off-topic mementos. Hence, we present the Off-Topic Memento Toolkit, which allows users to detect off-topic mementos within web archive collections. The mementos identified by this toolkit can then be separately removed from a collection or merely excluded from downstream analysis. The following similarity measures are available: byte count, word count, cosine similarity, Jaccard distance, Sørensen-Dice distance, Simhash using raw text content, Simhash using term frequency, and Latent Semantic Indexing via the gensim library. We document the implementation of each of these similarity measures. We possess a gold standard dataset generated by manual analysis, which contains both off-topic and on-topic mementos. Using this gold standard dataset, we establish a default threshold corresponding to the best F1 score for each measure. We also provide an overview of potential future directions that the toolkit may take.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2019

Sensitivity and Uncertainty Workflow of Full System SIERRA Models Supporting High Consequence Applications

Credibility of end-to-end CompSim (Computational Simulation) models and their agile execution requires an expressive framework to describe, communicate and execute complex computational tool chains representing the model. All stakeholders from system engineering and customers through model developers and V&V partners need views and functionalities of the workflow representing the model in a manner that is natural to their discipline. In the milestone and in this report we define workflow as a network of computation simulation activities executed autonomously on a distributed set of computational platforms. The FY19 ASC L2 Milestone (6802) for the Integrated Workflow (IWF) project was designed to integrate and improve existing capabilities or develop new functionalities to provide a wide range of stakeholders a coherent and intuitive platform capable of defining and executing CompSim modeling from analysis workflow definition to complex ensemble calculations. The main goal of the milestone was to advance the integrated workflow capabilities to support the weapon system analysts with a production deployment in FY20. Ensemble calculations supporting program decisions include sensitivity analysis, optimization and uncertainty quantification. The goal of the L2 milestone aligned with the ultimate goal of the IWF project is to foster cultural and technical shift toward and integrated CompSim capability based on automated workflows. Specific deliverables were defined in five broad categories: 1) Infrastructure, including development of distributed-computing workflow capability, 2) integration of Dakota (Sandia's sensitivity, optimization and UQ engine) with SAW (Sandia Analysis Workbench), 3) ARG (Automatic Report Generator introspecting analysis artifacts and generating human-readable extensible and archivable reports), 4) Libraries and Repositories aiding capability reuse, and 5) Exemplars to support training, capturing best practices and stress testing of the platform. A set of exemplars was defined to represent typical weapon system qualification CompSim projects. Analyzing the required capabilities and using the findings to plan implementation of required capabilities ensured optimal allocation of development resources focused on production deployment after the L2 is completed. It was recognized early that the end-to-end modeling applications pose a considerable number of diverse risks, and a formal risk tracking process was implemented. The project leveraged products, capabilities and development tasks of IWF partners. SAW, Dakota, Cubit, Sierra, Slycat, and NGA (NexGen Analytics, a small business) contributed to the integrated platform developed during this milestone effort. New products delivered include: a) NGW (Next Generation Workflow) for robust workflow definition and execution, b) Dakota wizards, editor and results visualization, and c) the automatic report generator ARG. User engagement was initiated early in the development process eliciting concrete requirements and actionable feedback to assure that the integrated CompSim capability will have high user acceptance and impact. The current integrated capabilities have been demonstrated and are continually being tested by a set of exemplars ranging from training scenarios to computationally demanding uncertainty analyses. The integrated workflow platform has been deployed on both SRN (Sandia Restricted Network) and SCN (Sandia Classified Network). Computational platforms where the system has been demonstrated span from Windows (Creo the CAD platform chosen by Sandia) to Trinity HPC (Sierra and CTH solvers). Follow up work will focus on deployment at SNL and other sites in the nuclear enterprise (LLNL, KCNSC), training and consulting support to democratize the analysis agility, process health and knowledge management benefits the NGW platform provides.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2004

Developing parallel GeoFEST(P) using the PYRAMID AMR library

The PYRAMID parallel unstructured adaptive mesh refinement (AMR) library has been coupled with the GeoFEST geophysical finite element simulation tool to support parallel active tectonics simulations. Specifically, we have demonstrated modeling of coseismic and postseismic surface displacement due to a simulated Earthquake for the Landers system of interacting faults in Southern California. The new software demonstrated a 25-times resolution improvement and a 4-times reduction in time to solution over the sequential baseline milestone case. Simulations on workstations using a few tens of thousands of stress displacement finite elements can now be expanded to multiple millions of elements with greater than 98% scaled efficiency on various parallel platforms over many hundreds of processors. Our most recent work has demonstrated that we can dynamically adapt the computational grid as stress grows on a fault. In this paper, we will describe the major issues and challenges associated with coupling these two programs to create GeoFEST(P). Performance and visualization results will also be described.

adaptive mesh refinement (AMR)↗