Search NASA⌕ Search

SEARCH · Search NASA

Results for “Directory”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

GUIDE: A Scalable Information Directory Service to Collect, Federate, and Analyze Logs for Operational Insights into a Leadership HPC Facility

In this paper, we describe the GUIDE framework to collect, federate, and analyze log data from the Oak Ridge Leadership Computing Facility's (OLCF), and how we use it derive insights into center operations. We collectsystem logs and extract monitoring data at every level of the various OLCF subsystems, and have developed a suite of pre-processing tools to make the raw data consumable. The cleansed logs are then ingested and federated into a central, scalable data warehouse, Splunk, that offers storage, indexing, querying, and visualization capabilities. We have further developed and deployed a set of analytics tools to analyze these multiple disparate log streams in concert, and derive operational insights. We describe our experience from developing and deploying the GUIDE infrastructure, and deriving valuable insights on the various subsystems, based on two years of operations in the production OLCF environment.

Vazhkudai, Sudharshan↗
DOE OSTI2018

Distributed metadata servers in a file system with separate metadata servers for file metadata and directory metadata

A file system is provided having a plurality of distributed metadata servers with separate metadata servers for file metadata and directory metadata. An exemplary distributed metadata storage system comprises a plurality of compute nodes, wherein at least one compute node comprises a file system client that generates a directory metadata request and/or a file metadata request; a plurality of directory metadata servers to serve directory metadata in response to a given directory metadata request; and a plurality of file metadata servers to serve file metadata in response to a given file metadata request. The plurality of directory metadata servers are distinct from the plurality of file metadata servers. The compute nodes optionally comprise an object client that requests object data from an object store system. The number of directory metadata servers and the number of file metadata servers are optionally determined independently to balance for different workloads, and can be dynamically readjusted for evolving workloads.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Label-based Virtual Directories In dCache

Traditional filesystems organize data in directories. These directories are typically a collection of files whose grouping is based on a single criterion, e.g., the starting date of an experiment, experiment name, beamline ID, measurement device, or instrument. However, each file in a directory can belong to several logical groups, such as a special event type, experiment condition, or a part of a selected dataset. dCache is a storage system developed to store large amounts of scientific data, used by many HEP and Photon Science experiments. With recent developments in dCache, we have introduced a concept of file tagging, which dynamically groups files with the same label into virtual directories. The file labels can be added, removed, renamed, and deleted through the admin interface or via REST API. The files in virtual directories are exposed through all protocols supported by dCache. This contribution will describe the details of the implementation for file tagging in dCache and present our future development plans on automatic metadata extractions, a feature that will significantly simplify data management. Additionally, we are exploring the future use of virtual directories as a way to translate scientific data catalogs into filesystem views for direct data analysis.

Sahakyan, Marina [DESY]↗
DOE OSTI2020

Distributed coherence directory subsystem with exclusive data regions

A processing system includes a first set of one or more processing units including a first processing unit, a second set of one or more processing units including a second processing unit, and a memory having an address space shared by the first and second sets. The processing system further includes a distributed coherence directory subsystem having a first coherence directory to support a first subset of one or more address regions of the address space and a second coherence directory to support a second subset of one or more address regions of the address space. In some implementations, the first coherence directory is implemented in the system so as to have a lower access latency for the first set, whereas the second coherence directory is implemented in the system so as to have a lower access latency for the second set.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Distributed coherence directory subsystem with exclusive data regions

A processing system includes a first set of one or more processing units including a first processing unit, a second set of one or more processing units including a second processing unit, and a memory having an address space shared by the first and second sets. The processing system further includes a distributed coherence directory subsystem having a first coherence directory to support a first subset of one or more address regions of the address space and a second coherence directory to support a second subset of one or more address regions of the address space. In some implementations, the first coherence directory is implemented in the system so as to have a lower access latency for the first set, whereas the second coherence directory is implemented in the system so as to have a lower access latency for the second set.

Eckert, Yasuko↗
DOE OSTI2021

Distributed directory of named data elements in coordination namespace

An approach is described that provides a distributed directory structure within a storage of an information handling system (a local node). A request is received with the request corresponding to a shared virtual address. The shared virtual address that is shared amongst a number of nodes that includes the local node and some remote nodes. A Global Address Space Directory (GASD) is retrieved that corresponds to a global virtual address space. The GASD is stored in a Coordination Namespace that is stored in a memory that is distributed amongst the nodes. A mapping that is included in the GASD is used to determine the node where the shared virtual address currently resides. The shared virtual address is then accessed from the node where it currently resides.

Johns, Charles Ray↗
DOE OSTI2020

Streaming Data Reorganization at Scale with DeltaFS Indexed Massive Directories

We report complex storage stacks providing data compression, indexing, and analytics help leverage the massive amounts of data generated today to derive insights. It is challenging to perform this computation, however, while fully utilizing the underlying storage media. This is because, while storage servers with large core counts are widely available, single-core performance and memory bandwidth per core grow slower than the core count per die. Computational storage offers a promising solution to this problem by utilizing dedicated compute resources along the storage processing path. We present DeltaFS Indexed Massive Directories (IMDs), a new approach to computational storage. DeltaFS IMDs harvest available (i.e., not dedicated) compute, memory, and network resources on the compute nodes of an application to perform computation on data. We demonstrate the efficiency of DeltaFS IMDs by using them to dynamically reorganize the output of a real-world simulation application across 131,072 CPU cores. DeltaFS IMDs speed up reads by 1,740x while only slightly slowing down the writing of data during simulation I/O for in situ data processing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

The Biosurveillance Analytics Resource Directory (BARD): Facilitating the use of epidemiological models for infectious disease surveillance

Epidemiological modeling for infectious disease is important for disease management and its routine implementation needs to be facilitated through better description of models in an operational context. A standardized model characterization process that allows selection or making manual comparisons of available models and their results is currently lacking. A key need is a universal framework to facilitate model description and understanding of its features. Los Alamos National Laboratory (LANL) has developed a comprehensive framework that can be used to characterize an infectious disease model in an operational context. The framework was developed through a consensus among a panel of subject matter experts. In this paper, we describe the framework, its application to model characterization, and the development of the Biosurveillance Analytics Resource Directory (BARD; http://brd.bsvgateway.org/brd/), to facilitate the rapid selection of operational models for specific infectious/communicable diseases. We offer this framework and associated database to stakeholders of the infectious disease modeling field as a tool for standardizing model description and facilitating the use of epidemiological models.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Input files and WRF run directory for a LASSO-CACTI simulation

Tar file containing files necessary for reproducing a given Weather Research and Forecasting (WRF) simulation from the Large-Eddy Simulation (LES) Atmospheric Radiation Measurement (ARM) Symbiotic Simulation and Observation (LASSO) deep-convection scenario for the Cloud, Aerosol, and Complex Terrain Interactions (CACTI) field campaign. The LASSO-CACTI simulations span grid spacings from 7.5 km to 100 m for convection near the Sierras de Córdoba mountain range, roughly centered on the ARM Mobile Facility. More information can be found at https://www.arm.gov/capabilities/modeling/lasso. Types of files in this tar include the initial and boundary conditions, WRF namelist, and other files present in a WRF run directory.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

DMS Instrument Directory Scanner

This program examines the source folder on instrument computers to find files and folders. It creates a tab-delimited text file for each instrument, listing the names of the files and folders found. The text files for the instruments are stored in a central location, and are made made visible via the DMS website to allow instrument operators to see the instrument data files on the instrument computer.

Monroe, Matthew↗
DOE OSTI2021

System Validation Platform Working Directories

A suite of software tools used to interface with power electronic equipment for laboratory testing and device certification. They are typically used as equipment drivers with using the System Validation Platform (SVP) graphical user interface (GUI). Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND2021-1185 O

Johnson, JayT↗
DOE OSTI2015

Global Social Media Directory. A Resource Guide

Social media platforms are internet-based applications focused on broadcasting user-generated content. While primarily web-based, these services are increasingly available on mobile platforms. Communities and individuals share information, photos, music, videos, provide commentary and ratings/reviews, and more. In essence, social media is about sharing information, consuming information, and repurposing content. Social media technologies identified in this report are centered on social networking services, media sharing, blogging and microblogging. The purpose of this Resource Guide is to provide baseline information about use and application of social media platforms around the globe. It is not intended to be comprehensive as social media evolves on an almost daily basis. The long-term goal of this work is to identify social media information about all geographic regions and nations. The primary objective is that of understanding the evolution and spread of social networking and user-generated content technologies internationally.

blogs↗
DOE OSTI2018

Data directory for Halbach magnets

The data for measurement and survey of the CBETA Halbach magnets is stored on the “DISCOVERY” network computer at BNL Superconducting Magnet Division (SMD), on the network share called “C_Beta”. This serves as storage for the measurements, under systematic filenames, as well as generated reports and other relevant per-magnet documentation such as travellers.

43 PARTICLE ACCELERATORS↗
DOE OSTI2022

Hawaii Wave Surge Energy Converter (HAWSEC) OSU O.H. Hinsdale Basin

The following information and metadata applies to both the Phase I (Hydrodynamics) and Phase II (Full System Power Take-Off) zip folders which contain testing data from the OSU (Oregon State University) O.H. Hinsdale Wave Research Laboratory, from both OSU and the University of Hawaii at Manoa (UH). See zip folders provided further below in the downloads section. For experimental data of the full system, including PTO, see Phase II dataset. There are two main directories in each Phases's zip folder: "OSU_data" and "UH_data". The "OSU_data" directory contains data collected from their DAQ (data acquisition system), which includes all wave gauge observations, as well as body motions derived from their Qualisys motion tracking system. The organization of the directory follows OSU's convention. Detailed information on the instrument setup can be found under "OSU_data/docs/setup/instm_locations". The experiments conducted are documented in the "OSU_data/docs/daq_logs", which provides the trial number to the corresponding data located under "OSU_data/data" in several formats (e.g., ".mat" and ".txt"). Inside the trial directory, data is provided for each of the instruments defined in "OSU_data/docs/setup/instm_locations". The "UH_data" directory contains data collected from their DAQ. The data is stored in a ".tdms" file format. There are free plug-ins for Microsoft Excel and MathWorks MATLAB to read the ".tdms" format. Below are a few links providing methods to read in the data, but a Google search should identify alternatives sources if these no longer exist (valid as of January 2024): Excel: http://www.ni.com/example/27944/en/ MATLAB: https://www.mathworks.com/matlabcentral/fileexchange/30023-tdms-reader The Excel plugin is recommend to get a quick overview of the data. The UH data is organized by directory name, in which the sub-directories for each experiment contains a directory whose name defines the wave height and period for the experimental data within. For example, a directory name "H02_T0275" corresponds to an experiment with wave height 0.1m and a period of 2.75s. For random wave data, the gamma value is also included in the directory name. For example, a directory name "H02_T0225_G18" corresponds to an experiment with a significant wave height of 0.2m, a peak period of 2.25s, and a gamma value of 1.8, with each spectra being a TMA spectrum. For the free decay experiments, the directory name is defined by the initial angular displacement. For example, a directory name "ang05_run01" corresponds to an experiment with an initial angular displacement of 5 degrees. There is a dataset in the UH data for each corresponding experiment defined in the OSU DAQ logs. The ".tdms" data is output from the DAQ at fixed intervals. Therefore, if multiple files are contained within the folder, the data will need to be stitched together. Within the UH dataset, there are two input channels from the OSU DAQ providing a random square wave signal for time synchronization ("ENV-WHT-0010") and a high/low signal ("ENV-WHT-0012") to identify when the wave maker is active (+5V). The UH data is logged as a collection of channel outputs. Channels not in use for the OSU testing (either Phase I or Phase II) are marked "nan" below. If the sensor is disconnected, it will record noise throughout the experiment. Below are the channel definitions in terms of what they measure: GPS Time = time CYL-POS-0001 = position between flap and fixed reference CYL-LCA-0001 = force between flap and hydraulic cylinder REC-LPT-0001 = nan REC-HPT-0001 = nan REC-HPT-0002 = nan REC-HPT-0003 = nan HHT-HPT-0001 = pressure at exhaust ("head" only) REC-FQC-0001 = nan REC-FQC-0002 = nan HHT-FQC-0001 = flow at exhaust ("head" only) ENV-WHT-0001 = nan ENV-WHT-0002 = nan ENV-WHT-0003 = nan ENV-WHT-0010 = random signal from OSU DAQ ENV-WHT-0012 = high/low signal from OSU DAQ Also included is a calibration curve to convert the string pot data to flap pi...

16 TIDAL AND WAVE POWER↗