Search NASA⌕ Search

SEARCH · Search NASA

Results for “File Transfer System”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Streaming Data from Experimental Facilities to Supercomputers for Real-Time Data Processing

In this paper we demonstrate direct data streaming from instruments and detectors at a large-scale experimental facility to a supercomputer for real-time data processing and feedback. Streaming data to supercomputers introduces the potential for novel scientific applications and workflow models, including the ability to provide real-time feedback from very large datasets during an experiment and the integration of real-time ML training and inference at scale. We discuss a successful demonstration for real-time processing of data from the Advanced Photon Source (APS) on the Polaris supercomputer using an EPICS-based streaming framework. We describe the capabilities of the streaming framework itself, and outline the architecture that allows us to process experimentally derived data on a supercomputer without file-based data transfers. We present throughput measurements that are indicative of system performance capable of sustaining the expected data production rates of the facility, as well as discuss some outstanding challenges and our future directions.

real-time processing↗
NASA NTRSDate not supplied

Optical Communications Operations Concept for the Artemis II Crewed Mission to the Moon

NASA’s Artemis II mission will fly a human crew of 4 to the Moon aboard the Orion spacecraft for the first time since the Apollo missions. The Orion spacecraft includes an optical communication payload, known on board as “OpCom,” which is part of NASA’s Orion Artemis II Optical Communications (O2O) demonstration. OpCom will be controlled directly from the Mission Control System (MCC) in Houston and will support forward links up to 20 Mbps and return links up to 250 Mbps for at least 1 hour per mission day. It will be used to transfer files and provide real-time video SD and HD video to/from MCC. We describe the OpCom system architecture and operations concept.

optical communications↗
NASA NTRS2004

Automated File Transfer and Storage Management Concepts for Space

This presentation will summarize work that has been done to prototype and analyze approaches for automated file transfer and storage management for space missions. The concepts were prototyped in an environment with data files being generated at the target mission rates and stored in onboard files. The space-to-ground link was implemented using a channel simulator to introduce representative mission delays and errors. The system was operated for days with data files building up on the spacecraft and periodically being transferred to ground storage during a limited contact time. Overall performance was measured to identify limits under which the entire data volume could be transferred automatically while still fitting into the mission s limited contact time. The overall concepts, measurements, and results will be presented.

Hogie, Keith↗
NASA NTRS2012

Personal Computer Transport Analysis Program

The Personal Computer Transport Analysis Program (PCTAP) is C++ software used for analysis of thermal fluid systems. The program predicts thermal fluid system and component transients. The output consists of temperatures, flow rates, pressures, delta pressures, tank quantities, and gas quantities in the air, along with air scrubbing component performance. PCTAP s solution process assumes that the tubes in the system are well insulated so that only the heat transfer between fluid and tube wall and between adjacent tubes is modeled. The system described in the model file is broken down into its individual components; i.e., tubes, cold plates, heat exchangers, etc. A solution vector is built from the components and a flow is then simulated with fluid being transferred from one component to the next. The solution vector of components in the model file is built at the initiation of the run. This solution vector is simply a list of components in the order of their inlet dependency on other components. The component parameters are updated in the order in which they appear in the list at every time step. Once the solution vectors have been determined, PCTAP cycles through the components in the solution vector, executing their outlet function for each time-step increment.

DiStefano, Frank, III↗
NASA NTRS1993

TOAD Editor

Transferable Output ASCII Data (TOAD) computer program (LAR-13755), implements format designed to facilitate transfer of data across communication networks and dissimilar host computer systems. Any data file conforming to TOAD format standard called TOAD file. TOAD Editor is interactive software tool for manipulating contents of TOAD files. Commonly used to extract filtered subsets of data for visualization of results of computation. Also offers such user-oriented features as on-line help, clear English error messages, startup file, macroinstructions defined by user, command history, user variables, UNDO features, and full complement of mathematical statistical, and conversion functions. Companion program, TOAD Gateway (LAR-14484), converts data files from variety of other file formats to that of TOAD. TOAD Editor written in FORTRAN 77.

Bingle, Bradford D.↗
NASA NTRS2013

Multi-User Space Link Extension (SLE) System

The Multi-User Space (MUS) Link Extension system, a software and data system, provides Space Link Extension (SLE) users with three space data transfer services in timely, complete, and offline modes as applicable according to standards defined by the Consultative Committee for Space Data Systems (CCSDS). MUS radically reduces the schedule, cost, and risk of implementing a new SLE user system, minimizes operating costs with a lights-out approach to SLE, and is designed to require no sustaining engineering expense during its lifetime unless changes in the CCSDS SLE standards, combined with new provider implementations, force changes. No software modification to MUS needs to be made to support a new mission. Any systems engineer with Linux experience can begin testing SLE user service instances with MUS starting from a personal computer (PC) within five days. For flight operators, MUS provides a familiar-looking Web page for entering SLE configuration data received from SLE. Operators can also use the Web page to back up a space mission's entire set of up to approximately 500 SLE service instances in less than five seconds, or to restore or transfer from another system the same amount of data from a MUS backup file in about the same amount of time. Missions operate each MUS SLE service instance independently by sending it MUS directives, which are legible, plain ASCII strings. MUS directives are usually (but not necessarily) sent through a TCP-IP (Transmission Control Protocol Internet Protocol) socket from a MOC (Mission Operations Center) or POCC (Payload Operations Control Center) system, under scripted control, during "lights-out" spacecraft operation. MUS permits the flight operations team to configure independently each of its data interfaces; not only commands and telemetry, but also MUS status messages to the MOC. Interfaces can use single- or multiple-client TCP/IP server sockets, TCP/IP client sockets, temporary disk files, the system log, or standard in, standard out, or standard error as applicable. By defining MUS templates in ASCII, the flight operations team can include any MUS system variable in telemetry or command headers or footers, and/or in status messages. Data fields can be arranged within messages in different sequences, according to the mission s needs. The only constraints imposed are on the format of MUS directive strings, and some bare minimum logical requirements that must be met in order for MUS to read the mission control center's spacecraft command inputs. The MUS system imposes no limits or constraints on the numbers and combinations of missions and SLE service instances that it will support simultaneously. At any time, flight operators may add, change, delete, bind, connect, or disconnect.

Perkins, Toby↗
DOE OSTI2024

THM: the MOOSE thermal hydraulics module

The MOOSE Thermal Hydraulics Module (THM) is designed to facilitate the development of thermal hydraulic system models. It provides the capability to assemble networks of coupled components such as pipes, junctions, valves, turbomachinery, and heat exchangers. Its library of components supports a single-phase, compressible flow model based on a variable-area formulation of the Euler equations of gas dynamics and discretized using a finite volume scheme. THM offers a flexible system for specifying closures such as friction factors or heat transfer coefficients, allowing the user to choose from built-in correlations or define their own in the input file. A control logic system can be used to control input parameters, necessary for implementing transient scenarios and mirroring real control systems in thermal hydraulic systems. THM can be coupled with other MOOSE-based applications for multiphysics calculations. This talk will give an introduction to the capabilities of THM and provide some examples of its usage and validation.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

MOOSE Thermal-Hydraulics Module - MOOSE workshop

The MOOSE Thermal Hydraulics Module (THM) is designed to facilitate the development of thermal hydraulic system models. It provides the capability to assemble networks of coupled components such as pipes, junctions, valves, turbomachinery, and heat exchangers. Its library of components supports a single-phase, compressible flow model based on a variable-area formulation of the Euler equations of gas dynamics and discretized using a finite volume scheme. THM offers a flexible system for specifying closures such as friction factors or heat transfer coefficients, allowing the user to choose from built-in correlations or define their own in the input file. A control logic system can be used to control input parameters, necessary for implementing transient scenarios and mirroring real control systems in thermal hydraulic systems. THM can be coupled with other MOOSE-based applications for multiphysics calculations. This training will give an introduction to the capabilities of THM and provide some examples of its usage and validation.

97 - MATHEMATICS AND COMPUTING↗
NASA NTRS1992

Zebra: A striped network file system

The design of Zebra, a striped network file system, is presented. Zebra applies ideas from log-structured file system (LFS) and RAID research to network file systems, resulting in a network file system that has scalable performance, uses its servers efficiently even when its applications are using small files, and provides high availability. Zebra stripes file data across multiple servers, so that the file transfer rate is not limited by the performance of a single server. High availability is achieved by maintaining parity information for the file system. If a server fails its contents can be reconstructed using the contents of the remaining servers and the parity information. Zebra differs from existing striped file systems in the way it stripes file data: Zebra does not stripe on a per-file basis; instead it stripes the stream of bytes written by each client. Clients write to the servers in units called stripe fragments, which are analogous to segments in an LFS. Stripe fragments contain file blocks that were written recently, without regard to which file they belong. This method of striping has numerous advantages over per-file striping, including increased server efficiency, efficient parity computation, and elimination of parity update.

Hartman, John H.↗
DOE OSTI2024

Fermilab s Transition to Token Authentication

Fermilab is the first High Energy Physics institution to transition from X.509 user certificates to authentication tokens in production systems. All of the experiments that Fermilab hosts are now using JSON Web Token (JWT) access tokens in their grid jobs. Many software components have been either updated or created for this transition, and most of the software is available to others as open source. The tokens are defined using the WLCG Common JWT Profile. Token attributes for all the tokens are stored in the Fermilab FERRY system which generates the configuration for the CILogon token issuer. High security-value refresh tokens are stored in Hashicorp Vault configured by htvault-config, and JWT access tokens are requested by the htgettoken client through its integration with HTCondor. The Fermilab job submission system jobsub was redesigned to be a lightweight wrapper around HTCondor. For automated job submissions a managed tokens service was created to reduce duplication of effort and knowledge of how to securely keep tokens active. The existing Fermilab file transfer tool ifdh was updated to work seamlessly with tokens, as well as the Fermilab POMS (Production Operations Management System) which is used to manage automatic job submission and the RCDS (Rapid Code Distribution System) which is used to distribute analysis code via the CernVM FileSystem. The dCache storage system was reconfigured to accept tokens for authentication in place of X.509 proxy certificates. As some services and sites have not yet implemented token support, proxy certificates are still sent with jobs for backwards compatibility but some experiments are beginning to transition to stop using them. There have been some glitches and learning curve issues but in general the system has been performing well and is being improved as operational problems are addressed.

Dykstra, David↗
NASA NTRS2001

Duplex Direct Data Distribution System

The NASA Glenn Research Center (GRC) is developing and demonstrating communications and network technologies that are helping to enable the near-Earth space Internet. GRC envisions several service categories. The first of these categories is direct data distribution or D3 (pronounced "D-cubed"). Commercially provided D3 will make it possible to download a data set from a spacecraft, like the International Space Station. as easily as one can extract a file from a remote server today, using a file transfer protocol. In a second category, NASA spacecraft will make use of commercial satellite communication (SATCOM) systems. Some of those services will come from purchasing time on unused transponders that cover landmasses. While it is likely there will be gaps in service coverage, Internet services should be available using these systems. This report addresses alternative methods of implementing a full duplex enhancement of the GRC developed experimental Ka-Band Direct Data Distribution (D3) space-to-ground communication link. The resulting duplex version is called the Duplex Direct Data Distribution (D4) system. The D4 system is intended to provide high-data-rate commercial direct or internet-based communications service between the NASA spacecraft in low earth orbit (LEO) and the respective principal investigators associated with these spacecraft. Candidate commercial services were assessed regarding their near-term potential to meet NASA requirements. Candidates included Ka-band and V-band geostationary orbit and non-geostationary orbit satellite relay services and direct downlink ("LEO teleport") services. End-to-end systems concepts were examined and characterized in terms of alternative link layer architectures. Alternatives included a Direct Link, a Relay Link, a Hybrid Link, and a Dual Mode Link. The direct link assessment examined sample ground terminal placements and antenna angle issues. The SATCOM-based alternatives examined existing or proposed commercial SATCOM services that could be available in the 2005 time frame. The alternatives were evaluated and compared in terms of average daily system throughput and cost per bit. Throughput was estimated based on hypothetical scenarios supporting the International Space Station and polar orbiting missions. The feasibility of using standard TCP and a modified TCP was evaluated and risks were identified. An estimate of the TCP acknowledgment data rate required to support a return channel rate of 622 Mbps was developed using OPNET.

Greenfield, Israel↗
DOE OSTI2025

Fermilab's Transition to Token Authentication

Fermilab is the first High Energy Physics institution to transition from X.509 user certificates to authentication tokens in production systems. All the experiments that Fermilab hosts are now using JSON Web Token (JWT) access tokens in their grid jobs. Many software components have been either updated or created for this transition, and most of the software is available to others as open source. The tokens are defined using the WLCG Common JWT Profile. Token attributes for all the tokens are stored in the Fermilab FERRY system which generates the configuration for the CILogon token issuer. High security-value refresh tokens are stored in Hashicorp Vault configured by htvault-config, and JWT access tokens are requested by the htgettoken client through its integration with HTCondor. The Fermilab job submission system jobsub was redesigned to be a lightweight wrapper around HTCondor. The grid workload management system GlideinWMS which is also based on HTCondor was updated to use tokens for pilot job submission. For automated job submissions a managed tokens service was created to reduce duplication of effort and knowledge of how to securely keep tokens active. The existing Fermilab file transfer tool ifdh was updated to work seamlessly with tokens, as well as the Fermilab POMS (Production Operations Management System) which is used to manage automatic job submission and the RCDS (Rapid Code Distribution System) which is used to distribute analysis code via the CernVM FileSystem. The dCache storage system was reconfigured to accept tokens for authentication in place of X.509 proxy certificates. As some services and sites have not yet implemented token support, proxy certificates are still sent with jobs for backwards compatibility, but some experiments are beginning to transition to stop using them.

Dykstra, Dave [Fermilab] (ORCID:0000000326539015)↗
DOE OSTI2022

Viability of S3 Object Storage for the ASC Program at Sandia

Recent efforts at Sandia such as DataSEA are creating search engines that enable analysts to query the institution’s massive archive of simulation and experiment data. The benefit of this work is that analysts will be able to retrieve all historical information about a system component that the institution has amassed over the years and make better-informed decisions in current work. As DataSEA gains momentum, it faces multiple technical challenges relating to capacity storage. From a raw capacity perspective, data producers will rapidly overwhelm the system with massive amounts of data. From an accessibility perspective, analysts will expect to be able to retrieve any portion of the bulk data, from any system on the enterprise network. Sandia’s Institutional Computing is mitigating storage problems at the enterprise level by procuring new capacity storage systems that can be accessed from anywhere on the enterprise network. These systems use the simple storage service, or S3, API for data transfers. While S3 uses objects instead of files, users can access it from their desktops or Sandia’s high-performance computing (HPC) platforms. S3 is particularly well suited for bulk storage in DataSEA, as datasets can be decomposed into object that can be referenced and retrieved individually, as needed by an analyst. In this report we describe our experiences working with S3 storage and provide information about how developers can leverage Sandia’s current systems. We present performance results from two sets of experiments. First, we measure S3 throughput when exchanging data between four different HPC platforms and two different enterprise S3 storage systems on the Sandia Restricted Network (SRN). Second, we measure the performance of S3 when communicating with a custom-built Ceph storage system that was constructed from HPC components. Overall, while S3 storage is significantly slower than traditional HPC storage, it provides significant accessibility benefits that will be valuable for archiving and exploiting historical data. There are multiple opportunities that arise from this work, including enhancing DataSEA to leverage S3 for bulk storage and adding native S3 support to Sandia’s IOSS library.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS1985

Introduction to the Space Physics Analysis Network (SPAN)

The Space Physics Analysis Network or SPAN is emerging as a viable method for solving an immediate communication problem for the space scientist. SPAN provides low-rate communication capability with co-investigators and colleagues, and access to space science data bases and computational facilities. The SPAN utilizes up-to-date hardware and software for computer-to-computer communications allowing binary file transfer and remote log-on capability to over 25 nationwide space science computer systems. SPAN is not discipline or mission dependent with participation from scientists in such fields as magnetospheric, ionospheric, planetary, and solar physics. Basic information on the network and its use are provided. It is anticipated that SPAN will grow rapidly over the next few years, not only from the standpoint of more network nodes, but as scientists become more proficient in the use of telescience, more capability will be needed to satisfy the demands.

Green, J. L.↗
NASA NTRS2005

Adopting Internet Standards for Orbital Use

After a year of testing and demonstrating a Cisco mobile access router intended for terrestrial use onboard the low-Earth-orbiting UK-DMC satellite as part of a larger merged ground/space IP-based internetwork, we reflect on and discuss the benefits and drawbacks of integration and standards reuse for small satellite missions. Benefits include ease of operation and the ability to leverage existing systems and infrastructure designed for general use, as well as reuse of existing, known, and well-understood security and operational models. Drawbacks include cases where integration work was needed to bridge the gaps in assumptions between different systems, and where performance considerations outweighed the benefits of reuse of pre-existing file transfer protocols. We find similarities with the terrestrial IP networks whose technologies we have adopted and also some significant differences in operational models and assumptions that must be considered.

Wood, Lloyd↗
NASA NTRS1993

Storage system architectures and their characteristics

Not all users storage requirements call for 20 MBS data transfer rates, multi-tier file or data migration schemes, or even automated retrieval of data. The number of available storage solutions reflects the broad range of user requirements. It is foolish to think that any one solution can address the complete range of requirements. For users with simple off-line storage requirements, the cost and complexity of high end solutions would provide no advantage over a more simple solution. The correct answer is to match the requirements of a particular storage need to the various attributes of the available solutions. The goal of this paper is to introduce basic concepts of archiving and storage management in combination with the most common architectures and to provide some insight into how these concepts and architectures address various storage problems. The intent is to provide potential consumers of storage technology with a framework within which to begin the hunt for a solution which meets their particular needs. This paper is not intended to be an exhaustive study or to address all possible solutions or new technologies, but is intended to be a more practical treatment of todays storage system alternatives. Since most commercial storage systems today are built on Open Systems concepts, the majority of these solutions are hosted on the UNIX operating system. For this reason, some of the architectural issues discussed focus around specific UNIX architectural concepts. However, most of the architectures are operating system independent and the conclusions are applicable to such architectures on any operating system.

Sarandrea, Bryan M.↗
NASA NTRS1996

Derived virtual devices: a secure distributed file system mechanism

This paper presents the design of derived virtual devices (DVDs). DVDs are the mechanism used by the Netstation Project to provide secure shared access to network-attached peripherals distributed in an untrusted network environment. DVDs improve Input/Output efficiency by allowing user processes to perform I/O operations directly from devices without intermediate transfer through the controlling operating system kernel. The security enforced at the device through the DVD mechanism includes resource boundary checking, user authentication, and restricted operations, e.g., read-only access. To illustrate the application of DVDs, we present the interactions between a network-attached disk and a file system designed to exploit the DVD abstraction. We further discuss third-party transfer as a mechanism intended to provide for efficient data transfer in a typical NAP environment. We show how DVDs facilitate third-party transfer, and provide the security required in a more open network environment.

VanMeter, Rodney↗
NASA NTRS2001

Registered File Support for Critical Operations Files at (Space Infrared Telescope Facility) SIRTF

The SIRTF Science Center's (SSC) Science Operations System (SOS) has to contend with nearly one hundred critical operations files via comprehensive file management services. The management is accomplished via the registered file system (otherwise known as TFS) which manages these files in a registered file repository composed of a virtual file system accessible via a TFS server and a file registration database. The TFS server provides controlled, reliable, and secure file transfer and storage by registering all file transactions and meta-data in the file registration database. An API is provided for application programs to communicate with TFS servers and the repository. A command line client implementing this API has been developed as a client tool. This paper describes the architecture, current implementation, but more importantly, the evolution of these services based on evolving community use cases and emerging information system technology.

file management services↗