Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed systems, data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

Integrated Software Systems for Crew Management During Extravehicular Activity in Planetary Terrain Exploration

Initial planetary explorations with the Apollo program had a veritable ground support army monitoring the safety and health of the 12 astronauts who performed lunar surface extravehicular activities (EVAs). Given the distances involved, this will not be possible on Mars. A spacesuit for Mars must be smart enough to replace that army. The next generation suits can do so using 2 software systems serving as virtual companions, LEGACI (Life support, Exploration Guidance Algorithm and Consumable Interrogator) and VIOLET (Voice Initiated Operator for Life support and Exploration Tracking). The system presented in this study integrates data inputs from a suite of sensors into the MIII suit s communications, avionics and informatics hardware for distribution to remote managers and data analysis. If successful, the system has application not only for Mars but for nearer term missions to the Moon, and the next generation suits used on ISS as well. Field tests are conducted to assess capabilities for next generation spacesuits at Johnson Space Center (JSC) as well as the Mars and Lunar analog (Devon Island, Canada). LEGACI integrates data inputs from a suite of noninvasive biosensors in the suit and the astronaut (heart rate, suit inlet/outlet lcg temperature and flowrate, suit outlet gas and dewpoint temperature, pCO2, suit O2 pressure, state vector (accelerometry) and others). In the Integrated Walkback Suit Tests held at NASA-JSC and the HMP tests at Devon Island, communication and informatics capabilities were tested (including routing by satellite from the suit at Devon Island to JSC in Houston via secure servers at VCU in Richmond, VA). Results. The input from all the sensors enable LEGACI to compute multiple independent assessments of metabolic rate, from which a "best" met rate is chosen based on statistical methods. This rate can compute detailed information about the suit, crew and EVA performance using test-derived algorithms. VIOLET gives LEGACI voice activation capability, allowing the crew to query the suit, and receive feedback and alerts that will lead to corrective action. LEGACI and VIOLET can also automatically control the astronaut's cooling and consumable use rate without crew input if desired. These findings suggest that non-invasive physiological and environmental sensors supported with data analysis can allow for more effective management of mission task performance during EVA. Integrated remote and local view of data metrics allow crewmember to receive real time feedback in synch with mission control in preventing performance shortcomings for EVA in exploration missions.

Kuznetz, Lawrence↗

Data collection system: Earth Resources Technology Satellite-1

Subjects covered at the meeting concerned results on the overall data collection system including sensors, interface hardware, power supplies, environmental enclosures, data transmission, processing and distribution, maintenance and integration in resources management systems.

Cooper, S.↗

User interface to administrative DRMS within a distributed environment

The implementation of a data base management system (DBMS) into a communications office to control and report on communication leased service contracts is discussed. The system user executes online programs to update five files residing on a UNIVAC 1100/82, through the forms mode features of the Tektronix 4025 terminal and IMSAI 8080 microcomputer. This user can select the appropriate form to the Tektronix 4025 screen, and enter new data, update existing data, or discontinue service. Selective online printing of 40 reports is accomplished by the system user to satisfy management, budget, and bill payment reporting requirements.

Martin, L. D.↗

The Importance of User Feedback in Sustaining Trusted Repositories

The NASA Earth Observation System Data and Information System (EOSDIS) has been operating since 1994 and is serving a global user community with well-managed Earth science data in a variety of scientific disciplines. EOSDIS processes, archives and distributes data and information products resulting from spaceborne and airborne instruments as well as in situ measurements from field campaigns. The Earth Science Data and Information System (ESDIS) Project at the NASA Goddard Space Flight Center manages EOSDIS with its 12 Distributed Active Archive Centers (DAACs) located across the United States. During the entire life of EOSDIS, the ESDIS Project and the DAACs have deployed many different mechanisms for user feedback, which have proven extremely valuable to their evolution and performance in their service to user communities. Some of the inputs from our user groups have resulted in fundamental changes in the architecture, design and operations of EOSDIS, while others have provided novel ideas for incremental changes. The EOSDIS DAACs have User Working Groups (UWGs) that represent broad user communities in the Earth science disciplines served by the DAACs. The UWGs meet periodically to assess and provide feedback on dataset and service priorities. As regular users of the data and services of the DAAC and experts in the scientific disciplines, the UWG members provide valuable inputs for planning and prioritizing the services, as well as addition of new datasets, for the benefit of the community. The EOSDIS is evaluated annually through an independently administered survey of its users resulting in the American Customer Satisfaction Index (ACSI). The survey provides an ACSI score as well as free-text suggestions from users, which are also helpful in making specific system improvements. In addition, each of the DAACs has a user services group that address on-going requests for help and other comments from users. The ESDIS Project has established a mechanism through the "earthdata" website (http://earthdata.nasa.gov) for users to provide feedback of any kind and these questions/comments are routed to the appropriate individuals in the Project or the DAACs. These various forms of receiving user feedback and responding to them continue to be extremely valuable in evolving and sustaining our Earth Science repository.

Behnke, Jeanne↗

A data and information system for processing, archival, and distribution of data for global change research

Work on this project was focused on information management techniques for Marshall Space Flight Center's EOSDIS Version 0 Distributed Active Archive Center (DAAC). The centerpiece of this effort has been participation in EOSDIS catalog interoperability research, the result of which is a distributed Information Management System (IMS) allowing the user to query the inventories of all the DAAC's from a single user interface. UAH has provided the MSFC DAAC database server for the distributed IMS, and has contributed to definition and development of the browse image display capabilities in the system's user interface. Another important area of research has been in generating value-based metadata through data mining. In addition, information management applications for local inventory and archive management, and for tracking data orders were provided.

Graves, Sara J.↗

Overview of ATLAS PanDA Workload Management

The Production and Distributed Analysis System (PanDA) plays a key role in the ATLAS distributed computing infrastructure. All ATLAS Monte-Carlo simulation and data reprocessing jobs pass through the PanDA system. We will describe how PanDA manages job execution on the grid using dynamic resource estimation and data replication together with intelligent brokerage in order to meet the scaling and automation requirements of ATLAS distributed computing. PanDA is also the primary ATLAS system for processing user and group analysis jobs, bringing further requirements for quick, flexible adaptation to the rapidly evolving analysis use cases of the early datataking phase, in addition to the high reliability, robustness and usability needed to provide efficient and transparent utilization of the grid for analysis users. We will describe how PanDA meets ATLAS requirements, the evolution of the system in light of operational experience, how the system has performed during the first LHC data-taking phase and plans for the future.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Implementing Connected Component Labeling as a User Defined Operator for SciDB

We have implemented a flexible User Defined Operator (UDO) for labeling connected components of a binary mask expressed as an array in SciDB, a parallel distributed database management system based on the array data model. This UDO is able to process very large multidimensional arrays by exploiting SciDB's memory management mechanism that efficiently manipulates arrays whose memory requirements far exceed available physical memory. The UDO takes as primary inputs a binary mask array and a binary stencil array that specifies the connectivity of a given cell to its neighbors. The UDO returns an array of the same shape as the input mask array with each foreground cell containing the label of the component it belongs to. By default, dimensions are treated as non-periodic, but the UDO also accepts optional input parameters to specify periodicity in any of the array dimensions. The UDO requires four stages to completely label connected components. In the first stage, labels are computed for each subarray or chunk of the mask array in parallel across SciDB instances using the weighted quick union (WQU) with half-path compression algorithm. In the second stage, labels around chunk boundaries from the first stage are stored in a temporary SciDB array that is then replicated across all SciDB instances. Equivalences are resolved by again applying the WQU algorithm to these boundary labels. In the third stage, relabeling is done for each chunk using the resolved equivalences. In the fourth stage, the resolved labels, which so far are "flattened" coordinates of the original binary mask array, are renamed with sequential integers for legibility. The UDO is demonstrated on a 3-D mask of O(1011) elements, with O(108) foreground cells and O(106) connected components. The operator completes in 19 minutes using 84 SciDB instances.

UDO↗

Data Management as a Cluster Middleware Centerpiece

Through earth and space modeling and the ongoing launches of satellites to gather data, NASA has become one of the largest producers of data in the world. These large data sets necessitated the creation of a Data Management System (DMS) to assist both the users and the administrators of the data. Halcyon Systems Inc. was contracted by the NASA Center for Computational Sciences (NCCS) to produce a Data Management System. The prototype of the DMS was produced by Halcyon Systems Inc. (Halcyon) for the Global Modeling and Assimilation Office (GMAO). The system, which was implemented and deployed within a relatively short period of time, has proven to be highly reliable and deployable. Following the prototype deployment, Halcyon was contacted by the NCCS to produce a production DMS version for their user community. The system is composed of several existing open source or government-sponsored components such as the San Diego Supercomputer Center s (SDSC) Storage Resource Broker (SRB), the Distributed Oceanographic Data System (DODS), and other components. Since Data Management is one of the foremost problems in cluster computing, the final package not only extends its capabilities as a Data Management System, but also to a cluster management system. This Cluster/Data Management System (CDMS) can be envisioned as the integration of existing packages.

Zero, Jose↗

NASA Langley Research Center's distributed mass storage system

There is a trend in institutions with high performance computing and data management requirements to explore mass storage systems with peripherals directly attached to a high speed network. The Distributed Mass Storage System (DMSS) Project at NASA LaRC is building such a system and expects to put it into production use by the end of 1993. This paper presents the design of the DMSS, some experiences in its development and use, and a performance analysis of its capabilities. The special features of this system are: (1) workstation class file servers running UniTree software; (2) third party I/O; (3) HIPPI network; (4) HIPPI/IPI3 disk array systems; (5) Storage Technology Corporation (STK) ACS 4400 automatic cartridge system; (6) CRAY Research Incorporated (CRI) CRAY Y-MP and CRAY-2 clients; (7) file server redundancy provision; and (8) a transition mechanism from the existent mass storage system to the DMSS.

Pao, Juliet Z.↗

Providing the Persistent Data Storage in a Software Engineering Environment Using Java/COBRA and a DBMS

An investigation was undertaken to build the software foundation for the WHERE (Web-based Hyper-text Environment for Requirements Engineering) project. The TCM (Toolkit for Conceptual Modeling) was chosen as the foundation software for the WHERE project which aims to provide an environment for facilitating collaboration among geographically distributed people involved in the Requirements Engineering process. The TCM is a collection of diagram and table editors and has been implemented in the C++ programming language. The C++ implementation of the TCM was translated into Java in order to allow the editors to be used for building various functionality of the WHERE project; the WHERE project intends to use the Web as its communication back- bone. One of the limitations of the translated software (TcmJava), which militated against its use in the WHERE project, was persistent data management mechanisms which it inherited from the original TCM; it was designed to be used in standalone applications. Before TcmJava editors could be used as a part of the multi-user, geographically distributed applications of the WHERE project, a persistent storage mechanism must be built which would allow data communication over the Internet, using the capabilities of the Web. An approach involving features of Java, CORBA (Common Object Request Broker), the Web, a middle-ware (Java Relational Binding (JRB)), and a database server was used to build the persistent data management infrastructure for the WHERE project. The developed infrastructure allows a TcmJava editor to be downloaded and run from a network host by using a JDK 1.1 (Java Developer's Kit) compatible Web-browser. The aforementioned editor establishes connection with a server by using the ORB (Object Request Broker) software and stores/retrieves data in/from the server. The server consists of a CORBA object or objects depending upon whether the data is to be made persistent on a single server or multiple servers. The CORBA object providing the persistent data server is implemented using the Java progranu-ning language. It uses the JRB to store/retrieve data in/from a relational database server. The persistent data management system provides transaction and user management facilities which allow multi-user, distributed access to the stored data in a secure manner.

Dhaliwal, Swarn S.↗

Data Grid Management Systems

The "Grid" is an emerging infrastructure for coordinating access across autonomous organizations to distributed, heterogeneous computation and data resources. Data grids are being built around the world as the next generation data handling systems for sharing, publishing, and preserving data residing on storage systems located in multiple administrative domains. A data grid provides logical namespaces for users, digital entities and storage resources to create persistent identifiers for controlling access, enabling discovery, and managing wide area latencies. This paper introduces data grids and describes data grid use cases. The relevance of data grids to digital libraries and persistent archives is demonstrated, and research issues in data grids and grid dataflow management systems are discussed.

Moore, Reagan W.↗

Integrating the PanDA Workload Management System with the Vera C. Rubin Observatory

The Vera C. Rubin Observatory will produce an unprecedented astronomical data set for studies of the deep and dynamic universe. Its Legacy Survey of Space and Time (LSST) will image the entire southern sky every three to four days and produce tens of petabytes of raw image data and associated calibration data over the course of the experiment’s run. More than 20 terabytes of data must be stored every night, and annual campaigns to reprocess the entire dataset since the beginning of the survey will be conducted over ten years. The Production and Distributed Analysis (PanDA) system was evaluated by the Rubin Observatory Data Management team and selected to serve the Observatory’s needs due to its demonstrated scalability and flexibility over the years, for its Directed Acyclic Graph (DAG) support, its support for multi-site processing, and its highly scalable complex workflows via the intelligent Data Delivery Service (iDDS). PanDA is also being evaluated for prompt processing where data must be processed within 60 seconds after image capture. This paper will briefly describe the Rubin Data Management system and its Data Facilities (DFs). Finally, it will describe in depth the work performed in order to integrate the PanDA system with the Rubin Observatory to be able to run the Rubin Science Pipelines using PanDA.

79 ASTRONOMY AND ASTROPHYSICS↗

EMASS (trademark): An expandable solution for NASA space data storage needs

The data acquisition, distribution, processing, and archiving requirements of NASA and other U.S. Government data centers present significant data management challenges that must be met in the 1990's. The Earth Observing System (EOS) project alone is expected to generate daily data volumes greater than 2 Terabytes (2 x 10(exp 12) Bytes). As the scientific community makes use of this data, their work will result in larger, increasingly complex data sets to be further exploited and managed. The challenge for data storage systems is to satisfy the initial data management requirements with cost effective solutions that provide for planned growth. The expendable architecture of the E-Systems Modular Automated Storage System (EMASS(TM)), a mass storage system which is designed to support NASA's data capture, storage, distribution, and management requirements into the 21st century is described.

Peterson, Anthony L.↗

NASA Exhibits

A series of NASA presentations for the Supercomputing 2001 conference are summarized. The topics include: (1) Mars Surveyor Landing Sites "Collaboratory"; (2) Parallel and Distributed CFD for Unsteady Flows with Moving Overset Grids; (3) IP Multicast for Seamless Support of Remote Science; (4) Consolidated Supercomputing Management Office; (5) Growler: A Component-Based Framework for Distributed/Collaborative Scientific Visualization and Computational Steering; (6) Data Mining on the Information Power Grid (IPG); (7) Debugging on the IPG; (8) Debakey Heart Assist Device: (9) Unsteady Turbopump for Reusable Launch Vehicle; (10) Exploratory Computing Environments Component Framework; (11) OVERSET Computational Fluid Dynamics Tools; (12) Control and Observation in Distributed Environments; (13) Multi-Level Parallelism Scaling on NASA's Origin 1024 CPU System; (14) Computing, Information, & Communications Technology; (15) NAS Grid Benchmarks; (16) IPG: A Large-Scale Distributed Computing and Data Management System; and (17) ILab: Parameter Study Creation and Submission on the IPG.

Deardorff, Glenn↗

Space station data system analysis/architecture study. Task 3: Trade studies, DR-5, volume 2

Results of a Space Station Data System Analysis/Architecture Study for the Goddard Space Flight Center are presented. This study, which emphasized a system engineering design for a complete, end-to-end data system, was divided into six tasks: (1); Functional requirements definition; (2) Options development; (3) Trade studies; (4) System definitions; (5) Program plan; and (6) Study maintenance. The Task inter-relationship and documentation flow are described. Information in volume 2 is devoted to Task 3: trade Studies. Trade Studies have been carried out in the following areas: (1) software development test and integration capability; (2) fault tolerant computing; (3) space qualified computers; (4) distributed data base management system; (5) system integration test and verification; (6) crew workstations; (7) mass storage; (8) command and resource management; and (9) space communications. Results are presented for each task.

Source record↗

Advanced optical disk storage technology

There is a growing need within the Air Force for more and better data storage solutions. Rome Laboratory, the Air Force's Center of Excellence for C3I technology, has sponsored the development of a number of operational prototypes to deal with this growing problem. This paper will briefly summarize the various prototype developments with examples of full mil-spec and best commercial practice. These prototypes have successfully operated under severe space, airborne and tactical field environments. From a technical perspective these prototypes have included rewritable optical media ranging from a 5.25-inch diameter format up to the 14-inch diameter disk format. Implementations include an airborne sensor recorder, a deployable optical jukebox and a parallel array of optical disk drives. They include stand-alone peripheral devices to centralized, hierarchical storage management systems for distributed data processing applications.

Haritatos, Fred N.↗

Building near-real-time processing pipelines with the spark-MPI platform

Advances in detectors and computational technologies provide new opportunities for applied research and the fundamental sciences. Concurrently, dramatic increases in the three V's (Volume, Velocity, and Variety) of experimental data and the scale of computational tasks produced the demand for new real-time processing systems at experimental facilities. Recently, this demand was addressed by the Spark-MPI approach connecting the Spark data-intensive platform with the MPI high-performance framework. In contrast with existing data management and analytics systems, Spark introduced a new middleware based on resilient distributed datasets (RDDs), which decoupled various data sources from high-level processing algorithms. The RDD middleware significantly advanced the scope of data-intensive applications, spreading from SQL queries to machine learning to graph processing. Spark-MPI further extended the Spark ecosystem with the MPI applications using the Process Management Interface. The paper explores this integrated platform within the context of online ptychographic and tomographic reconstruction pipelines.

97 MATHEMATICS AND COMPUTING↗

Software and computing for Run 3 of the ATLAS experiment at the LHC

The ATLAS experiment has developed extensive software and distributed computing systems for Run 3 of the LHC. These systems are described in detail, including software infrastructure and workflows, distributed data and workload management, database infrastructure, and validation. The use of these systems to prepare the data for physics analysis and assess its quality are described, along with the software tools used for data analysis itself. An outlook for the development of these projects towards Run 4 is also provided.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗