Search NASA⌕ Search

SEARCH · Search NASA

Results for “DISTRIBUTED DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

A Future Vision of a Data Acquisition: Distributed Sensing, Processing, and Health Monitoring

This paper presents a vision fo a highly enhanced data acquisition and health monitoring system at NASA Stennis Space Center (SSC) rocket engine test facility. This vision includes the use of advanced processing capabilities in conjunction with highly autonomous distributed sensing and intelligence, to monitor and evaluate the health of data in the context of it's associated process. This method is expected to significantly reduce data acquisitions costs and improve system reliability. A Universal Signal Conditioning Amplifier (USCA) based system, under development at Kennedy Space Center, is being evaluated for adaptation to the SSC testing infrastructure. Kennedy's USCA architecture offers many advantages including flexible and auto-configuring data acquisition with improved calibration and verifiability. Possible enhancements at SSC may include multiplexing the distributed USCAs to reduce per channel cost, and the use of IEEE-485 to Allen-Bradley Control Net Gateways for interfacing with the resident control systems.

Figueroa, Fernando↗

Expressing Parallelism with ROOT

The need for processing the ever-increasing amount of data generated by the LHC experiments in a more efficient way has motivated ROOT to further develop its support for parallelism. Such support is being tackled both for shared-memory and distributed-memory environments. The incarnations of the aforementioned parallelism are multi-threading, multi-processing and cluster-wide executions. In the area of multi-threading, we discuss the new implicit parallelism and related interfaces, as well as the new building blocks to safely operate with ROOT objects in a multi-threaded environment. Regarding multi-processing, we review the new MultiProc framework, comparing it with similar tools (e.g. multiprocessing module in Python). Finally, as an alternative to PROOF for cluster-wide executions, we introduce the efforts on integrating ROOT with state-of-the-art distributed data processing technologies like Spark, both in terms of programming model and runtime design (with EOS as one of the main components). For all the levels of parallelism, we discuss, based on real-life examples and measurements, how our proposals can increase the productivity of scientists.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Tropical rainfall measuring mission - US/Japan science operations

The Tropical Rainfall Measuring Mission (TRMM) is a joint-venture between the National Aeronautics and Space Administration (NASA) and the National Space Development Agency (NASDA) of Japan. This cooperative program provides the resources required to place in a low-earth orbit a 3500 kilogram (kg) satellite. The satellite carries a complement of instruments designed to measure the tropical and subtropical rainfall as required to provide data products which will lead to climate modeling. This paper describes the mission, the spacecraft and instrument complement, and the data processing, distribution, and archiving system. Included in the paper is the possible, but not approved, an experimental data relay link using a NASDA data relay satellite.

Keating, Thomas↗

Standard services for the capture, processing, and distribution of packetized telemetry data

Standard functional services for the capture, processing, and distribution of packetized data are discussed with particular reference to the future implementation of packet processing systems, such as those for the Space Station Freedom. The major functions are listed under the following major categories: input processing, packet processing, and output processing. A functional block diagram of a packet data processing facility is presented, showing the distribution of the various processing functions as well as the primary data flow through the facility.

Stallings, William H.↗

On-board B-ISDN fast packet switching architectures. Phase 2: Development. Proof-of-concept architecture definition report

For the next-generation packet switched communications satellite system with onboard processing and spot-beam operation, a reliable onboard fast packet switch is essential to route packets from different uplink beams to different downlink beams. The rapid emergence of point-to-point services such as video distribution, and the large demand for video conference, distributed data processing, and network management makes the multicast function essential to a fast packet switch (FPS). The satellite's inherent broadcast features gives the satellite network an advantage over the terrestrial network in providing multicast services. This report evaluates alternate multicast FPS architectures for onboard baseband switching applications and selects a candidate for subsequent breadboard development. Architecture evaluation and selection will be based on the study performed in phase 1, 'Onboard B-ISDN Fast Packet Switching Architectures', and other switch architectures which have become commercially available as large scale integration (LSI) devices.

Shyy, Dong-Jye↗

Advanced telemetry systems for payloads. Technology needs, objectives and issues

The current trends in advanced payload telemetry are the new developments in advanced modulation/coding, the applications of intelligent techniques, data distribution processing, and advanced signal processing methodologies. Concerted efforts will be required to design ultra-reliable man-rated software to cope with these applications. The intelligence embedded and distributed throughout various segments of the telemetry system will need to be overridden by an operator in case of life-threatening situations, making it a real-time integration issue. Suitable MIL standards on physical interfaces and protocols will be adopted to suit the payload telemetry system. New technologies and techniques will be developed for fast retrieval of mass data. Currently, these technology issues are being addressed to provide more efficient, reliable, and reconfigurable systems. There is a need, however, to change the operation culture. The current role of NASA as a leader in developing all the new innovative hardware should be altered to save both time and money. We should use all the available hardware/software developed by the industry and use the existing standards rather than inventing our own.

Source record↗

Localization of Ad-Hoc Lunar Constellations in Communication Failure Modes for Distributed Spacecraft Autonomy

As lunar missions increase in complexity inspired by NASA’s Artemis Program, they will require reliable and sufficient capability of the Position, Navigation, and Timing (PNT) system to support their scientific objectives. In addition, NASA's Commercial Lunar Payload Services (CLPS) program initiates the proliferation of public and private exploration partnerships using small satellites from commercial and private organizations, expanding traditionally confined low Earth orbit to be used for missions beyond geosynchronous orbit (Zucherman et al., 2022). Therefore, the Lunar PNT system is also required to provide navigation services compatible with the smaller platforms being sent by the public and private sectors, like CubeSats. However, traditional approaches to deep space missions’ navigation based on ground radio facilities have difficulties in providing sufficient support for the increasing number of users and communication at a distance from the Earth (Kaplev et al., 2022). In particular, the existing Lunar navigation technologies such as weak signal global positioning system (GPS) and deep space network (DSN) are not able to ensure operations of the upcoming small-scale Lunar missions due to their limitations in localization performance as well as capacity aspects. Another way to provide Lunar PNT service is to create a dedicated Lunar global navigation satellite system (GNSS) constellation, like GNSS systems on Earth. Space agencies like NASA, ESA, and JAXA are now developing the lunar communications relay and navigation systems (LCRNS) and Lunar navigation satellite systems (LNSS). In their systems, satellites will be deployed in moon orbits to provide the communication, positioning, navigation, and timing (CPNT) service at the lunar south pole region where the Artemis base camp will be expected (Murata et al., 2022). Meanwhile, common challenges considered in lunar PNT research arise from poor geometry of the terrestrial GNSS satellites when seen from the lunar user, highly perturbed lunar orbits, and limitations in power, size, and cost of the equipment on lunar satellites (Iiyama et al., 2023). It is also not clear if there will be enough Lunar users to support the cost and resources this would require as the Low-cost surface missions may not be able to support the large power, mass, and weight requirements that these navigation solutions entail (Niemoeller et al., 2022). As an alternative, existing Lunar science and exploration assets could be used to create a low-cost, autonomous, ad-hoc, and on-demand mission-centric Lunar PNT swarm capable of providing PNT services to these low-cost lunar missions (Hagenau et al., 2021). Introducing the non-dedicated and ad-hoc Lunar navigation constellation gives a way to provide PNT services on-demand. The non-dedicated swarm assets of Lunar constellations are designed to localize themselves with minimal interaction with Earth by adding cooperative autonomous localization to lunar missions, freeing up valuable bandwidth and ground segment resources. An autonomous localization of Lunar constellations is based on the concept of the decentralized PNT system with a distributed extended Kalman filter (DEKF) approach to state estimation for minimal onboard operating costs. In the distributed data processing algorithm, computation is broken down and assigned to each satellite, resulting in a considerably decreased computational amount while maintaining the accuracy of the orbit ephemeris and clock offsets as the result of centralized data processing (Wen et al., 2019). The DEKF requires spacecraft to perform two-way ranging operations with each other to communicate simultaneously, leveraging neighbor two-way intersatellite link (ISL) measurements such as pseudoranges to, and relative velocities between, visible satellites as sensor values (Frank et al., 2021). The Lunar autonomous PNT simulation (LAPS) demonstrated the feasibility of orbital asset localization among ad-hoc Lunar small-sat constellations based on the DEKF in Hagenau et al. (2021) and evaluated the matching algorithm proposed by Frank et al. (2021) in scheduling position estimation updates. In previous papers, all assets and measurements are assumed to be always available without consideration of the impact of intermittent and permanent communication failure. This study presents localization performance with increasing levels of network degradation for swarm assets and users to demonstrate the robustness of the decentralized Lunar PNT service in more realistic scenarios. Main issues arising from communication failure include spacecraft permanent or transient loss, antenna failures, message delays, etc. We tested four possible reasons for network degradation for 7 days in 21 satellites frozen with an altitude of 5500 km, evenly spaced around 3 circular, 40 inclination orbital planes where each spacecraft has two directional antennas. As anchor nodes with an independent estimate of their position are required in the DEKF approach, two ground nodes in each pole and one node in the gateway were implemented in the simulation. First, the most probable failure scenario involves the loss of a single spacecraft due to solar interference and technical malfunctions of the assets. Losing the availability of a single spacecraft means losing the two-way ISL measurement of the asset in the DEKF update. In order to provide the best possible quality of PNT service with limited time and resources, the distributed Lunar constellations must schedule the communication activities. The scheduler leverages mixed-integer linear programming (MILP) for the coordination and scheduling of the desired “as-needed” localization service (Niemoeller et al., 2022). We assume the scheduler has completely excluded the spacecraft information before the DEKF update in the failure scenario. When a random spacecraft has been turned off at a specific time, the robustness of the autonomous Lunar PNT system is evaluated. The simulation results give an 11.5% degradation in median position accuracy compared to the idealized performance excluding the asset loss. Second, a large number of assets may vanish due to major hardware problems or meteor strikes around the moon. A multiple spacecraft loss can degrade the localization performance very fast by losing the communication ability to do cross-plane measurements and in-plane measurements in a 3-plane constellation. When the matching-based scheduler is aware of ISL availability, we investigate a large number of in-plane and cross-plane asset vanishments both in close proximity and equally spaced throughout the orbital plane. According to the simulations, the loss of in-plane measurements gives 40.2% degradation while cross-plane measurements degrade 50.5% of asset localization performance among available assets. Therefore, it is concluded that cross-plane measurements are more important in improving the position estimation accuracy. Third, spacecraft failure information can be lost due to the internal message delay, resulting in the DEKF update scheduler to solve the matching problem with unavailable assets. The DEKF update cycle is comprised of network setup, communication, and computations where a global broadcast network and a 2-way ISL network setup take 6 minutes in total (Frank et al., 2021). Once the broadcast network successfully transmits and receives information, a random spacecraft may lose its availability right before solving the matching problem. This means the matching solution is no longer optimal, resulting in degradation in the localization performance. A numerical assessment shows the matching-based scheduler with knowing failure holds 11.5% of position accuracy degradation, whereas the scheduler without knowing failure gives 34% degraded localization performance without asset loss. Fourth, a transient loss of a single or multiple spacecraft may occur due to their antenna outages. After losing the two-way ISL availability for a few DEKF update cycles, the availability of spacecraft can easily be recovered as their states have been independently updated using measurements from anchor nodes. It is likely that the longer failure will result in worse localization performance. We have tested the transient failure of a random single asset for 30 min in the simulation, which is losing 3 update cycles in the DEKF system. From the simulation results, the position accuracy has been degraded to 4.84% which is better than the degraded localization performance of 11.5% from the permanent loss scenario among available assets. In conclusion, the autonomous Lunar PNT system based on the DEKF approach shows the ability to maintain resilience and robustness in the possible communication failure scenarios, ensuring that localization accuracy is preserved across various network degradation and outages. Future studies on investigating user localization performance near the South Pole and the broadcast network system will be continued in the following months.

Yeji Kim↗

CMS Analysis and Data Reduction with Apache Spark

Experimental Particle Physics has been at the forefront of analyzing the world's largest datasets for decades. The HEP community was among the first to develop suitable software and computing tools for this task. In recent times, new toolkits and systems for distributed data processing, collectively called "Big Data" technologies have emerged from industry and open source projects to support the analysis of Petabyte and Exabyte datasets in industry. While the principles of data analysis in HEP have not changed (filtering and transforming experiment-specific data formats), these new technologies use different approaches and tools, promising a fresh look at analysis of very large datasets that could potentially reduce the time-to-physics with increased interactivity. Moreover these new tools are typically actively developed by large communities, often profiting of industry resources, and under open source licensing. These factors result in a boost for adoption and maturity of the tools and for the communities supporting them, at the same time helping in reducing the cost of ownership for the end-users. In this talk, we are presenting studies of using Apache Spark for end user data analysis. We are studying the HEP analysis workflow separated into two thrusts: the reduction of centrally produced experiment datasets and the end-analysis up to the publication plot. Studying the first thrust, CMS is working together with CERN openlab and Intel on the CMS Big Data Reduction Facility. The goal is to reduce 1 PB of official CMS data to 1 TB of ntuple output for analysis. We are presenting the progress of this 2-year project with first results of scaling up Spark-based HEP analysis. Studying the second thrust, we are presenting studies on using Apache Spark for a CMS Dark Matter physics search, comparing Spark's feasibility, usability and performance to the ROOT-based analysis.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Avionics test bed development plan

A development plan for a proposed avionics test bed facility for the early investigation and evaluation of new concepts for the control of large space structures, orbiter attached flex body experiments, and orbiter enhancements is presented. A distributed data processing facility that utilizes the current laboratory resources for the test bed development is outlined. Future studies required for implementation, the management system for project control, and the baseline system configuration are defined. A background analysis of the specific hardware system for the preliminary baseline avionics test bed system is included.

Harris, L. H.↗

Avionics test bed development plan

The plan is for a facility for the early investigation and evaluation of new concepts for the control of large space structures, orbiter attached flex body experiments, and orbiter enhancements. This plan outlines a distributed data processing facility that will utilize the current JSC laboratory resources for the test bed development. The future studies required for implementation, the management system for project control, and the baseline system configuration are described.

Harris, L. H.↗

Advanced optical disk storage technology

There is a growing need within the Air Force for more and better data storage solutions. Rome Laboratory, the Air Force's Center of Excellence for C3I technology, has sponsored the development of a number of operational prototypes to deal with this growing problem. This paper will briefly summarize the various prototype developments with examples of full mil-spec and best commercial practice. These prototypes have successfully operated under severe space, airborne and tactical field environments. From a technical perspective these prototypes have included rewritable optical media ranging from a 5.25-inch diameter format up to the 14-inch diameter disk format. Implementations include an airborne sensor recorder, a deployable optical jukebox and a parallel array of optical disk drives. They include stand-alone peripheral devices to centralized, hierarchical storage management systems for distributed data processing applications.

Haritatos, Fred N.↗

Model Output Is Data, Too: And Other Futuristic Views

The NASA space science data environment has been undergoing significant changes during the past decade: Data has become significantly more voluminous and more varied. At the same time it became much more openly available to the scientific community. As a result a number of efforts have been made to document the data content at NSSDC/SPDF, PDS, the ISTP program and more recently as part of the newly developing Virtual Observatories. Concurrently, our need to access data has become much more immediate dictating a distributed environment where users can obtain their data directly from the production sites. But where does this change lead? And what more needs to be done to further enable scientific productivity? A view of our data system 5- years in the future will be offered that envisions distributed data processing services whereby the user can obtain the type of data that is needed not necessarily what is produced. Moreover, in an ideal system the means should exist to treat models as instruments allowing the ready mixing of their output with spacecraft generated data.

Szabo, A.↗

A data and information system for processing, archival, and distribution of data for global change research

Work on this project was focused on information management techniques for Marshall Space Flight Center's EOSDIS Version 0 Distributed Active Archive Center (DAAC). The centerpiece of this effort has been participation in EOSDIS catalog interoperability research, the result of which is a distributed Information Management System (IMS) allowing the user to query the inventories of all the DAAC's from a single user interface. UAH has provided the MSFC DAAC database server for the distributed IMS, and has contributed to definition and development of the browse image display capabilities in the system's user interface. Another important area of research has been in generating value-based metadata through data mining. In addition, information management applications for local inventory and archive management, and for tracking data orders were provided.

Graves, Sara J.↗

The Suomi National Polar-Orbiting Partnership (SNPP): Continuing NASA Research and Applications

The Suomi National Polar-orbiting Partnership (SNPP) satellite was successfully launched into a polar orbit on October 28, 2011 carrying 5 remote sensing instruments designed to provide data to improve weather forecasts and to increase understanding of long-term climate change. SNPP provides operational continuity of satellite-based observations for NOAA's Polar-orbiting Operational Environmental Satellites (POES) and continues the long-term record of climate quality observations established by NASA's Earth Observing System (EOS) satellites. In the 2003 to 2011 pre-launch timeframe, NASA's SNPP Science Team assessed the adequacy of the operational Raw Data Records (RDRs), Sensor Data Records (SDRs), and Environmental Data Records (EDRs) from the SNPP instruments for use in NASA Earth Science research, examined the operational algorithms used to produce those data records, and proposed a path forward for the production of climate quality products from SNPP. In order to perform these tasks, a distributed data system, the NASA Science Data Segment (SDS), ingested RDRs, SDRs, and EDRs from the NOAA Archive and Distribution and Interface Data Processing Segments, ADS and IDPS, respectively. The SDS also obtained operational algorithms for evaluation purposes from the NOAA Government Resource for Algorithm Verification, Independent Testing and Evaluation (GRAVITE). Within the NASA SDS, five Product Evaluation and Test Elements (PEATEs) received, ingested, and stored data and performed NASA's data processing, evaluation, and analysis activities. The distributed nature of this data distribution system was established by physically housing each PEATE within one of five Climate Analysis Research Systems (CARS) located at either at a NASA or a university institution. The CARS were organized around 5 key EDRs directly in support of the following NASA Earth Science focus areas: atmospheric sounding, ocean, land, ozone, and atmospheric composition products. The PEATES provided the system level interface with members of the NASA SNPP Science Team and other science investigators within each CARS. A sixth Earth Radiation Budget CARS was established at NASA Langley Research Center (NASA LaRC) to support instrument performance, data evaluation, and analysis for the SNPP Clouds and the Earth's Radiant Budget Energy System (CERES) instrument. Following the 2011 launch of SNPP, spacecraft commissioning, and instrument activation, the NASA SNPP Science Team evaluated the operational RDRs, SDRs, and EDRs produced by the NOAA ADS and IDPS. A key part in that evaluation was the NASA Science Team's independent processing of operational RDRs and SDRs to EDRs using the latest NASA science algorithms. The NASA science evaluation was completed in the December 2012 to April 2014 timeframe with the release of a series of NASA Science Team Discipline Reports. In summary, these reports indicated that the RDRs produced by the SNPP instruments were of sufficiently high quality to be used to create data products suitable for NASA Earth System science and applications. However, the quality of the SDRs and EDRs were found to vary greatly when considering suitability for NASA science. The need for improvements in operational algorithms, adoption of different algorithmic approaches, greater monitoring of on-orbit instrument calibration, greater attention to data product validation, and data reprocessing were prominent findings in the reports. In response to these findings, NASA, in late 2013, directed the NASA SNPP Science Team to use SNPP instrument data to develop data products of sufficiently high quality to enable the continuation of EOS time series data records and to develop innovative, practical applications of SNPP data. This direction necessitated a transition of the SDS data system from its pre-launch assessment mode to one of full data processing and production. To do this, the PEATES, which served as NASA's data product testing environment during the prelaunch and early on-orbit periods, were transitioned to Science Investigator-led Processing Systems (SIPS). The distributed data architecture was maintained in this new system by locating the SIPS at the same institutions at which the CARS and PEATES were located. The SIPS acquire raw SNPP instrument Level 0 (i.e. RDR) data over the full SNPP mission from the NOAA ADS and IDPS through the NASA SDS Data Distribution and Depository Element (SD3E). The SIPS process those data into NASA Level 1, Level 2, and global, gridded Level 3 standard products using peer-reviewed algorithms provided by members of the NASA Science Team. The SIPS work with the NASA SNPP Science Team in obtaining enhanced, refined, or alternate real-time algorithms to support the capabilities of the Direct Readout Laboratory (DRL). All data products, algorithm source codes, coefficients, and auxiliary data used in product generation are archived in an assigned NASA Distributed Active Archive Center (DAAC).

remote sensing↗

Advanced Distributed Measurements and Data Processing at the Vibro-Acoustic Test Facility, GRC Space Power Facility, Sandusky, Ohio - an Architecture and an Example

A large-scale, distributed, high-speed data acquisition system (HSDAS) is currently being installed at the Space Power Facility (SPF) at NASA Glenn Research Center s Plum Brook Station in Sandusky, OH. This installation is being done as part of a facility construction project to add Vibro-acoustic Test Capabilities (VTC) to the current thermal-vacuum testing capability of SPF in support of the Orion Project s requirement for Space Environments Testing (SET). The HSDAS architecture is a modular design, which utilizes fully-remotely managed components, enables the system to support multiple test locations with a wide-range of measurement types and a very large system channel count. The architecture of the system is presented along with details on system scalability and measurement verification. In addition, the ability of the system to automate many of its processes such as measurement verification and measurement system analysis is also discussed.

Hill, Gerald M.↗

Flexible Revision Of Data-Processing Communications

Distributed Objects Protocol Layer (DOPL) software system implements protocol for communications among software objects programmed in C++ computer language. DOPL is application-program interface (API). Enables communication of data without regard for which specific computer(s) and operating system(s) is (are) in use. No extension of the C++ language necessary for use of DOPL.

Burleigh, Scott C.↗

EOSDIS Data for Oceans Research

The JPL Physical Oceanography Distributed Active Archive Center (DAAC) is one of nine centers processing data for the Earth Observation System. As the physical oceanography data center, it archives, processes, and distributes data to researchers from various satellites. Data holdings can be searched through the Global Change Master Directory, and an advanced information management system is now in prototype.

physical oceanography EOS↗

Performance of the Landsat-Data Collection System in a Total System Context

The author has identified the following significant results. This experiment was, and continues to be, an integration of the LANDSAT-DCS with the data collection and processing system of the Geological Survey. Although an experimental demonstration, it was a successful integration of a satellite relay system that is capable of continental data collection, and an existing governmental nationwide operational data processing and distributing networks. The Survey's data processing system uses a large general purpose computer with insufficient redundancy for 24-hour a day, 7 day a week operation. This is significant, but soluble obstacle to converting the experimental integration of the system to an operational integration.

Paulson, R. W.↗