Search NASA⌕ Search

SEARCH · Search NASA

Results for “DISTRIBUTED DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

An Internet Protocol-Based Software System for Real-Time, Closed-Loop, Multi-Spacecraft Mission Simulation Applications

The paper will provide an overview of the web-based distributed simulation software system developed for end-to-end, multi-spacecraft mission design, analysis, and test at the NASA Goddard Space Flight Center (GSFC). This software system was developed for an internal research and development (IR&D) activity at GSFC called the Distributed Space Systems (DSS) Distributed Synthesis Environment (DSE). The long-term goal of the DSS-DSE is to integrate existing GSFC stand-alone test beds, models, and simulation systems to create a "hands on", end-to-end simulation environment for mission design, trade studies and simulations. The short-term goal of the DSE was therefore to develop the system architecture, and then to prototype the core software simulation capability based on a distributed computing approach, with demonstrations of some key capabilities by the end of Fiscal Year 2002 (FY02). To achieve the DSS-DSE IR&D objective, the team adopted a reference model and mission upon which FY02 capabilities were developed. The software was prototyped according to the reference model, and demonstrations were conducted for the reference mission to validate interfaces, concepts, etc. The reference model, illustrated in Fig. 1, included both space and ground elements, with functional capabilities such as spacecraft dynamics and control, science data collection, space-to-space and space-to-ground communications, mission operations, science operations, and data processing, archival and distribution addressed.

Davis, George↗

Collectives for Multiple Resource Job Scheduling Across Heterogeneous Servers

Efficient management of large-scale, distributed data storage and processing systems is a major challenge for many computational applications. Many of these systems are characterized by multi-resource tasks processed across a heterogeneous network. Conventional approaches, such as load balancing, work well for centralized, single resource problems, but breakdown in the more general case. In addition, most approaches are often based on heuristics which do not directly attempt to optimize the world utility. In this paper, we propose an agent based control system using the theory of collectives. We configure the servers of our network with agents who make local job scheduling decisions. These decisions are based on local goals which are constructed to be aligned with the objective of optimizing the overall efficiency of the system. We demonstrate that multi-agent systems in which all the agents attempt to optimize the same global utility function (team game) only marginally outperform conventional load balancing. On the other hand, agents configured using collectives outperform both team games and load balancing (by up to four times for the latter), despite their distributed nature and their limited access to information.

Tumer, K.↗

Pre-Launch GOES-R Risk Reduction Activities for the Geostationary Lightning Mapper

The GOES-R Geostationary Lightning Mapper (GLM) is a new instrument planned for GOES-R that will greatly improve storm hazard nowcasting and increase warning lead time day and night. Daytime detection of lightning is a particularly significant technological advance given the fact that the solar illuminated cloud-top signal can exceed the intensity of the lightning signal by a factor of one hundred. Our approach is detailed across three broad themes which include: Data Processing Algorithm Readiness, Forecast Applications, and Radiance Data Mining. These themes address how the data will be processed and distributed, and the algorithms and models for developing, producing, and using the data products. These pre-launch risk reduction activities will accelerate the operational and research use of the GLM data once GOES-R begins on-orbit operations. The GLM will provide unprecedented capabilities for tracking thunderstorms and earlier warning of impending severe and hazardous weather threats. By providing direct information on lightning initiation, propagation, extent, and rate, the GLM will also capture the updraft dynamics and life cycle of convective storms, as well as internal ice precipitation processes. The GLM provides information directly from the heart of the thunderstorm as opposed to cloud-top only. Nowcasting applications enabled by the GLM data will expedite the warning and response time of emergency management systems, improve the dispatch of electric power utility repair crews, and improve airline routing around thunderstorms thereby improving safety and efficiency, saving fuel and reducing delays. The use of GLM data will assist the Bureau of Land Management (BLM) and the Forest Service in quickly detecting lightning ground strikes that have a high probability of causing fires. Finally, GLM data will help assess the role of thunderstorms and deep convection in global climate, and will improve regional air quality and global chemistry/climate modeling. The GLM has a robust design that benefits and improves upon its strong heritage of NASA-developed LEO predecessors, the Optical Transient Detector (OTD) and the Lightning Imaging Sensor (LIS). GLM will have a substantially larger number of pixels within the focal plane, two lens systems, and multiple Real-Time Event Processors REPS for on-board event detection and data compression to provide continuous observations of the Americas and adjacent oceans.

Goodman, S. J.↗

Some Aeronautical Communications Experiments

Classically there has existed an asymmetry between the computing and communicating sides of aerospace systems. Over the past few decades, this asymmetry has shifted to favoring communication link technologies, meaning that advancements in available central processing units (CPUs), storage devices, and internal data buses have stagnated. Indeed, the increased emphasis placed on refining subsystem performance such as with antenna bandwidth in phased arrays, amplifier power efficiency, software defined radio (SDR) flexibility and encoding for data compression and error correction has given rise to successful debuts of multi-gigabit-per-second data return from long space-link distances. These accomplishments are easily quantifiable through link budgets and illustrate what is possible, but also reveal the deficiencies in overall communications capabilities. In particular, the ever-accelerating presence of aerospace vehicles gives rise to newer and larger classes of challenges to address the needs of 21st century systems. Furthermore remote sensing and imaging capabilities have far outpaced our ability to transmit their products to the ground, so we are increasingly dependent on pre-processing and downselection to contend with the communications bottleneck. No longer may we depend upon the constrained logistics in delivering end-to-end data delivery through manual reconfigurations, static event scheduling and execution on a per-vehicle basis, for these methods do not scale and therefore must give way to dynamic, networked approaches with an overall systems view in mind. Emerging mission requirements exhibit a trend toward multiple smaller-scale vehicles working together to perform dissimilar observations. Such operations necessitate sensor fusion across a constellation, and where data processing may be distributed throughout a fairly disconnected network whose topology changes over time in non-deterministic manners. Individual communications link performance is still very relevant to deploying an effective communications system, but now must be embedded within a greater architecture of capability to optimally utilize the bandwidth available from each link to generate an ultimate end-to-end quality of service. The deleterious effects of timing uncertainty across the arrangement presents a challenge to measurement synchronization and delivery, so a successful deployed system needs to be tolerant to the delays inherent in time-of-light between elements and digital processing latencies existing at each node. In this presentation we share the flight test results from a high performance Gbps laser communications terminal evaluated with a suite of store and forward capabilities called High-rate Delay Tolerant Networking (HDTN). The communications payload is operated over Lake Erie across a range of configurations including several convergence layers, and is evaluated to determine recovery time after link disruptions, information loss, efficiency and speed. The effectiveness of utilizing a flying laboratory to increase the Technology Readiness Level (TRL) of an integrated system in relevant environments is discussed, as well as the value of conducting aeronautics experiments to retire risk for technology infusion into space missions. Upcoming flight campaigns will be presented, including opportunities to demonstrate secure command and control, data intensive hyperspectral imaging, quantum link characterization, 4k High Definition (HD) video streaming and internetworked space-ground-aero relay operations. These experiments will pave the way for future missions which will depend upon interoperability across disparate government and privately owned networks, involve contention with uncertain and dynamic timing, and require agility to autonomously configure optimal parameters across networks of ever-increasing size and complexity to ensure data delivery. https://www1.grc.nasa.gov/space/scan/acs/tech-studies/dtn/

Daniel Raible↗

Real-Time Wireless Data Acquisition System

Current and future aerospace requirements demand the creation of a new breed of sensing devices, with emphasis on reduced weight, power consumption, and physical size. This new generation of sensors must possess a high degree of intelligence to provide critical data efficiently and in real-time. Intelligence will include self-calibration, self-health assessment, and pre-processing of raw data at the sensor level. Most of these features are already incorporated in the Wireless Sensors Network (SensorNet(TradeMark)), developed by the Instrumentation Group at Kennedy Space Center (KSC). A system based on the SensorNet(TradeMark) architecture consists of data collection point(s) called Central Stations (CS) and intelligent sensors called Remote Stations (RS) where one or more CSs can be accommodated depending on the specific application. The CS's major function is to establish communications with the Remote Stations and to poll each RS for data and health information. The CS also collects, stores and distributes these data to the appropriate systems requiring the information. The system has the ability to perform point-to-point, multi-point and relay mode communications with an autonomous self-diagnosis of each communications link. Upon detection of a communication failure, the system automatically reconfigures to establish new communication paths. These communication paths are automatically and autonomously selected as the best paths by the system based on the existing operating environment. The data acquisition system currently under development at KSC consists of the SensorNet(TradeMark) wireless sensors as the remote stations and the central station called the Radio Frequency Health Node (RFHN). The RFF1N is the central station which remotely communicates with the SensorNet(TradeMark) sensors to control them and to receive data. The system's salient feature is the ability to provide deterministic sensor data with accurate time stamps for both time critical and non-time critical applications. Current wireless standards such as Zigbee(TradeMark) and Bluetooth(Registered TradeMark) do not have these capabilities and can not meet the needs that are provided by the SensorNet technology. Additionally, the system has the ability to automatically reconfigure the wireless communication link to a secondary frequency if interference is encountered and can autonomously search for a sensor that was perceived to be lost using the relay capabilities of the sensors and the secondary frequency. The RFHN and the SensorNet designs are based on modular architectures that allow for future increases in capability and the ability to expand or upgrade with relative ease. The RFHN and SensorNet sensors .can also perform data processing which forms a distributed processing architecture allowing the system to pass along information rather than just sending "raw data points" to the next higher level system. With a relatively small size, weight and power consumption, this system has the potential for both spacecraft and aircraft applications as well as ground applications that require time critical data.

Valencia, Emilio J.↗

Monitoring techniques and alarm procedures for CMS services and sites in WLCG

The CMS offline computing system is composed of roughly 80 sites (including most experienced T3s) and a number of central services to distribute, process and analyze data worldwide. A high level of stability and reliability is required from the underlying infrastructure and services, partially covered by local or automated monitoring and alarming systems such as Lemon and SLS, the former collects metrics from sensors installed on computing nodes and triggers alarms when values are out of range, the latter measures the quality of service and warns managers when service is affected. CMS has established computing shift procedures with personnel operating worldwide from remote Computing Centers, under the supervision of the Computing Run Coordinator at CERN. This dedicated 24/7 computing shift personnel is contributing to detect and react timely on any unexpected error and hence ensure that CMS workflows are carried out efficiently and in a sustained manner. Synergy among all the involved actors is exploited to ensure the 24/7 monitoring, alarming and troubleshooting of the CMS computing sites and services. We review the deployment of the monitoring and alarming procedures, and report on the experience gained throughout the first two years of LHC operation. We describe the efficiency of the communication tools employed, the coherent monitoring framework, the proactive alarming systems and the proficient troubleshooting procedures that helped the CMS Computing facilities and infrastructure to operate at high reliability levels.

Molina-Perez, J.↗

An Efficient Storage-Driven Machine Learning Model for Performance in the Era of Multimodal Scientific Data

Scientific workflows are increasingly relying on machine learning (ML), simulation, and hybrid techniques to predict, understand, and optimize the behavior of complex experiments. High-performance computing has greatly improved researchers’ ability to acquire diverse data modalities in these workflows. Recent studies suggest that the performance of machine learning models can be improved by integrating data from various sources. Unfortunately, these workloads pose unprecedent pressure on the network storage to meet the demands associated with accessing these multimodal data. To mitigate the impact of intensive IO, we propose a solution that utilizes a multi-tier High-Performance Computing (HPC) distributed storage and data processing framework, placing computation where the data resides for better performance. By adopting this project, the scientific community will gain new opportunities to explore multimodal storage-driven possibilities, integrating multiple scientific data sources with advanced streaming frameworks. Additionally, our framework effectively utilizes computing resources and bridges the gaps identified by HPC experts. Our proposed approach tackles scalability and persistence challenges by leveraging native persistency, which has posed difficulties in traditional approaches. Furthermore, we seek to enhance fault-tolerance and load-balance of computations by leveraging real-time streaming in diverse scientific computing environments, thereby propelling advanced scientific computing research into the next generation.

97 MATHEMATICS AND COMPUTING↗

Data collection system: Earth Resources Technology Satellite-1

Subjects covered at the meeting concerned results on the overall data collection system including sensors, interface hardware, power supplies, environmental enclosures, data transmission, processing and distribution, maintenance and integration in resources management systems.

Cooper, S.↗

The McDonald Observatory lunar laser ranging project

A summary of the activities of the McDonald lunar laser ranging station at Fort Davis for the FY 77-78 fiscal year is presented. The lunar laser experiment uses the observatory 2.7m reflecting telescope on a thrice-per-day, 21-day-per-lunation schedule. Data are recorded on magnetic tapes and sent to the University of Texas at Austin where the data is processed. After processing, the data is distributed to interested analysis centers and later to the National Space Science Data Center where it is available for routine distribution. Detailed reports are published on the McDonald operations after every fourth lunation or approximately once every 115 days. These reports contain a day-by-day documentation of the ranging activity, detailed discussions of the equipment development efforts, and an abundance of other information as is needed to document and archive this important data type.

Silverberg, E. C.↗

A concept for a transparent data acquisition and distribution system for spaceflight applications

The paper describes the Modular Data System concept of the NASA End-to-End Data System program. Data from a single instrument along with the necessary ancillary data will be assembled into self-contained packets and subsequently transmitted over various communications links (space telemetry channel, ground communications circuits, etc.) to the experimenter's facility in real time; reliable error control coding will be included in each link transmission to protect the integrity of the data packets. A major objective is to make the entire data acquisition and distribution process transparent to the experimenter in the sense that the output terminal of the distribution system will be physically, logically and electrically identical to that of the experiment output channel.

Greene, E. P.↗

CNPQ/INPE LANDSAT system

The current status of the Brazilian LANDSAT facilities is described and main accomplishments are outlined. Receiving, recording, and processing substations and data distribution centers are discussed. Examples of the preliminary TM product produced by the Brazilian station are given.

Parada, N. D. J.↗

B-2 and other current flight test control center system architectures

The topics covered include the following: (1) telemetry processing independent of source; (2) generic system/software; (3) time homogeneity; (4) low latency networks; (5) distributed processing; (6) all data available to all workstations; and (7) recall of data in real time.

Hill, Jerry↗

CCD data acquisition systems at Lick and Keck Observatories

This paper will describe and compare two distinct but related CCD data acquisition systems (DAS) currently under development at Lick and Keck Observatories. Although these two systems have a number of major architectural differences, they share a considerable amount of common hardware and software. Both of these new systems build on a large body of proven software that is the foundation of the existing CCD DAS currently in use at Lick Observatory. Both will provide support for reading up to four on-chip amplifiers per CCD and/or reading out mosaics of CCD chips. In addition, they will provide the capability for interactive, real-time adjustment of CCD waveforms for engineering purposes. Each of these two systems is composed of three major subsystems: (1) an instrument computer and its software; (2) a data capture computer and its software; and (3) a CCD/dewar controller and its software. The instrument computer is a Unix workstation, and the functions it provides include user interfaces, the interactive real-time display of CCD images, and the recording of image and FITS header data to disk and/or tape. The data capture computer is responsible for the packaging and high-speed transfer of the CCD pixel data stream into a bulk RAM, and the subsequent transfer of this data to the instrument computer. The CCD/dewar controller generates the waveforms for clocking the CCD, digitizes the pixel data, and transmits it via high-speed link to the data capture computer. It is also responsible for monitoring and controlling the dewar temperature and cryogen levels. Given the number of different types of processors and high-speed data links employed in both systems, a major emphasis of this paper will be on the various forms of interprocessor communications utilized for data transfer and distributed process synchronization.

Kibrick, R. I.↗

Radio-science performance analysis software

The Radio Science Systems Group (RSSG) provides various support functions for several flight project radio-science teams. Among these support functions are uplink and sequence planning, real-time operations monitoring and support, data validation, archiving and distribution functions, and data processing and analysis. This article describes the support functions that encompass radio-science data performance analysis. The primary tool used by the RSSG to fulfill this support function is the STBLTY program set. STBLTY is used to reconstruct observable frequencies and calculate model frequencies, frequency residuals, frequency stability in terms of Allan deviation, reconstructed phase, frequency and phase power spectral density, and frequency drift rates. In the case of one-way data, using an ultrastable oscillator (USO) as a frequency reference, the program set computes the spacecraft transmitted frequency and maintains a database containing the in-flight history of the USO measurements. The program set also produces graphical displays. Some examples and discussions on operating the program set on Galileo and Ulysses data will be presented.

Morabito, D. D.↗

ASF archive issues: Current status, past history, and questions for the future

The Alaska SAR Facility (ASF) collects, processes, archives, and distributes data from synthetic aperture radar (SAR) satellites in support of scientific research. ASF has been in operation since 1991 and presently has an archive of over 100 terabytes of data. ASF is performing an analysis of its magnetic tape storage system to ensure long-term preservation of this archive. Future satellite missions have the possibility of doubling to tripling the amounts of data that ASF acquires. ASF is examining the current data systems and the high volume storage, and exploring future concerns and solutions.

Goula, Crystal A.↗

Cloud Surprises in Moving NASA EOSDIS Applications into Amazon Web Services

NASA ESDIS has been moving a variety of data ingest, distribution, and science data processing applications into a cloud environment over the last 2 years. As expected, there have been a number of challenges in migrating primarily on-premises applications into a cloud-based environment, related to architecture and taking advantage of cloud-based services. What was not expected is a number of issues that were beyond purely technical application re-architectures. We ran into surprising network policy limitations, billing challenges in a government-based cost model, and difficulty in obtaining certificates in an NASA security-compliant manner. On the other hand, this approach has allowed us to move a number of applications from local hosting to the cloud in a matter of hours (yes, hours!!), and our CMR application now services 95% of granule searches and an astonishing 99% of all collection searches in under a second. And most surprising of all, well, you'll just have to wait and see the realization that caught our entire team off guard!

Cloud↗

PBL Height from AIRS, GPS RO, and MERRA-2 Products in NASA GES DISC and Their 10 Year Seasonal Mean Intercomparison

Within the planetary boundary layer (PBL), surface forcing response, drag, turbulence, and vertical mixing are important processes and play a more critical role here than in the overlying “free atmosphere”. The PBL Height (PBLH) is an important parameter in climate models, weather forecasts, and air quality prediction. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) provides data processing, archiving, and distribution services for numerous Earth science products. PBLH is a parameter in three products served by GES DISC, which are from the Atmospheric Infrared Sounder (AIRS), the Global Positioning System (GPS) radio occultation (RO) experiment, and the NASA reanalysis product Modern-Era Retrospective analysis for Research and Applications – 2 (MERRA-2). These products have different spatial and temporal resolutions and coverages, and their PBLH definitions are also different. To better serve the PBL research community, we have summarized the specifications of these products. A ten-year seasonal mean intercomparison is also conducted to provide further guidance to users. The intercomparison results show that MERRA-2 has a much shallower PBL than AIRS and GPS RO. An experimental study indicates the different PBLH definition in MERRA-2 caused smaller values of PBLH. The improvement of the water vapor retrieval in AIRS version 7 over version 6 results in the version 7 PBLH agreeing better with GPS RO and MERRA-2 than version 6, especially near the equator and low latitudes.

Feng Ding↗

SPHEREx: NASA’s Near-Infrared Spectrophotmetric All-Sky Survey

SPHEREx, the Spectro-Photometer for the History of the Universe, Epoch of Reionization, and ices Explorer, isa NASA MIDEX mission planned for launch in 2024. SPHEREx will carry out the first all-sky spectral survey at wavelengths between 0.75μm and 5μm with spectral resolving power ~40 between 0.75 and 3.8μm and ~120 between 3.8 and 5μm At the end of its two-year mission, SPHEREx will provide 14 billion 0.75-to-5μm spectra of each 6."2 x 6."2 pixel on the sky. This paper updates an earlier description of SPHEREx presenting changes made during the mission's Preliminary Design Phase, including a discussion of instrument integration and test flow and a summary of the data processing, analysis, and distribution plans.

Zemcov, Michael↗