Design of False Data Injection Attack on Distributed Process Estimation
Not provided.
Computer Science↗
SEARCH · Search NASA
Results for “DISTRIBUTED DATA PROCESSING”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
SparseX - A tool for distributed processing of multidimensional sparse data
Developed for atmospheric science: specifically, a tool for processing sparse photon propagation data
Corzine, Riley Zhou↗
An automated system for processing and distributing SIRTF telemetry data
Explore the source record for details and available documents.
MIPL↗
Land and Atmosphere Near-Real-Time Capability for Earth Observing System
The past decade has seen a rapid increase in availability and usage of near-real-time data from satellite sensors. The EOSDIS (Earth Observing System Data and Information System) was not originally designed to provide data with sufficiently low latency to satisfy the requirements for near-real-time users. The EOS (Earth Observing System) instruments aboard the Terra, Aqua and Aura satellites make global measurements daily, which are processed into higher-level 'standard' products within 8-40 hours of observation and then made available to users, primarily earth science researchers. However, applications users, operational agencies, and even researchers desire EOS products in near-real-time to support research and applications, including numerical weather and climate prediction and forecasting, monitoring of natural hazards, ecological/invasive species, agriculture, air quality, disaster relief and homeland security. These users often need data much sooner than routine science processing allows, usually within 3 hours, and are willing to trade science product quality for timely access. While Direct Broadcast provides more timely access to data, it does not provide global coverage. In 2002, a joint initiative between NASA (National Aeronautics and Space Administration), NOAA (National Oceanic and Atmospheric Administration), and the DOD (Department of Defense) was undertaken to provide data from EOS instruments in near-real-time. The NRTPE (Near Real Time Processing Effort) provided products within 3 hours of observation on a best-effort basis. As the popularity of these near-real-time products and applications grew, multiple near-real-time systems began to spring up such as the Rapid Response System. In recognizing the dependence of customers on this data and the need for highly reliable and timely data access, NASA's Earth Science Division sponsored the Earth Science Data and Information System Project (ESDIS)-led development of a new near-real-time system called LANCE (Land, Atmosphere Near-Real-Time Capability for EOS) in 2009. LANCE consists of special processing elements, co-located with selected EOSDIS data centers and processing facilities. A primary goal of LANCE is to bring multiple near-real-time systems under one umbrella, offering commonality in data access, quality control, and latency. LANCE now processes and distributes data from the Moderate Resolution Imaging Spectroradiometer (MODIS), Atmospheric Infrared Sounder (AIRS), Advanced Microwave Scanning Radiometer Earth Observing System (AMSR-E), Microwave Limb Sounder (MLS) and Ozone Monitoring Instrument (OMI) instruments within 3 hours of satellite observation. The Rapid Response System and the Fire Information for Resource Management System (FIRMS) capabilities will be incorporated into LANCE in 2011. LANCE maintains a central website to facilitate easy access to data and user services. LANCE products are extensively tested and compared with science products before being made available to users. Each element also plans to implement redundant network, power and server infrastructure to ensure high availability of data and services. Through the user registration system, users are informed of any data outages and when new products or services will be available for access. Building on a significant investment by NASA in developing science algorithms and products, LANCE creates products that have a demonstrated utility for applications requiring near-real-time data. From lower level data products such as calibrated geolocated radiances to higher-level products such as sea ice extent, snow cover, and cloud cover, users have integrated LANCE data into forecast models and decision support systems. The table above shows the current near-real-time product categories by instrument. The ESDIS Project continues to improve the LANCE system and use the experience gained through practice to seek adjustments to improve the quality and performance of the system. For example, anGC-compliant Web Map Service (WMS) will be added shortly that will allow users to download geo-referenced MODIS images for arbitrary bounding boxes. Further, an OGC-compliant Web Coverage Service (WCS) will be added later this year that will expedite user access to arbitrary data subsets or re-formatted products. AIRS images are now served through WMS and available in multiple formats (PNG, GeoTIFF, KMZ). NASA has established a LANCE User Working Group to steer the development of the system and create a forum for sharing ideas and experiences that are expected to further improve the LANCE capabilities. The LANCE system has proved a success by satisfying the growing needs of the applications and operational communities for land and atmosphere data in near-real-time. NASA's Earth Sciences Division was able to leverage existing science research capabilities to provide the near-real-time community with products and imagery that support monitoring of disasters in a timely manner.
Murphy, Kevin J.↗
QuakeSim 2.0
QuakeSim 2.0 improves understanding of earthquake processes by providing modeling tools and integrating model applications and various heterogeneous data sources within a Web services environment. QuakeSim is a multisource, synergistic, data-intensive environment for modeling the behavior of earthquake faults individually, and as part of complex interacting systems. Remotely sensed geodetic data products may be explored, compared with faults and landscape features, mined by pattern analysis applications, and integrated with models and pattern analysis applications in a rich Web-based and visualization environment. Integration of heterogeneous data products with pattern informatics tools enables efficient development of models. Federated database components and visualization tools allow rapid exploration of large datasets, while pattern informatics enables identification of subtle, but important, features in large data sets. QuakeSim is valuable for earthquake investigations and modeling in its current state, and also serves as a prototype and nucleus for broader systems under development. The framework provides access to physics-based simulation tools that model the earthquake cycle and related crustal deformation. Spaceborne GPS and Inter ferometric Synthetic Aperture (InSAR) data provide information on near-term crustal deformation, while paleoseismic geologic data provide longerterm information on earthquake fault processes. These data sources are integrated into QuakeSim's QuakeTables database system, and are accessible by users or various model applications. UAVSAR repeat pass interferometry data products are added to the QuakeTables database, and are available through a browseable map interface or Representational State Transfer (REST) interfaces. Model applications can retrieve data from Quake Tables, or from third-party GPS velocity data services; alternatively, users can manually input parameters into the models. Pattern analysis of GPS and seismicity data has proved useful for mid-term forecasting of earthquakes, and for detecting subtle changes in crustal deformation. The GPS time series analysis has also proved useful as a data-quality tool, enabling the discovery of station anomalies and data processing and distribution errors. Improved visualization tools enable more efficient data exploration and understanding. Tools provide flexibility to science users for exploring data in new ways through download links, but also facilitate standard, intuitive, and routine uses for science users and end users such as emergency responders.
Donnellan, Andrea↗
IDC Reengineering Phase 2 Project Scope
The Comprehensive Nuclear-Test-Ban Treaty Organization (CTBTO) operates the International Data Centre (IDC) to support treaty verification activities by Member States. The IDC collects, stores, processes, and distributes data from seismic, hydroacoustic, infrasound, and radionuclide stations. A significant component of the IDC is the seismic, hydroacoustic, and infrasound (SHI) processing system. The IDC has recognized the need to reengineer the SHI processing system and has defined a three-phase reengineering effort. IDC Reengineering Phase 1 (RP1) started in 2011 with the goal to significantly modernize SHI automatic and interactive data processing software, focusing on enhancements to individual components of the system. Reengineering Phase 2 (RP2), started in 2014, addresses the task of specifying a unified architecture for all SHI software, across processing stages, to pave the way for further software development and sustainment in the future. A key objective of RP2 is an architecture sufficient to provide a basis for a cost estimate for the development or enhancement of the software components and subsystems. This report provides a brief description of the scope of the IDC Reengineering Phase 2 project. It describes the goals and objectives of reengineering, the system definition, and the technical scope of the system.This report expands upon the “Scope for IDC Reengineering Phase 2”, July 2014, ECS-DIS-WGB-43-PTS-MATERIAL-12-ADD-1, produced by the IDC.
58 GEOSCIENCES↗
EMASS (trademark): An expandable solution for NASA space data storage needs
The data acquisition, distribution, processing, and archiving requirements of NASA and other U.S. Government data centers present significant data management challenges that must be met in the 1990's. The Earth Observing System (EOS) project alone is expected to generate daily data volumes greater than 2 Terabytes (2 x 10(exp 12) Bytes). As the scientific community makes use of this data, their work will result in larger, increasingly complex data sets to be further exploited and managed. The challenge for data storage systems is to satisfy the initial data management requirements with cost effective solutions that provide for planned growth. The expendable architecture of the E-Systems Modular Automated Storage System (EMASS(TM)), a mass storage system which is designed to support NASA's data capture, storage, distribution, and management requirements into the 21st century is described.
Peterson, Anthony L.↗
EMASS (tm): An expandable solution for NASA space data storage needs
The data acquisition, distribution, processing, and archiving requirements of NASA and other U.S. Government data centers present significant data management challenges that must be met in the 1990's. The Earth Observing System (EOS) project alone is expected to generate daily data volumes greater than 2 Terabytes (2(10)(exp 12) Bytes). As the scientific community makes use of this data their work product will result in larger, increasingly complex data sets to be further exploited and managed. The challenge for data storage systems is to satisfy the initial data management requirements with cost effective solutions that provide for planned growth. This paper describes the expandable architecture of the E-Systems Modular Automated Storage System (EMASS (TM)), a mass storage system which is designed to support NASA's data capture, storage, distribution, and management requirements into the 21st century.
Peterson, Anthony L.↗
Extracting Resilience Metrics From Distribution Utility Data Using Outage and Restore Process Statistics
Resilience curves track the accumulation and restoration of outages during an event on an electric distribution grid. We show that a resilience curve generated from utility data can always be decomposed into an outage process and a restore process and that these processes generally overlap in time. We use many events in real utility data to characterize the statistics of these processes, and derive formulas based on these statistics for resilience metrics such as restore duration, customer hours not served, and outage and restore rates. The formulas express the mean value of these metrics as a function of the number of outages in the event. We also give a formula for the variability of restore duration, which allows us to predict a maximum restore duration with 95% confidence. Overall, we give a simple and general way to decompose resilience curves into outage and restore processes and then show how to use these processes to extract resilience metrics from standard distribution system data.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Near Real-Time Two and Three-Dimensional Electrical Geophysical Monitoring of Natural and Engineered Processes Associated with Subsurface Remediation - 16052
Legacy soil and groundwater contamination associated with nuclear weapons production is projected to be one of the largest cleanup liabilities to the U.S. Department of Energy for the foreseeable future. Remediation of subsurface contaminants is complicated by the combined effects of geochemical heterogeneity, geological heterogeneity, and the difficulty and expense of adequate borehole access. Each of these complications contributes to uncertainty in the distribution of contaminants and the performance of in situ treatments, which increases risk to human health and the environment and cost to closure in general. Electrical resistivity tomography (ERT) is a method of remotely imaging the electrical properties of the subsurface, which are governed by both geochemical and geological structure, thereby providing useful proxies for understanding contaminant distribution and the behavior in situ remediation processes. Recent advancements in both data collection and data processing capabilities are enabling ERT monitoring to be executed and provided to site operators in near real-time. To summarize the process, ERT surveys are rapidly and continuously collected during a time-sensitive operation such as an amendment injection. At the completion of each survey, data are transferred by wireless internet to offsite supercomputing resources for parallel tomographic inversion. Inversion results are then transferred back to onsite operators and/or other locations for visualization. In this paper, we demonstrate the utility of time-lapse ERT for monitoring fluid transport (both gas and liquid phase). We then demonstrate two real-time imaging applications for monitoring both natural and engineered processes at the Hanford Site within the vadose and the saturated zones. We expect this new capability for autonomous real-time imaging to enable ERT to be used as a cost-effective, short-term rapid feedback mechanism for guiding subsurface remediation injections, in addition to longer term monitoring of post-injection performance and environmental impact, all using the same electrode array and instrumentation. (authors)
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
The Airborne Visible/Infrared Imaging Spectrometer (AVIRIS)
AVIRIS is a facility consisting of a flight system, a ground data system, a calibration facility, and a full-time operations team. The facility was developed by JPL under funding from NASA. NASA also provides funding for operations and maintenance. The flight system is a whisk-broom imager that acquires data in 224 narrow, contiguous spectral bands covering the solar reflected portion of the electromagnetic spectrum. It is flown aboard the NASA high altitude ER-2 research aircraft. The ground data system is a facility dedicated to the processing and distribution of data acquired by AVIRIS. It operates year round at JPL. The calibration facility consists of a calibration laboratory at JPL and a suite of field instruments and procedures for performing inflight calibration of AVIRIS. A small team of engineers, technicians, and scientists supports a yearly operations schedule that includes 6 months of flight operations, 6 months of routine ground maintenance of the flight system, and year-round data processing and distribution. Details of the AVIRIS system, its performance history, and future plans are described.
Vane, Gregg↗
Maximum ADPE Approach for a High Rate CCSDS Return Link Processing System
The earth observing system data and operations system (EDOS) multi-mission data processing and distribution system for the earth observing system is considered. The EDOS was based on the Consultative Committee for Space Data Systems (CCSDS) protocols. The development included the challenge of developing and demonstrating a 150 Mbps CCSDS return link processing capability for the support of the first EDOS delivery. The approach used general-purpose automated data processing equipment (ADPE) and minimized the use of customized hardware. The way in which the system was developed is described. The principle design decisions and the performance benchmark results are presented.
Krimchansky, Alexander↗
Tropical Rainfall Measuring Mission
Tropical rainfall affects the lives and economics of a majority of the Earth's population. Tropical rain systems, such as hurricanes, typhoons, and monsoons, are crucial to sustaining the livelihoods of those living in the tropics. Excess rainfall can cause floods and great property and crop damage, whereas too little rainfall can cause drought and crop failure. The latent heat release during the process of precipitation is a major source of energy that drives the atmospheric circulation. This latent heat can intensify weather systems, affecting weather thousands of kilometers away, thus making tropical rainfall an important indicator of atmospheric circulation and short-term climate change. Tropical forests and the underlying soils are major sources of many of the atmosphere's trace constituents. Together, the forests and the atmosphere act as a water-energy regulating system. Most of the rainfall is returned to the atmosphere through evaporation and transpiration, and the atmospheric trace constituents take part in the recycling process. Hence, the hydrological cycle provides a direct link between tropical rainfall and the global cycles of carbon, nitrogen, and sulfur, all important trace materials for the Earth's system. Because rainfall is such an important component in the interactions between the ocean, atmosphere, land, and the biosphere, accurate measurements of rainfall are crucial to understanding the workings of the Earth-atmosphere system. The large spatial and temporal variability of rainfall systems, however, poses a major challenge to estimating global rainfall. So far, there has been a lack of rain gauge networks, especially over the oceans, which points to satellite measurement as the only means by which global observation of rainfall can be made. The Tropical Rainfall Measuring Mission (TRMM), jointly sponsored by the National Aeronautics and Space Administration (NASA) of the United States and the National Space Development Agency (NASDA) of Japan, provides visible, infrared, and microwave observations of tropical and subtropical rain systems.The satellite observations are complemented by ground radar and rain gauge measurements to validate satellite rain estimation techniques. Goddard Space Flight Center's involvement includes the observatory, four instruments, integration and testing of the observatory, data processing and distribution, and satellite operations. TRMM has a design lifetime of three years. Data generated from TRMM and archived at the GDAAC are useful not only for hydrologists, atmospheric scientists, and climatologists, but also for the health community studying infectious diseases, the ocean research community, and the agricultural community.
Source record↗
GPS Global Tracking Network Operations At JPL During the 1992 IGS Experiment and Beyond
The GPS Network Operations center at JPL Provides resources for the collection, distribution and maintenance of GPS data from global network of GPS receivers. This includes the ability to remotely diagnosis receiver problems, communicate with receivers and operators of receivers, to automatically collect, process and distribute data for 14 stations in the IGS network. This amounts to 35 megabytes of raw, finished data and products daily.
GPS receiver Global Network↗
Science opportunities using the NASA scatterometer on N-ROSS
The National Aeronautics and Space Administration scatterometer (NSCAT) is to be flown as part of the Navy Remote Ocean Sensing System (N-ROSS) scheduled for launch in 1989. The NSCAT will provide frequent accurate and high-resolution measurements of vector winds over the global oceans. NSCAT data will be applicable to a wide range of studies in oceanography, meteorology, and instrument science. The N-ROSS mission, is outlined, are described. The capabilities of the NSCAT flight instrument and an associated NASA research ground data-processing and distribution system, and representative oceanographic meteorological, and instrument science studies that may benefit from NSCAT data are surveyed.
Freilich, M. H.↗
EJFAT Scientific Perspective
Presented new computing model to the test by deploying the EJFAT system alongside a data-stream processing framework running the production-level CLAS12 event reconstruction application. In this experiment, a continuous stream of CLAS12 Level-1 identified events was processed in real-time using the EJFAT load balancer, distributing the workload across 90 computing nodes located across the U.S. This marks the first-ever large-scale, real-time distributed data stream processing experiment, demonstrating that scientific data-streaming pipelines can efficiently scale across four dimensions, thanks to EJFAT’s advanced hardware and software capabilities.
Gyurjyan, Vardan [Thomas Jefferson National Accele↗
TRMM Data from the Goddard Earth Sciences (GES) DISC DAAC: Tropical Rainfall Measuring Mission (TRMM)
Tropical rainfall affects the lives and economies of a majority of the Earth's population. Tropical rain systems, such as hurricanes, typhoons, and monsoons, are crucial to sustaining the livelihoods of those living in the tropics. Excess rainfall can cause floods and great property and crop damage, whereas too little rainfall can cause drought and crop failure. The latent heat release during the process of precipitation is a major source of energy that drives the atmospheric circulation. This latent heat can intensify weather systems, affecting weather thousands of kilometers away, thus making tropical rainfall an important indicator of atmospheric circulation and short-term climate change. The Tropical Rainfall Measuring Mission (TRMM), jointly sponsored by the National Aeronautics and Space Administration (NASA) of the United States and the National Space Development Agency (NASDA) of Japan, provides visible, infrared, and microwave observations of tropical and subtropical rain systems. The satellite observations are complemented by ground radar and rain gauge measurements to validate satellite rain estimation techniques. Goddard Space Flight Center's involvement includes the observatory, four instruments, integration and testing of the observatory, data processing and distribution, and satellite operations. TRMM has a design lifetime of three years. It is currently in its fifth year of operation. Data generated from TRMM and archived at the GES DAAC are useful not only for hydrologists, atmospheric scientists, and climatologists, but also for the health community studying infectious diseases, the ocean research community, and the agricultural community.
Source record↗
Atmospheric Infrared Sounder Version 7 Near-Real-Time Product and Imagery Released by NASA GES DISC
The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) has been the home of data processing, archive, and distribution services for data from the Atmospheric Infrared Sounder (AIRS) mission since its launch in 2002. The GES DISC provides service to both AIRS routine and Near Real-Time (NRT) products. The AIRS NRT products are an important element in the Land, Atmosphere Near real-time Capability for EOS (LANCE). In collaboration with AIRS Project, the GES DISC has just released products from the Version 7 algorithm. The new version algorithm provides significant improvements over the previous version. The most substantial advances are: improved consistency between day and night water vapor; improved total column ozone and temperature; improved infrared-only (IR-only) retrievals, especially in high latitude regions; an improved Stochastic Cloud Clearing Neural Network used as a first guess at the initial value in the iterative retrieval process; and removal of ambiguity in surface classification in the IR-only retrieval algorithm. In addition, the GES DISC produces AIRS NRT imagery. The AIRS NRT imagery are generated by mosaicking and mapping the available AIRS 6-minute retrieval granules to a global projection. The images are constantly refreshed when new granules are produced. The AIRS NRT Viewer and LANCE Worldview provide visualization services to online users for AIRS NRT imagery. The data products used to generate this imagery include atmospheric temperature, humidity, precipitation, cloud, Dust Score, CO, and SO2. In this presentation, we will demonstrate visualization of the AIRS NRT imagery from the new Version 7, and demonstrate some improvements over the previous version. Progress on improving the AIRS NRT imagery, a collaboration project with the AIRS Applications Development Team at NASA Jet Propulsion Laboratory (JPL), will also be presented.
Feng Ding↗