Search NASA⌕ Search

SEARCH · Search NASA

Results for “Massive Data Movement”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

Parallel algorithms for interactive manipulation of digital terrain models

Interactive three-dimensional graphics applications, such as terrain data representation and manipulation, require extensive arithmetic processing. Massively parallel machines are attractive for this application since they offer high computational rates, and grid connected architectures provide a natural mapping for grid based terrain models. Presented here are algorithms for data movement on the massive parallel processor (MPP) in support of pan and zoom functions over large data grids. It is an extension of earlier work that demonstrated real-time performance of graphics functions on grids that were equal in size to the physical dimensions of the MPP. When the dimensions of a data grid exceed the processing array size, data is packed in the array memory. Windows of the total data grid are interactively selected for processing. Movement of packed data is needed to distribute items across the array for efficient parallel processing. Execution time for data movement was found to exceed that for arithmetic aspects of graphics functions. Performance figures are given for routines written in MPP Pascal.

Davis, E. W.↗

Dynamic Load Balancing For Grid Partitioning on a SP-2 Multiprocessor: A Framework

Computational requirements of full scale computational fluid dynamics change as computation progresses on a parallel machine. The change in computational intensity causes workload imbalance of processors, which in turn requires a large amount of data movement at runtime. If parallel CFD is to be successful on a parallel or massively parallel machine, balancing of the runtime load is indispensable. Here a framework is presented for dynamic load balancing for CFD applications, called Jove. One processor is designated as a decision maker Jove while others are assigned to computational fluid dynamics. Processors running CFD send flags to Jove in a predetermined number of iterations to initiate load balancing. Jove starts working on load balancing while other processors continue working with the current data and load distribution. Jove goes through several steps to decide if the new data should be taken, including preliminary evaluate, partition, processor reassignment, cost evaluation, and decision. Jove running on a single IBM SP2 node has been completely implemented. Preliminary experimental results show that the Jove approach to dynamic load balancing can be effective for full scale grid partitioning on the target machine IBM SP2.

Sohn, Andrew↗

Dynamic Load Balancing for Grid Partitioning on a SP-2 Multiprocessor: A Framework

Computational requirements of full scale computational fluid dynamics change as computation progresses on a parallel machine. The change in computational intensity causes workload imbalance of processors, which in turn requires a large amount of data movement at runtime. If parallel CFD is to be successful on a parallel or massively parallel machine, balancing of the runtime load is indispensable. Here a framework is presented for dynamic load balancing for CFD applications, called Jove. One processor is designated as a decision maker Jove while others are assigned to computational fluid dynamics. Processors running CFD send flags to Jove in a predetermined number of iterations to initiate load balancing. Jove starts working on load balancing while other processors continue working with the current data and load distribution. Jove goes through several steps to decide if the new data should be taken, including preliminary evaluate, partition, processor reassignment, cost evaluation, and decision. Jove running on a single EBM SP2 node has been completely implemented. Preliminary experimental results show that the Jove approach to dynamic load balancing can be effective for full scale grid partitioning on the target machine IBM SP2.

Sohn, Andrew↗

Dynamic Load Balancing for Finite Element Calculations on Parallel Computers

Computational requirements of full scale computational fluid dynamics change as computation progresses on a parallel machine. The change in computational intensity causes workload imbalance of processors, which in turn requires a large amount of data movement at runtime. If parallel CFD is to be successful on a parallel or massively parallel machine, balancing of the runtime load is indispensable. Here a frame work is presented for dynamic load balancing for CFD applications, called Jove. One processor is designated as a decision maker Jove while others are assigned to computational fluid dynamics. Processors running CFD send flags to Jove in a predetermined number of iterations to initiate load balancing. Jove starts working on load balancing while other processors continue working with the current data and load distribution. Jove goes through several steps to decide if the new data should be taken, including preliminary evaluate, partition, processor reassignment, cost evaluation, and decision. Jove running on a single SP2 node has been completely implemented. Preliminary experimental results show that the Jove approach to dynamic load balancing can be effective for full scale grid partitioning on the target machine SP2.

Pramono, Eddy↗

Early Warning of Food Security Crises in Urban Areas: The Case of Harare, Zimbabwe, 2007

In 2007, the citizens of Harare, Zimbabwe began experiencing an intense food security crisis. The crisis, due to a complex mix of poor government policies, high inflation rates and production decline due to drought, resulted in a massive increase in the number of food insecure people in Harare. The international humanitarian aid response to this crisis was largely successful due to the early agreement among donors and humanitarian aid officials as to the size and nature of the problem. Remote sensing enabled an early and decisive movement of resources greatly assisting the delivery of food aid in a timely manner. Remote sensing data gave a clear and compelling assessment of significant crop production shortfalls, and provided donors of humanitarian assistance a single number around which they could come to agreement. This use of remote sensing data typifies how remote sensing may be used in early warning systems in Africa.

Brown, Molly E.↗

Towards the Interoperability of Web, Database, and Mass Storage Technologies for Petabyte Archives

At the San Diego Supercomputer Center, a massive data analysis system (MDAS) is being developed to support data-intensive applications that manipulate terabyte sized data sets. The objective is to support scientific application access to data whether it is located at a Web site, stored as an object in a database, and/or storage in an archival storage system. We are developing a suite of demonstration programs which illustrate how Web, database (DBMS), and archival storage (mass storage) technologies can be integrated. An application presentation interface is being designed that integrates data access to all of these sources. We have developed a data movement interface between the Illustra object-relational database and the NSL UniTree archival storage system running in a production mode at the San Diego Supercomputer Center. With this interface, an Illustra client can transparently access data on UniTree under the control of the Illustr DBMS server. The current implementation is based on the creation of a new DBMS storage manager class, and a set of library functions that allow the manipulation and migration of data stored as Illustra 'large objects'. We have extended this interface to allow a Web client application to control data movement between its local disk, the Web server, the DBMS Illustra server, and the UniTree mass storage environment. This paper describes some of the current approaches successfully integrating these technologies. This framework is measured against a representative sample of environmental data extracted from the San Diego Ba Environmental Data Repository. Practical lessons are drawn and critical research areas are highlighted.

Moore, Reagan↗

An Expanded RXTE Survey of Long-Term X-ray Variability in Seyfert 1 Galaxies

The first seven years of RXTE monitoring of Seyfert 1 active galactic nuclei have been systematically analyzed to yield five homogenous samples of 2-12 keV light curves, probing hard X-ray variability on successively longer durations from approx. 1 day to approx. 3.5 years. 2-10 keV variability on time scales of approx. 1 day, as probed by ASCA, are included. All sources exhibit stronger X-ray variability towards longer time scales, with variability amplitudes saturating at the longest time scales, but the increase is greater for relatively higher luminosity sources. The well-documented anticorrelation between variability amplitude and luminosity is confirmed on all time scales. However, anticorrelations between variability amplitude and black hole mass estimate are evident on only the shortest time scales probed. The data are consistent with the models of power spectral density (PSD) movement described in Markowitz et al. (2003) and McHardy et al. (2004), whereby Seyfert 1 galaxies variability can be described by a single, universal PSD shape whose cutoff frequency scales with black hole mass. The best-fitting scaling relations between variability time scale, black hole mass and X-ray luminosity support an average accretion rate of 2% of the Eddington limit for the sample. Nearly all sources exhibit stronger variability in the relatively soft 2-4 keV band compared to the 7-12 keV band on all time scales. Color-flux diagrams support also Seyfert 1s' softening as they brighten. There are indications that relatively less luminous or less massive sources exhibit a greater degree of spectral variability for a given increase in overall flux.

Markowitz, A.↗

Climatic and Demographic Consequences of the Massive Volcanic Eruption of 1258

Somewhere in the tropics, a volcano exploded violently during the year 1258, producing a massive stratospheric aerosol veil that eventually blanketed the globe. Arctic and Antarctic ice cores suggest that this was the world's largest volcanic eruption of the past millennium. According to contemporary chronicles, the stratospheric dry fog possibly manifested itself in Europe as a persistently cloudy aspect of the sky and also through an apparently total darkening of the eclipsed Moon. Based on a sudden temperature drop for several months in England, the eruption's initiation date can be inferred to have been probably January 1258. The frequent cold and rain that year led to severe crop damage and famine throughout much of Europe. Pestilence repeatedly broke out in 1258 and 1259; it occurred also in the Middle East, reportedly there as plague. Another very cold winter followed in 1260-1261. The troubled period's wars, famines, pestilences, and earthquakes appear to have contributed in part to the rise of the European flagellant movement of 1260, one of the most bizarre social phenomena of the Middle Ages. Analogies can be drawn with the climatic aftereffects and European social unrest following another great tropical eruption, Tambora in 1815. Some generalizations about the climatic impacts of tropical eruptions are made from these and other data.

Stothers, Richard B.↗

Airborne Radar Observations of Hurricane Georges during Landfall over the Dominican Republic

On 22 September 1998 hurricane Georges made landfall on the Dominican Republic (DR). Georges cost the DR at least 500 lives, made more than 155,000 people homeless and caused extensive damage to the country's main industries, tourism and agriculture. There was considerable wind damage, with wind gusts up to 58 m/s in Santa Domingo on the south coast, but most of the damage and deaths resulted from mudslides and the flooding of rivers. While this may have been the worst natural disaster to strike the DR, the sustained rapid storm movement saved the island from worse damage. Georges had previously affected several islands in the Lesser Antilles and Puerto Rico, but it had retained much of its circulation strength. Forty raingauge stations across the DR measured rainfall totals from Georges between 0.7 and 41 cm, the latter at the capital Santo Domingo, located on the south coast. At Herrera the maximum 1 h rainfall rate was 72 mm/h. It is suspected that much higher rain rates occurred in DR's mountainous interior. Before landfall the eye was clearly evident in satellite imagery. When the eye moved over southeastern DR, it filled rapidly, and the cloud top height decreased in all storm sectors except in the southern inflow sector, where a long-lived MCS, with a diameter larger than that of the eyewall, slowly became enwrapped in the hurricane circulation. The eye closure was most rapid between 16-18 UTC, when the eyewall circulation felt the mountainous terrain of the Cordillera Central, which rises up to 3,093 m. The estimated central pressure increased from 962 hPa at 15 UTC to 986 hPa at 03Z on 23 Sept, and the maximum sustained surface wind speed decreased from 54 to 36 in s-1 during the same period. The island of Hispaniola has a cross-track width of about 250 km, much wider than the diameter of the eyewall anvil (about 100 km before landfall). So the event can truly be considered to be a landfalling case, even though Georges recovered after crossing Hispaniola, albeit never to the same strength. This talk will summarize satellite and ground observations of Georges, as it passed the DR, and it will focus on EDOP data. In particular, we will try to estimate the rainfall rate over the mountainous terrain of the DR. And we will use detailed sounding data to explain the presence and characteristics of the massive MCS to the south, as well as the upper-level updrafts apparent over this MCS and over the mountains of the DR.

Geerts, B.↗

Contributions of GRACE to Climate Monitoring

The NASA/German Gravity Recovery and Climate Experiment (GRACE) was launched in March 2002. Rather than looking downward, GRACE continuously monitors the locations of and precise distance between twin satellites which orbit in tandem about 200 km apart. Variations in mass near Earth's surface cause heterogeneities in its gravity field, which in turn affect the orbits of satellites. Thus scientists can use GRACE data to map Earth's gravity field with enough accuracy to discern month to month changes caused by ocean circulation and redistribution of water stored on and in the land. Other gravitational influences, such as atmospheric circulation, post-glacial rebound, and solid earth movements are either independently determined and removed or are negligible on a monthly to sub-decadal timescale. Despite its coarse spatial (>150,000 sq km at mid-latitudes) and temporal (approx monthly) resolutions, GRACE has enabled significant advancements in the oceanic, hydrologic, and cryospheric science, and has great potential for climate monitoring, because it is the only global observing system able to measure ocean bottom pressures, total terrestrial water storage, and ice mass changes. The best known GRACE results are estimates of Greenland and Antarctic ice sheet loss rates. Previously, scientists had estimated ice mass losses using ground and satellite based altimetry and surface mass balance estimates based on snowfall accumulation and glacier discharge. While such measurements are still very useful for their spatial detail, they are imperfectly correlated with large-scale ice mass changes, due to snow and ice compaction and incomplete spatial coverage. GRACE enables scientists to generate monthly time series of Greenland and Antarctic ice mass, which have confirmed the shrinking of the polar ice sheets, one of the most obvious and indisputable manifestations of climate change. Further, GRACE has located and quantified hot spots of ice loss in southeastern Greenland and western Antarctica. For 2002 to present, the rate of ice mass loss has been 200 to 300 GT/yr in Greenland and 70 to 210 GT/yr in Antarctica, and some scientists are suggesting that the rates are accelerating. Similarly, GRACE has been used to monitor mass changes in alpine glaciers. Tamisiea et al. first characterized glacier melt along the southern coast of Alaska, more recently estimated to be occurring at a rate of 84 GT/yr. Chen et al. estimated that Patagonian glaciers are melting at a rate of 28 GT/yr, and estimated that the high mountains of central Asia lose ice at a rate of 47 GT/yr. Tapley et al. and Wahr et al. presented the first GRACE based estimates of changes in column-integrated terrestrial water storage (TWS; the sum of ground-water, soil moisture, surface waters, snow, ice, and water stored in vegetation) at continental scales. Since then, dozens of studies have shown that GRACE based estimates of regional to continental scale TWS variations agree with independent information, and some innovative uses of GRACE data have been developed. Rodell et al. (2004) and Swenson and Wahr (2006) demonstrated that by combining GRACE derived terrestrial water storage changes with observations of precipitation and runoff in a river basin scale water budget, it was possible to produce new estimates of evapotranspiration and atmospheric moisture convergence, essential climate variables that are difficult to estimate accurately. Similarly, GRACE has been used to constrain estimates of global river discharge and the contribution of changes in TWS to sea level rise. Crowley et al. observed a negative correlation between interannual TWS anomalies in the Amazon and the Congo River basin. Yeh et al. and Rodell et al. estimated regionally averaged groundwater storage variations based on GRACE and auxiliary observations. Rodell et al. and Tiwari et al. applied that method to quantify massive groundwater depletion in northern India caused by over reliance on aquifers for irration, and Famiglietti et al. found a similar situation in California's Central Valley. Zaitchik et al. and Lo et al. described approaches to use GRACE to constrain hydrological models, enabling integration of GRACE data with other observations and achieving much higher spatial and temporal resolutions than GRACE alone. Such approaches are now supporting applications including drought and water resources monitoring. Oceanography has likewise benefitted from the independent nature of GRACE observations. One application is measurement of the mass component of sea level rise, which complements radar altimetry and in situ measurements. GRACE also measures ocean bottom pressures (OBP), which help to refine understanding and modeling of ocean circulation and the ocean's fresh water budget, among other things. For example, Hayakawa et al. showed that GRACE observes OBP patterns absent from the background models of oceanic variability. Morison et al. used GRACE to describe important decadal scale shifts in circulation and an ongoing trend of freshening of the western Arctic, important indicators of climate variability. The research of Song and Zlotnicki and Chambers and Willis on GRACE-derived ocean bottom pressures in the sub-polar gyre led to the discovery of an ENSO teleconnection and a long-term change in OBP in the North Pacific sub-polar gyre that was not predicted by an ocean model. Further, Chambers and Willis were able to identify an internal redistribution of mass between Atlantic and Pacific Oceans lasting at least six years, which was not predicted by ocean models and was the first direct evidence of sustained mass transport from one ocean basin to another on periods longer than a year. Boening et al. observed a record increase in OBP over part of the southeastern Pacific in late 2009 and early 2010, primarily caused by wind stress curl associated with a strong and persistent anticyclone and likely related to the concurrent Central Pacific El Nino. GRACE has far surpassed its 5-year design lifetime, but it will likely succumb to the aging of batteries and instrument systems sometime in the next few years. NASA has begun initial development of a follow-on to GRACE with very similar design, which could launch as soon as 2016 and would provide continuity in the data record while improving resolution slightly. Higher resolution time variable gravity missions are also on the drawing board .

Rodell, Matthew↗

Autonomous Ocean World Exploration: Advancement of a Virtual Testbed

The search for life (extinct or extant) and potentially habitable bodies in our solar system and beyond is one of the 12 priority science questions outlined in the National Acadamies’ 2022 decadal survey [5]. Extraterrestrial destinations containing liquid water present an opportunity to search for life as we know it, and in recent years an increasing number of such locations have been discovered within our solar system. Several Jovian moons—Europa, Ganymede, and Callisto [10]—and the Saturnian moons Enceladus [8] and Titan [9] are known or suspected to harbor massive subsurface oceans. Of these "ocean worlds", Europa is the focus of at least one planned NASA orbiter mission, Europa Clipper [4], and an early lander mission concept, the Europa Lander [2, 3]. Whereas most robotic missions to the Moon and Mars (e.g. orbiters, rovers, landers) to date have had ground controllers on Earth tightly involved in mission operations, missions to more distant worlds will require a high degree of onboard autonomy due to long communication lags and blackouts, harsh environments (radiation, cold), and more limited battery and hardware life. The past decade has seen great advances in both AI technologies and computing scalability and performance that offer promising solutions for spacecraft autonomy and motivate the software system and research programs described in this paper. The Ocean Worlds Autonomy Testbed for Exploration, Research, and Simulation (OceanWATERS) [1], which has been in development at the NASA Ames Research Center since 2018, is a virtual environment for testing lander autonomy solutions. It is built on the Robot Operating System (ROS), runs on consumer-grade Linux workstations, and was released as open source in 2020. OceanWATERS provides a physical and visual simulation of a prototypical lander in a Europa-like environment (Figure 1). The lander was modeled after requirements and specifications made in JPL’s Europa Lander Study of 2016 [3]. Simulated lander systems include stereo cameras and spotlights mounted on an antenna mast that pans and tilts, a 6 degrees of freedom (DoF) robotic arm with a force-torque sensor and two interchangeable end effectors, and a battery pack power system. The environment consists of multiple terrain models including a highly detailed model sourced from the FROST dataset [11], simulation of surrounding planetary bodies based on an ephemeris model, and lighting from the sun with associated surface illumination, reflectance, and shadows. Operations supported by OceanWATERS include panoramic and directed imaging of the environment and lander workspace, Cartesian and joint-level arm commanding, grinding of the terrain surface (e.g. digging a trench), and scooping of ground material (Figure 2) which can be discarded or collected as science samples in a receptacle that can be emptied (science operations themselves are not simulated). These operations are realized as ROS Actions and are complimented by a wide selection of telemetry that is continually produced by each lander subsystem. The power system model is driven by the open-source Generic Software Architecture for Prognostics (GSAP) [11] that predicts the battery’s remaining useful life and other characteristics. As a testbed for high-level autonomy, OceanWATERS provides an execution framework based on PLEXIL [12], an open-source plan specification language and execution engine developed largely at Ames. NASA's initial development of OceanWATERS, as well the Ocean Worlds Lander Autonomy Testbed (OWLAT) [6], a complimentary physical testbed developed at JPL, was the first step in a plan for realizing candidate onboard autonomy solutions for such planetary landers. In 2020 NASA solicited applications for its Autonomous Robotics Research for Ocean Worlds (ARROW) program, and in 2021 the similar Concepts for Ocean worlds Life Detection Technology (COLDTech) program. Collectively six research teams, based in universities and companies across the United States, were awarded grants to develop and demonstrate autonomy solutions on OceanWATERS and OWLAT. These 1–2-year projects have now finished or are nearing completion, and a wide variety of autonomy challenges in ocean world surface missions were addressed. Prototyped and demonstrated solutions have included autonomous discovery, response and adaptation to system faults and unexpected environmental events, world model synthesis through perception, plan synthesis using learned models, methods to optimize sample target selection and prioritize science data transmission, extension of PLEXIL for stochastic decision-making, and an integration of a model of JPL’s mission-ready COLDArm [7]. Technologies used in these projects include many forms of machine learning, causal reasoning, automated planning, Markov decision processes, formal methods, and other advanced techniques. A more detailed summary of the ARROW and COLDTech projects is given herein. OceanWATERS has had significant enhancements since its open-source release in 2020. Many of its new features were driven or shaped by feedback from the ARROW and COLDTech teams and requirements of their projects. In support of enabling autonomous adaptation to spacecraft faults (a specific capability solicited by both programs), a fault injection and detection framework was developed that supports a wide and growing range of fault types such as locked joints, image loss, and battery failures. The power system model was completed and integrated into the simulator, starting as a single-cell battery model and later upgraded to a multi-cell model with associated faults such as cell disconnection. Arm/terrain interaction was improved by adding a force-torque sensor and associated faults, and an analytic dig force model based on the Balovnev bucket force equations. Environment fidelity was increased by modeling terrain deformation resulting from digging and scooping; visual improvements were made in textures, lighting, and shadows. To facilitate interoperation with OWLAT, a unified command and telemetry interface between the testbeds was developed at the ROS level, along with a PLEXIL interface. The number of lander operations was greatly expanded (e.g. with Cartesian-based arm and antenna movement), and a framework was designed for users to build their own lander actions. A GUI for PLEXIL plan selection was created (Figure 3), and an expansive set of plans were added, such as those that illustrate patterns for fault handling. This paper provides a self-contained high-level description of OceanWATERS, focusing on more detailed coverage of the aforementioned enhancements. It provides a high-level summary of the projects undertaken by participants in the ARROW and COLDTech programs and how these efforts have helped shape OceanWATERS. Finally, potential future work and directions for the testbed are listed, as likely informed by the recent planetary science decadal survey [5].

K Michael Dalal↗