Search NASA⌕ Search

SEARCH · Search NASA

Results for “data-intensive”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Software architecture for large scale, distributed, data-intensive systems

This paper presents our experience with OODT, a novel software architectual style, and middlware-based implementation for data-intensive systems. To date, OODT has been successfully evaluated in several different science domains including Cancer Research with the National Cancer Institute (NCI), and Planetary Science with NASA's Planetary Data System (PDS).

software architecture↗

GLIDE: a grid-based light-weight infrastructure for data-intensive environments

The promise of the grid is that it will enable public access and sharing of immense amounts of computational and data resources among dynamic coalitions of individuals and institutions. However, the current grid solutions make several limiting assumptions that curtail their widespread adoption. To address these limitations, we present GLIDE, a prototype light-weight, data-intensive middleware infrastructure that enables access to the robust data and computational power of the grid on DREAM platforms.

PRISM-MW↗

Data-Intensive Science meets Inquiry-Driven Pedagogy: Interactive Big Data Exploration, Threshold Concepts, and Liminality

Threshold concepts in any discipline are the core concepts an individual must understand in order to master a discipline. By their very nature, these concepts are troublesome, irreversible, integrative, bounded, discursive, and reconstitutive. Although grasping threshold concepts can be extremely challenging for each learner as s/he moves through stages of cognitive development relative to a given discipline, the learner's grasp of these concepts determines the extent to which s/he is prepared to work competently and creatively within the field itself. The movement of individuals from a state of ignorance of these core concepts to one of mastery occurs not along a linear path but in iterative cycles of knowledge creation and adjustment in liminal spaces - conceptual spaces through which learners move from the vaguest awareness of concepts to mastery, accompanied by understanding of their relevance, connectivity, and usefulness relative to questions and constructs in a given discipline. For example, challenges in the teaching and learning of atmospheric science can be traced to threshold concepts in fluid dynamics. In particular, Dynamic Meteorology is one of the most challenging courses for graduate students and undergraduates majoring in Atmospheric Science. Dynamic Meteorology introduces threshold concepts - those that prove troublesome for the majority of students but that are essential, associated with fundamental relationships between forces and motion in the atmosphere and requiring the application of basic classical statics, dynamics, and thermodynamic principles to the three dimensionally varying atmospheric structure. With the explosive growth of data available in atmospheric science, driven largely by satellite Earth observations and high-resolution numerical simulations, paradigms such as that of dataintensive science have emerged. These paradigm shifts are based on the growing realization that current infrastructure, tools and processes will not allow us to analyze and fully utilize the complex and voluminous data that is being gathered. In this emerging paradigm, the scientific discovery process is driven by knowledge extracted from large volumes of data. In this presentation, we contend that this paradigm naturally lends to inquiry-driven pedagogy where knowledge is discovered through inductive engagement with large volumes of data rather than reached through traditional, deductive, hypothesis-driven analyses. In particular, data-intensive techniques married with an inductive methodology allow for exploration on a scale that is not possible in the traditional classroom with its typical problem sets and static, limited data samples. In addition, we identify existing gaps and possible solutions for addressing the infrastructure and tools as well as a pedagogical framework through which to implement this inductive approach.

Ramachandran, Rahul↗

Nonparametric analysis of Minnesota spruce and aspen tree data and LANDSAT data

The application of nonparametric methods in data-intensive problems faced by NASA is described. The theoretical development of efficient multivariate density estimators and the novel use of color graphics workstations are reviewed. The use of nonparametric density estimates for data representation and for Bayesian classification are described and illustrated. Progress in building a data analysis system in a workstation environment is reviewed and preliminary runs presented.

Scott, D. W.↗

Distributed Large Data-Object Environments: End-to-End Performance Analysis of High Speed Distributed Storage Systems in Wide Area ATM Networks

We have developed and deployed a distributed-parallel storage system (DPSS) in several high speed asynchronous transfer mode (ATM) wide area networks (WAN) testbeds to support several different types of data-intensive applications. Architecturally, the DPSS is a network striped disk array, but is fairly unique in that its implementation allows applications complete freedom to determine optimal data layout, replication and/or coding redundancy strategy, security policy, and dynamic reconfiguration. In conjunction with the DPSS, we have developed a 'top-to-bottom, end-to-end' performance monitoring and analysis methodology that has allowed us to characterize all aspects of the DPSS operating in high speed ATM networks. In particular, we have run a variety of performance monitoring experiments involving the DPSS in the MAGIC testbed, which is a large scale, high speed, ATM network and we describe our experience using the monitoring methodology to identify and correct problems that limit the performance of high speed distributed applications. Finally, the DPSS is part of an overall architecture for using high speed, WAN's for enabling the routine, location independent use of large data-objects. Since this is part of the motivation for a distributed storage system, we describe this architecture.

Johnston, William↗

Towards the Interoperability of Web, Database, and Mass Storage Technologies for Petabyte Archives

At the San Diego Supercomputer Center, a massive data analysis system (MDAS) is being developed to support data-intensive applications that manipulate terabyte sized data sets. The objective is to support scientific application access to data whether it is located at a Web site, stored as an object in a database, and/or storage in an archival storage system. We are developing a suite of demonstration programs which illustrate how Web, database (DBMS), and archival storage (mass storage) technologies can be integrated. An application presentation interface is being designed that integrates data access to all of these sources. We have developed a data movement interface between the Illustra object-relational database and the NSL UniTree archival storage system running in a production mode at the San Diego Supercomputer Center. With this interface, an Illustra client can transparently access data on UniTree under the control of the Illustr DBMS server. The current implementation is based on the creation of a new DBMS storage manager class, and a set of library functions that allow the manipulation and migration of data stored as Illustra 'large objects'. We have extended this interface to allow a Web client application to control data movement between its local disk, the Web server, the DBMS Illustra server, and the UniTree mass storage environment. This paper describes some of the current approaches successfully integrating these technologies. This framework is measured against a representative sample of environmental data extracted from the San Diego Ba Environmental Data Repository. Practical lessons are drawn and critical research areas are highlighted.

Moore, Reagan↗

Bridging the Gap from Networking Technologies to Applications: Workshop Report

The objective of the Next Generation Internet (NGI) Federal program is threefold, encompassing development of networking technologies, high-performance network testbeds, and revolutionary applications. There have been notable advances in emerging network technologies and several nationwide testbeds have been established, but the integration of emerging technologies into applications is lagging. To help bridge this gap between developers of NGI networking technologies and developers of NGI applications, the NASA Research and Education Network (NREN) project hosted a two-day workshop at NASA Ames Research Center in August 1999. This paper presents a summary of the results of this workshop and also describes some of the challenges NREN is facing while incorporating new technologies into HPCC and other NASA applications. The workshop focused on three technologies - Quality of Service (QoS), advanced multicast, and security-and five major NGI application areas - telemedicine, digital earth, digital video, distributed data-intensive applications, and computational infrastructure applications. Network technology experts, application developers, and NGI testbed representatives came together at the workshop to promote cross-fertilization between the groups. Presentations on the first day, including an overview of the three technologies, application case studies and testbed status reports, laid the foundation for discussions on the second day. The objective of these latter discussions, held within smaller breakout groups, was to establish a coherent picture of the current status of the various pieces of each of the three technologies, to create a roadmap outlining future technology development, and to offer technological guidance to application developers. In this paper we first present a brief overview of the NGI applications that were represented at the workshop, focusing on the identification of technological advances that have successfully been incorporated in each application and technological challenges that remain. Next we present the technology roadmaps that were created at the workshop, summarizing the status of various mechanisms that are currently under development and forecasting when various advances are likely to occur within the next one-to-three-year time span. Then we identify issues that were raised at the workshop that might hinder technology development or that might impede integration into NGI applications. We also report some specific guidelines that were offered at the workshop to enable application developers to integrate and effectively use emerging NGI technology building blocks. Finally, we describe NREN activities to incorporate emerging technologies into NASA applications. These activities include support for other NASA High-Performance Computing and Communications Program areas such as IPG (Information Power Grid), support for NASA science enterprises such as Earth science and Mars program prototyping activities, support for satellite/terrestrial networking applications such as the TransAtlantic and TransPacific demonstrations and the Interplanetary Internet, support for NASA telemedicine applications such as the Virtual Collaborative Clinic, and participation in NGI advanced technology testbed initiatives such as the QBone and the NTON/Supernet. For each activity we highlight the primary technological challenge that is associated with it.

Johnson, Marjory J.↗

Finite Element Analysis of Reverberation Chambers

The primary motivating factor behind the initiation of this work was to provide a deterministic means of establishing the validity of the statistical methods that are recommended for the determination of fields that interact in -an avionics system. The application of finite element analysis to reverberation chambers is the initial step required to establish a reasonable course of inquiry in this particularly data-intensive study. The use of computational electromagnetics provides a high degree of control of the "experimental" parameters that can be utilized in a simulation of reverberating structures. As the work evolved there were four primary focus areas they are: 1. The eigenvalue problem for the source free problem. 2. The development of a complex efficient eigensolver. 3. The application of a source for the TE and TM fields for statistical characterization. 4. The examination of shielding effectiveness in a reverberating environment. One early purpose of this work was to establish the utility of finite element techniques in the development of an extended low frequency statistical model for reverberation phenomena. By employing finite element techniques, structures of arbitrary complexity can be analyzed due to the use of triangular shape functions in the spatial discretization. The effects of both frequency stirring and mechanical stirring are presented. It is suggested that for the low frequency operation the typical tuner size is inadequate to provide a sufficiently random field and that frequency stirring should be used. The results of the finite element analysis of the reverberation chamber illustrate io-W the potential utility of a 2D representation for enhancing the basic statistical characteristics of the chamber when operating in a low frequency regime. The basic field statistics are verified for frequency stirring over a wide range of frequencies. Mechanical stirring is shown to provide an effective frequency deviation.

Bunting, Charles F.↗

Enabling Earth Science: The Facilities and People of the NCCS

The NCCS's mass data storage system allows scientists to store and manage the vast amounts of data generated by these computations, and its high-speed network connections allow the data to be accessed quickly from the NCCS archives. Some NCCS users perform studies that are directly related to their ability to run computationally expensive and data-intensive simulations. Because the number and type of questions scientists research often are limited by computing power, the NCCS continually pursues the latest technologies in computing, mass storage, and networking technologies. Just as important as the processors, tapes, and routers of the NCCS are the personnel who administer this hardware, create and manage accounts, maintain security, and assist the scientists, often working one on one with them.

Source record↗

Phased-Array Satcom Antennas Developed for Aeronautical Applications

The Advanced Communications (AC) for Aeronautics research at the NASA Glenn Research Center integrates both aeronautics and space communications technologies to achieve the national objective of upgrading the present National Airspace System infrastructure by responding to the agency's aviation capacity and safety goals. One concept for future air traffic management, free flight, presents a significantly increased demand for communications systems capacity and performance in comparison to current air traffic management practices. Current aeronautical communications systems are incapable of supporting the anticipated demands, and the new digital data communications links that are being developed, or are in the early stages of implementation, are not primarily designed to carry the data-intensive free flight air traffic management (ATM) communications loads. Emerging satellite communications technologies are the best potential long-term solution to provide the capacity and performance necessary to enable a mature free flight concept to be deployed. NASA AC/ATM funded the development of a Boeing-designed Ku-band transmit phased-array antenna, a combined in-house and contract effort. Glenn designed and integrated an Aeronautical Mobile Satellite Communications terminal based on the transmit phased-array antenna and a companion receive phased-array antenna previously developed by Boeing.

Kerczewski, Robert J.↗

Secure, Mobile, Wireless Network Technology Designed, Developed, and Demonstrated

The inability to seamlessly disseminate data securely over a high-integrity, wireless broadband network has been identified as a primary technical barrier to providing an order-of-magnitude increase in aviation capacity and safety. Secure, autonomous communications to and from aircraft will enable advanced, automated, data-intensive air traffic management concepts, increase National Air Space (NAS) capacity, and potentially reduce the overall cost of air travel operations. For the first time ever, secure, mobile, network technology was designed, developed, and demonstrated with state-ofthe- art protocols and applications by a diverse, cooperative Government-industry team led by the NASA Glenn Research Center. This revolutionary technology solution will make fundamentally new airplane system capabilities possible by enabling secure, seamless network connections from platforms in motion (e.g., cars, ships, aircraft, and satellites) to existing terrestrial systems without the need for manual reconfiguration. Called Mobile Router, the new technology autonomously connects and configures networks as they traverse from one operating theater to another. The Mobile Router demonstration aboard the Neah Bay, a U.S. Coast Guard vessel stationed in Cleveland, Ohio, accomplished secure, seamless interoperability of mobile network systems across multiple domains without manual system reconfiguration. The Neah Bay was chosen because of its low cost and communications mission similarity to low-Earth-orbiting satellite platforms. This technology was successfully advanced from technology readiness level (TRL) 2 (concept and/or application formation) to TRL 6 (system model or prototype demonstration in a relevant environment). The secure, seamless interoperability offered by the Mobile Router and encryption device will enable several new, vehicle-specific and systemwide technologies to perform such things as remote, autonomous aircraft performance monitoring and early detection and mitigation of potential equipment malfunctions. As an additional benefit, team advancements were incorporated into open standards, ensuring technology transfer. Low-cost, commercial products incorporating the new technology are already available. Furthermore, these products are fully interoperable with legacy network technology equipment currently being used throughout the world.

Ivancic, William D.↗

Long-range planning for the Deep Space Network

Conduct of space exploration is undergoing a significant transformation. Initial reconnaissance missions are giving way to long duration observations with data-intensive instruments, in situ investigations and complex operations. To keep pace, a transformation in the Deep Space Network is in order.

DSN long range plan↗

The Montage architecture for grid-enabled science processing of large, distributed datasets

Montage is an Earth Science Technology Office (ESTO) Computational Technologies (CT) Round III Grand Challenge investigation to deploy a portable, compute-intensive, custom astronomical image mosaicking service for the National Virtual Observatory (NVO). Although Montage is developing a compute- and data-intensive service for the astronomy community, we are also helping to address a problem that spans both Earth and Space science, namely how to efficiently access and process multi-terabyte, distributed datasets. In both communities, the datasets are massive, and are stored in distributed archives that are, in most cases, remote from the available Computational resources. Therefore, state of the art computational grid technologies are a key element of the Montage portal architecture. This paper describes the aspects of the Montage design that are applicable to both the Earth and Space science communities.

virtual observatory↗

High-Capacity Communications from Martian Distances

High capacity communications from Martian distances, required for the envisioned human exploration and desirable for data-intensive science missions, is challenging. NASA s Deep Space Network currently requires large antennas to close RF telemetry links operating at kilobit-per-second data rates. To accommodate higher rate communications, NASA is considering means to achieve greater effective aperture at its ground stations. This report, focusing on the return link from Mars to Earth, demonstrates that without excessive research and development expenditure, operational Mars-to-Earth RF communications systems can achieve data rates up to 1 Gbps by 2020 using technology that today is at technology readiness level (TRL) 4-5. Advanced technology to achieve the needed increase in spacecraft power and transmit aperture is feasible at an only moderate increase in spacecraft mass and technology risk. In addition, both power-efficient, near-capacity coding and modulation and greater aperture from the DSN array will be required. In accord with these results and conclusions, investment in the following technologies is recommended:(1) lightweight (1 kg/sq m density) spacecraft antenna systems; (2) a Ka-band receive ground array consisting of relatively small (10-15 m) antennas; (3) coding and modulation technology that reduces spacecraft power by at least 3 dB; and (4) efficient generation of kilowatt-level spacecraft RF power.

Williams, W. Dan↗

Framework Resources Multiply Computing Power

As an early proponent of grid computing, Ames Research Center awarded Small Business Innovation Research (SBIR) funding to 3DGeo Development Inc., of Santa Clara, California, (now FusionGeo Inc., of The Woodlands, Texas) to demonstrate a virtual computer environment that linked geographically dispersed computer systems over the Internet to help solve large computational problems. By adding to an existing product, FusionGeo enabled access to resources for calculation- or data-intensive applications whenever and wherever they were needed. Commercially available as Accelerated Imaging and Modeling, the product is used by oil companies and seismic service companies, which require large processing and data storage capacities.

Source record↗

NASA Center for Climate Simulation (NCCS) Presentation

The NASA Center for Climate Simulation (NCCS) offers integrated supercomputing, visualization, and data interaction technologies to enhance NASA's weather and climate prediction capabilities. It serves hundreds of users at NASA Goddard Space Flight Center, as well as other NASA centers, laboratories, and universities across the US. Over the past year, NCCS has continued expanding its data-centric computing environment to meet the increasingly data-intensive challenges of climate science. We doubled our Discover supercomputer's peak performance to more than 800 teraflops by adding 7,680 Intel Xeon Sandy Bridge processor-cores and most recently 240 Intel Xeon Phi Many Integrated Core (MIG) co-processors. A supercomputing-class analysis system named Dali gives users rapid access to their data on Discover and high-performance software including the Ultra-scale Visualization Climate Data Analysis Tools (UV-CDAT), with interfaces from user desktops and a 17- by 6-foot visualization wall. NCCS also is exploring highly efficient climate data services and management with a new MapReduce/Hadoop cluster while augmenting its data distribution to the science community. Using NCCS resources, NASA completed its modeling contributions to the Intergovernmental Panel on Climate Change (IPCG) Fifth Assessment Report this summer as part of the ongoing Coupled Modellntercomparison Project Phase 5 (CMIP5). Ensembles of simulations run on Discover reached back to the year 1000 to test model accuracy and projected climate change through the year 2300 based on four different scenarios of greenhouse gases, aerosols, and land use. The data resulting from several thousand IPCC/CMIP5 simulations, as well as a variety of other simulation, reanalysis, and observationdatasets, are available to scientists and decision makers through an enhanced NCCS Earth System Grid Federation Gateway. Worldwide downloads have totaled over 110 terabytes of data.

Webster, William P.↗

QuakeSim 2.0

QuakeSim 2.0 improves understanding of earthquake processes by providing modeling tools and integrating model applications and various heterogeneous data sources within a Web services environment. QuakeSim is a multisource, synergistic, data-intensive environment for modeling the behavior of earthquake faults individually, and as part of complex interacting systems. Remotely sensed geodetic data products may be explored, compared with faults and landscape features, mined by pattern analysis applications, and integrated with models and pattern analysis applications in a rich Web-based and visualization environment. Integration of heterogeneous data products with pattern informatics tools enables efficient development of models. Federated database components and visualization tools allow rapid exploration of large datasets, while pattern informatics enables identification of subtle, but important, features in large data sets. QuakeSim is valuable for earthquake investigations and modeling in its current state, and also serves as a prototype and nucleus for broader systems under development. The framework provides access to physics-based simulation tools that model the earthquake cycle and related crustal deformation. Spaceborne GPS and Inter ferometric Synthetic Aperture (InSAR) data provide information on near-term crustal deformation, while paleoseismic geologic data provide longerterm information on earthquake fault processes. These data sources are integrated into QuakeSim's QuakeTables database system, and are accessible by users or various model applications. UAVSAR repeat pass interferometry data products are added to the QuakeTables database, and are available through a browseable map interface or Representational State Transfer (REST) interfaces. Model applications can retrieve data from Quake Tables, or from third-party GPS velocity data services; alternatively, users can manually input parameters into the models. Pattern analysis of GPS and seismicity data has proved useful for mid-term forecasting of earthquakes, and for detecting subtle changes in crustal deformation. The GPS time series analysis has also proved useful as a data-quality tool, enabling the discovery of station anomalies and data processing and distribution errors. Improved visualization tools enable more efficient data exploration and understanding. Tools provide flexibility to science users for exploring data in new ways through download links, but also facilitate standard, intuitive, and routine uses for science users and end users such as emergency responders.

Donnellan, Andrea↗