Results for “Data migration” — Page 3

NASA NTRS2017

Technical Challenges and Lessons from the Migration of the GLOBE Data and Information System to Utilize Cloud Computing Service

The Global Learning and Observation to Benefit the Environment (GLOBE) Data and Information System supports an international science and education program with capabilities to accept local environment observations, archive, display and visualize them along with global satellite observations. Since its inception twenty years ago, the Web and database system has been upgraded periodically to accommodate the changes in technology and the steady growth of GLOBEs education community and collection of observations. Recently, near the end-of-life of the system hardware, new commercial computer platform options were explored and a decision made to utilize Cloud services. Now the GLOBE DIS has been fully deployed and maintained using Amazon Cloud services for over two years now. This paper reviews the early risks, actual challenges, and some unexpected findings as a result of the GLOBE DIS migration. We describe the plans, cost drivers and estimates, highlight adjustments that were made and suggest improvements. We present the trade studies for provisioning, for load balancing, networks, processing, storage, as well as production, staging and backup systems. We outline the migration teams skills and required level of effort for transition, and resulting changes in the overall maintenance and operations activities. Examples include incremental adjustments to processing capacity and frequency of backups, and efforts previously expended on hardware maintenance that were refocused onto application-specific enhancements.

GLOBE Data↗

NASA NTRS2016

Technical Challenges and Lessons from the Migration of the GLOBE Data and Information System to Utilize Cloud Computing Service

The Global Learning and Observation to Benefit the Environment (GLOBE) Data and Information System supports an international science and education program with capabilities to accept local environment observations, archive, display and visualize them along with global satellite observations. Since its inception twenty years ago, the Web and database system has been upgraded periodically to accommodate the changes in technology and the steady growth of GLOBEs education community and collection of observations. Recently, near the end-of-life of the system hardware, new commercial computer platform options were explored and a decision made to utilize Cloud services. Now the GLOBE DIS has been fully deployed and maintained using Amazon Cloud services for over two years now. This paper reviews the early risks, actual challenges, and some unexpected findings as a result of the GLOBE DIS migration. We describe the plans, cost drivers and estimates, highlight adjustments that were made and suggest improvements. We present the trade studies for provisioning, for load balancing, networks, processing, storage, as well as production, staging and backup systems. We outline the migration teams skills and required level of effort for transition, and resulting changes in the overall maintenance and operations activities. Examples include incremental adjustments to processing capacity and frequency of backups, and efforts previously expended on hardware maintenance that were refocused onto application-specific enhancements.

GLOBE↗

NASA NTRSDate not supplied

LAADS DAAC Migrates to the Cloud: Lessons Learned from Communicating About Earth Science Data on the Cloud

The Level-1 and Atmosphere Archive Distribution System (LAADS) Distributed Active Archive Center (DAAC) is migrating data to the cloud. As one of twelve DAACS supported by NASA’s Earth Science Data and Information System (ESDIS), LAADS is using moving away from on premise data storage facilities to migrating to Amazon Web Services, where the massive archive of data from the Moderate Imaging Spectroradiometer (MODIS) and the Visible Infrared Imaging Radiometer Suite (VIIRS) will be available for download and post-processing transformations online. The migration is happening in three phases and LAADS is concluding its beta testing period. This poster shows the lessons learned from communicating with a select group of users about how to effectively educate data users on using data in the cloud.

Tassia Owen↗

NASA NTRS2017

Trade Study: Storing NASA HDF5/netCDF-4 Data in the Amazon Cloud and Retrieving Data via Hyrax Server / THREDDS Data Server

As part of the overall effort to understand implications of migrating ESDIS data and services to the cloud we are testing several common OPeNDAP and HDF use cases against three architectures for general performance and cost characteristics. The architectures include retrieving entire files, retrieving datasets using HTTP range gets, and retrieving elements of datasets (chunks) with HTTP range gets. We will describe these architectures and discuss our approach to estimating cost.

HDF↗

NASA NTRS2022

Continental Patterns of Bird Migration Linked to Climate Variability

For nearly 100 years, avian migration studies have divided North America into three or four primary flyways, at times based on subjective approaches or just for convenience. Those studies often fail to adequately reflect a critical characterization of migration —phenology. This shortcoming has been partly due to the lack of reliable continental-scale data, a gap filled by our current study. Here, we leveraged unique radar-based data quantifying migration phenology and used an objective regionalization approach to revisit the traditional spatial framework. Consequently, we identified two regions with distinct inter annual variability of spring migration across the contiguous U.S. This new data-driven framework has enabled us to explore the climatic cues affecting the inter annual variability of migration phenology, “specific to each region” across North America. For example, our “two-region” approach allowed us to identify an east-west dipole pattern in migratory behavior linked to atmospheric Ross by waves. Also, we revealed a low-frequency variability in migration movements over the western U.S. that is inversely related with temperature and the Pacific Decadal Oscillation (PDO). Our spatial platform would facilitate future work on better understanding the mechanisms responsible for broad-scale migration phenology and its potential future changes.

Atmosphere↗

NASA NTRS2000

KDD Services at the Goddard Earth Sciences Distributed Active Archive Center

NASA's Goddard Earth Sciences Distributed Active Archive Center (GES DAAC) processes, stores and distributes earth science data from a variety of remote sensing satellites. End users of the data range from instrument scientists to global change and climate researchers to federal agencies and foreign governments. Many of these users apply data mining techniques to large volumes of data (up to 1 TB) received from the GES DAAC. However, rapid advances in processing power are enabling increases in data processing that are outpacing tape drive performance and network capacity. As a result, the proportion of data that can be distributed to users continues to decrease. As mitigation, we are migrating more data mining and mining preparation activities into the data center in order to reduce the data volume that needs to be distributed and to offer the users a more useful and manageable product. This migration of activities faces a number of technical and human-factor challenges. As data reduction and mining algorithms are normally quite specific to the user's research needs, the user's algorithm must be integrated virtually unchanged into the archive environment. Also, the archive itself is busy with everyday data archive and distribution activities and cannot be dedicated to, or even impacted by, the mining activities. Therefore, we schedule KDD 'campaigns' (similar to reprocessing campaigns), during which we schedule a wholesale retrieval of specific data products, offering users the opportunity to extract information from the data being retrieved during the campaign.

Lynnes, Christopher↗

NASA NTRS1993

Study of bubble behavior in weightlessness (effects of thermal gradient and acoustic stationary wave) (M-16)

The aim of this experiment is to understand how bubbles behave in a thermal gradient and acoustic stationary wave under microgravity. In microgravity, bubble or bubbles in a liquid will not rise upward as they do on Earth but will rest where they are formed because there exists no gravity-induced buoyancy. We are interested in how bubbles move and in the mechanisms which support the movement. We will try two ways to make bubbles migrate. The first experiment concerns behavior of bubbles in a thermal gradient. It is well known than an effect of surface tension which is masked by gravity on the ground becomes dominant in microgravity. The surface tension on the side of the bubble at a lower temperature is stronger than at a higher temperature. The bubble migrates toward the higher temperature side due to the surface tension difference. The migration speed depends on the so-called Marangoni number, which is a function of the temperature difference, the bubble diameter, liquid viscosity, and thermal diffusivity. At present, some experimental data about migration speeds in liquids with very small Marangoni numbers were obtained in space experiments, but cases of large Marangoni number are rarely obtained. In our experiment a couple of bubbles are to be injected into a cell filled with silicon oil, and the temperature gradient is to be made gradually in the cell by a heater and a cooler. We will be able to determine migration speeds in a very wide range of Marangoni numbers, as well as study interactions between the bubbles. We will observe bubble movements affected by hydrodynamical and thermal interactions, the two kinds of interactions which occur simultaneously. These observation data will be useful for analyzing the interactions as well as understanding the behavior of particles or drops in materials processing. The second experiment concerns bubble movement in an acoustic stationary wave. It is known that a bubble in a stationary wave moves toward the node or the loop according to whether its diameter is larger or smaller than that of the main resonant radius. In our experiment fine bubbles will be observed to move according to an acoustic field formed in a cylindrical cell. The existence of bubbles varies the acoustic speed, and the interactive force between bubbles will make the bubble behavior collective and complicated. This experiment will be very useful to development of bubble removable technology as well as to the understanding of bubble behavior.

Azuma, H.↗

NASA NTRSDate not supplied

Flight Test Design and Implementation for Airspace Independent Surveillance Through a Distributed Ground Based Sensor Network

The paper presents a system architecture for distributed sensing, networking and computing, its hardware implementation, and execution of initial flight experiments to validate theoretical findings. It induces development of distributed sensing requirements, framework, and architecture, development of distributed ground node hardware prototypes, integration of all nodes and testing of baseline functionalities, integration of in-house developed perception, migration and tracking software packages, establishing flight scenario and flyable path for a selected UAS, flying the air vehicle along the path, recording sensors measurements, pre-processing them and transferring the resulting data to an optimal computing center. It also addresses the challenges related to pre-flight hardware calibration, clock synchronization, sensor registration and establishing a communication network. Sensors data processing results demonstrate the functionality of the presented distributed architecture and satisfactory performance of the applied technologies.

Target tracking↗

NASA NTRSDate not supplied

Flight Test Design and Implementation for Independent Surveillance of an Airspace Through a Distributed Ground Sensing Network

The paper presents a system architecture for distributed sensing, networking and computing, its hardware implementation, and execution of initial flight experiments to validate theoretical findings. It induces development of distributed sensing requirements, framework, and architecture, development of distributed ground node hardware prototypes, integration of all nodes and testing of baseline functionalities, integration of in-house developed perception, migration and tracking software packages, establishing flight scenario and flyable path for a selected UAS, flying the air vehicle along the path, recording sensors measurements, pre-processing them and transferring the resulting data to an optimal computing center. It also addresses the challenges related to pre-flight hardware calibration, clock synchronization, sensor registration and establishing a communication network. Sensors data processing results demonstrate the functionality of the presented distributed architecture and satisfactory performance of the applied technologies.

Distributed sensing↗

DOE OSTI2025

Migratory Strategy is a Key Factor Driving Interactions at Wind Energy Facilities in At-Risk North American Bats

Animal migration remains poorly understood for many organisms, impeding understanding of movement dynamics and limiting conservation actions. We develop a framework that scales from movements of individuals to the dynamics of continental migration using data synthesis of endogenous markers, which we apply to three North American bat species with unexplained high rates of fatalities at wind energy facilities. The two species experiencing the highest fatality rates exhibit a "pell-mell" migration strategy in which individuals move from summer habitats in multiple directions, both to higher and lower latitudes, during autumn. We link movements to higher latitudes to encounters with wind energy facilities and report that the timing of pell-mell migration strongly overlaps with that of fatalities at the continental level. These findings support the hypothesis that migration distance and strategy are drivers of increased interactions with wind energy facilities, highlighting the significance of understanding migratory strategy to achieve conservation goals.

08 HYDROGEN↗

NASA NTRS1996

Towards the Interoperability of Web, Database, and Mass Storage Technologies for Petabyte Archives

At the San Diego Supercomputer Center, a massive data analysis system (MDAS) is being developed to support data-intensive applications that manipulate terabyte sized data sets. The objective is to support scientific application access to data whether it is located at a Web site, stored as an object in a database, and/or storage in an archival storage system. We are developing a suite of demonstration programs which illustrate how Web, database (DBMS), and archival storage (mass storage) technologies can be integrated. An application presentation interface is being designed that integrates data access to all of these sources. We have developed a data movement interface between the Illustra object-relational database and the NSL UniTree archival storage system running in a production mode at the San Diego Supercomputer Center. With this interface, an Illustra client can transparently access data on UniTree under the control of the Illustr DBMS server. The current implementation is based on the creation of a new DBMS storage manager class, and a set of library functions that allow the manipulation and migration of data stored as Illustra 'large objects'. We have extended this interface to allow a Web client application to control data movement between its local disk, the Web server, the DBMS Illustra server, and the UniTree mass storage environment. This paper describes some of the current approaches successfully integrating these technologies. This framework is measured against a representative sample of environmental data extracted from the San Diego Ba Environmental Data Repository. Practical lessons are drawn and critical research areas are highlighted.

Moore, Reagan↗

DOE OSTI2025

BaZrO3-Ni-defect-formation

These data include migration energies for interstitial Ni defects in doped and undoped BaZrO3, as well as formation energies for Ni-related defects and defect complexes.

08 HYDROGEN↗

NASA NTRS1994

Performance evaluation of the NASA/KSC CAD/CAE and office automation LAN's

This study's objective is the performance evaluation of the existing CAD/CAE (Computer Aided Design/Computer Aided Engineering) network at NASA/KSC. This evaluation also includes a similar study of the Office Automation network, since it is being planned to integrate this network into the CAD/CAE network. The Microsoft mail facility which is presently on the CAD/CAE network was monitored to determine its present usage. This performance evaluation of the various networks will aid the NASA/KSC network managers in planning for the integration of future workload requirements into the CAD/CAE network and determining the effectiveness of the planned FDDI (Fiber Distributed Data Interface) migration.

Zobrist, George W.↗

NASA NTRS2005

NASA GTE TRACE-P Augmentation

There were three major tasks and objectives that the Tropospheric Trace Gas and Airborne Measurement Group's (TTGAMG) worked on for different aspects of this grant: 1) Migrate the data acquisition system from HP-UX to Linux, thus reducing future costs as the result of software and operating system (OS) upgrades and improving upon usability as membership in the group changes; 2) Rework the Optical Parametric Oscillator (OPO) cavities. These are the OPOs that are integral to the Georgia Institute of Technology Airborne Laser Induced Fluorescent Experiment (GITALIFE) that the TTGAMG flew on TRACE-P. The objective was to improve upon optimizing the pump laser energy and narrowing the linewidth of the UV wavelength generated by the OPOs; 3) Improve and expand the interactive website on http://tmbk2.eas.gatech.edu by adding 3-D graphing, improve the response time for Joe Surfer Dude, improve performance, usability, and expand the database. If I were to assign a letter grade to each of the above tasks, I would give the TTGAMG two Bs and an A to the tasks listed above.

Sandholm, Scott↗

NASA NTRS2009

Some Improvements in Utilization of Flash Memory Devices

Two developments improve the utilization of flash memory devices in the face of the following limitations: (1) a flash write element (page) differs in size from a flash erase element (block), (2) a block must be erased before its is rewritten, (3) lifetime of a flash memory is typically limited to about 1,000,000 erases, (4) as many as 2 percent of the blocks of a given device may fail before the expected end of its life, and (5) to ensure reliability of reading and writing, power must not be interrupted during minimum specified reading and writing times. The first development comprises interrelated software components that regulate reading, writing, and erasure operations to minimize migration of data and unevenness in wear; perform erasures during idle times; quickly make erased blocks available for writing; detect and report failed blocks; maintain the overall state of a flash memory to satisfy real-time performance requirements; and detect and initialize a new flash memory device. The second development is a combination of hardware and software that senses the failure of a main power supply and draws power from a capacitive storage circuit designed to hold enough energy to sustain operation until reading or writing is completed.

Gender, Thomas K.↗

NASA NTRSDate not supplied

Concepts for Distributed Sensing and Collaborative Airspace Autonomy in Advanced Urban Air Mobility

Emerging concepts for advanced urban air mobility envision responsive air transportation capabilities that will safely move people and cargo in locations presently underserved by aviation. Expanding aviation services to these locales, particularly for high-density autonomous flight operations over urban centers, will require advances beyond the state-of-the-art techniques for airborne sensing. The emerging field of distributed sensing and ‘smart spaces’ – where sensing, processing, communication, and actuation are embedded in the environment in which agents are acting – may provide attractive alternatives over traditional aviation solutions. This paper outlines the challenges and opportunities for distributed sensing and smart space concepts to meet the emerging needs of advanced urban operations in the national airspace. We present an overview of distributed sensing concepts and research currently being investigated under this endeavor.

Distributed sensing↗

NASA NTRSDate not supplied

A Structurally-Adaptive Framework for Distributed Airborne Sensing over Real-time Collaborative Information Sharing Networks

The emergence and maturation of wireless communication technologies continue to transform the aviation industry and are enabling new solutions to challenges faced by NASA’s Advanced Air Mobility (AAM) initiative. AAM is leading towards high-density autonomous aircraft operations in areas underserved by traditional aviation, such as over densely populated urban centers. In this paper, we build on concepts from distributed sensing and smart spaces - where sensing, processing, and communication are embedded in an environment, and agents are operating within the space can exploit these capabilities in real-time through collaborative information sharing networks. Building from these concepts, we propose a framework to enable a dynamic, topologically-adaptive, and distributed estimation system for man-rated aviation to address challenges faced by autonomous AAM operations. This paper presents the initial concept of operations and system design for this framework, presents a mathematical formulation for abstraction of the problem, identifies requirements and constraints for operation, and presents algorithmic constructs to demonstrate operation. The initial framework design will focus on supporting precision navigation and independent surveillance supporting conformance monitoring of aircraft in airspace corridors and vertiport airspaces. Preliminary results from this framework shows promise in addressing gaps in current technologies needed to enable future AAM concepts, while promising greater capabilities, performance, robustness, and safety over current aviation systems and operations.

Distributed sensing↗

NASA NTRSDate not supplied

Using OPeNDAP In The Cloud to Connect NASA Data Centers and Support Open Data Access

NASA DAACs (Distributed Active Archive Centers), including the Goddard Earth Sciences Data Information and Services Center (GES DISC), are currently transitioning from on-premises servers to a shared Earthdata Cloud in order to build more interoperability, cross-collaboration, and streamlined services between their data centers. Migrating its on-premises OPeNDAP service to the cloud is a critical component of making this interconnectedness between NASA DAACs possible. To improve their cloud services, GES DISC is leveraging open-source platforms like Github to collect user feedback, create use cases, and develop resources to enable real-time learning for users about OPeNDAP in the cloud. This presentation gives an overview of the OPeNDAP in the cloud, resources developed to access this service, and considerations for improved user guidance and experience to further support NASA's commitment to the Open-Source Science Initiative (OSSI).

Christopher Battisto↗