Search NASA⌕ Search

SEARCH · Search NASA

Results for “large-scale scientific datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

Harnessing Big Data to Support the Conservation and Rehabilitation of Mangrove Forests Globally

Mangrove forests are found on sheltered coastlines in tropical, subtropical, and some warm temperate regions. These forests support unique biodiversity and provide a range of benefits to coastal communities, but as a result of large-scale conversion for aquaculture, agriculture, and urbanization, mangroves are considered increasingly threatened ecosystems. Scientific advances have led to accurate and comprehensive global datasets on mangrove extent, structure, and condition, and these can support evaluation of ecosystem services and stimulate greater conservation and rehabilitation efforts. To increase the utility and uptake of these products, in this Perspective we provide an overview of these recent and forthcoming global datasets and explore the challenges of translating these new analyses into policy action and on-the-ground conservation. We describe a new platform for visualizing and disseminating these datasets to the global science community, non-governmental organizations, government officials, and rehabilitation practitioners and highlight future directions and collaborations to increase the uptake and impact of large-scale mangrove research.

mangrove↗

The architecture of the High Performance Storage System (HPSS)

The rapid growth in the size of datasets has caused a serious imbalance in I/O and storage system performance and functionality relative to application requirements and the capabilities of other system components. The High Performance Storage System (HPSS) is a scalable, next-generation storage system that will meet the functionality and performance requirements or large-scale scientific and commercial computing environments. Our goal is to improve the performance and capacity of storage by two orders of magnitude or more over what is available in the general or mass marketplace today. We are also providing corresponding improvements in architecture and functionality. This paper describes the architecture and functionality of HPSS.

Teaff, Danny↗

A Lightweight I/O Scheme to Facilitate Spatial and Temporal Queries of Scientific Data Analytics

In the era of petascale computing, more scientific applications are being deployed on leadership scale computing platforms to enhance the scientific productivity. Many I/O techniques have been designed to address the growing I/O bottleneck on large-scale systems by handling massive scientific data in a holistic manner. While such techniques have been leveraged in a wide range of applications, they have not been shown as adequate for many mission critical applications, particularly in data post-processing stage. One of the examples is that some scientific applications generate datasets composed of a vast amount of small data elements that are organized along many spatial and temporal dimensions but require sophisticated data analytics on one or more dimensions. Including such dimensional knowledge into data organization can be beneficial to the efficiency of data post-processing, which is often missing from exiting I/O techniques. In this study, we propose a novel I/O scheme named STAR (Spatial and Temporal AggRegation) to enable high performance data queries for scientific analytics. STAR is able to dive into the massive data, identify the spatial and temporal relationships among data variables, and accordingly organize them into an optimized multi-dimensional data structure before storing to the storage. This technique not only facilitates the common access patterns of data analytics, but also further reduces the application turnaround time. In particular, STAR is able to enable efficient data queries along the time dimension, a practice common in scientific analytics but not yet supported by existing I/O techniques. In our case study with a critical climate modeling application GEOS-5, the experimental results on Jaguar supercomputer demonstrate an improvement up to 73 times for the read performance compared to the original I/O method.

Temporal Queries↗

Venus mapping

Semicontrolled image mosaics of Venus, based on Magellan data, are being compiled at 1:50,000,000, 1:10,000,000, 1:5,000,000, and 1:1,000,000 scales to support the Magellan Radar Investigator (RADIG) team. The mosaics are semicontrolled in the sense that data gaps were not filled and significant cosmetic inconsistencies exist. Contours are based on preliminary radar altimetry data that is subjected to revision and improvement. Final maps to support geologic mapping and other scientific investigations, to be compiled as the dataset becomes complete, will be sponsored by the Planetary Geology and Geophysics Program and/or the Venus Data Analysis Program. All maps, both semicontrolled and final, will be published as I-maps by the United States Geological Survey. All of the mapping is based on existing knowledge of the spacecraft orbit; photogrammetric triangulation, a traditional basis for geodetic control on planets where framing cameras were used, is not feasible with the radar images of Venus, although an eventual shift of coordinate system to a revised spin-axis location is anticipated. This is expected to be small enough that it will affect only large-scale maps.

Batson, R. M.↗

Investigating Low-Altitude Constellations of Ad-Hoc Lunar PNT System for Distributed Spacecraft Autonomy

In this study, we examine a low-altitude Lunar Position, Navigation, and Timing (LPNT) constellations and the localization performance of Centralized Extended Kalman Filter (CEKF) and Decentralized Extended Kalman Filter (DEKF) algorithms. The primary investigation involves a 100-node swarm operating at a 100 km altitude, in contrast to previous studies that examined a 21-node asset in a frozen-orbit at 5,500 km. The autonomous operation of large-scale swarm is based on two-way Inter-Satellite Link (ISL) measurements, which involve pseudoranges and relative velocities among swarm nodes. We perform a numerical assessment of the two filtering approaches, utilizing ‘fully sampled’ measurements from all available assets as well as ‘two ISL’ measurements where each spacecraft is restricted to only two antennas. This research includes an analysis of CEKF under 2-ISL constraints and evaluates the performance of DEKF in a 100-node swarm, which has not been explored in previous studies. In addition, we examine the impact of increasing the sampling frequency for DEKF, showing that the update cycle can be shortened from a 10-minute interval. A novel approach for ‘2-ISL limited’ DEKF will also be introduced, using a matching formulation that exhaustively enumerates all potential matches. This study provides valuable insights into large-scale distributed swarm operations, considering various filter configurations, sampling frequencies, matching strategies, and scalability of CEKF and DEKF for low-altitude LPNT applications. The Lunar PNT technology plays a key role in providing reliable and robust navigation services on the Moon's surface and the South pole, where the primary Lunar missions are planned. To support upcoming Lunar missions, including small satellites from NASA's Commercial Lunar Payload Services program, the Lunar PNT system must be adaptable to smaller platforms like CubeSats. Driven by the growing involvement of public and private exploration partnerships, the traditional low Earth orbit missions are shifting to beyond geosynchronous orbit [1]. These upcoming missions aim to foster a sustainable and innovative exploration program, in collaboration with commercial and international partners, to facilitate human expansion throughout the solar system and return new knowledge and opportunities to Earth [2]. As part of this trend, there are increasing efforts to utilize science missions in Lunar orbit to develop a non-dedicated and ad-hoc PNT network system. Two traditional approaches, the Deep Space Network (DSN) and the weak signal Global Positioning System (GPS), are established deep-space navigation technologies for missions beyond the geosynchronous orbit. Beginning in 1958, the DSN was developed to communicate with the Explorer 1 spacecraft based on the use of radiometric tracking in spacecraft navigation [3]. The DSN is capable of providing nearly unfettered coverage to spacecraft beyond low-Earth orbit (LEO), however, increased space mission volume has created concerns about future expectations of DSN usage for spacecraft navigation [4]. For cislunar mission applications, the position accuracy using DSN achieves 100 m (3σ) with at least three geometrically diverse ground stations when using radiometric tracking alone [5]. The DSN's dependence on Earth-based ground stations restricts its operational capabilities to periods of Earth visibility. This limitation, coupled with its poor localization performance, renders the DSN unsuitable for future lunar missions that demand continuous tracking and precise positioning. To satisfy the increasing requirements of DSN in Lunar applications, spacecrafts are also required to improve their onboard antenna power and efficiency of the transmission. However, there is an important aggregate cost trade between adding capabilities to every spacecraft and adding to a capacity on the ground that serves multiple spacecraft [6]. A weak GPS system can provide PNT service while the user spacecraft is bound to the Moon, leveraging a single, steerable high gain antenna with the relatively narrow beam which includes all the sources in its field of view [7]. However, the higher the altitude the receiver is above the GPS constellations, the poorer and the weaker are the relative geometry and the received signal powers, respectively, leading to a significant navigation accuracy reduction [8]. The transmitted power becomes weaker with increasing distance from the Earth as well as signals tracked from one of the side lobes of the GPS antenna pattern. As a results, the number of visible satellites and relative geometric condition of the GPS satellites at very high altitude drops dramatically and reduces the navigation solution accuracy. Therefore, the weak GPS system is also not an ideal way to provide PNT service to upcoming Lunar missions when considering its limited geometric condition and the recued navigation accuracy. Another navigation approach on the Moon is being developed, similar to the Global Navigation Satellite System (GNSS) on Earth, aiming to offer navigation service with continuous 24/7 coverage across the entire Lunar surface. For example, lunar communications relay and navigation systems (LCRNS) by NASA and Lunar navigation satellite systems (LNSS) by JAXA are designed to serve as dedicated Position, Navigation, and Timing (PNT) systems for the Moon. However, designing a dedicated LNSS and PNT service involves additional challenges, which are unique to the lunar environment, including limited payload capacity for the CubeSat platform, i.e., the size, weight, and power (SWaP) of the onboard clock, limited lunar ground monitoring stations, and limited financial investment as compared to the legacy Earth-GPS [9]. NASA’s focus on utilizing CubeSat platforms on the Moon leads to an alternative Lunar navigation platform that leverages the existing Lunar science and exploration assets. The small satellites used in Lunar missions can be used to create a low-cost, autonomous, ad-hoc, and on-demand mission-centric Lunar PNT swarm capable of providing PNT services to these low-cost lunar missions [10]. As upcoming Lunar missions will often operate at low-altitude about 30 km to 100 km for scientific observations and mapping purposes, the low-altitude orbital constellations could be employed to create an ad-hoc Lunar PNT system. However, several issues must be addressed, such as the instability of these orbits, which often require maintenance or are only suitable for short-duration missions, operating for fewer than 90 days. Additionally, at an altitude of 100 km, the satellites have a limited period during which they are above the horizon and capable of providing PNT service to users. The implementation of a non-dedicated, ad-hoc Lunar navigation constellation facilitates on-demand PNT services. A preliminary study of ad-hoc Lunar PNT system was conducted using 21 spacecraft in 5,5000 km altitude frozen orbits to test its feasibility and a basic performance of orbital asset localization among ad-hoc Lunar constellations in small satellites format [10]. These swarm assets are designed for autonomous localization with minimal Earth interaction, reducing dependency on bandwidth and ground resources. The design in [10] demonstrated the feasibility of a decentralized PNT approach, specifically employing a DEKF approach for state estimation, which helps minimize onboard operating costs. The DEKF method distributes computation across individual satellites, which lightens the computational load while maintaining accuracy in orbit ephemeris and clock offsets, similar to centralized systems [11]. In a follow-on study [12], each spacecraft was limited to 2 communications antennae, forcing the selection of measurements and scheduling spacecraft activities to perform the measurements. A matching algorithm is implemented to select the best measurements and schedule position estimation updates. The decentralized localization performance is also investigated with increasing levels of network degradation for swarm assets considering the impact of intermittent and permanent communication failure, to demonstrate the robustness and fidelity of the decentralized Lunar PNT service [13]. This study confirmed that the ad-hoc PNT constellations in frozen orbit are highly robust and resilient to communication failures. However, unlike frozen orbit swarm assets, the low-altitude satellites have a limited ground view at an altitude of 100 km, where the ad-hoc Lunar constellation consists of 98 low-altitude satellites, evenly distributed across seven circular polar orbital planes, alongside two satellites in a frozen orbit at an altitude of 5,500 km (Figure 1). Therefore, the number of satellites visible to ground users is significantly limited in low-altitude orbit constellations. As each visibility of a spacecraft remains intact for only a few ticks before it moves out of the field of view, the ground user encounters challenges in maintaining continuous navigation service, resulting in sparse availability and provision of Lunar PNT system. Consequently, service availability is primarily restricted to the Lunar South Pole region (Figure 2). Given these limitations and concerns, the localization performance of low-altitude swarm assets will be assessed in this study. We focus on the investigation of the localization performance of low-altitude swarm assets and ground users near the Lunar South Pole. The overall flow of the Lunar PNT simulation incorporates the DEKF approach of asset localization and the weighted least-squares approach in user localization (Figure 3). The autonomous Lunar PNT simulation is primarily implemented in MATLAB, where the DEKF based on the matching scheduler is implemented with Google’s OR-tools as a model builder and Gurobi optimization tool as a backend solver. The General Mission Analysis Tool (GMAT) is utilized to generate ephemeris data for swarm assets, and accounts for satellite orbital details, mass, and perturbations like solar radiation pressure and drag coefficients. Each ephemeris dataset is produced in the Moon International Celestial Reference Frame (ICRF) inertial coordinate system. For state estimation, the distributed swarm assets rely on two-way Inter-Satellite Link (ISL) measurements, which involve tracking pseudoranges and relative velocities between visible satellites and anchor nodes during each observation. Numerical evaluations of the decentralized localization process are conducted to demonstrate the feasibility of the low-altitude PNT system in providing reliable navigation services. The main approach involves using DEKF and CEKF to localize 100 satellites in low-altitude constellations, where the CEKF is implemented to serve as a baseline for comparing the performance of distributed algorithms. In both cases, we evaluate ‘fully sampled’ measurements from all available assets, and ‘two ISL’ measurements when spacecraft are constrained to have only two antennas. We test four estimation techniques: CEKF fully sampled, CEKF two ISL, DEKF fully sampled, and DEKF two ISL filters. As the DEKF update cycle is comprised of network setup, communication, and computations, a global broadcast network and 2-way ISL network setup will take from 4 to 6 minutes as maximum [12]. In this simulation, the DEKF update cycle is set to 10 minutes, including a 4-minute latency for obtaining and computing the actual measurement updates. We experiment an increased update cycle to demonstrate the feasibility and evaluate the impact on localization performance using various tuning values for measurement noise covariances (Figures 4 and 5). By comparing centralized and decentralized approaches using a matching algorithm, we analyze the influence of cross-correlation factors in the covariance matrix, assuming 100% reliability of all assets and measurements. The increased frequency and the adjustments of tuning parameters reveal distinct error patterns between the two scenarios. The localization accuracy of the swarm assets and ground users is assessed by taking the median error across 100 assets and one ground user (84.9°S, 137.5°E) over 7-day simulation period (Table 1). Since the user localization accuracy is significantly affected by the performance of the swarm assets, it is crucial to maintain high localization accuracy within the swarm. This study will continue to explore decentralized filtering for autonomous LPNT operations, with further investigation of an 'iterative' matching approach which enumerates every valid matching pair, planned for the following month.

Yeji Kim↗

Solar Coronal Structure Study

The subject of this investigation is to study the physics of the solar corona through the analysis of the EUV and UV data produced by two flights (12 May 1992 and 25 April 1994) of the Lockheed Solar Plasma Diagnostics Experiment (SPDE) sounding rocket payload, in combination with Yohkoh and ground-based data. Each rocket flight produced both spectral and imaging data. These joint datasets are useful for understanding the physical state of various features in the solar atmosphere at different heights ranging from the photosphere to the corona at the time of the, rocket flights, which took place during the declining phase of a solar cycle, 2-4 years before the minimum. The investigation is narrowly focused on comparing the physics of small- and medium-scale strong-field structures with that of large-scale, weak fields. As we close th is investigation, we have to recall that our present position in the understanding of basic solar physics problems (such as coronal heating) is much different from that in 1995 (when we proposed this investigation), due largely to the great success of SOHO and TRACE. In other words, several topics and techniques we proposed can now be better realized with data from these missions. For this reason, at some point of our work, we started concentrating on the 1992 data, which are more unique and have more supporting data. As a result, we discontinued the investigation on small-scale structures, i.e., bright points, since high-resolution TRACE images have addressed more important physics than SPDE EUV images could do. In the final year, we still spent long time calibrating the 1992 data. The work was complicated because of the old-fashioned film, which had problems not encountered with more modern CCD detectors. After our considerable effort on calibration, we were able to focus on several scientific topics, relying heavily on the SPDE UV images. They include the relation between filaments and filament channels, the identification of hot loops, and the physical conditions of such loops especially at their foot-points. A total of four papers were completed from this contract which are listed in the last section.

Nitta, Nariaki↗

SPACE: the SPectroscopic, All-Sky Cosmic Explorer

We describe the scientific motivations, the mission concept and the instrumentation of SPACE, a class-M mission proposed for concept study at the first call of the ESA Cosmic-Vision 2015-2025 planning cycle. SPACE aims at producing the largest three-dimensional evolutionary map of the Universe over the past 10 billion years by taking near-IR spectra and measuring redshifts of more than half a billion galaxies at 0 < z < 2 down to AB approximately 23 over 37r sr of the sky. In addition, SPACE will also target a smaller sky field, performing a deep spectroscopic survey of millions of galaxies to AB approximately 26 and at 2 < z < l0+. Owing to the depth, redshift range, volume coverage and quality of its spectra, SPACE will reveal with unique sensitivity most of the fundamental cosmological signatures, including the power spectrum of density fluctuations and its turnover, the baryonic acoustic oscillations imprinted when matter and radiation decoupled, the distance-luminosity relation of cosmological supernovae, the evolution of the cosmic expansion rate, the growth rate of cosmic large-scale structure, the large scale distribution of galaxies. The datasets from the SPACE mission will represent a long lasting legacy that will be data mined for many years to come.

Cimatti, A.↗

Australian Soil Moisture Field Experiments in Support of Soil Moisture Satellite Observations

Large-scale field campaigns provide the critical fink between our understanding retrieval algorithms developed at the point scale, and algorithms suitable for satellite applications at vastly larger pixel scales. Retrievals of land parameters must deal with the substantial sub-pixel heterogeneity that is present in most regions. This is particularly the case for soil moisture remote sensing, because of the long microwave wavelengths (L-band) that are optimal. Yet, airborne L-band imagers have generally been large, heavy, and required heavy-lift aircraft resources that are expensive and difficult to schedule. Indeed, US soil moisture campaigns, have been constrained by these factors, and European campaigns have used non-imagers due to instrument and aircraft size constraints. Despite these factors, these campaigns established that large-scale soil moisture remote sensing was possible, laying the groundwork for satellite missions. Starting in 2005, a series of airborne field campaigns have been conducted in Australia: to improve our understanding of soil moisture remote sensing at large scales over heterogeneous areas. These field data have been used to test and refine retrieval algorithms for soil moisture satellite missions, and most recently with the launch of the European Space Agency's Soil Moisture Ocean Salinity (SMOS) mission, to provide validation measurements over a multi-pixel area. The campaigns to date have included a preparatory campaign in 2005, two National Airborne Field Experiments (NAFE), (2005 and 2006), two campaigns to the Simpson Desert (2008 and 2009), and one Australian Airborne Cal/val Experiment for SMOS (AACES), just concluded in the austral spring of 2010. The primary airborne sensor for each campaign has been the Polarimetric L-band Microwave Radiometer (PLMR), a 6-beam pushbroom imager that is small enough to be compatible with light aircraft, greatly facilitating the execution of the series of campaigns, and a key to their success. An L-band imaging radar is being added to the complement to provide simultaneous active-passive L-band observations, for algorithm development activities in support of NASA's upcoming Soil Moisture Active Passive (.S"M) mission. This paper will describe the campaigns, their objectives, their datasets, and some of the unique advantages of working with small/light sensors and aircraft. We will also review the main scientific findings, including improvements to the SMOS retrieval algorithm enabled by NAFE observations and the evaluation of the Simpson Desert as a calibration target for L-band satellite missions. Plans for upcoming campaigns will also be discussed.

Kim, Edward↗

NASA's Hyperwall Revealing the Big Picture

NASA:s hyperwall is a sophisticated visualization tool used to display large datasets. The hyperwall, or video wall, is capable of displaying multiple high-definition data visualizations and/or images simultaneously across an arrangement of screens. Functioning as a key component at many NASA exhibits, the hyperwall is used to help explain phenomena, ideas, or examples of world change. The traveling version of the hyperwall is typically comprised of nine 42-50" flat-screen monitors arranged in a 3x3 array (as depicted below). However, it is not limited to monitor size or number; screen sizes can be as large as 52" and the arrangement of screens can include more than nine monitors. Generally, NASA satellite and model data are used to highlight particular themes in atmospheric, land, and ocean science. Many of the existing hyperwall stories reveal change across space and time, while others display large-scale still-images accompanied by descriptive, story-telling captions. Hyperwall content on a variety of Earth Science topics already exists and is made available to the public at: eospso.gsfc.nasa.gov/hyperwall. Keynote and PowerPoint presentations as well as Summary of Story files are available for download on each existing topic. New hyperwall content and accompanying files will continue being developed to promote scientific literacy across a diverse group of audience members. NASA invites the use of content accessible through this website but requests the user to acknowledge any and all data sources referenced in the content being used.

Sellers, Piers↗

Using Docker Containers to Extend Reproducibility Architecture for the NASA Earth Exchange (NEX)

NASA Earth Exchange (NEX) is a data, supercomputing and knowledge collaboratory that houses NASA satellite, climate and ancillary data where a focused community can come together to address large-scale challenges in Earth sciences. As NEX has been growing into a petabyte-size platform for analysis, experiments and data production, it has been increasingly important to enable users to easily retrace their steps, identify what datasets were produced by which process chains, and give them ability to readily reproduce their results. This can be a tedious and difficult task even for a small project, but is almost impossible on large processing pipelines. We have developed an initial reproducibility and knowledge capture solution for the NEX, however, if users want to move the code to another system, whether it is their home institution cluster, laptop or the cloud, they have to find, build and install all the required dependencies that would run their code. This can be a very tedious and tricky process and is a big impediment to moving code to data and reproducibility outside the original system. The NEX team has tried to assist users who wanted to move their code into OpenNEX on Amazon cloud by creating custom virtual machines with all the software and dependencies installed, but this, while solving some of the issues, creates a new bottleneck that requires the NEX team to be involved with any new request, updates to virtual machines and general maintenance support. In this presentation, we will describe a solution that integrates NEX and Docker to bridge the gap in code-to-data migration. The core of the solution is saemi-automatic conversion of science codes, tools and services that are already tracked and described in the NEX provenance system, to Docker - an open-source Linux container software. Docker is available on most computer platforms, easy to install and capable of seamlessly creating and/or executing any application packaged in the appropriate format. We believe this is an important step towards seamless process deployment in heterogeneous environments that will enhance community access to NASA data and tools in a scalable way, promote software reuse, and improve reproducibility of scientific results.

earth exchange↗