Search NASA⌕ Search

Engineering topics

Humphrey, Marty

Publications and source records attributed to Humphrey, Marty.

The FLUXNET2015 dataset and the ONEFlux processing pipeline for eddy covariance data

The FLUXNET2015 dataset provides ecosystem-scale data on CO2, water, and energy exchange between the biosphere and the atmosphere, and other meteorological and biological measurements, from 212 sites around the globe (over 1500 site-years, up to and including year 2014). These sites, independently managed and operated, voluntarily contributed their data to create global datasets. Data were quality controlled and processed using uniform methods, to improve consistency and intercomparability across sites. The dataset is already being used in a number of applications, including ecophysiology studies, remote sensing studies, and development of ecosystem and Earth system models. FLUXNET2015 includes derived-data products, such as gap-filled time series, ecosystem respiration and photosynthetic uptake estimates, estimation of uncertainties, and metadata about the measurements, presented for the first time in this paper. In addition, 206 of these sites are for the first time distributed under a Creative Commons (CC-BY 4.0) license. This paper details this enhanced dataset and the processing methods, now made available as open-source codes, making the dataset more accessible, transparent, and reproducible.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Connecting AmeriFlux to the Globe, Extending the Partnership with Global Flux Network FLUXNET

We propose a project that will extend and expand the activities and contributions of AmeriFlux, to the global stage by working with our international partners to continue operating and expand the functioning of global FLUXNET database. One goal is to produce the next-generation FLUXNET dataset. This will be an integrated dataset on carbon, water and energy fluxes, on meteorological conditions and on the structure and function of the ecosystems; we have to potential to construct a new database, double the size of the La Thuile dataset, with over 2000 site-years of trace gas flux measurements, and supporting meteorological and site meta-data from over 400 sites world-wide. The production of the next FLUXNET database is critical to the activities of this ROSE’s solicitation because these data will drive many of the carbon cycle synthesis activities and the model parameterization and validation activities. A second goal is to build the next-generation, data-system that will house, distribute and query data and discover new information by the broader biogeoscience user community. The third goal is to use these data in synthesis activities that improve our understanding on how climate, weather, ecological, physiological and soil factors govern the exchange of carbon, water and energy between vegetation and the atmosphere at multiple time and space scales. The fourth goal is to act as a coordinator of the project to recruit data, to organize a workshop/conference and to build a community of data sharing and interaction across vast cultural spectrum of participating scientists. The rationale for proposing this work is based on the fact that the vast databases that are produced by flux networks cannot perpetuate and evolve without continued support of the intellectual activities of scientists who produce data, expert in databases, and scientists who use data for modeling. The actuality of a flux network takes human intervention to recruit data from different countries and cultures, to build trust to share data in open-access format, to support data users, and to collaborate. Specifically, we will develop an updated data portal that is easy to navigate and conducive for distributing, visualization, evaluation and discovery of the multiple types of flux and environmental data and information. We will foster collaboration and interactions with our national and international partners and the recruitment of data by: 1) continuing the periodic publication of the newsletter, FLUXLETTER; 2) by hosting an international conference; 3) by coordinating and launching multi-investigator data synthesis activities; and 4) by developing tools that coordinate the development of research papers resulting from the dataset. Data products include raw and gap-filled data at their native time steps (30 minute averages), data products that are integrated on daily and annual time scales and value added products like ecosystem respiration and canopy photosynthesis. Uncertainty analysis and quality control metrics will be produced in coordination with AmeriFlux and ICOS. We will also work with colleagues to expand the scope of FLUXNET beyond carbon dioxide and water vapor, and start archiving the growing methane flux databases that are being collected, world-wide. The interpretation of the flux data relies on good meta-data. We will work with data providers to populate and expand the meta-data archives. We will also work with collaborators to incorporate data from remote sensing products such as MODIS and Landsat, from corresponding climate and weather stations. Another goal is to incorporate new data on canopy structure, with recent LIDAR acquisitions, to populate the data archive with automated soil respiration and understory flux measurements, where available and local climate records for corresponding sites. And, we will collaborate with the TRY database, by merging flux data with plant trait information.

54 ENVIRONMENTAL SCIENCES↗

Hunting Data Rogues at Scale: Data Quality Control for Observational Data in Research Infrastructures

Data quality control is one of the most time consuming activities within Research Infrastructures (RIs), especially when involving observational data and multiple data providers. In this work we report on our ongoing development of data rogues, a scalable approach to manage data quality issues for observational data within RIs. The motivation for this work started with the creation of the FLUXNET2015 dataset, which includes carbon, water, and energy fluxes plus micrometeorological and ancillary data measured in over 200 sites around the world. To create an uniform dataset, including derived data products, extensive work on data quality control was needed. The unpredictable nature of observational data quality issues makes the automation of data quality control inherently difficult. Developed based on this experience, the data rogues methodology allows for increased automation of quality control activities by systematically identifying, cataloging, and documenting implementations of solutions to data issues. We believe this methodology can be extended and applied to others domains and types of data, making the automation of data quality control a more tractable problem.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

International Soil Carbon Network (ISCN) Database v3-1

The ISCN is an international scientific community devoted to the advancement of soil carbon research. The ISCN manages an open-access, community-driven soil carbon database. This is version 3-1 of the ISCN Database, released in December 2015. It gathers 38 separate dataset contributions, totalling 67,112 sites with data from 71,198 soil profiles and 431,324 soil layers. For more information about the ISCN, its scientific community and resources, data policies and partner networks visit: http://iscn.fluxdata.org/.

Nave, Luke↗

Fault Tolerance and Scaling in e-Science Cloud Applications: Observations from the Continuing Development of MODISAzure

It can be natural to believe that many of the traditional issues of scale have been eliminated or at least greatly reduced via cloud computing. That is, if one can create a seemingly wellfunctioning cloud application that operates correctly on small or moderate-sized problems, then the very nature of cloud programming abstractions means that the same application will run as well on potentially significantly larger problems. In this paper, we present our experiences taking MODISAzure, our satellite data processing system built on the Windows Azure cloud computing platform, from the proof-of-concept stage to a point of being able to run on significantly larger problem sizes (e.g., from national-scale data sizes to global-scale data sizes). To our knowledge, this is the longest-running eScience application on the nascent Windows Azure platform. We found that while many infrastructure-level issues were thankfully masked from us by the cloud infrastructure, it was valuable to design additional redundancy and fault-tolerance capabilities such as transparent idempotent task retry and logging to support debugging of user code encountering unanticipated data issues. Further, we found that using a commercial cloud means anticipating inconsistent performance and black-box behavior of virtualized compute instances, as well as leveraging changing platform capabilities over time. We believe that the experiences presented in this paper can help future eScience cloud application developers on Windows Azure and other commercial cloud providers.

97 MATHEMATICS AND COMPUTING↗