Search NASA⌕ Search

SEARCH · Search NASA

Results for “climate data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Climate Data Analysis Tools

Climate Data Analysis Tools (CDAT) is a software infrastructure that uses an object-oriented scripting language to link together separate software subsystems and packages thus forming an integrated environment for solving model diagnosis problems, The power of the system comes from Python and its ability to seamlissly interconnect software. Python provides a general purpose and full-featured scripting language with a variety of user interfaces including command-line interaction, stand-alone scripts (applications) and fraphical user interfaces (GUI). The CDAT subsystems, implemented as modules, provide access to and management of gridded data (Climate Data Management Systems or CDMS); large-array numerical operations (Numerical Python); and visualization (Visualization and Control System or VCS).

Source record↗

Climate Data Analysis Tools

Climate Data Analysis Tools (CDAT) is a software infrastructure that uses an object-oriented scripting language to link together separate software subsystems and packages thus forming an integrated environment for solving model diagnosis problems, The power of the system comes from Python and its ability to seamlissly interconnect software. Python provides a general purpose and full-featured scripting language with a variety of user interfaces including command-line interaction, stand-alone scripts (applications) and fraphical user interfaces (GUI). The CDAT subsystems, implemented as modules, provide access to and management of gridded data (Climate Data Management Systems or CDMS); large-array numerical operations (Numerical Python); and visualization (Visualization and Control System or VCS).

Source record↗

2014 Earth System Grid Federation and Ultrascale Visualization Climate Data Analysis Tools Conference Report

The climate and weather data science community met December 9–11, 2014, in Livermore, California, for the fourth annual Earth System Grid Federation (ESGF) and Ultrascale Visualization Climate Data Analysis Tools (UV-CDAT) Face-to-Face (F2F) Conference, hosted by the Department of Energy, National Aeronautics and Space Administration, National Oceanic and Atmospheric Administration, the European Infrastructure for the European Network of Earth System Modelling, and the Australian Department of Education. Both ESGF and UVCDATremain global collaborations committed to developing a new generation of open-source software infrastructure that provides distributed access and analysis to simulated and observed data from the climate and weather communities. The tools and infrastructure created under these international multi-agency collaborations are critical to understanding extreme weather conditions and long-term climate change. In addition, the F2F conference fosters a stronger climate and weather data science community and facilitates a stronger federated software infrastructure. The 2014 F2F conference detailed the progress of ESGF, UV-CDAT, and other community efforts over the year and sets new priorities and requirements for existing and impending national and international community projects, such as the Coupled Model Intercomparison Project Phase Six. Specifically discussed at the conference were project capabilities and enhancements needs for data distribution, analysis, visualization, hardware and network infrastructure, standards, and resources.

54 ENVIRONMENTAL SCIENCES↗

Xarray Climate Data Analysis Tools

xCDAT is an extension of xarray for climate data analysis on structured grids. It serves as a modern successor to the Community Data Analysis Tools (CDAT) library. Xarray is an "open source project and Python package that introduces labels in the form of dimensions, coordinates, and attributes on top of raw NumPy-like arrays, which allows for more intuitive, more concise, and less error-prone user experience. Xarray includes a large and growing library of domain-agnostic functions for advanced analytics and visualization with these data structures" (source: https://xarray.dev/). The goal of xCDAT is to provide generalizable features and utilities for simple and robust analysis of climate data. xCDAT's design philosophy is focused on reducing the overhead required to accomplish certain tasks in xarray. Some key xCDAT features are inspired by or ported from the core CDAT library, while others leverage powerful libraries in the xarray ecosystem (e.g., xESMF and cf_xarray) to deliver robust APIs.

Vo, Tom↗

3rd Annual Earth System Grid Federation and 3rd Annual Earth System Grid Federation and Ultrascale Visualization Climate Data Analysis Tools Face-to-Face Meeting Report December 2013

The climate and weather data science community gathered December 3–5, 2013, at Lawrence Livermore National Laboratory, in Livermore, California, for the third annual Earth System Grid Federation (ESGF) and Ultra-scale Visualization Climate Data Analysis Tools (UV-CDAT) Face-to-Face (F2F) Meeting, which was hosted by the Department of Energy, National Aeronautics and Space Administration, National Oceanic and Atmospheric Administration, the European Infrastructure for the European Network of Earth System Modelling, and the Australian Department of Education. Both ESGF and UV-CDAT are global collaborations designed to develop a new generation of open-source software infrastructure that provides distributed access and analysis to observed and simulated data from the climate and weather communities. The tools and infrastructure developed under these international multi-agency collaborations are critical to understanding extreme weather conditions and long-term climate change, while the F2F meetings help to build a stronger climate and weather data science community and stronger federated software infrastructure. The 2013 F2F meeting determined requirements for existing and impending national and international community projects; enhancements needed for data distribution, analysis, and visualization infrastructure; and standards and resources needed for better collaborations.

54 ENVIRONMENTAL SCIENCES↗

xCDAT: A Python Package for Simple and Robust Analysis of Climate Data

xCDAT (Xarray Climate Data Analysis Tools) is an open-source Python package that extends Xarray (Hoyer & Hamman, 2017) for climate data analysis on structured grids. xCDAT streamlines analysis of climate data by exposing common climate analysis operations through a set of straightforward APIs. Some of xCDAT’s key features include spatial averaging, temporal averaging, and regridding. These features are inspired by the Community Data Analysis Tools (CDAT) library (Dean N. Williams et al., 2009) (D. N. Williams, 2014) (Doutriaux et al., 2019) and leverage powerful packages in the Xarray ecosystem including xESMF (Zhuang et al., 2023), xgcm (Abernathey et al., 2022), and CF xarray (Cherian et al., 2023). To ensure general compatibility across various climate models, xCDAT operates on datasets that are compliant with the Climate and Forecast (CF) metadata conventions (Hassell et al., 2017).

54 ENVIRONMENTAL SCIENCES↗

Ultra-scale Visualization Climate Data Analysis Tools (UV-CDAT)

This report summarizes work carried out by the Ultra-scale Visualization Climate Data Analysis Tools (UV-CDAT) Team for the period of January 1, 2011 through June 30, 2011. It discusses highlights, overall progress, period goals, and collaborations and lists papers and presentations. To learn more about our project, please visit our UV-CDAT website (URL: http://uv-cdat.org). This report will be forwarded to the program manager for the Department of Energy (DOE) Office of Biological and Environmental Research (BER), national and international collaborators and stakeholders, and to researchers working on a wide range of other climate model, reanalysis, and observation evaluation activities. The UV-CDAT executive committee consists of Dean N. Williams of Lawrence Livermore National Laboratory (LLNL); Dave Bader and Galen Shipman of Oak Ridge National Laboratory (ORNL); Phil Jones and James Ahrens of Los Alamos National Laboratory (LANL), Claudio Silva of Polytechnic Institute of New York University (NYU-Poly); and Berk Geveci of Kitware, Inc. The UV-CDAT team consists of researchers and scientists with diverse domain knowledge whose home institutions also include the National Aeronautics and Space Administration (NASA) and the University of Utah. All work is accomplished under DOE open-source guidelines and in close collaboration with the project's stakeholders, domain researchers, and scientists. Working directly with BER climate science analysis projects, this consortium will develop and deploy data and computational resources useful to a wide variety of stakeholders, including scientists, policymakers, and the general public. Members of this consortium already collaborate with other institutions and universities in researching data discovery, management, visualization, workflow analysis, and provenance. The UV-CDAT team will address the following high-level visualization requirements: (1) Alternative parallel streaming statistics and analysis pipelines - Data parallelism, Task parallelism, Visualization parallelism; (2) Optimized parallel input/output (I/O); (3) Remote interactive execution; (4) Advanced intercomparison visualization; (5) Data provenance processing and capture; and (6) Interfaces for scientists - Workflow data analysis and visualization construction tools, and Visualization interfaces.

54 ENVIRONMENTAL SCIENCES↗

Climate Data Analysis Tools Facilitate Scientific Investigations

Insightful analysis of geophysical phenomena often depends on the choice of software tools to access, manipulate, and visualize data sets of interest. Our data exploration tasks can be efficiently executed in a single computational environment by the freely distributed Climate Data Analysis Tools (CDAT; http://www-pcmdi.llnl.gov/software-portal/). Although CDAT has been designed primarily for climate science applications, its capabilities are increasingly being enhanced to address the data needs of other geophysical sciences. The CDAT software is based on the object-oriented Python computer language, but the software extends existing constructs to achieve capabilities that are relevant to any gridded geophysical data. Depending on the users preference, CDAT can be fully deployed by either a script or graphical user interface.

54 ENVIRONMENTAL SCIENCES↗

Ultrascale Visualization Climate Data Analysis Tools (UV-CDAT) Final Report

A partnership across government, academic, and private sectors has created a novel system that enables climate researchers to solve current and emerging data analysis and visualization challenges. The Ultrascale Visualization Climate Data Analysis Tools (UV-CDAT) software project utilizes the Python application programming interface (API) combined with C/C++/Fortran implementations for performance-critical software that offers the best compromise between "scalability" and “ease-of-use.” The UV-CDAT system is highly extensible and customizable for high-performance interactive and batch visualization and analysis for climate science and other disciplines of geosciences. For complex, climate data-intensive computing, UV-CDAT’s inclusive framework supports Message Passing Interface (MPI) parallelism as well as taskfarming and other forms of parallelism. More specifically, the UV-CDAT framework supports the execution of Python scripts running in parallel using the MPI executable commands and leverages Department of Energy (DOE)-funded general-purpose, scalable parallel visualization tools such as ParaView and VisIt. This is the first system to be successfully designed in this way and with these features. The climate community leverages these tools and others, in support of a parallel client-server paradigm, allowing extreme-scale, server-side computing for maximum possible speed-up.

54 ENVIRONMENTAL SCIENCES↗

Ultrascale Visualization Climate Data Analysis Tools (UV-CDAT): Semi-Annual Progress Report

This report summarizes work carried out by the Ultrascale Visualization Climate Data Analysis Tools (UV-CDAT) Team for the period of July 1, 2011 through December 31, 2011. It discusses highlights, overall progress, period goals, and collaborations and lists papers and presentations. The UV-CDAT team is positioned to address the following high-level visualization requirements: (1) Alternative parallel streaming statistics and analysis pipelines - Data parallelism, Task parallelism, Visualization parallelism; (2) Optimized parallel input/output (I/O); (3) Remote interactive execution; (4) Advanced intercomparison visualization; (5) Data provenance processing and capture; and (6) Interfaces for scientists - Workflow data analysis and visualization construction tools, Visualization interfaces.

54 ENVIRONMENTAL SCIENCES↗

The Ultra-scale Visualization Climate Data Analysis Tools (UV-CDAT): Data Analysis and Visualization for Geoscience Data

To support interactive visualization and analysis of complex, large-scale climate data sets, UV-CDAT integrates a powerful set of scientific computing libraries and applications to foster more efficient knowledge discovery. Connected through a provenance framework, the UV-CDAT components can be loosely coupled for fast integration or tightly coupled for greater functionality and communication with other components. This framework addresses many challenges in the interactive visual analysis of distributed large-scale data for the climate community.

97 MATHEMATICS AND COMPUTING↗

Ultra-scale Visualization Climate Data Analysis Tools (UV-CDAT)

For the past three years, a large analysis and visualization effort—funded by the Department of Energy’s Office of Biological and Environmental Research (BER), the National Aeronautics and Space Administration (NASA), and the National Oceanic and Atmospheric Administration (NOAA)—has brought together a wide variety of industry-standard scientific computing libraries and applications to create Ultra-scale Visualization Climate Data Analysis Tools (UV-CDAT) to serve the global climate simulation and observational research communities. To support interactive analysis and visualization, all components connect through a provenance application–programming interface to capture meaningful history and workflow. Components can be loosely coupled into the framework for fast integration or tightly coupled for greater system functionality and communication with other components. The overarching goal of UV-CDAT is to provide a new paradigm for access to and analysis of massive, distributed scientific data collections by leveraging distributed data architectures located throughout the world. The UV-CDAT framework addresses challenges in analysis and visualization and incorporates new opportunities, including parallelism for better efficiency, higher speed, and more accurate scientific inferences. Today, it provides more than 600 users access to more analysis and visualization products than any other single source.

54 ENVIRONMENTAL SCIENCES↗

Climate data, analysis and models for the study of natural variability and anthropogenic change

Gridded Temperature Under prior/current support, we completed and published (Jones et al., 2012) the fourth major update to our global land dataset of near-surface air temperatures, CRUTEM4. This is one of the most widely used records of the climate system, having been updated, maintained and further developed with DoE support since the 1980s. We have continued to update the CRUTEM4 (Jones et al., 2012) database that is combined with marine data to produce HadCRUT4 (Morice et al., 2012). The emphasis in our use of station temperature data is to access as many land series that have been homogenized by National Meteorological Services (NMSs, including NCDC/NOAA, Asheville, NC). Unlike the three US groups monitoring surface temperatures in a similar way, we do not infill areas that have no or missing data. We can only infill such regions in CRUTEM4 by accessing more station temperature series. During early 2014, we have begun the extensive task of updating as many of these series as possible using data provided by some NMSs and also through a number of research projects and programs around the world. All the station data used in CRUTEM4 have been available since 2009, but in Osborn and Jones (2014) we have made this more usable using a Google Earth interface (http://www.cru.uea.ac.uk/cru/data/crutem/ge/ ). We have recently completed the update of our infilled land multi-variable dataset (CRU TS 3.10, Harris et al., 2014). This additionally produces complete land fields (except for the Antarctic) for temperature, precipitation, diurnal temperature range, vapour pressure and sunshine/cloud. Using this dataset we have calculated sc-PDSI (self-calibrating Palmer Drought Severity Index) data and compared with other PDSI datasets (Trenberth et al., 2014). Also using CRU TS 3.10 and Reanalysis datasets, we showed no overall increase in global temperature variability despite changing regional patterns (Huntingford et al., 2013). Harris et al. (2014) is an update of an earlier dataset (Mitchell and Jones, 2005) which also had earlier DoE support. The earlier dataset has been cited over 1700 times according to ResearcherID on 31/July/2014 and the recent paper has already been cited 22 times. Analyses of Temperature Data Using the ERA-Interim estimate of the absolute surface air temperature of the Earth (instead of in the more normal form of anomalies) we compared the result against estimates we produced in 1999 with earlier DoE support. The two estimates are surprisingly close (differing by a couple of tenths of a degree Celsius), with the average temperature of the world (for 1981-2010) being very close to 14°C (Jones and Harpham, 2013). We have assessed ERA-Interim against station temperatures from manned and automatic weather station measurements across the Antarctic (Jones and Lister, 2014). Agreement is generally excellent across the Antarctic Peninsula and the sparsely sampled western parts of Antarctica. Differences tend to occur over eastern Antarctica where ERA-Interim is biased warm (up to 6°C) in the interior of the continent and biased cool (up to 6°C) for some of the coastal locations. Opportunities presented themselves during 2012 for collaborative work with a couple of Chinese groups. Three papers develop new temperature series for China as a whole and also for the eastern third of China (Wang et al., 2014, Cao et al., 2013 and Zhao et al., 2014). A dataset of ~400 daily Chinese temperature stations has been added to the CRU datasets. The latter paper finds that urban effects are generally about 10% of the long-term warming trend across eastern China. A fourth paper (Wang et al., 2013) illustrates issues with comparisons between reanalyses and surface temperatures across China, a method that has been widely used by some to suggest urban heating effects are much larger in the region. ERA-Interim can be used but NCEP/NCAR comparisons are very dependent on the period analysed. Earlier a new temperature dataset of homogenized records was developed for China (Li et al., 2009). Urbanization has also been addressed for London (Jones and Lister, 2009) where two rural sites have not warmed more than a city centre site since 1900. Additionally, in Ethymiadis and Jones (2010) we show that land air temperatures agree with marine data around coastal areas, further illustrating that urbanization is not a major component of large-scale surface air temperature change. Early instrumental data (before the development of modern thermometer screens) have always been suspected of being biased warm in summer, due to possible direct exposure to the sun. Two studies (Böhm et al., 2010 and Brunet et al., 2010) show this for the Greater Alpine Region (GAR) and for mainland Spain respectively. The issue is important before about 1870 in the GAR and before about 1900 in Spain. After correction for the problems, summer temperature estimates before these dates are cooler by about 0.4°C. In Jones and Wigley (2010), we discussed the importance of the biases in global temperature estimation. Exposure and to a lesser extent urbanization are the most important biases for the land areas, but both are dwarfed by the necessary adjustments for bucket SST measurements before about 1950. Individual station homogeneity is only important at the local scale. This was additionally illustrated by Hawkins and Jones (2013) where we replicated the temperature record developed by Guy Stewart Callendar in papers in 1938 and 1961. Analyses of Daily Climate Data Work here indicates that ERA-Interim (at least in Europe, Cornes and Jones, 2013, discussed in more detail in this proposal) can be used to monitor extremes (using the ETCCDI software – see Zhang et al., 2011). Additionally, also as a result of Chinese collaboration, a new method of daily temperature homogenization has been developed (Li et al., 2014). In Cornes and Jones (2011) we assessed storm activity in the northeast Atlantic region using daily gridded data. Even though the grid resolution is coarse (5° by 5° lat/long) the changes in storm activity are similar to those developed from the pressure triangle approach with station data. Analyses of humidity and pressure data In Simmons et al. (2010) we showed a reduction in relative humidity over low-latitude and mid-latitude land areas for the 10 years to 2008, based on monthly anomalies of surface air temperature and humidity from ECMWF reanalyses (ERA-40 and ERA-Interim) and our earlier land-only dataset (CRUTEM3) and synoptic humidity observations (HadCRUH). Updates of this station-based humidity dataset (now called HadISDH) extend the record, showing continued reductions (Willett et al., 2013). Analyses of Proxy Temperature Data In Vinther et al. (2010), relationships between the seasonal stable isotope data from Greenland Ice Cores and Greenland and Icelandic instrumental temperatures were investigated for the past 150-200 years. The winter season stable isotope data are found to be influenced by the North Atlantic Oscillation (NAO) and very closely related to SW Greenland temperatures. The summer season stable isotope data display higher correlations with Icelandic summer temperatures and North Atlantic SST conditions than with local SW Greenland temperatures. In Jones et al. (2014) we use these winter isotope reconstructions to show the expected inverse correlation (due to the NAO) with winter-season documentary reconstructions from the Netherlands and Sweden over the last 800 years. Finally, in this section Jones et al. (2013) shows the agreement between tree-ring width measurements from Northern Sweden and Finland and an assessment of the link to explosive volcanic eruptions. An instrumental record for the region in the early 19th century indicates that the summer of 1816 was only slightly below normal, explaining why this year has normal growth for both ring width and density. GCM/RCM/Reanalysis Evaluation In this section we have intercompared daily temperature extremes across Europe in Cornes and Jones (2013) using station data, E-OBS and ERA-Interim. We have additionally considered the impact of the urban issue on the global scale using the results of the Compo et al. (2011) Reanalyses, 20CR. These only make use of SST and station pressure data. Across the world’s land areas, they indicate similar warming since 1900 to that which has occurred (Compo et al., 2013), again illustrating that urbanization is not the cause of the long-term warming. Changes in HadCRUH global land surface specific humidity and CRUTEM3 surface temperatures from 1973 to 1999 were compared to the CMIP3 archive of climate model simulations with 20th Century forcings (Willett et al., 2010). The models reproduce the magnitude of observed interannual variance over all large regions. Observed and modelled trends and temperature-humidity relationships are comparable with the exception of the extra-tropical Southern Hemisphere where observations exhibit no trend but models exhibit moistening.

54 ENVIRONMENTAL SCIENCES↗

Why High Performance Visual Data Analytics is both Relevant and Difficult

Data visualization, as well as data analysis and data analytics, are all an integral part of the scientific process. Collectively, these technologies provide the means to gain insight into data of ever-increasing size and complexity. Over the past two decades, a substantial amount of visualization, analysis, and analytics R&D has focused on the challenges posed by increasing data size and complexity, as well as on the increasing complexity of a rapidly changing computational platform landscape. While some of this research focuses on solely on technologies, such as indexing and searching or novel analysis or visualization algorithms, other R&D projects focus on applying technological advances to specific application problems. Some of the most interesting and productive results occur when these two activities R&D and application are conducted in a collaborative fashion, where application needs drive R&D, and R&D results are immediately applicable to real world problems.

97 MATHEMATICS AND COMPUTING↗

Elastic Changepoint Detection for Globally-indexed Functional Time Series Data with Climate Applications

Changepoint detection is a vital tool in the application of climate data analysis. Numerous types of climate observation data are most properly represented by functional time series, implying a need for accurate changepoint detection methods applicable to functional time series data. Such data taken at a global scale often contain both spatial heterogeneity and dependence as well as phase (time) misalignment. In this report, we present methods which can detect spatially-dependent changepoints while allowing different estimates of change time and change strength depending on location. Additionally, we provide extensions to this spatially-predicted model which controls for phase variability among observations. Our methods provide the ability to detect a single change, or control for epidemic changes (where a “return-to-normal” change is more likely to be detected than the initial change). We showcase results analyzing the June 1991 eruption of Mt. Pinatubo, where our methods demonstrate the ability to accurately detect both single and epidemic changepoints even in the presence of strong seasonal variability. We find that our spatially-predicted model improves the detection of relevant changepoints versus methods which do not take spatial information into account, and we find that controlling for phase variability helps to control the false discovery rate during the detection process.

54 ENVIRONMENTAL SCIENCES↗