Search NASA⌕ Search

SEARCH · Search NASA

Results for “data and knowledge visualization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22

EXO-DAT: AN INFORMATION SYSTEM IN SUPPORT OF THE CoRoT/EXOPLANET SCIENCE

Exo-Dat is a database and an information system created primarily in support of the exoplanet program of the COnvection ROtation and planetary Transits (CoRoT) mission. In the directions of CoRoT pointings, it provides a united interface to several sets of data: stellar published catalogs, photometric and spectroscopic data obtained during the mission preparation, results from the mission and from follow-up observations, and several mission-specific technical parameters. The new photometric data constitute the subcatalog Exo-Cat, and give consistent 4-color photometry of 14.0 million stars with a completeness to 19th magnitude in the r-filter. It covers several zones in the galactic plane around CoRoT pointings, with a total area of 209 deg{sup 2}. This Exo-Dat information system provides essential technical support to the ongoing CoRoT light-curve analyses and ground-based follow-up by supplying additional complementary information such as the prior knowledge of the star's fundamental parameters or its contamination level inside the large CoRoT photometric mask. The database is fully interfaced with VO tools and thus benefits from existing visualization and analysis tools like TOPCAT or ALADIN. It is accessible to the CoRoT community through the Web, and will be gradually opened to the public. It is the ideal tool to prepare the foreseen statistical studies of the properties of the exoplanetary systems. As a VO-compliant system, such analyses could thus benefit from the most up-to-date classifier tools.

79 ASTRONOMY AND ASTROPHYSICS↗

Numerical Modeling of the Lake Mary Road Bridge for Foundation Reuse Assessment

This project uses numerical techniques to assess the structural integrity and capacity of the bridge foundations and, as a result, reduces the risk associated with reusing the same foundation for a new superstructure. Nondestructive test methods of different types were used in combination with the numerical modeling and analysis. The onsite tests included visual inspection, tomography, ground penetrating radar, drilling boreholes and coreholes, and the laboratory tests on recovered samples. The results were utilized to identify the current geometry of the structure with foundation, including the hidden geometry of the abutments and piers, and soil and foundation material properties. This data was used to build the numerical models and run computational analyses on a high performance computer cluster to assess the structural integrity of the bridge and foundations including the suitability of the foundation for reuse with a new superstructure and traffic that will increase the load on the foundations. Computational analysis is more cost-effective and gives an advantage of getting more detailed knowledge about the structural response. It also enables to go beyond non-destructive testing and find the failure conditions without destroying the structure under consideration.

foundation reuse↗

Toward Understanding Tip Leakage Flows in Small Compressor Cores Including Stator Leakage Flow

The focus of this work was to provide additional data to supplement the work reported in NASA/CR-2015-218868 (Berdanier and Key, 2015b). The aim of that project was to characterize the fundamental flow physics and the overall performance effects due to increased rotor tip clearance heights in axial compressors. Data have been collected in the three-stage axial research compressor at Purdue University with a specific focus on analyzing the multistage effects resulting from the tip leakage flow. Three separate rotor tip clearances were studied with nominal tip clearance gaps of 1.5 percent, 3.0 percent, and 4.0 percent based on a constant annulus height. Overall compressor performance was previously investigated at four corrected speedlines (100 percent, 90 percent, 80 percent, and 68 percent) for each of the three tip clearance configurations. This study extends the previously published results to include detailed steady and time-resolved pressure data at two loading conditions, nominal loading (NL) and high loading (HL), on the 100 percent corrected speedline for the intermediate clearance level (3.0 percent). Steady detailed radial traverses of total pressure at the exit of each stator row are supported by flow visualization techniques to identify regions of flow recirculation and separation. Furthermore, detailed radial traverses of time-resolved total pressures at the exit of each rotor row have been measured with a fast-response pressure probe. These data were combined with existing three-component velocity measurements to identify a novel technique for calculating blockage in a multistage compressor. Time-resolved static pressure measurements have been collected over the rotor tips for all rotors with each of the three tip clearance configurations for up to five loading conditions along the 100 percent corrected speedline using fast-response piezoresistive pressure sensors. These time-resolved static pressure measurements reveal new knowledge about the trajectory of the tip leakage flow through the rotor passage. Further, these data extend previous measurements identifying a modulation of the tip leakage flow due to upstream stator wake propagation. Finally, a novel instrumentation technique has been implemented to measure pressures in the shrouded stator cavities. These data provide boundary conditions relating to the flow across the shrouded stator knife seal teeth. Moreover, the utilization of fast-response pressure sensors provides a new look at the time-resolved pressure field, leading to instantaneous differential pressures across the seal teeth. Ultimately, the data collected for this project represent a unique data set which contributes to build a better understanding of the tip leakage flow field and its associated loss mechanisms. These data will facilitate future engine design goals leading to small blade heights in the rear stages of high pressure compressors and aid in the development of new blade designs which are desensitized to the performance penalties attributed to rotor tip leakage flows.

Berdanier, Reid A.↗

Chimbuko: A Workflow-Level Scalable Performance Trace Analysis Tool

Due to the sheer volume of data it is typically impractical to analyze the detailed performance of an HPC application running at-scale. While conventional small-scale benchmarking and scaling studies are often sufficient for simple applications, many modern workflow-based applications couple multiple elements with competing resource demands and complex inter-communication patterns for which performance cannot easily be studied in isolation and at small scale. This work discusses Chimbuko, a performance analysis framework that provides real-time, in situ anomaly detection. By focusing specifically on performance anomalies and their origin (aka provenance), data volumes are dramatically reduced without losing necessary details. To the best of our knowledge, Chimbuko is the first online, distributed, and scalable workflow-level performance trace analysis framework. We demonstrate the tool's usefulness on Oak Ridge National Laboratory's Summit system.

97 MATHEMATICS AND COMPUTING↗

A measure of active interfaces in supported catalysts for high-temperature reactions

Formulating knowledge of structure-function relationships in heterogeneous catalysis is central to the design of efficient catalysts; yet, the elucidation of dominant reaction sites has remained as a challenge. Here, in this paper, we present a methodology that can be used to visualize metal-gas and metal-oxide-gas interfaces in three dimensions and to quantify their catalytic activity levels. As a case study, CH 4 oxidation occurring in a Pt/CeO 2 system is chosen. By employing thermally robust Pt@CeO 2 model catalysts with size-tunable and monodisperse cores, and gas-permeable shells, we reconstruct a series of structures in 3D via electron tomography and match the information to activity data and theoretical calculations. This strategy reveals that two different interfaces catalyze the CH 4 oxidation and that their contribution to the overall rate changes with the Pt size, temperature, and gas atmosphere. Our results provide an analytic platform on which to explore reaction pathways and mechanisms applicable to multiple reactions and materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Using Machine Learning to Identify Optimum Prospects for Offshore Geologic CO 2 Storage and Enhanced Oil Recovery in the Gulf of Mexico

The SECARB Offshore Partnership is a government-industry partnership focused on assembling the knowledge base required for secure, long-term, large-scale CO 2 subsea storage in saline formations and mature oil reservoirs. The SECARB Offshore Partnership is evaluating potential storage opportunities in the Central and Eastern planning areas of the outer continental shelf (OCS) of the Gulf of Mexico. The evaluation focuses on active and depleted oil and gas fields and potentially associated CO 2 -enhanced oil recovery (CO 2 -EOR), as well as, deep saline storage resources. As part of Subtask 3.0 (Offshore Storage Resources Characterization) and Subtask 4.0 (Risk Assessment, Simulation, and Modeling), Oklahoma State University (OSU) is developing a machine learning system that employs the SAS Institute’s Viya platform with visual data mining and machine learning to identify and characterize optimum prospects for enhanced oil recovery and stacked storage in saline reservoirs. This report summarizes this effort to date and discusses the many variables that can be used to characterize these storage objectives, as well as, the design vision of the Viya machine learning system.

02 PETROLEUM↗

WHONDRS-GUI: a web application for global survey of surface water metabolites

Background The Worldwide Hydrobiogeochemistry Observation Network for Dynamic River Systems (WHONDRS) is a consortium that aims to understand complex hydrologic, biogeochemical, and microbial connections within river corridors experiencing perturbations such as dam operations, floods, and droughts. For one ongoing WHONDRS sampling campaign, surface water metabolite and microbiome samples are collected through a global survey to generate knowledge across diverse river corridors. Metabolomics analysis and a suite of geochemical analyses have been performed for collected samples through the Environmental Molecular Sciences Laboratory (EMSL). The obtained knowledge and data package inform mechanistic and data-driven models to enhance predictions of outcomes of hydrologic perturbations and watershed function, one of the most critical components in model-data integration. To support efforts of the multi-domain integration and make the ever-growing data package more accessible for researchers across the world, a Shiny/R Graphical User Interface (GUI) called WHONDRS-GUI was created. Results The web application can be run on any modern web browser without any programming or operational system requirements, thus providing an open, well-structured, discoverable dataset for WHONDRS. Together with a context-aware dynamic user interface, the WHONDRS-GUI has functionality for searching, compiling, integrating, visualizing and exporting different data types that can easily be used by the community. The web application and data package are available at https://data.ess-dive.lbl.gov/view/doi:10.15485/1484811 , which enables users to simultaneously obtain access to the data and code and to subsequently run the web app locally. The WHONDRS-GUI is also available for online use at Shiny Server ( https://xmlin.shinyapps.io/whondrs/ ).

59 BASIC BIOLOGICAL SCIENCES↗

Bridging Informatics and Earth Science: a Look at Gregory Leptoukh's Contributions

With the tragic passing this year of Gregory Leptoukh, the Earth and Space Sciences community lost a tireless participant in--and advocate for--science informatics. Throughout his career at NASA, Dr. Leptoukh established a theme of bridging the gulf between the informatics and science communities. Nowhere is this more evident than his leadership in the development of Giovanni (GES DISC Interactive Online Visualization ANd aNalysis Infrastructure). Giovanni is an online tool that serves to hide the often-complex technical details of data format and structure, making science data easier to explore and use by Earth scientists. To date Giovanni has been acknowledged as a contributor in 500-odd scientific articles. In recent years, Leptoukh concentrated his efforts on multi-sensor data inter-comparison, merging and fusion. This work exposed several challenges at the intersection of data and science. One of these was the ease with which a naive user might generate spurious comparisons, a potential hazard that was the genesis of the Multi-sensor Data Synergy Advisor (MDSA). The MDSA uses semantic ontologies and inference rules to organize knowledge about dataset quality and other salient characteristics in order to advise users on potential caveats for comparing or merging two datasets. Recently, Leptoukh also led the development of AeroStat, an online Giovanni instance to investigate aerosols via statistics from station and satellite comparisons and merged maps of data from more than one instrument. Aerostat offers a neural net based bias adjustment to harmonize the data by removing systematic offsets between datasets before merging. These examples exhibit Leptoukh's talent for adopting advanced computer technologies in the service of making science data more accessible to researchers. In this, he set an example that is at once both vital and challenging for the ESSI community to emulate.

Source record↗

Intelligent Process Visualization through Nuclear Operation Process Modeling, Reasoning, and Object Detection from Field Videos (Final Report)

This report is a deliverable for the “Final Report” task of DOE NEET Project 19-16790, "Context-Aware Safety Information Display for Nuclear Field Workers." This project's overall goal is to test the hypothesis that integrating computer vision and process reasoning methods will enable proactive visualization of the safe operation and maintenance processes of Nuclear Power Plants (NPP) for field workers. Augmented Reality (AR) glasses adopting such proactive safety information visualization techniques can significantly increase personnel safety and reduce the NPP’s operating costs. The current practice of monitoring NPPs requires workers to switch between digital models, data, and physical workspaces in identifying relevant but potentially occluded objects and in assessing the risks of operation and maintenance processes. On the other hand, frequently changed field conditions require field workers to report to supervisors for real-time guidance. Such guidance is essential to ensure that changing conditions will not invalidate or endanger the work order and other ongoing processes that may jeopardize NPP operations. Additionally, incorrect recognition of equipment objects can result in communication errors and safety problems. AR techniques can assist engineers in viewing the physical workspaces with objects labeled with detailed operation procedures and safety reminders during field operations. The project team developed an “Intelligent Context-Aware Safety Information Display” (ICAD) for supporting Nuclear Power Plant (NPP) field workers in achieving safe and efficient execution of a series of operational tasks in uncertain and changing workspaces of an NPP. Before designing the ICAD-AR prototype, the project team synthesized NPP operational knowledge models through literature review studies, surveys, interviews with domain experts, and knowledge modeling. The project team conducted an extensive study of the operational procedures of various NPPs, and digital technologies that can support the safe and efficient execution of those procedures in different NPP operational contexts. This literature review helped the project team conduct surveys and interviews with nuclear engineers and field workers to identify three categories of information. The NPP knowledge modeling efforts reveal that the three categories of information identified have different levels of importance in a typical procedure of carrying out a series of tasks to achieve a specific NPP operation goal (e.g., shutdown, mode changes). These three categories of information include 1) Workspace dynamics – the changing spatial arrangements of workspaces, tools, protection equipment, and supporting materials, 2) Workflow prognostics – the dynamic dependencies between different parts of an NPP that functionally support and influence each other in terms of safety and efficiency, and 3) Hazards – objects and spaces that contain hazardous materials or physical conditions that can pose risks to workers or mechanical systems. The project team has profiled the importance levels of these categories of information into a knowledge model. This knowledge model specifies what types of information are more critical for a given task in a given workspace so that computers can automatically identify critical objects and sensors in a scene for delivering context-ware safety information to field workers through AR devices. Significant research development of this project results in technical research outcomes and a prototyping system that illustrates the technical feasibility of establishing an ICAD-AR system supporting the proactive safety information display for nuclear field workers. This final report summarizes the project team’s technological achievements in the past three years. Overall, the project team completed the development and integration of five techniques into a prototype ICAD Augmented Reality (ICAD-AR) system and demonstrated the developed system’s real-time execution in a mechanical room. The project team completed the analysis of using this prototype in other types of workspaces based on 3D image data and digital design models collected from two additional workspaces (a water treatment plant and a flow loop training facility). The integrated techniques include 1) Natural Language Processing (NLP) algorithms supporting the generation and updates of nuclear fieldwork process models based on text analysis of work packages and operation manuals; 2) sensor log analysis for predicting control actions in given sensor reading contexts; 3) computer vision algorithms for automatic localization and navigation of workers; 4) object detection algorithms for identifying task-related objects and correlated sensors for safety checking; 5) AR technique as a platform for supporting the integration. The testing results of these five techniques have shown that 1) the sensor log analysis model can predict the next control action with an accuracy of 0.883; 2) the trained natural language processing model can extract more than 80% of the critical information from paper-based procedures (PBPs); 3) the navigation algorithm with the integration of Visual Inertial Odometry (VIO) and Non-Recursive Bayesian Filter methods make operator’s trajectory estimation resilient to drift error; 4) the computer vision algorithm can detect task-specific and safety-critical objects with an average accuracy of 95.3%. The project team used work procedures collected from a flow loop training facility and two datasets collected from two mechanical rooms simulating the workspaces of NPPs to demonstrate the technical capabilities of the developed ICAD-AR prototype. The demonstration validated the technical feasibility of establishing the ICAD-AR system for nuclear field workers and identified the challenges in 1) automatic text analysis of work packages; 2) use of limited samples of sensor logs for predicting the proper timings of control actions; 3) reliably tracking workers and their task progress in mechanical rooms with many similar objects.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Interface Metaphors for Interactive Machine Learning

To promote more interactive and dynamic machine learn- ing, we revisit the notion of user-interface metaphors. User-interface metaphors provide intuitive constructs for supporting user needs through interface design elements. A user-interface metaphor provides a visual or action pattern that leverages a user’s knowledge of another domain. Metaphors suggest both the visual representations that should be used in a display as well as the interactions that should be afforded to the user. We argue that user-interface metaphors can also offer a method of extracting interaction-based user feedback for use in machine learning. Metaphors offer indirect, context-based information that can be used in addition to explicit user inputs, such as user-provided labels. Implicit information from user interactions with metaphors can augment explicit user input for active learning paradigms. Or it might be leveraged in systems where explicit user inputs are more challenging to obtain. Each interaction with the metaphor provides an opportunity to gather data and learn. We argue this approach is especially important in streaming applications, where we desire machine learning systems that can adapt to dynamic, changing data.

Interactive machine learning↗

HIDES: Hybrid Intrusion Detector for Energy Systems

The establishment of a resilient electric grid accompanied by a secure communications network is an ongoing battle as advanced persistent threats continue to exploit existing vulnerabilities in legacy supervisory control and data acquisition system (SCADA) infrastructure. Traditional intrusion detection systems (IDSs) lack consistent performance because of the continuously evolving attack surface of SCADA systems. These shortcomings can be overcome by integrating logical system behavior, protocol-specific knowledge, and data-based learning to develop a comprehensive IDS solution. In this paper, we present a Hybrid Intrusion Detector for Energy Systems by integrating a network-based IDS, state-of-the-art machine learning-based IDS, and model-based IDS to detect unknown and stealthy cyberattacks targeting the SCADA networks. The proposed IDS uses synchrophasor measurements and cyber logs to learn patterns of different scenarios based on spatiotemporal behaviors of power systems. As a proof of concept, we implement and validate the proposed IDS by leveraging resources available at the National Renewable Energy Laboratory's Energy Systems Integration Facility test bed. Experimental results show promising performance in detecting cyberattacks while providing realtime visualization of power system measurements and cyber logs.

machine-learning intrusion detection system↗

Understanding protein-complex assembly through grand canonical maximum entropy modeling

Inside a cell, heterotypic proteins assemble in inhomogeneous, crowded systems where the abundance of these proteins vary with cell types. While some protein complexes form putative structures that can be visualized with imaging, there are far more protein complexes that are yet to be solved because of their dynamic associations with one another. Nevertheless, it is possible to infer these protein complexes through a physical model. However, it is often not clear to physicists what kind of data from biology is necessary for such a modeling endeavor. Here, we aim to model these clusters of coarse-grained protein assemblies from multiple subunits through the constraints of interactions among the subunits and the chemical potential of each subunit. We obtained the constraints on the interactions among subunits from the known protein structures. We inferred the chemical potential that dictates the particle number distribution of each protein subunit from the knowledge of protein abundance from experimental data. Guided by the maximum entropy principle, we formulated an inverse statistical mechanical method to infer the distribution of particle numbers from the data of protein abundance as chemical potentials for a grand canonical multicomponent mixture. Using grand canonical Monte Carlo simulations, we captured a distribution of high-order clusters in a protein complex of succinate dehydrogenase with four known subunits. The complexity of hierarchical clusters varies with the relative protein abundance of each subunit in distinctive cell types such as lung, heart, and brain. When the crowding content increases, we observed that crowding stabilizes emergent clusters that do not exist in dilute conditions. We, therefore, proposed a testable hypothesis that the hierarchical complexity of protein clusters on a molecular scale is a plausible biomarker of predicting the phenotypes of a cell.

59 BASIC BIOLOGICAL SCIENCES↗

NASA GeneLab: Open Science for Life in Space

NASA’s GeneLab helps scientists understand how the fundamental building blocks of life – DNA, RNA, proteins, and metabolites – change from exposure to the space environment including microgravity and cosmic radiation exposure. GeneLab does so by providing fully coordinated epigenomics, genomics, transcriptomics, proteomics, and metabolomics data (collectively known as omics data) alongside essential metadata describing each spaceflight and space-relevant experiment. The open-access GeneLab repository currently consists of over 300 omics datasets generated by biological experiments, involving various model organisms, that are relevant to spaceflight. In order to maximize the intelligibility of these data, particularly for users with limited bioinformatics knowledge, GeneLab has started processing and analyzing these datasets to generate differential gene expression data and identify biological and physiological pathways that are dysregulated as a result of spaceflight. To aide GeneLab’s efforts to harmonize and democratize space-relevant omics data, over 130 scientists have joined one of four GeneLab Analysis Working Groups (Animal AWG, Plant AWG, Microbe AWG, Multi-Omics AWG) and together helped develop and adopted standard data analysis workflows for all data types available in GeneLab. Currently, the GeneLab Data System includes a data repository with federated search capability, an online controlled-access toolshed powered by "Galaxy" for users to process data with vetted standard workflows, a workspace for data sharing, a data submission portal, and the ability to browse and visualize transcriptomics processed data. The user interface was designed to be accessible to a broad variety of users, including high school and college students who can use it to learn about omics data analysis and space biology. The visualization portal enhances GeneLab’s ability to democratize omics data by removing the need for bioinformatics expertise to interpret transcriptomics data hosted on GeneLab. This presentation will provide an over-view of NASA’s GeneLab including how to navigate the GeneLab Data System and will conclude by providing resources for opportunities to work with GeneLab and NASA at large.

Amanda M Saravia-Butler↗

GRDC. A Collaborative Framework for Radiological Background and Contextual Data Analysis

The Radiation Mobile Analysis Platform (RadMAP) is unique in its capability to collect both high quality radiological data from both gamma-ray detectors and fast neutron detectors and a broad array of contextual data that includes positioning and stance data, high-resolution 3D radiological data from weather sensors, LiDAR, and visual and hyperspectral cameras. The datasets obtained from RadMAP are both voluminous and complex and require analyses from highly diverse communities within both the national laboratory and academic communities. Maintaining a high level of transparency will enable analysis products to further enrich the RadMAP dataset. It is in this spirit of open and collaborative data that the RadMAP team proposed to collect, calibrate, and make available online data from the RadMAP system. The Berkeley Data Cloud (BDC) is a cloud-based data management framework that enables web-based data browsing visualization, and connects curated datasets to custom workflows such that analysis products can be managed and disseminated while maintaining user access rights. BDC enables cloud-based analyses of large datasets in a manner that simulates real-time data collection, such that BDC can be used to test algorithm performance on real and source-injected datasets. Using the BDC framework, a subset of the RadMAP datasets have been disseminated via the Gamma Ray Data Cloud (GRDC) that is hosted through the National Energy Research Science Computing (NERSC) Center, enabling data access to over 40 users at 10 institutions.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Grist : grid-based data mining for astronomy

The Grist project is developing a grid-technology based system as a research environment for astronomy with massive and complex datasets. This knowledge extraction system will consist of a library of distributed grid services controlled by a workflow system, compliant with standards emerging from the grid computing, web services, and virtual observatory communities. This new technology is being used to find high redshift quasars, study peculiar variable objects, search for transients in real time, and fit SDSS QSO spectra to measure black hole masses. Grist services are also a component of the 'hyperatlas' project to serve high-resolution multi-wavelength imagery over the Internet. In support of these science and outreach objectives, the Grist framework will provide the enabling fabric to tie together distributed grid services in the areas of data access, federation, mining, subsetting, source extraction, image mosaicking, statistics, and visualization.

grid computing↗

SciDAC's Earth System Grid Center for Enabling Technologies Semiannual Progress Report October 1, 2010 through March 31, 2011

This report summarizes work carried out by the Earth System Grid Center for Enabling Technologies (ESG-CET) from October 1, 2010 through March 31, 2011. It discusses ESG-CET highlights for the reporting period, overall progress, period goals, and collaborations, and lists papers and presentations. To learn more about our project and to find previous reports, please visit the ESG-CET Web sites: http://esg-pcmdi.llnl.gov/ and/or https://wiki.ucar.edu/display/esgcet/Home. This report will be forwarded to managers in the Department of Energy (DOE) Scientific Discovery through Advanced Computing (SciDAC) program and the Office of Biological and Environmental Research (OBER), as well as national and international collaborators and stakeholders (e.g., those involved in the Coupled Model Intercomparison Project, phase 5 (CMIP5) for the Intergovernmental Panel on Climate Change (IPCC) 5th Assessment Report (AR5); the Community Earth System Model (CESM); the Climate Science Computational End Station (CCES); SciDAC II: A Scalable and Extensible Earth System Model for Climate Change Science; the North American Regional Climate Change Assessment Program (NARCCAP); the Atmospheric Radiation Measurement (ARM) program; the National Aeronautics and Space Administration (NASA), the National Oceanic and Atmospheric Administration (NOAA)), and also to researchers working on a variety of other climate model and observation evaluation activities. The ESG-CET executive committee consists of Dean N. Williams, Lawrence Livermore National Laboratory (LLNL); Ian Foster, Argonne National Laboratory (ANL); and Don Middleton, National Center for Atmospheric Research (NCAR). The ESG-CET team is a group of researchers and scientists with diverse domain knowledge, whose home institutions include eight laboratories and two universities: ANL, Los Alamos National Laboratory (LANL), Lawrence Berkeley National Laboratory (LBNL), LLNL, NASA/Jet Propulsion Laboratory (JPL), NCAR, Oak Ridge National Laboratory (ORNL), Pacific Marine Environmental Laboratory (PMEL)/NOAA, Rensselaer Polytechnic Institute (RPI), and University of Southern California, Information Sciences Institute (USC/ISI). All ESG-CET work is accomplished under DOE open-source guidelines and in close collaboration with the project's stakeholders, domain researchers, and scientists. Through the ESG project, the ESG-CET team has developed and delivered a production environment for climate data from multiple climate model sources (e.g., CMIP (IPCC), CESM, ocean model data (e.g., Parallel Ocean Program), observation data (e.g., Atmospheric Infrared Sounder, Microwave Limb Sounder), and analysis and visualization tools) that serves a worldwide climate research community. Data holdings are distributed across multiple sites including LANL, LBNL, LLNL, NCAR, and ORNL as well as unfunded partners sites such as the Australian National University (ANU) National Computational Infrastructure (NCI), the British Atmospheric Data Center (BADC), the Geophysical Fluid Dynamics Laboratory/NOAA, the Max Planck Institute for Meteorology (MPI-M), the German Climate Computing Centre (DKRZ), and NASA/JPL. As we transition from development activities to production and operations, the ESG-CET team is tasked with making data available to all users who want to understand it, process it, extract value from it, visualize it, and/or communicate it to others. This ongoing effort is extremely large and complex, but it will be incredibly valuable for building 'science gateways' to critical climate resources (such as CESM, CMIP5, ARM, NARCCAP, Atmospheric Infrared Sounder (AIRS), etc.) for processing the next IPCC assessment report. Continued ESG progress will result in a production-scale system that will empower scientists to attempt new and exciting data exchanges, which could ultimately lead to breakthrough climate science discoveries.

54 ENVIRONMENTAL SCIENCES↗

Build Less Code, Deliver More Science: An Experience Report on Composing Scientific Environments using Component-based and Commodity Software Platforms

Modern scientific software is daunting in its diversity and complexity. From massively parallel simulations running on the world’s largest supercomputers, to visualizations and user support environments that manage ever growing complex data collections, the challenges for software engineers are plentiful. While high performance simulators are necessarily specialized codes to maximize performance on specific supercomputer architectures, we argue the vast majority of supporting infrastructure, data management and analysis tools can leverage commodity open source and component-based technologies. This approach can significantly drive down the effort and costs of building complex, collaborative scientific user environments, as well as increase their reliability and extensibility. In this paper we describe our experiences in creating an initial user environment for scientists involved in modeling the detailed effects of climate change on the environment of selected geographical regions. Our approach composes the user environment using the Velo scientific knowledge management platform and the MeDICi Integration Framework for scientific workflows. These established platforms leverage component-based technologies and extend commodity open source platforms with abstractions and capabilities that make them amenable for broad use in science. Using this approach we were able to deliver an operational user environment capable of running thousands of simulations in a 7 month period, and achieve significant software reuse.

Gorton, Ian↗

GeoNEX: A Cloud Gateway for Near Real-time Processing of Geostationary Satellite Products

The emergence of a new generation of geostationary satellite sensors provides land andatmosphere monitoring capabilities similar to MODIS and VIIRS with far greater temporal resolution (5-15 minutes). However, processing such large volume, highly dynamic datasets requires computing capabilities that (1) better support data access and knowledge discovery for scientists; (2) provide resources to enable real-time processing for emergency response (wildfire, smoke, dust, etc.); and (3) provide reliable and scalable services for the broader user community. This paper presents an implementation of GeoNEX (Geostationary NASA-NOAA Earth Exchange) services that integrate scientific algorithms with Amazon Web Services (AWS) to provide near realtime monitoring (~5 minute latency) capability in a hybrid cloud-computing environment. It offers a user-friendly, manageable and extendable interface and benefits from the scalability provided by Amazon Web Services. Four use cases are presented to illustrate how to (1) search and access geostationary data; (2) configure computing infrastructure to enable near real-time processing; (3) disseminate and utilize research results, visualizations, and animations to concurrent users; and (4) use a Jupyter Notebook-like interface for data exploration and rapid prototyping. As an example of (3), the Wildfire Automated Biomass Burning Algorithm (WF_ABBA) was implemented on GOES-16 and -17 data to produce an active fire map every 5 minutes over the conterminous US. Details of the implementation strategies, architectures, and challenges of the use cases are discussed.

GeoNEX↗