Search NASA⌕ Search

SEARCH · Search NASA

Results for “Scientific data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

Science Operations of the International Ultraviolet Explorer (IUE) Observatory

The fundamental operational objective of the International Ultraviolet Explorer (IUE) program is to support competitively selected astronomical research program. Through the IUE program, researchers make IUE observations, have their scientific data reduced in a meaningful way, and receive data products in a form amenable to the pursuit of scientific research. The IUE Observatory is key to the program since it is the central control and support facility for all science support functions within the IUE project.

Source record↗

Validation of Orthorectified Interferometric Radar Imagery and Digital Elevation Models

This work was performed under NASA's Verification and Validation (V&V) Program as an independent check of data supplied by EarthWatch, Incorporated, through the Earth Science Enterprise Scientific Data Purchase (SDP) Program. This document serves as the basis of reporting results associated with validation of orthorectified interferometric interferometric radar imagery and digital elevation models (DEM). This validation covers all datasets provided under the first campaign (Central America & Virginia Beach) plus three earlier missions (Indonesia, Red River: and Denver) for a total of 13 missions.

Smith Charles M.↗

ICER-3D Hyperspectral Image Compression Software

Software has been developed to implement the ICER-3D algorithm. ICER-3D effects progressive, three-dimensional (3D), wavelet-based compression of hyperspectral images. If a compressed data stream is truncated, the progressive nature of the algorithm enables reconstruction of hyperspectral data at fidelity commensurate with the given data volume. The ICER-3D software is capable of providing either lossless or lossy compression, and incorporates an error-containment scheme to limit the effects of data loss during transmission. The compression algorithm, which was derived from the ICER image compression algorithm, includes wavelet-transform, context-modeling, and entropy coding subalgorithms. The 3D wavelet decomposition structure used by ICER-3D exploits correlations in all three dimensions of sets of hyperspectral image data, while facilitating elimination of spectral ringing artifacts, using a technique summarized in "Improving 3D Wavelet-Based Compression of Spectral Images" (NPO-41381), NASA Tech Briefs, Vol. 33, No. 3 (March 2009), page 7a. Correlation is further exploited by a context-modeling subalgorithm, which exploits spectral dependencies in the wavelet-transformed hyperspectral data, using an algorithm that is summarized in "Context Modeler for Wavelet Compression of Hyperspectral Images" (NPO-43239), which follows this article. An important feature of ICER-3D is a scheme for limiting the adverse effects of loss of data during transmission. In this scheme, as in the similar scheme used by ICER, the spatial-frequency domain is partitioned into rectangular error-containment regions. In ICER-3D, the partitions extend through all the wavelength bands. The data in each partition are compressed independently of those in the other partitions, so that loss or corruption of data from any partition does not affect the other partitions. Furthermore, because compression is progressive within each partition, when data are lost, any data from that partition received prior to the loss can be used to reconstruct that partition at lower fidelity. By virtue of the compression improvement it achieves relative to previous means of onboard data compression, this software enables (1) increased return of hyperspectral scientific data in the presence of limits on the rates of transmission of data from spacecraft to Earth via radio communication links and/or (2) reduction in spacecraft radio-communication power and/or cost through reduction in the amounts of data required to be downlinked and stored onboard prior to downlink. The software is also suitable for compressing hyperspectral images for ground storage or archival purposes.

Xie, Hua↗

The Use of Handheld X-Ray Fluorescence (XRF) Technology in Unraveling the Eruptive History of the San Francisco Volcanic Field, Arizona

While traditional geologic mapping includes the examination of structural relationships between rock units in the field, more advanced technology now enables us to simultaneously collect and combine analytical datasets with field observations. Information about tectonomagmatic processes can be gleaned from these combined data products. Historically, construction of multi-layered field maps that include sample data has been accomplished serially (first map and collect samples, analyze samples, combine data, and finally, readjust maps and conclusions about geologic history based on combined data sets). New instruments that can be used in the field, such as a handheld xray fluorescence (XRF) unit, are now available. Targeted use of such instruments enables geologists to collect preliminary geochemical data while in the field so that they can optimize scientific data return from each field traverse. Our study tests the application of this technology and projects the benefits gained by real-time geochemical data in the field. The integrated data set produces a richer geologic map and facilitates a stronger contextual picture for field geologists when collecting field observations and samples for future laboratory work. Real-time geochemical data on samples also provide valuable insight regarding sampling decisions by the field geologist

Young, Kelsey E.↗

Open-Source Data Engineering at NASA: CCMC's Approach to Managing Petabyte-Scale Heliophysics Data

The Community Coordinated Modeling Center (CCMC) at NASA Goddard Space Flight Center (GSFC) leads heliophysics research by providing open access to numerous models and their outputs. Our resources are available on-demand and continuously updated with real-time data, covering sun-earth interactions across multiple domains. These domains include coronal, heliosphere, inner and global magnetosphere, ionosphere, thermosphere, and lower atmosphere interactions. Operating in a hybrid environment, CCMC utilizes both self-owned hardware and Amazon Web Services (AWS) cloud infrastructure. Managing petabytes of data across multiple locations necessitates robust data engineering solutions. To address this challenge, CCMC has adopted industry-standard and open-source tools. We use Apache Airflow as our primary data engineering platform, Python for scripting and data processing, and GitLab for version control and CI/CD. Additionally, we employ Kubernetes for containerized services, Grafana and Prometheus for metrics and monitoring, and Terraform and Puppet for reproducible infrastructure as code. This presentation will discuss lessons learned from our data engineering experiences, platforms evaluated but found unsuitable for our scientific data requirements, and specific techniques developed to enhance data transfer speed and reliability. By using these technologies effectively, CCMC continues to advance heliophysics research through efficient data management and open-access modeling.

space weather↗

Active workflow system for near real-time extreme-scale science. In: PPAA '14 Proceedings of the first workshop on Parallel programming for analytics applications

In recent years, streaming-based data processing has been gaining substantial traction for dealing with overwhelming data generated by real-time applications, from both enterprise sources and scientific computing. In this work, however, we look at an emerging class of scientific data with Near Real-Time (NRT) requirement, in which data is typically generated in a bursty fashion with the near real-time constraints being applied primarily between bursts, rather than within a stream. A key challenge for this types of data sources is that the processing time per data element is not uniform, and not always feasible to predict. Given the observations on the increasing unpredictability of compute load and system dynamics, this work looks to adapt streaming-based approach to the context of this new class of large experiments and simulations that have complex run-time control and analysis issues. In particular, we deploy a novel two-tier scheme for handling the increasing unpredictability of runtime behaviors: Instead of relying on determining what and where to run the scientific workflows beforehand or partial dynamically, the decision will also be adaptively enhanced online according to system runtime status. This is enabled by embedding workflow along with data streams. Specifically, we break data outputs generated from experiments or simulations into multiple self-describing "chunks", which we call active data objects. As such, if there is a transient hotspot observed, a data object with unfinished workflow pipeline can break its previous schedule and search for a least loaded location to continue the execution. Our preliminary experiment results based on synthetic workloads demonstrate the proposed active workflow system as a very promising solution by outperforming the state-of-the-art semi-dynamic workflow schedulers with an improved workflow completion time, as well as a good scalability.

Zhang, Yanwei↗

Development of a Web-Based Visualization Platform for Climate Research Using Google Earth

Recently, it has become easier to access climate data from satellites, ground measurements, and models from various data centers, However, searching. accessing, and prc(essing heterogeneous data from different sources are very tim -consuming tasks. There is lack of a comprehensive visual platform to acquire distributed and heterogeneous scientific data and to render processed images from a single accessing point for climate studies. This paper. documents the design and implementation of a Web-based visual, interoperable, and scalable platform that is able to access climatological fields from models, satellites, and ground stations from a number of data sources using Google Earth (GE) as a common graphical interface. The development is based on the TCP/IP protocol and various data sharing open sources, such as OPeNDAP, GDS, Web Processing Service (WPS), and Web Mapping Service (WMS). The visualization capability of integrating various measurements into cE extends dramatically the awareness and visibility of scientific results. Using embedded geographic information in the GE, the designed system improves our understanding of the relationships of different elements in a four dimensional domain. The system enables easy and convenient synergistic research on a virtual platform for professionals and the general public, gr$tly advancing global data sharing and scientific research collaboration.

Sun, Xiaojuan↗

Outlier analyses of the Protein Data Bank archive using a probability-density-ranking approach

Outlier analyses are central to scientific data assessments. Conventional outlier identification methods do not work effectively for Protein Data Bank (PDB) data, which are characterized by heavy skewness and the presence of bounds and/or long tails. We have developed a data-driven nonparametric method to identify outliers in PDB data based on kernel probability density estimation. Unlike conventional outlier analyses based on location and scale, Probability Density Ranking can be used for robust assessments of distance from other observations. Analyzing PDB data from the vantage points of probability and frequency enables proper outlier identification, which is important for quality control during deposition-validation-biocuration of new three-dimensional structure data. Ranking of Probability Density also permits use of Most Probable Range as a robust measure of data dispersion that is more compact than Interquartile Range. The Probability-Density-Ranking approach can be employed to analyze outliers and data-spread on any large data set with continuous distribution.

59 BASIC BIOLOGICAL SCIENCES↗

On-demand data analytics in HPC environments at leadership computing facilities: Challenges and experiences

Scaling-up scientific data analysis and machine learning algorithms for data-driven discovery is a grand challenge that we face today. Despite the growing need for analysis from science domains that are generating `Big Data' from instruments and simulations, building high-performance analytical workflows of data-intensive algorithms have been daunting because: (i) the `Big Data' hardware and software architecture landscape is constantly evolving, (ii) newer architectures impose new programming models, and (iii) data-parallel kernels of analysis algorithms and their performance facets on different architectures are poorly understood. To address these problems, we have: (i) identified scalable data-parallel kernels of popular data analysis algorithms, (ii) implemented `Mini-Apps' of those kernels using different programming models (e.g. Map Reduce, MPI, etc.), (iii) benchmarked and validated the performance of the kernels in diverse architectures. In this paper, we discuss two of those Mini-Apps and show the execution of principal component analysis built as a workflow of the Mini-Apps. We show that Mini-Apps enable scientists to (i) write domain-specific data analysis code that scales on most HPC hardware and (ii) and offers the ability (most times with over a 10x speed-up) to analyze data sizes 100 times the size of what off-the-shelf desktop/workstations of today can handle.

Harney, John↗

The Landsat Data Purchase and ESAD

The Earth Science Applications Directorate (ESAD) purchases satellite imagery for The Scientific Data Purchase (SDP) Project. SDP allows a variety of customers in turn to purchase Landsat, IKONOS, and other data. The SDP customer base includes includes private companies, universities, and government agencies. SDP customers are required to register and receive clearance.

Policelli, Fritz↗

The Virtual Climate Data Server (vCDS): An iRODS-Based Data Management Software Appliance Supporting Climate Data Services and Virtualization-as-a-Service in the NASA Center for Climate Simulation

Scientific data services are becoming an important part of the NASA Center for Climate Simulation's mission. Our technological response to this expanding role is built around the concept of a Virtual Climate Data Server (vCDS), repetitive provisioning, image-based deployment and distribution, and virtualization-as-a-service. The vCDS is an iRODS-based data server specialized to the needs of a particular data-centric application. We use RPM scripts to build vCDS images in our local computing environment, our local Virtual Machine Environment, NASA s Nebula Cloud Services, and Amazon's Elastic Compute Cloud. Once provisioned into one or more of these virtualized resource classes, vCDSs can use iRODS s federation capabilities to create an integrated ecosystem of managed collections that is scalable and adaptable to changing resource requirements. This approach enables platform- or software-asa- service deployment of vCDS and allows the NCCS to offer virtualization-as-a-service: a capacity to respond in an agile way to new customer requests for data services.

Schnase, John L.↗

Society of abdominal radiology gastrointestinal bleeding disease-focused panel consensus recommendations for CTA technical parameters in the evaluation of acute overt gastrointestinal bleeding

Purpose: To formulate consensus recommendations for CT angiography technical parameters used to evaluate overt gastrointestinal (GI) bleeding. Methods: An electronic questionnaire consisting of 17 questions was sent to a panel of 16 radiologists with expertise on the imaging of GI bleeding from the Society of Abdominal Radiology GI Bleeding disease-focused panel to obtain consensus agreement on issues related to CTA technical parameters for imaging overt GI bleeding. A multi-round Delphi method of voting was performed to obtain consensus which was defined as ≥ 80% agreement. Results: Consensus agreement was reached in 15/17 (89%) of the questions including the technique for the administration of IV contrast, the number of phases, scan timing, and image reconstruction. Conclusions: A panel of experts on the imaging of GI bleeding from the Society of Abdominal Radiology was able to reach consensus on the majority of technical parameters used for CTA of overt GI bleeding. These recommendations should improve the quality of patient care by adopting these minimal technical requirements for optimal exam performance and lead to less variation in the performance of these exams which will facilitate collecting and comparing published data from different centers. These recommendations will need revisions as additional scientific data become available.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Experiences with Testing the Largest Ground System NASA Has Ever Built

In the 1980s, the National Aeronautics and Space Administration (NASA) embarked upon a major Earth-focused program called Mission to Planet Earth. The Goddard Space Flight Center (GSFC) was selected to manage and develop a key component - the Earth Observing System (EOS). The EOS consisted of four major missions designed to monitor the Earth. The missions included 4 spacecraft. Terra (launched December 1999), Aqua (launched May 2002), ICESat (Ice, Cloud, and Land Elevation Satellite, launched January 2003), and Aura (scheduled for launch January 2004). The purpose of these missions was to provide support for NASA s long-term research effort for determining how human-induced and natural changes affect our global environment. The EOS Data and Information System (EOSDIS), a globally distributed, large-scale scientific system, was built to support EOS. Its primary function is to capture, collect, process, and distribute the most voluminous set of remotely sensed scientific data to date estimated to be 350 Gbytes per day. The EOSDIS is composed of a diverse set of elements with functional capabilities that require the implementation of a complex set of computers, high-speed networks, mission-unique equipment, and associated Information Technology (IT) software along with mission-specific software. All missions are constrained by schedule, budget, and staffing resources, and rigorous testing has been shown to be critical to the success of each mission. This paper addresses the challenges associated with the planning, test definition. resource scheduling, execution, and discrepancy reporting involved in the mission readiness testing of a ground system on the scale of EOSDIS. The size and complexity of the mission systems supporting the Aqua flight operations, for example, combined with the limited resources available, prompted the project to challenge the prevailing testing culture. The resulting success of the Aqua Mission Readiness Testing (MRT) program was due in no small measure to re-structuring the traditional programmatic and technical approach to a more efficient and robust program. Programmatically, it meant gaining the endorsement, commitment, and cooperation of the numerous subsystem element managers and other stakeholder organizations. Technically, it required an MRT program that was agile, could rapidly adapt to requirements changes, and was flexible in its overall approach. Furthermore, this paper addresses the following questions: 1. What are the key ingredients (e.g., test tools, organization) needed to conduct a successful MRT program? 2. What distinguishes EOS MRT from the traditional system testing approach? 3. Where should the focus of testing be since it is infeasible to test every element or subsystem? 4. How can MRT be applied effectively to other systems or missions? To provide answers to these questions, this paper relies heavily on real-life, hands-on experiences ("lessons learned") gained during mission readiness testing of the Terra ground system and, most recently, the Aqua and ICESat missions. Moreover, this paper explores how lessons learned were turned into lessons applied for the upcoming Aura mission. Although derived from the EOS missions, MRT techniques and strategies can be applied to enhance the testing of other missions.

Lehtonen, Ken↗

Acquisition of Earth Science Remote Sensing Observations from Commercial Sources: Lessons Learned from the Space Imaging IKONOS Example

In an effort to more full explore the potential of commercial remotely sensed land data sources, the NASA Earth Science Enterprise (ESE) implemented an experimental Scientific Data Purchase (SDP) that solicited bids from the private sector to meet ESE-user data needs. The images from the Space Imaging IKONOS system provided a particularly good match to the current ESE missions such as Terra and Landsat 7 and therefore serve as a focal point in this analysis.

Goward, Samuel N.↗

An Overview of the Performance of the Chandra X-ray Observatory

The Chandra X-ray Observatory is the X-ray component of NASA's Great Observatory Program which includes the recently launched Spitzer Infrared Telescope, the Hubble Space Telescope (HST) for observations in the visible, and the Compton Gamma-Ray Observatory (CGRO) which, after providing years of useful data has reentered the atmosphere. All these facilities provide, or provided, scientific data to the international astronomical community in response to peer-reviewed proposals for their use. The Chandra X-ray Observatory was the result of the efforts of many academic, commercial, and government organizations primarily in the United States but also in Europe. NASA s Marshall Space Flight Center (MSFC) manages the Project and provides Project Science; Northrop Grumman Space Technology (NGST - formerly TRW) served as prime contractor responsible for providing the spacecraft, the telescope, and assembling and testing the Observatory; and the Smithsonian Astrophysical Observatory (SAO) provides technical support and is responsible for ground operations including the Chandra X-ray Center (CXC). Telescope and instrument teams at SAO, the Massachusetts Institute of Technology (MIT), the Pennsylvania State University (PSU), the Space Research Institute of the Netherlands (SRON), the Max-Planck Institut fur extraterrestrische Physik (MPE), and the University of Kiel support also provide technical support to the Chandra Project. We present here a detailed description of the hardware, its on-orbit performance, and a brief overview of some of the remarkable discoveries that illustrate that performance.

Weisskopf, M. C.↗

Optimal A-Train Data Utilization: A Use Case of Aura OMI L2G and MERRA-2 Aerosol Products

Ozone Monitoring Instrument (OMI) aboard NASA's Aura mission measures ozone column and profile, aerosols, clouds, surface UV irradiance, and the trace gases including NO2, SO2, HCHO, BrO, and OClO using UltraViolet electromagnetic spectrum (280 - 400 nm) with a daily global coverage and a pixel spatial resolution of 13 km × 24 km at nadir, and it's been one of the key instruments to study the Earth's atmospheric composition and chemistry. The second Modern-Era Retrospective analysis for Research and Applications (MERRA-2) is NASA's atmospheric reanalysis using an upgraded version of Goddard Earth Observing System Model, version 5 (GEOS-5) data assimilation system. Compared to its predecessor MERRA, MERRA-2 is enhanced with more aspects of the Earth system among which is aerosol assimilation. When comparing between satellite pixel measurements and modeled grid data, how to properly handle counterpart pairing is critical considering their spatial and temporal variations. The comparison between satellite and model data by simply using Level 3 (L3) products may result biases due to lack of detailed temporal information. It has been preferred to inter-compare or implement satellite derived physical quantity (i.e., Level 2 (L2) Swath type) directly with/to model measurements with higher temporal and spatial resolution as possible. However, this has posed a challenge in the community to handle. Rather than directly handling the L2 or L3 data, there is a Level 2G (L2G) product conserving L2 pixel scientific data quality but in Grid type with the global coverage. In this presentation, we would like to demonstrate the optimal utilization of OMI L2G daily aerosol products by comparing with MERRA-2 hourly aerosol simulations matched well in both space and time.

MERRA-2 reanalysis↗

Task 51 - Cloud-Optimized Format Study

The cloud infrastructure provides a number of capabilities that can dramatically improve access and use of Earth Observation data. However, in many cases, data may need to be reorganized and/or reformatted in order to make them tractable to support cloud-native analysis/access patterns. The purpose of this study is to examine the pros and cons of different formats for storing data on the cloud. The evaluation will focus on both enabling high-performance data access and usage as well as to meet the existing scientific data stewardship needs of EOSDIS.

Durbin, Chris↗

Operations Concept for responding to Urgent Requests for NASA-ISRO Synthetic Aperture Radar (NISAR)

This mission is a collaboration between the National Aeronautics and Space Administration (NASA) and the Indian Space Research Organization (ISRO). This spacecraft will carry two instruments that will operate at radar wavelengths (L and S band) and will provide data for understanding changes in the Earth’s land surface. The scientific data from this mission will revolutionize our understanding of the causes and consequences of land surface changes on Earth.

Hammer, Brian↗