Search NASA⌕ Search

SEARCH · Search NASA

Results for “data formats”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
NASA NTRS1996

A Description of the Software Element of the NASA EME Flight Tests

In support of NASA's Fly-By-Light/Power-By-Wire (FBL/PBW) program, a series of flight tests were conducted by NASA Langley Research Center in February, 1995. The NASA Boeing 757 was flown past known RF transmitters to measure both external and internal radiated fields. The aircraft was instrumented with strategically located sensors for acquiring data on shielding effectiveness and internal coupling. The data are intended to support computational and statistical modeling codes used to predict internal field levels of an electromagnetic environment (EME) on aircraft. The software was an integral part of the flight tests, as well as the data reduction process. The software, which provided flight test instrument control, data acquisition, and a user interface, executes on a Hewlett Packard (HP) 300 series workstation and uses BP VEEtest development software and the C programming language. Software tools were developed for data processing and analysis, and to provide a database organized by frequency bands, test runs, and sensors. This paper describes the data acquisition system on board the aircraft and concentrates on the software portion. Hardware and software interfaces are illustrated and discussed. Particular attention is given to data acquisition and data format. The data reduction process is discussed in detail to provide insight into the characteristics, quality, and limitations of the data. An analysis of obstacles encountered during the data reduction process is presented.

Koppen, Sandra V.↗
NASA NTRS2018

The Best Educational Tool for Interdisciplinary Earth Science Giovanni

Accessing and using NASA Earth science data has commonly presented a challenge to many educators and students, due to issues such as heterogeneous data formats, complex data structures, large volumes of data storage, special programming requirements, and diverse analytical software options that often require a significant investment in time and resources, especially for novices. By facilitating data access and evaluation, as well as promoting open access to create a more level playing field for non-funded scientists, NASA Earth observation data can be more readily used for scientific discovery and societal benefits. To advance this goal, the NASA Goddard Earth Sciences (GES) Data and Information Services Center (DISC) developed the Geospatial Interactive Online Visualization ANd aNalysis Infrastructure (Giovanni). To date, Giovanni has assisted researchers around the world publish over 1300 peer-reviewed papers in a wide range of Earth science disciplines. In this presentation, we will demonstrate how easy it is to use Giovanni for the rapid creation of many different analyses of both weather and climate events.

interdisciplinary↗
NASA NTRS2017

Reusing JPSS Ground System Components to Process Aura Ozone Monitoring Instrument Science Products

New Earth observation instruments are planned to enable advancements in Earth science research over the next decade. Diversity of Earth observing instruments and their observing platforms will continue to increase as new instrument technologies emerge and are deployed as part of National programs such as Joint Polar Satellite System (JPSS), Geostationary Operational Environmental Satellite system (GOES), Landsat as well as the potential for many CubeSat and aircraft missions. The practical use and value of these observational data often extends well beyond their original purpose. The practicing community needs intuitive and standardized tools to enable quick unfettered development of tailored products for specific applications and decision support systems. However, the associated data processing system can take years to develop and requires inherent knowledge and the ability to integrate increasingly diverse data types from multiple sources. This paper describes the adaptation of a large-scale data processing system built for supporting JPSS algorithm calibration and validation (CalVal) node to a simplified science data system for rapid application. The new configurable data system reuses scalable JAVA technologies built for the JPSS Government Resource for Algorithm Verification, Independent Test, and Evaluation (GRAVITE) system to run within a laptop environment and support product generation and data processing of AURA Ozone Monitoring Instrument (OMI) science products. Of particular interest are the root requirements necessary for integrating experimental algorithms and Hierarchical Data Format (HDF) data access libraries into a science data production system. This study demonstrates the ability to reuse existing Ground System technologies to support future missions with minimal changes.

Science Data Systems↗
DOE OSTI2022

LANL Nuclear Data Manager Format Specification v1.0

The LANL nuclear data manager tool is a Python utility that downloads nuclear data from the Nuclear Data Team’s website at https://nucleardata.lanl.gov, arranges it, and configures directory listings for nuclear codes such as the MCNP code to consume. In this document, the Version 1.0 API that this tool relies on will be discussed, which will show how an external provider of nuclear data could set up their website to provide a compatible implementation.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1986

The CDF software package

The concepts that are fundamental to the new Common Data Format are outlined. With Pilot Climate Data System (PCDS) Version 4.0, the Common Data Format (CDF) will supersede the Climate Data File (also CDF) or earlier versions. This new format incorporates generalizations in both design and terminology that make it applicable to multidisciplinary data sets. Furthermore, the new CDF will be made available to programmers as a software package that shields them from the low-level details of file formats. The CDF interface routines create an abstract conceptual environment for the scientific programmer. The principle concept for visualizing a CDF is known as the basic grid. A basic grid is an n-dimensional block by means of which a CDF is constructed. The size of the block may vary from one CDF to another, but is consistent within any individual CDF. Thus, the basic grid serves as a fundamental uniform building block for a CDF. The number of grid dimensions and the size of each dimension are chosen by the scientist/programmer to represent the patterns by which data are structured. The uniform grid structure appears to the programmer to be propagated into each record for each variable. The CDF stores a variable value for each lattice point of the grid for each record. These data values are inserted and retrieved simply by specifying the variable's identifier, a record number, and the indices that specify the lattice point of interest.

Wilson, C. E.↗
NASA NTRS2000

IMAGE Software Suite

The IMAGE Mission is generating a truely unique set of magnetospheric measurement through a first-of-its-kind complement of remote, global observations. These data are being distributed in the Universal Data Format (UDF), which consists of data, calibration, and documentation. This is an open dataset, available to all by request to the National Space Science Data Center (NSSDC) at NASA Goddard Space Flight Center. Browse data, which consists of summary observations, is also available through the NSSDC in the Common Data Format (CDF) and graphic representations of the browse data. Access to the browse data can be achieved through the NSSDC CDAWeb services or by use of NSSDC provided software tools. This presentation documents the software tools, being provided by the IMAGE team, for use in viewing and analyzing the UDF telemetry data. Like the IMAGE data, these tools are openly available. What these tools can do, how they can be obtained, and how they are expected to evolve will be discussed.

Gallagher, Dennis L.↗
DOE OSTI2021

A Guide to Using GitHub for Developing and Versioning Data Standards and Reporting Formats

Abstract Data standardization combined with descriptive metadata facilitate data reuse, which is the ultimate goal of the Findable, Accessible, Interoperable, and Reusable (FAIR) principles. Community data or metadata standards are increasingly created through an approach that emphasizes collaboration between various stakeholders. Such an approach requires platforms for collaboration on the development process that centers on sharing information and receiving feedback. Our objective in this study was to conduct a systematic review to identify data standards and reporting formats that use version control for developing data standards and to summarize common practices, particularly in earth and environmental sciences. Out of 108 data standards and reporting formats identified in our review, 32 used GitHub as the version control platform, and no other platforms were used. We found no universally accepted methodology for developing and publishing data standards. Many GitHub repositories did not use key features that could help developers to gather user feedback, or to create and revise standards that build on previous work. We provide guidance for community‐driven standard development and associated documentation on GitHub based on a systematic review of existing practices.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2001

New tracking implementation in the Deep Space Network

As part of the Network Simplification Project, the tracking system of the Deep Space Network is being upgraded. This upgrade replaces the discrete logic sequential ranging system with a system that is based on commercial Digital Signal Processor boards. The new implementation allows both sequential and pseudo-noise types of ranging. The other major change is a modernization of the data formatting. Previously, there were several types of interfaces, delivering both intermediate data and processed data (called 'observables'). All of these interfaces were bit-packed blocks, which do not allow for easy expansion, and many of these interfaces required knowledge of the specific hardware implementations. The new interface supports four classes of data: raw (direct from the measuring equipment), derived (the observable data), interferometric (multiple antenna measurements), and filtered (data whose values depend on multiple measurements). All of the measurements are reported at the sky frequency or phase level, so that no knowledge of the actual hardware is required. The data is formatted into Standard Formatted Data Units, as defined by the Consultative Committee for Space Data Systems, so that expansion and cross-center usage is greatly enhanced.

ranging↗
NASA NTRS2007

Defining the Core Archive Data Standards of the International Planetary Data Alliance (IPDA)

A goal of the International Planetary Data Alliance (lPDA) is to develop a set of archive data standards that enable the sharing of scientific data across international agencies and missions. To help achieve this goal, the IPDA steering committee initiated a six month proj ect to write requirements for and draft an information model based on the Planetary Data System (PDS) archive data standards. The project had a special emphasis on data formats. A set of use case scenarios were first developed from which a set of requirements were derived for the IPDA archive data standards. The special emphasis on data formats was addressed by identifying data formats that have been used by PDS nodes and other agencies in the creation of successful data sets for the Planetary Data System (PDS). The dependency of the IPDA information model on the PDS archive standards required the compilation of a formal specification of the archive standards currently in use by the PDS. An ontology modelling tool was chosen to capture the information model from various sources including the Planetary Science Data Dictionary [I] and the PDS Standards Reference [2]. Exports of the modelling information from the tool database were used to produce the information model document using an object-oriented notation for presenting the model. The tool exports can also be used for software development and are directly accessible by semantic web applications.

ontology↗
NASA NTRS2003

Data Access Tools And Services At The Goddard Distributed Active Archive Center (GDAAC)

As one of the largest providers of Earth Science data from the Earth Observing System, GDAAC provides the latest data from the Moderate Resolution Imaging Spectroradiometer (MODIS), Atmospheric Infrared Sounder (AIRS), Solar Radiation and Climate Experiment (SORCE) data products via GDAAC's data pool (50TB of disk cache). In order to make this huge volume of data more accessible to the public and science communities, the GDAAC offers multiple data access tools and services: Open Source Project for Network Data Access Protocol (OPeNDAP), Grid Analysis and Display System (GrADS/DODS) (GDS), Live Access Server (LAS), OpenGlS Web Map Server (WMS) and Near Archive Data Mining (NADM). The objective is to assist users in retrieving electronically a smaller, usable portion of data for further analysis. The OPeNDAP server, formerly known as the Distributed Oceanographic Data System (DODS), allows the user to retrieve data without worrying about the data format. OPeNDAP is capable of server-side subsetting of HDF, HDF-EOS, netCDF, JGOFS, ASCII, DSP, FITS and binary data formats. The GrADS/DODS server is capable of serving the same data formats as OPeNDAP. GDS has an additional feature of server-side analysis. Users can analyze the data on the server there by decreasing the computational load on their client's system. The LAS is a flexible server that allows user to graphically visualize data on the fly, to request different file formats and to compare variables from distributed locations. Users of LAS have options to use other available graphics viewers such as IDL, Matlab or GrADS. WMS is based on the OPeNDAP for serving geospatial information. WMS supports OpenGlS protocol to provide data in GIs-friendly formats for analysis and visualization. NADM is another access to the GDAAC's data pool. NADM gives users the capability to use a browser to upload their C, FORTRAN or IDL algorithms, test the algorithms, and mine data in the data pool. With NADM, the GDAAC provides an environment physically close to the data source. NADM will benefit users with mining or offer data reduction algorithms by reducing large volumes of data before transmission over the network to the user.

Pham, Long↗
NASA NTRS2017

Development of Gridded Innovations and Observations Supplement to MERRA-2

Atmospheric reanalysis have become an important source of data for weather and climate research, owing to the continuity of the data, but especially because of the multitude of observational data included (radiosondes, commercial aircraft, retrieved data products and radiances). However, the presence of assimilated observations can vary based on numerous factors, and so it is difficult or impossible for a researcher to say with any degree of certainty how many and what type of observations contributed to the reanalysis data they are using at any give point in time or space. For example, quality control, transmission interruptions, and station outages can occasionally affect data availability. While orbital paths can be known, drift in certain instruments and the large number of available instruments makes it challenging to know which satellite is observing any region at any point in the diurnal cycle. Furthermore, there is information from the statistics generated by the data assimilation that can help understand the model and the quality of the reanalysis. Typically, the assimilated observations and their innovations are in observation-space data formats and have not been made easily available to reanalysis users.A test data set has been developed to make the MERRA-2 assimilated observations available for rapid and general use, by simplifying the data format. The observations are binned to a grid similar as MERRA-2 and saved as netCDF. This data collection includes the mean and number of observations in the bin as well as its variance. The data will also include the innovations from the data assimilation, the forecast departure and the analysis increment, as well as bias correction (for satellite radiances). We refer to this proof-of-concept data as the MERRA-2 Gridded Innovations and Observations (GIO). In this paper, we present the data format and its strengths and limitations with some initial testing and validation of the methodology.

SSMI↗
NASA NTRS2018

Development of Gridded Innovations and Observations Supplement to MERRA-2

Atmospheric reanalysis have become an important source of data for weather and climate research, owing to the continuity of the data, but especially because of the multitude of observational data included (radiosondes, commercial aircraft, retrieved data products and radiances). However, the presence of assimilated observations can vary based on numerous factors, and so it is difficult or impossible for a researcher to say with any degree of certainty how many and what type of observations contributed to the reanalysis data they are using at any give point in time or space, or their contribution to the eventual analyzed fields. For example, quality control, transmission interruptions, and station outages can occasionally affect data availability. While orbital paths are known, drift in certain instruments, cloud clearing, and the large number of available instruments makes it challenging to know which satellite is observing any region at any point in the diurnal cycle. Furthermore, there is information from the statistics generated by the data assimilation that can help understand the model and the quality of the reanalysis. Typically, the assimilated observations and their innovations are saved in observation-space data formats and are not easily made available to reanalysis users.Here, we present an early version of a data set has been developed to make the MERRA-2 assimilated observations available for rapid and general use, by simplifying the data format. The observations are binned to a grid similar as MERRA-2 and saved as netCDF. This data collection includes the mean and number of observations in the bin as well as its variance. The data will also include the innovations from the data assimilation, the forecast departure and the analysis increment, as well as bias correction (for satellite radiances). In this paper, we present the data format (called MERRA-2 Gridded Innovations and Observations or GIO) and its strengths and limitations with some initial testing and validation of the methodology.

Bosilovich, Michael G.↗
NASA NTRS2007

Real-Time Data Display

RT-Display is a MATLAB-based data acquisition environment designed to use a variety of commercial off-the-shelf (COTS) hardware to digitize analog signals to a standard data format usable by other post-acquisition data analysis tools. This software presents the acquired data in real time using a variety of signal-processing algorithms. The acquired data is stored in a standard Operator Interactive Signal Processing Software (OISPS) data-formatted file. RT-Display is primarily configured to use the Agilent VXI (or equivalent) data acquisition boards used in such systems as MIDDAS (Multi-channel Integrated Dynamic Data Acquisition System). The software is generalized and deployable in almost any testing environment, without limitations or proprietary configuration for a specific test program or project. With the Agilent hardware configured and in place, users can start the program and, in one step, immediately begin digitizing multiple channels of data. Once the acquisition is completed, data is converted into a common binary format that also can be translated to specific formats used by external analysis software, such as OISPS and PC-Signal (product of AI Signal Research Inc.). RT-Display at the time of this reporting was certified on Agilent hardware capable of acquisition up to 196,608 samples per second. Data signals are presented to the user on-screen simultaneously for 16 channels. Each channel can be viewed individually, with a maximum capability of 160 signal channels (depending on hardware configuration). Current signal presentations include: time data, fast Fourier transforms (FFT), and power spectral density plots (PSD). Additional processing algorithms can be easily incorporated into this environment.

Pedings, Marc↗
NASA NTRS2001

Evaluation of Incremental Releases of ECS User Interfaces and the Development of HDF/HDF-EOS Tutorials

During the reporting period, the PI has continued to serve on numerous review panels, task forces, and committees with the goal of providing input and guidance for the Earth Observing System Data and Information System (EOSDIS) program at NASA Headquarters and NASA Goddard Space Flight Center (GSFC). In addition, the PI has worked together with personnel at Simpson Weather Associates (SWA) to help create an on-line HDF/HDF-EOS tutorial for beginning and non-expert users of both the Hierarchical Data Format (HDF) and HDF-EOS data format and software libraries. Finally, the PI has worked together with personnel at SWA and the Information Technology and Systems Center (ITSC) at the University of Alabama in Huntsville (UAH) on a feasibility study regarding the use of data mining software to ascertain features from the gridded output from numerical meteorological forecast models. A summary of these activities is provided.

Emmitt, G. D.↗
NASA NTRS2006

Adding Hierarchical Objects to Relational Database General-Purpose XML-Based Information Managements

NETMARK is a flexible, high-throughput software system for managing, storing, and rapid searching of unstructured and semi-structured documents. NETMARK transforms such documents from their original highly complex, constantly changing, heterogeneous data formats into well-structured, common data formats in using Hypertext Markup Language (HTML) and/or Extensible Markup Language (XML). The software implements an object-relational database system that combines the best practices of the relational model utilizing Structured Query Language (SQL) with those of the object-oriented, semantic database model for creating complex data. In particular, NETMARK takes advantage of the Oracle 8i object-relational database model using physical-address data types for very efficient keyword searches of records across both context and content. NETMARK also supports multiple international standards such as WEBDAV for drag-and-drop file management and SOAP for integrated information management using Web services. The document-organization and -searching capabilities afforded by NETMARK are likely to make this software attractive for use in disciplines as diverse as science, auditing, and law enforcement.

Lin, Shu-Chun↗
NASA NTRSDate not supplied

Additional Metadata Guidelines to Improve the Structure and Usability of HDF and NetCDF Files

The Hierarchical Data Format (HDF) and Network Common Data Form (NetCDF) are data file formats created to aid users in the creation or use of scientific data. These file formats are useful for handling large data volumes and hosting extensive metadata as global attributes or variables and are popular with the modeling community. HDF and NetCDF files are largely used with remote sensing data and have been used to support measurements from numerous campaigns, from satellite to aircraft or ground and mobile based measurements. The files from airborne field studies, however, vary greatly in terms of the file structure and the amount and content of metadata. Information relevant to the file that can be useful to the user such as the data producer, location where data was taken, variable descriptions, or information about the instrument might not be included in the file. This metadata might be present in another file in the dataset containing the same data using the International Consortium for Atmospheric Research on Transport and Transformation (ICARTT) format. Recently, the Aerosols, Clouds, and their Interactions for Earth System Models (MACIE) group started a grassroots effort to develop a set of requirements for the HDF and NetCDF files for field studies, aiming to make the data products more interoperable and usable. Particularly, these requirements seek to make the files more compliant to Climate and Forecast (CF) metadata conventions and to standardize the file structure and the global and variable attributes. These requirements would help to ensure that HDF and NetCDF files contain adequate metadata to better support their use for research, e.g., the modeling community, and to enhance the usability and interoperability of data for research communities at large. To be presented are the details of the MACIE requirements as well as examples of the implementation of these requirements for merge files and lidar observation data files.

Sean Leavor↗
NASA NTRSDate not supplied

Extending CF Conventions to Enhance Data FAIRness for Atmospheric Composition Observations

The Hierarchical Data Format (HDF) and Network Common Data Form (NetCDF) are data file formats created to aid users in the creation or use of scientific data. These file formats are useful for handling large data volumes and hosting extensive metadata as global, group, or variable attributes and are popular with the modeling community. HDF and NetCDF files are widely used with atmospheric remote sensing data and have been used to support measurements from numerous field campaigns, from satellite to aircraft or ground and mobile based measurements. The files from airborne field studies, however, vary greatly in terms of the file structure and the amount and content of their metadata. Information relevant to the file that can be useful to the user such as the data producer, location where data was taken, variable descriptions, or information about the instrument might not be included in the file. Recently, the Measurements of Aerosols, Clouds, and their Interactions for Earth System Models (MACIE) group started a grassroots effort to develop a CF-based template for the HDF and NetCDF files for field studies, with the aim of making the data products more interoperable and usable. This template seeks to make the files more compliant to Climate and Forecast (CF) metadata conventions and to standardize the file structure and the global and variable attributes. The template would help to ensure that HDF and NetCDF files contain adequate metadata to better support their use for research, e.g., the modeling community, and to enhance the usability and interoperability of data for research communities at large. The draft template has been applied to recent field studies for various instruments and their merge files in support of the Atmosphere Observing System (AOS) project. The details of the revised template are to be presented, as well as examples of the implementation of these requirements for merge files and lidar observation data files and issues revealed during the implementation process.

Sean Leavor↗