Search NASA⌕ Search

SEARCH · Search NASA

Results for “data processing and archiving”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
NASA NTRS2017

GES DISC Data Recipes in Jupyter Notebooks

The Earth Science Data and Information System (ESDIS) Project manages twelve Distributed Active Archive Centers (DAACs) which are geographically dispersed across the United States. The DAACs are responsible for ingesting, processing, archiving, and distributing Earth science data produced from various sources (satellites, aircraft, field measurements, etc.). In response to projections of an exponential increase in data production, there has been a recent effort to prototype various DAAC activities in the cloud computing environment. This, in turn, led to the creation of an initiative, called the Cloud Analysis Toolkit to Enable Earth Science (CATEES), to develop a Python software package in order to transition Earth science data processing to the cloud. This project, in particular, supports CATEES and has two primary goals. One, to transition data recipes created by the Goddard Earth Science Data and Information Service Center (GES DISC) into an interactive and educational environment using JupyterNotebooks. Two, to acclimate Earth scientists to cloud computing. To accomplish these goals, we create JupyterNotebooks to compartmentalize the different steps of data analysis and help users obtain and parse data from the command line. We also develop a Docker container, comprised of Jupyter Notebooks, Python dependencies, and command line tools, and configure it into an easy-to-deploy package. The end result is an end-to-end product that simulates the use case of end users working in the cloud computing environment.

discoverability↗
NASA NTRS1984

Industrial use of land observation satellite systems

The principal industrial users of land observation satellite systems are the geological industries; oil/gas, mining, and engineering/environmental companies. The primary system used is LANDSAT/MSS. Currently, use is also being made of the limited amounts of SKYLAB photography, SEASAT and SIR-A radar, and the new LANDSAT/TM data available. Although considered experimental, LANDSAT data is now used operationally by several hundred exploration and engineering companies worldwide as a vastly improved geological mapping tool to help direct more expensive geophysical and drilling phases, leading to more efficient decision-making and results. Future needs include global LANDSAT/TM; higher spatial resolution; stereo and radar; improved data handling, processing distribution and archiving systems, and integrated geographical information systems (GIS). For a promising future, governments must provide overall continuity (government and/or private sector) of such systems, insure continued government R and D, and commit to operating internationally under the civil Open Skies policy.

Henderson, F. B., III↗
NASA NTRS1988

MODIS information, data and control system (MIDACS) operations concepts

The MODIS Information, Data, and Control System (MIDACS) Operations Concepts Document provides a basis for the mutual understanding between the users and the designers of the MIDACS, including the requirements, operating environment, external interfaces, and development plan. In defining the concepts and scope of the system, how the MIDACS will operate as an element of the Earth Observing System (EOS) within the EosDIS environment is described. This version follows an earlier release of a preliminary draft version. The individual operations concepts for planning and scheduling, control and monitoring, data acquisition and processing, calibration and validation, data archive and distribution, and user access do not yet fully represent the requirements of the data system needed to achieve the scientific objectives of the MODIS instruments and science teams. The teams are not yet formed; however, it is possible to develop the operations concepts based on the present concept of EosDIS, the level 1 and level 2 Functional Requirements Documents, and through interviews and meetings with key members of the scientific community. The operations concepts were exercised through the application of representative scenarios.

Han, D.↗
NASA NTRS1992

Mission to Planet Earth

Mission to Planet Earth (MTPE) is NASA's concept for an international science program to produce the understanding needed to predict changes in the earth's environment. NASA and its interagency and international partners will place satellites carrying advanced sensors in strategic earth orbits to gather multidisciplinary data. A sophisticated data system will process and archive an unprecedented amount of information about the earth and how it works as a system. Increased understanding of the earth system is a basic human responsibility, a prerequisite to informed management of the planet's resources and to the preservation of the global environment.

Wilson, Gregory S.↗
NASA NTRS1990

Science operations for future space astrophysics missions

Plans for astrophysics science operations during the decade of the nineties are described from the point of view of a scientist who wishes to make a space-borne astronomical observation or to use archival astronomical data. In the process of preparing a proposal making an observation, and carrying out data processing, analysis, and dissemination of results, the scientist will be able to use a variety of services and infrastructure, including the 'Astrophysics Data System'. The current status and plans for these science operations services are described.

Riegler, Guenther H.↗
NASA NTRS1993

SpaceOps 1992: Proceedings of the Second International Symposium on Ground Data Systems for Space Mission Operations

The Second International Symposium featured 135 oral presentations in these 12 categories: Future Missions and Operations; System-Level Architectures; Mission-Specific Systems; Mission and Science Planning and Sequencing; Mission Control; Operations Automation and Emerging Technologies; Data Acquisition; Navigation; Operations Support Services; Engineering Data Analysis of Space Vehicle and Ground Systems; Telemetry Processing, Mission Data Management, and Data Archiving; and Operations Management. Topics focused on improvements in the productivity, effectiveness, efficiency, and quality of mission operations, ground systems, and data acquisition. Also emphasized were accomplishments in management of human factors; use of information systems to improve data retrieval, reporting, and archiving; design and implementation of logistics support for mission operations; and the use of telescience and teleoperations.

Source record↗
NASA NTRS1994

Mission to Planet Earth

Mission to Planet Earth (MTPE) is NASA's concept for an international science program to produce the understanding needed to predict changes in the Earth's environment. NASA and its interagency and international partners will place satellites carrying advanced sensors in strategic Earth orbits to gather multidisciplinary data. A sophisticated data system will process and archive an unprecedented amount of information about the Earth and how it works as a system. Increased understanding of the Earth system is a basic human responsibility, a prerequisite to informed management of the planet's resources and to the preservation of the global environment.

Tilford, Shelby G.↗
NASA NTRSDate not supplied

EOSDIS Archive & Data Stewardship

NASA’s Earth Observing System Data and Information System’s (EOSDIS) was built to archive and distribute earth science data from flight and research programs. At the close of FY2020, the data collection had grown to over 42 petabytes distributed across the US. This presentation describes fundamentals associated with managing a free and open archive of this size for a worldwide, multi-discipline user community. The presentation is prepared for the Committee on Earth Observation Satellites (CEOS), which strives to enhance international coordination and data exchange and to optimize societal benefit. The Working Group on Information System and Services (WGISS) is a forum within CEOS for the collaboration with other international and domestic agencies io the development of Earth observation data archives, systems and services. This presentation reviews the construct of the EOSDIS archives, formats for long term archive, preservation of appropriate data and documents, and challenges facing the community.

earth science↗
NASA NTRSDate not supplied

Overview of the NASA Terrestrial Environments Support Role: Data and Applications

This presentation focuses on the terrestrial environment component of the Natural Environments Branch’s (NEB’s) support role and highlights the instrumentation and data sources used, how the data are quality controlled and archived, and the analyses and use-cases of these datasets. A detailed overview is provided about some individual data sources that are used by the branch, which includes a network of instruments (e.g., wind towers, balloons, and profilers) and models, such as the Earth Global Reference Atmospheric Model (Earth-GRAM). Additionally, the data archival and quality control processes that maintain the high-quality climatological datasets from these sources are discussed, as are the implemented and pending updates to this process. Examples of the different applications for these datasets are presented to showcase the analytical and operational capabilities of the NEB and demonstrate the culmination of the entire process. This includes the Range Reference Atmospheres created and maintained by the NEB, sea-state analyses for crewed return, certification of observation systems for space flight, and the day-of-launch support process. The NEB at NASA Marshall Space Flight Center (MSFC) supports the NASA Engineering Directorate (ED) and Exploration Systems Development Mission Directorate (ESDMD) by providing the necessary environmental constraints to facilitate the safe design and operation of assets required for space travel. This includes, but is not limited to, the environmental considerations for assets residing at the launch sites, temporarily stationed at a launch site, traversing the [extra-]terrestrial atmosphere and open space, and arriving at their destination. As such, the Terrestrial and Planetary Environments Team within the NEB is but a part of the whole, delegated to quantify the environment that assets may encounter whilst on Earth and within its atmosphere. However, nearly all ESDMD assets are subjected to terrestrial conditions, which cover a diverse range of meteorological regimes and extremes. The team also supports day-of-launch operations for the Space Launch System (SLS), providing real-time monitoring of the winds and thermodynamics along the launch path using an amalgamation of observed, remotely sensed, and modeled output.

Systems Engineering↗
NASA NTRS2010

Doppler Lidar for Wind Measurements on Venus

NASA Langley Research Center has a long history of developing 2-micron laser transmitter for wind sensing. With support from NASA Laser Risk Reduction Program (LRRP) and Instrument Incubator Program (IIP), NASA Langley Research Center has developed a state-of-the-art compact lidar transceiver for a pulsed coherent Doppler lidar system for wind measurement. The transmitter portion of the transceiver employs the high-pulse-energy, Ho:Tm:LuLiF, partially conductively cooled laser technology developed at NASA Langley. The transceiver is capable of 250 mJ pulses at 10 Hz. It is very similar to the technology envisioned for coherent Doppler lidar wind measurements from Earth and Mars orbit. The transceiver is coupled to the large optics and data acquisition system in the NASA Langley VALIDAR mobile trailer. The large optics consists of a 15-cm off-axis beam expanding telescope, and a full-hemispheric scanner. Vertical and horizontal vector winds are measured, as well as relative backscatter. The data acquisition system employs frequency domain velocity estimation and pulse accumulation. It permits real-time display of the processed winds and archival of all data. This lidar system was recently deployed at Howard University facility in Beltsville, Mary-land, along with other wind lidar systems. Coherent Doppler wind lidar ground-based wind measurements and comparisons with other sensors will be presented. A simulation and data product for wind measurement at Venus will be presented.

Singh, Upendra N.↗
NASA NTRS2015

Development of a Climatology of Vertically Complete Wind Profiles from Doppler Radar Wind Profiler Systems

This paper describes in detail the QC and splicing methodology for KSC's 50- and 915-MHz DRWP measurements that generates an extensive archive of vertically complete profiles from 0.20-18.45 km. The concurrent POR from each archive extends from April 2000 to December 2009. MSFC NE applies separate but similar QC processes to each of the 50- and 915-MHz DRWP archives. DRWP literature and data examination provide the basis for developing and applying the automated and manual QC processes on both archives. Depending on the month, the QC'ed 50- and 915-MHz DRWP archives retain 52-65% and 16-30% of the possible data, respectively. The 50- and 915-MHz DRWP QC archives retain 84-91% and 85-95%, respectively, of all the available data provided that data exist in the non- QC'ed archives. Next, MSFC NE applies an algorithm to splice concurrent measurements from both DRWP sources. Last, MSFC NE generates a composite profile from the (up to) five available spliced profiles to effectively characterize boundary layer winds and to utilize all possible 915-MHz DRWP measurements at each timestamp. During a given month, roughly 23,000-32,000 complete profiles exist from 0.25-18.45 km from the composite profiles' archive, and approximately 5,000- 27,000 complete profiles exist from an archive utilizing an individual 915-MHz DRWP. One can extract a variety of profile combinations (pairs, triplets, etc.) from this sample for a given application. The sample of vertically complete DRWP wind measurements not only gives launch vehicle customers greater confidence in loads and trajectory assessments versus using balloon output, but also provides flexibility to simulate different DOL situations across applicable altitudes. In addition to increasing sample size and providing more flexibility for DOL simulations in the vehicle design phase, the spliced DRWP database provides any upcoming launch vehicle program with the capability to utilize DRWP profiles on DOL to compute vehicle steering commands, provided the program applies the procedures that this report describes to new DRWP data on DOL. Decker et al. (2015) details how SLS is proposing to use DRWP data and splicing techniques on DOL. Although automation could enhance the current DOL 50-MHz DRWP QC process and could streamline any future DOL 915-MHz DRWP QC and splicing process, the DOL community would still require manual intervention to ensure that the vehicle only uses valid profiles. If a program desires to use high spatial resolution profiles, then the algorithm could randomly add high-frequency components to the DRWP profiles. The spliced DRWP database provides lots of flexibility in how one performs DOL simulations, and the algorithms that this report provides will assist the aerospace and atmospheric communities that are interested in utilizing the DRWP.

Barbre, Robert E., Jr.↗
NASA NTRS2009

An Intelligent Archive Testbed Incorporating Data Mining

Many significant advances have occurred during the last two decades in remote sensing instrumentation, computation, storage, and communication technology. A series of Earth observing satellites have been launched by U.S. and international agencies and have been operating and collecting global data on a regular basis. These advances have created a data rich environment for scientific research and applications. NASA s Earth Observing System (EOS) Data and Information System (EOSDIS) has been operational since August 1994 with support for pre-EOS data. Currently, EOSDIS supports all the EOS missions including Terra (1999), Aqua (2002), ICESat (2002) and Aura (2004). EOSDIS has been effectively capturing, processing and archiving several terabytes of standard data products each day. It has also been distributing these data products at a rate of several terabytes per day to a diverse and globally distributed user community (Ramapriyan et al. 2009). There are other NASA-sponsored data system activities including measurement-based systems such as the Ocean Data Processing System and the Precipitation Processing system, and several projects under the Research, Education and Applications Solutions Network (REASoN), Making Earth Science Data Records for Use in Research Environments (MEaSUREs), and the Advancing Collaborative Connections for Earth-Sun System Science (ACCESS) programs. Together, these activities provide a rich set of resources constituting a value chain for users to obtain data at various levels ranging from raw radiances to interdisciplinary model outputs. The result has been a significant leap in our understanding of the Earth systems that all humans depend on for their enjoyment, livelihood, and survival. The trend in the community today is towards many distributed sets of providers of data and services. Despite this, visions for the future include users being able to locate, fuse and utilize data with location transparency and high degree of interoperability, and being able to convert data to information and usable knowledge in an efficient, convenient manner, aided significantly by automation (Ramapriyan et al. 2004; NASA 2005). We can look upon the distributed provider environment with capabilities to convert data to information and to knowledge as an Intelligent Archive in the Context of a Knowledge Building system (IA-KBS). Some of the key capabilities of an IA-KBS are: Virtual Product Generation, Significant Event Detection, Automated Data Quality Assessment, Large-Scale Data Mining, Dynamic Feedback Loop, and Data Discovery and Efficient Requesting (Ramapriyan et al. 2004).

Ramapriyan, H.↗
NASA NTRS1999

The Transport of Solar Ions Through the Earth's Magnetosphere

This report covers the initial phase of an investigation that was originally selected by NASA Headquarters for funding by a grant but was later transferred to NASA GSFC for continued funding under a new and separate contract. The principal objective of the investigation, led by Dr. O.W. Lennartsson, is to extract information about the solar origin plasma in Earth's magnetosphere, specifically about the entry and transport of this plasma, using energetic (10 eV/e to 18 keV/e) ion composition data from the Lockheed Plasma Composition Experiment on the NASA/ESA International Sun-Earth Explorer One (ISEE 1) satellite. These data were acquired many years ago, from November 1977 through March of 1982, but, because of subsequent failures of similar experiments on several other spacecraft, they are still the only substantial ion composition data available from Earth's magnetotail, beyond 10 R(sub E), in the critically important sub-kev to keV energy range. All of the Lockheed data now exist in a compacted scientific format, suitable for large-scale statistical investigations, which has been archived both at Lockheed Martin in Palo Alto and at the National Space Science Data Center (NSSDC) in Greenbelt. The completion of the archiving, by processing the remaining half of the data, was made possible by separate funding through a temporary NASA program for data restoration and was given priority over the data analysis by a no-cost extension of the subject grant. By chance, the period of performance coincided with an international study of source and loss processes of magnetospheric plasma, sponsored by the International Space Science Institute (ISSI) in Bern, Switzerland, for which Dr. Lennartsson was invited to serve as one of 12 co-chairs. This study meshed well with the continued analysis of the NASA/Lockheed ISEE ion composition data and provided a natural forum for a broader discussion of the results from this unique experiment. What follows is arranged, for the most part, in the context of the ISSI project.

Lennartsson, O. W.↗
NASA NTRS2000

EOSDIS: Archive and Distribution Systems in the Year 2000

Earth Science Enterprise (ESE) is a long-term NASA research mission to study the processes leading to global climate change. The Earth Observing System (EOS) is a NASA campaign of satellite observatories that are a major component of ESE. The EOS Data and Information System (EOSDIS) is another component of ESE that will provide the Earth science community with easy, affordable, and reliable access to Earth science data. EOSDIS is a distributed system, with major facilities at seven Distributed Active Archive Centers (DAACs) located throughout the United States. The EOSDIS software architecture is being designed to receive, process, and archive several terabytes of science data on a daily basis. Thousands of science users and perhaps several hundred thousands of non-science users are expected to access the system. The first major set of data to be archived in the EOSDIS is from Landsat-7. Another EOS satellite, Terra, was launched on December 18, 1999. With the Terra launch, the EOSDIS will be required to support approximately one terabyte of data into and out of the archives per day. Since EOS is a multi-mission program, including the launch of more satellites and many other missions, the role of the archive systems becomes larger and more critical. In 1995, at the fourth convening of NASA Mass Storage Systems and Technologies Conference, the development plans for the EOSDIS information system and archive were described. Five years later, many changes have occurred in the effort to field an operational system. It is interesting to reflect on some of the changes driving the archive technology and system development for EOSDIS. This paper principally describes the Data Server subsystem including how the other subsystems access the archive, the nature of the data repository, and the mass-storage I/O management. The paper reviews the system architecture (both hardware and software) of the basic components of the archive. It discusses the operations concept, code development, and testing phase of the system. Finally, it describes the future plans for the archive.

Behnke, Jeanne↗
NASA NTRS1999

"An On-Line Access System for TRMM Data"

The Tropical Rainfall Measuring Mission (TRMM), jointly sponsored by the National Aeronautics and Space Administration (NASA) of United States and the National Space Development Agency (NASDA) of Japan, provides visible, infrared, and microwave observations of tropical and subtropical rain system. The satellite observations are complemented by ground radar and rain gauge measurements to validate satellite rain estimation techniques. TRMM satellite was launched on November 27, 1997(EST). Data from TRMM are being archived, processed, and disseminated by DAAC at the Goddard Space Flight Center (GSFC) of NASA. The Goddard DAAC has been archiving Tropical Rainfall Measuring Mission (TRMM) data since the launch. The TRMM level 1 and level 2 products first have been released to public on June 15, 1998. Now, most of TRMM products, including some ground validation products are released to public. TRMM data are useful for forecast model research, disaster mitigation, climatological studies, agricultural predictions, and many other applications. An online system or user interface is designed for easy, friendly, and quick display, access, and ordering of the TRMM data. Components, features, and limitations of the system will be discussed and demonstrated. The TRMM standard products and variety of the subsets and ancillary data will be shown, and their availability will be discussed.

Rui, Hualan↗
NASA NTRS1992

The ALEXIS data processing package: An IDL based system

The Array of Low Energy X-ray Imaging Sensors (ALEXIS) experiment consists of a mini-satellite containing six wide angle EUV/ultrasoft x-ray telescopes. Its purpose is to map out the sky in three narrow (approximately 5 percent) bandpasses around 66, 71, and 93 eV. The 66 and 71 eV bandpasses are centered on intense Fe emission lines which are characteristic of million degree plasmas such as the one thought to produce the soft x-ray background. The 93 eV bandpass is not near any strong emission lines and is more sensitive to continuum sources. The mission will be launched on the Pegasus Air Launched Vehicle in the second half of 1992 into a 400-nautical-mile, high inclination orbit and will be controlled entirely from a small ground station located at Los Alamos. The project is a collaborative effort between Los Alamos National Laboratory, Sandia National Laboratory, and the University of California-Berkeley Space Sciences Laboratory. The six telescopes are arranged in three pairs. As the satellite spins twice a minute they scan the entire anti-solar hemisphere. Each f/1 telescope consists of a spherical, multilayer-coated mirror with a curved, microchannel plate detector located at the prime focus. The multilayer coatings determine the bandpasses of the telescopes. The field of view of each telescope is 30 degrees with a spatial resolution of 0.5 degree, limited by spherical aberration. The data processing requirements for ALEXIS are large. Each event is one of the six telescopes is telemetered to the ground with its time of arrival and position on the detector. This information must be folded with the aspect solution for the satellite to reconstruct the direction on the sky from which the photon came. Because of the way the six telescopes scan the sky, the effective exposure calculation is also very computationally intensive. ALEXIS may generate up to 100 megabytes of raw data per day, which are converted into a gigabyte per day of processed data. While the processing job for ALEXIS is sizable, the programming staff is small. To maximize programming efficiency, and to make the best use of tools available in the public domain, we chose IDL as our software development platform. IDL was used from the start of instrument development through flight. We use IDL as a top-level executive for the processing tasks (replacing Unix shell scripts), as a device independent graphics engine, as a database manager, and as a final data manipulator. IDL routines spawn special purpose C programs to perform detailed telemetry deconvolution and other specialized functions. We discuss the use of IDL and C within the processing and archiving strategy for the ALEXIS data anlaysis system as implemented on a SPARCstation platform. We also show results from our End-to-End software simulation capability as processed by our analysis codes.

Bloch, J. J.↗
NASA NTRS2017

Data Recovery Effort of Nimbus Era Observations by the NASA GES DISC

NASA launched seven Nimbus meteorological satellites in the 1960s and 70s. These satellites carried instruments for making observations of the Earth in the visible, infrared, ultraviolet, and microwave wavelengths. The original data archive consisted of a combination of magnetic tapes and various film media. As these media are well past their expected end of life, the valuable data they contain are now being migrated to the GES DISC modern online archive. The process involves recovering the digital data files from the tapes as well as scanning images of the data from film strips. This presentation will address the status and challenges of recovering the Nimbus data. The old data products were written on now obsolete hardware systems and outdated file formats. They lack any metadata standards and each product is often written in its own proprietary file structure. This requires creating metadata by reading the contents of the old data files. The job is tedious and laborious, as documentation may be incomplete, data files and tapes are sometimes corrupted, or were improperly copied at the time they were created.

Nimbus↗