Search NASA⌕ Search

SEARCH · Search NASA

Results for “science data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Holistic and Pragmatic Standards Processes Enable Interdisciplinary Science

Open, interdisciplinary science inevitably relies heavily on standards. Standards are those often unseen agreements that we take for granted when systems and processes are working fine. Yet standards work is perpetual, laborious, and sometimes contentious, especially for standards to work across diverse disciplines. Standards development, maintenance, and implementation is a complex, ongoing socio-technical process. NASA has developed a progressively open science policy and strategy that calls for the establishment of a data standards process reaching across the five diverse divisions of the Science Mission Directorate. This is a delicate exercise. We, therefore, seek to apply a holistic yet pragmatic approach to developing and maintaining a standards process. We adopt an ecological philosophy that focuses on the interactions within the data ecosystem and how standards facilitate those interactions. We couple high-level analysis with on the ground experimentation. We began by 1) mapping information ecosystem components (e.g. data centers, missions, services, protocols, users), 2)establishing how the components interact (e.g. sharing (meta)data, funding, personnel exchange), and 3) modelling system dynamics (e.g. creation of products from multiple data centers, redundant processes, shared services). The goal is to apply understanding of the ecosystem to real world applications (e.g. planning a new mission, implementing new policy requirements, improving process efficiency, etc.). We have also conducted studies of historical standardization efforts, documenting lessons learned and cautionary tales. We then contrast this more abstract work with real examples. We reviewed and assessed multiple existing standards development processes both within and external to NASA. We now work to implement an initial test process which can be further optimized. We seek to define a consistent approach for assigning persistent identifiers for research objects, especially for the purposes of citation. The experience from this relatively ‘simple’ test case adds a pragmatic perspective on how researchers and engineers actually work. This presentation will review the details of this methodology, our initial findings, and how they might apply to other interdisciplinary standardization efforts.

Mark A Parsons↗

Ion mass spectrometer experiment for ISIS-2 spacecraft

The International Satellite for Ionospheric Studies (ISIS) program of NASA was the longest duration program in NASA history. A number of satellites were flown under this program, the last being called ISIS-2, which was launched on April 1, 1971 and operated successfully for over 13 years. An experiment called the Ion Mass Spectrometer (IMS) was flown on the ISIS-2 spacecraft. It operated for 10 years providing a large data base of positive ion composition and ion flow velocities along the orbit of the satellite, the latter being circular at 1400 km with a 90 degree inclination. The data were processed and reside in the National Space Sciences Data Center.

Hoffman, John H.↗

The Pilot Land Data System: Report of the Program Planning Workshops

An advisory report to be used by NASA in developing a program plan for a Pilot Land Data System (PLDS) was developed. The purpose of the PLDS is to improve the ability of NASA and NASA sponsored researchers to conduct land-related research. The goal of the planning workshops was to provide and coordinate planning and concept development between the land related science and computer science disciplines, to discuss the architecture of the PLDs, requirements for information science technology, and system evaluation. The findings and recommendations of the Working Group are presented. The pilot program establishes a limited scale distributed information system to explore scientific, technical, and management approaches to satisfying the needs of the land science community. The PLDS paves the way for a land data system to improve data access, processing, transfer, and analysis, which land sciences information synthesis occurs on a scale not previously permitted because of limits to data assembly and access.

Source record↗

Search Enhancements using Natural Language Processing Techniques

NASA Goddard Earth Sciences Data and Information Services Center (GESDISC) is one of the 12 NASA Science Mission Directorate Data Centers. The main goal of GESDISC is to provide earth science data, information, and services to the earth science data community. Consequently, data discovery is at the center of our mission and our search engine is the primary tool for our users to interact, find, and access our data. Existing search approaches are largely focused on hard-matching of keywords in the search query with dataset metadata. Here we propose to expand the search by introducing a complementary natural language processing (NLP) search. At the heart of our proposed NLP search, we trained a joint embedding using scientific text corpus and a curated set of dataset metadata. The embedding learns the association between words in our dataset metadata and those of the scientific text corpus. This enables us to go beyond simple hard-matching of a query and data set metadata and have a notion of “similarity” between the search query and the datasets. We further integrated our NLP search into the Elastic Search (ES) framework leveraging similarity search capabilities offered through the “dense_vector” field type. Our preliminary evaluations show that our proposed NLP search has the potential to be utilized to complement the existing search engine and serve as a base for a dataset recommendation system.

Armin Mehrabian↗

The Development of Two Science Investigator-led Processing Systems (SIPS) for NASA's Earth Observation System (EOS)

In 2001, NASA Goddard Space Flight Center's Laboratory for Terrestrial Physics started the construction of a science Investigator-led Processing System (SIPS) for processing data from the Ozone Monitoring Instrument (OMI) which will launch on the Aura platform in mid 2004. The Ozone Monitoring Instrument (OMI) is a contribution of the Netherlands Agency for Aerospace Programs (NIVR) in collaboration with the Finnish Meteorological Institute (FMI) to the Earth Observing System (EOS) Aura mission. It will continue the Total Ozone Monitoring System (TOMS) record for total ozone and other atmospheric parameters related to ozone chemistry and climate. OMI measurements will be highly synergistic with the other instruments on the EOS Aura platform. The LTP previously developed the Moderate Resolution Imaging Spectrometer (MODIS) Data Processing System (MODAPS), which has been in full operations since the launches of the Terra and Aqua spacecrafts in December, 1999 and May, 2002 respectively. During that time, it has continually evolved to better support the needs of the MODIS team. We now run multiple instances of the system managing faster than real time reprocessings of the data as well as continuing forward processing. The new OMI Data Processing System (OMIDAPS) was adapted from the MODAPS. It will ingest raw data from the satellite ground station and process it to produce calibrated, geolocated higher level data products. These data products will be transmitted to the Goddard Distributed Active Archive Center (GDAAC) instance of the Earth Observing System (EOS) Data and Information System (EOSDIS) for long term archive and distribution to the public. The OMIDAPS will also provide data distribution to the OMI Science Team for quality assessment, algorithm improvement, calibration, etc. We have taken advantage of lessons learned from the MODIS experience and software already developed for MODIS. We made some changes in the hardware system organization, database and software to adapt the system for OMI. We replaced the fundamental database system, Sybase, with an Open Source RDBMS called PostgreSQL, and based the entire OMIDAPS on a cluster of Linux based commodity computers rather than the large SGI servers that MODAPS uses. Rather than relying on a central I/O server host, the new system distributes its data archive among multiple server hosts in the cluster. OMI is also customizing the graphical user interfaces and reporting structure to more closely meet the needs of the OMI Science Team. Prior to 2003, simulated OMI data and the science algorithms were not ready for production testing. We initially constructed a prototype system and tested using a 25 year dataset of Total Ozone Mapping Spectrometer (TOMS) and Solar Backscatter Ultraviolet Instrument (SBUV) data. This prototype system provided a platform to support the adaptation of the algorithms for OMI, and provided reprocessing of the historical data aiding in its analysis. In a recent reanalysis of the TOMS data, the OMIDAPS processed 108,000 full orbits of data through 4 processing steps per orbit, producing about 800,000 files (400 GiB) of level 2 and greater data files. More recently we have installed two instances of the OMIDAPS for integration and testing of OM1 science processes as they get delivered from the Science Team. A Test instance of the OMIDAPS has also supported a series of "Interface Confidence Tests" (ICTs) and End-to-End Ground System tests to ensure the launch readiness of the system. This paper will discuss the high-level hardware, software, and database organization of the OMIDAPS and how it builds on the MODAPS heritage system. It will also provide an overview of the testing and implementation of the production OMIDAPS.

Tilmes, Curt↗

Utilizing Earth Observations for Societal Issues

Over the last four decades a tremendous progress has been made in the Earth science space-based remote sensing observations, technologies and algorithms. Such advancements have improved the predictability by providing lead-time and accuracy of forecast in weather, climate, natural hazards, and natural resources. It has further reduced or bounded the overall uncertainties by partially improving our understanding of planet Earth as an integrated system that is governed by non-linear and chaotic behavior. Many countries such as the US, European Community, Japan, China, Russia, India has and others have invested billions of dollars in developing and launching space-based assets in the low earth (LEO) and geostationary (GEO) orbits. However, the wealth of this scientific knowledge that has potential of extracting monumental socio-economic benefits from such large investments have been slow in reaching the public and decision makers. For instance, there are a number of areas such as water resources and availability, energy forecasting, aviation safety, agricultural competitiveness, disaster management, air quality and public health, which can directly take advantage. Nevertheless, we all live in a global economy that depends on access to the best available Earth Science information for all inhabitants of this planet. This presentation discusses a process to transition Earth science data and products for societal needs including NASA's experience in achieving such objectives. It is important to mention that there are many challenges and issues that pertain to a number of areas such as: (1) difficulties in making a speedy transition of data and information from observations and models to relevant Decision Support Systems (DSS) or tools, (2) data and models inter-operability issues, (3) limitations of spatial, spectral and temporal resolution, (4) communication limitations as dictated by the availability of image processing and data compression techniques. Additionally, the most critical element amongst all is the organizational and management boundaries that must be resolved at local, state, national and international levels to implement and realize free flow of such vital information.

Habib, Shahid↗

Processing AIRS Scientific Data Through Level 3

The Atmospheric Infra-Red Sounder (AIRS) Science Processing System (SPS) is a collection of computer programs, known as product generation executives (PGEs). The AIRS SPS PGEs are used for processing measurements received from the AIRS suite of infrared and microwave instruments orbiting the Earth onboard NASA's Aqua spacecraft. Early stages of the AIRS SPS development were described in a prior NASA Tech Briefs article: Initial Processing of Infrared Spectral Data (NPO-35243), Vol. 28, No. 11 (November 2004), page 39. In summary: Starting from Level 0 (representing raw AIRS data), the AIRS SPS PGEs and the data products they produce are identified by alphanumeric labels (1A, 1B, 2, and 3) representing successive stages or levels of processing. The previous NASA Tech Briefs article described processing through Level 2, the output of which comprises geo-located atmospheric data products such as temperature and humidity profiles among others. The AIRS Level 3 PGE samples selected information from the Level 2 standard products to produce a single global gridded product. One Level 3 product is generated for each day s collection of Level 2 data. In addition, daily Level 3 products are aggregated into two multiday products: an eight-day (half the orbital repeat cycle) product and monthly (calendar month) product.

Granger, Stephanie↗

Meeting SPD-41a Requirements for BPS

In April 2021 BPS Formed the Science Data Management Work Group. This Poster describes the process and forward steps.

BPS Science Data Management↗

Total and Spectral Solar Irradiance Sensor (TSIS) Project Status

TSIS-1 studies the Sun's energy input to Earth and how solar variability affects climate. TSIS-1 will measure both the total amount of light that falls on Earth, known as the total solar irradiance (TSI), and how that light is distributed among ultraviolet, visible and infrared wavelengths, called solar spectral irradiance (SSI). TSIS-1 will provide the most accurate measurements of sunlight and continue the long-term climate data record. TSIS-1 includes two instruments: the Total Irradiance Monitor (TIM) and the Spectral Irradiance Monitor (SIM), integrated into a single payload on the International Space Station (ISS). The TSIS-1 TIM and SIM instruments are upgraded versions of the two instruments that are flying on the Solar Radiation and Climate Experiment (SORCE) mission launched in January 2003. NASA Goddard's TSIS project responsibilities include project management, system engineering, safety and mission assurance, and engineering oversight for TSIS-1. TSIS-1 was installed on the International Space Station in December 2017. At the end of the 90-day commissioning phase, responsibility for TSIS-1 operations transitions to the Earth Science Mission Operations (ESMO) project at Goddard for its 5-year operations. NASA contracts with the University of Colorado Laboratory for Atmospheric and Space Physics (LASP) for the design, development and testing of TSIS-1, support for ISS integration, science operations of the TSIS-1 instrument, data processing, data evaluation, calibration and delivery to the Goddard Earth Science Data and Information Services Center (GES DISC).

TSIS↗

Reusing Information Management Services for Recommended Decadal Study Missions to Facilitate Aerosol and Cloud Studies

NASA Earth Sciences Division (ESD) has made great investments in the development and maintenance of data management systems and information technologies, to maximize the use of NASA generated Earth science data. With information management system infrastructure in place, mature and operational, very small delta costs are required to fully support data archival, processing, and data support services required by the recommended Decadal Study missions. This presentation describes the services and capabilities of the Goddard Space Flight Center (GSFC) Earth Sciences Data and Information Services Center (GES DISC) and the reusability for these future missions. The GES DISC has developed a series of modular, reusable data management components currently in use. They include data archive and distribution (Simple, Scalable, Script-based, Science [S4] Product Archive aka S4PA), data processing (S4 Processor for Measurements aka S4PM), data search (Mirador), data browse, visualization, and analysis (Giovanni), and data mining services. Information management system components are based on atmospheric scientist inputs. Large development and maintenance cost savings can be realized through their reuse in future missions.

Kempler, Steve↗

The 1994 research and technology report at the Goddard Space Flight Center

The breadth of subject material in this 1994 edition of the Research and Technology Report illustrates the broad scope of activities at the Goddard Space Flight Center. The numerous entries dealing with data processing and visualization show the strong emphasis on data and its interpretation. Reports are presented in the following sections: data processing and visualization; space sciences - high energy astronomy, solar system, and new techniques; earth system science - atmospheres, oceans and ice, solid earth, and soils and vegetation; networks, planning, and information systems - mission scheduling and operations, spacecraft operation and status, software engineering, and infrastructure support; engineering and materials - spacecraft subsystems, launch vehicles, thermal control, new mechanisms, and testing and evaluation; and flight projects.

Soffen, Gerald↗

Systems Architecture for Fully Autonomous Space Missions

The NASA Goddard Space Flight Center is working to develop a revolutionary new system architecture concept in support of fully autonomous missions. As part of GSFC's contribution to the New Millenium Program (NMP) Space Technology 7 Autonomy and on-Board Processing (ST7-A) Concept Definition Study, the system incorporates the latest commercial Internet and software development ideas and extends them into NASA ground and space segment architectures. The unique challenges facing the exploration of remote and inaccessible locales and the need to incorporate corresponding autonomy technologies within reasonable cost necessitate the re-thinking of traditional mission architectures. A measure of the resiliency of this architecture in its application to a broad range of future autonomy missions will depend on its effectiveness in leveraging from commercial tools developed for the personal computer and Internet markets. Specialized test stations and supporting software come to past as spacecraft take advantage of the extensive tools and research investments of billion-dollar commercial ventures. The projected improvements of the Internet and supporting infrastructure go hand-in-hand with market pressures that provide continuity in research. By taking advantage of consumer-oriented methods and processes, space-flight missions will continue to leverage on investments tailored to provide better services at reduced cost. The application of ground and space segment architectures each based on Local Area Networks (LAN), the use of personal computer-based operating systems, and the execution of activities and operations through a Wide Area Network (Internet) enable a revolution in spacecraft mission formulation, implementation, and flight operations. Hardware and software design, development, integration, test, and flight operations are all tied-in closely to a common thread that enables the smooth transitioning between program phases. The application of commercial software development techniques lays the foundation for delivery of product-oriented flight software modules and models. Software can then be readily applied to support the on-board autonomy required for mission self-management. An on-board intelligent system, based on advanced scripting languages, facilitates the mission autonomy required to offload ground system resources, and enables the spacecraft to manage itself safely through an efficient and effective process of reactive planning, science data acquisition, synthesis, and transmission to the ground. Autonomous ground systems in turn coordinate and support schedule contact times with the spacecraft. Specific autonomy software modules on-board include mission and science planners, instrument and subsystem control, and fault tolerance response software, all residing within a distributed computing environment supported through the flight LAN. Autonomy also requires the minimization of human intervention between users on the ground and the spacecraft, and hence calls for the elimination of the traditional operations control center as a funnel for data manipulation. Basic goal-oriented commands are sent directly from the user to the spacecraft through a distributed internet-based payload operations "center". The ensuing architecture calls for the use of spacecraft as point extensions on the Internet. This paper will detail the system architecture implementation chosen to enable cost-effective autonomous missions with applicability to a broad range of conditions. It will define the structure needed for implementation of such missions, including software and hardware infrastructures. The overall architecture is then laid out as a common thread in the mission life cycle from formulation through implementation and flight operations.

Esper, Jamie↗

A Framework for Propagation of Uncertainties in the Kepler Data Analysis Pipeline

The Kepler space telescope is designed to detect Earth-like planets around Sun-like stars using transit photometry by simultaneously observing 100,000 stellar targets nearly continuously over a three and a half year period. The 96-megapixel focal plane consists of 42 charge-coupled devices (CCD) each containing two 1024 x 1100 pixel arrays. Cross-correlations between calibrated pixels are introduced by common calibrations performed on each CCD requiring downstream data products access to the calibrated pixel covariance matrix in order to properly estimate uncertainties. The prohibitively large covariance matrices corresponding to the ~75,000 calibrated pixels per CCD preclude calculating and storing the covariance in standard lock-step fashion. We present a novel framework used to implement standard propagation of uncertainties (POU) in the Kepler Science Operations Center (SOC) data processing pipeline. The POU framework captures the variance of the raw pixel data and the kernel of each subsequent calibration transformation allowing the full covariance matrix of any subset of calibrated pixels to be recalled on-the-fly at any step in the calibration process. Singular value decomposition (SVD) is used to compress and low-pass filter the raw uncertainty data as well as any data dependent kernels. The combination of POU framework and SVD compression provide downstream consumers of the calibrated pixel data access to the full covariance matrix of any subset of the calibrated pixels traceable to pixel level measurement uncertainties without having to store, retrieve and operate on prohibitively large covariance matrices. We describe the POU Framework and SVD compression scheme and its implementation in the Kepler SOC pipeline.

Clarke, Bruce D.↗

Reuse of Software Assets for the NASA Earth Science Decadal Survey Missions

Software assets from existing Earth science missions can be reused for the new decadal survey missions that are being planned by NASA in response to the 2007 Earth Science National Research Council (NRC) Study. The new missions will require the development of software to curate, process, and disseminate the data to science users of interest and to the broader NASA mission community. In this paper, we discuss new tools and a blossoming community that are being developed by the Earth Science Data System (ESDS) Software Reuse Working Group (SRWG) to improve capabilities for reusing NASA software assets.

Mattmann, Chris A.↗