Automating Manual Quality Control (QC) of Tropospheric Doppler Radar Wind Profiler (TDRWP) Archive Data with Machine Learning
Explore the source record for details and available documents.
Corey Walker↗
SEARCH · Search NASA
Results for “data archive”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Using Cloud Resources to Examine the Sampling Differences Among Boundary Layer Height Data Archived at the GES DISC
Explore the source record for details and available documents.
Planetary Boundary Layer↗
Demystifying Kepler Data: A Primer for Systematic Artifact Mitigation
The Kepler spacecraft has collected data of high photometric precision and cadence almost continuously since operations began on 2009 May 2. Primarily designed to detect planetary transits and asteroseismological signals from solar-like stars, Kepler has provided high quality data for many areas of investigation. Unconditioned simple aperture time-series photometry are however affected by systematic structure. Examples of these systematics are differential velocity aberration, thermal gradients across the spacecraft, and pointing variations. While exhibiting some impact on Kepler's primary science, these systematics can critically handicap potentially ground-breaking scientific gains in other astrophysical areas, especially over long timescales greater than 10 days. As the data archive grows to provide light curves for 10(exp 5) stars of many years in length, Kepler will only fulfill its broad potential for stellar astrophysics if these systematics are understood and mitigated. Post-launch developments in the Kepler archive, data reduction pipeline and open source data analysis software have occurred to remove or reduce systematic artifacts. This paper provides a conceptual primer for users of the Kepler data archive to understand and recognize systematic artifacts within light curves and some methods for their removal. Specific examples of artifact mitigation are provided using data available within the archive. Through the methods defined here, the Kepler community will find a road map to maximizing the quality and employment of the Kepler legacy archive.
Kinemuchi, K.↗
The data facility of the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS)
AVIRIS operations at the Jet Propulsion Laboratory include a significant data task. The AVIRIS data facility is responsible for data archiving, data calibration, quality monitoring and distribution. Since 1987, the data facility has archived over one terabyte of AVIRIS data and distributed these data to science investigators as requested. In this paper we describe recent improvements in the AVIRIS data facility.
Nielsen, Pia J.↗
Laying the Foundations for FAIR-er Science: ISA and the LSDA Data Submission Process in NASA's Evolving Data Management Environment
The Life Sciences Data Archive (LSDA) archives data resulting from research on the effects of spaceflight on humans and the development of countermeasures to mitigate spaceflight hazards. Archivists work with researchers to ensure that unique and high value data products and their metadata are preserved and managed to support current and future research. Currently, LSDA is updating its procedures and data submission requirements in response to the evolving data preservation environment at NASA. LSDA is implementing best practices for research data management through the establishment of clear data submission guidelines, integration of the FAIR (Findability, Accessibility, Interoperability, Reusability) principles, and use of the ISA (Investigation, Study, Assay) research metadata framework for data discoverability and transparency into the data management processes. These changes directly impact LSDA’s requirements for research data submissions. The newly revised Research Data Submission Agreement (RDSA), formerly the Data Submission Agreement (DSA), introduces ISA-compatible metadata collection standards to LSDA’s process. Adherence to LSDA’s data submission guidelines enhances the FAIR-ness of the repository’s collections for future users. This presentation will discuss (1) how submission of research data and associated metadata are impacted by current data management policies, (2) benefits of the adoption of FAIR principles and the ISA metadata framework for retrospective studies utilizing existing LSDA datasets and historic data collections, and (3) the support LSDA will provide to researchers during this transition.
LSDA↗
Quantitative 3-D imaging topogrammetry for telemedicine applications
The technology to reliably transmit high-resolution visual imagery over short to medium distances in real time has led to the serious considerations of the use of telemedicine, telepresence, and telerobotics in the delivery of health care. These concepts may involve, and evolve toward: consultation from remote expert teaching centers; diagnosis; triage; real-time remote advice to the surgeon; and real-time remote surgical instrument manipulation (telerobotics with virtual reality). Further extrapolation leads to teledesign and telereplication of spare surgical parts through quantitative teleimaging of 3-D surfaces tied to CAD/CAM devices and an artificially intelligent archival data base of 'normal' shapes. The ability to generate 'topogrames' or 3-D surface numerical tables of coordinate values capable of creating computer-generated virtual holographic-like displays, machine part replication, and statistical diagnostic shape assessment is critical to the progression of telemedicine. Any virtual reality simulation will remain in 'video-game' realm until realistic dimensional and spatial relational inputs from real measurements in vivo during surgeries are added to an ever-growing statistical data archive. The challenges of managing and interpreting this 3-D data base, which would include radiographic and surface quantitative data, are considerable. As technology drives toward dynamic and continuous 3-D surface measurements, presenting millions of X, Y, Z data points per second of flexing, stretching, moving human organs, the knowledge base and interpretive capabilities of 'brilliant robots' to work as a surgeon's tireless assistants becomes imaginable. The brilliant robot would 'see' what the surgeon sees--and more, for the robot could quantify its 3-D sensing and would 'see' in a wider spectral range than humans, and could zoom its 'eyes' from the macro world to long-distance microscopy. Unerring robot hands could rapidly perform machine-aided suturing with precision micro-sewing machines, splice neural connections with laser welds, micro-bore through constricted vessels, and computer combine ultrasound, microradiography, and 3-D mini-borescopes to quickly assess and trace vascular problems in situ. The spatial relationships between organs, robotic arms, and end-effector diagnostic, manipulative, and surgical instruments would be constantly monitored by the robot 'brain' using inputs from its multiple 3-D quantitative 'eyes' remote sensing, as well as by contact and proximity force measuring devices. Methods to create accurate and quantitative 3-D topograms at continuous video data rates are described.
Altschuler, Bruce R.↗
Mars Observer data production, transfer, and archival: The data production assembly line
This paper describes the data production, transfer, and archival process designed for the Mars Observer Flight Project. It addresses the developmental and operational aspects of the archive collection production process. The developmental aspects cover the design and packaging of data products for archival and distribution to the planetary community. Also discussed is the design and development of a data transfer and volume production process capable of handling the large throughput and complexity of the Mars Observer data products. The operational aspects cover the main functions of the process: creating data and engineering products, collecting the data products and ancillary products in a central repository, producing archive volumes, validating volumes, archiving, and distributing the data to the planetary community.
Childs, David B.↗
Integration of multi-discipline data processing for earth observing systems
The first steps taken to ensure the controlled evolution of existing facilities toward greater interoperability and sharing of resources among NASA-supported earth science and applications data systems (ESADS) are described. Recommendations made by the various panels during the 1987 ESADS Workshop are presented. The panels were concerned with directories and catalogs, data archives, data manipulation software, computational facilities, data storage media, database management, and networking. Consideration was also given to the tracking and tuning of overall development and management coordination issues.
Kahn, Ralph↗
The Isothermal Dendritic Growth Experiment (IDGE)
The Isothermal Dendritic Growth Experiment (IDGE) constituted a series of three NASA-supported microgravity experiments, all of which flew aboard the space shuttle, Columbia. This experimental space flight series was designed and operated to grow and record dendrite solidification in the absence of gravity-induced convective heat transfer, and thereby produce a wealth of benchmark-quality data for testing solidification scaling laws. The data and analysis performed on the dendritic growth speed and tip size in Succinontrie (SCN) demonstrates that although the theory yields predictions that are reasonably in agreement with experiment, there are significant discrepancies. However, some of these discrepancies can be explained by accurately describing the diffusion of heat. The key finding involves recognition that the actual three-dimensional shape of dendrites includes time-dependent side-branching and a tip region that is not a paraboloid of revolution. Thus, the role of heat transfer in dendritic growth is validated, with the caveat that a more realistic model of the dendrite then a paraboloid is needed to account for heat flow in an experimentally observed dendrite. We are currently conducting additional analysis to further confirm and demonstrate these conclusions. The data and analyses for the growth selection physics remain much less definitive. From the first flight, the data indicated that the selection parameter, sigma*, is not exactly a constant, but exhibits a slight dependence on the supercooling. Additional data from the second flight are being examined to investigate the selection of a unique dendrite speed, tip size and shape. The IDGE flight series is now complete. We are currently completing analyses and moving towards final data archiving. It is gratifying to see that the IDGE published results and archived data sets are being used actively by other scientists and engineers. In addition, we are also pleased to report that the techniques and IDGE hardware system that the authors developed with NASA, are being currently employed on both designated flight experiments, like EDSE, and on flight definition experiments, like TDSE.
Glicksman, Martin E.↗
Airborne Visible/Infrared Imaging Spectrometer (AVIRIS): Recent Improvements to the Sensor and Data Facility
AVIRIS operations at the Jet Propulsion Laboratory consist primarily of a sensor task and a data facility task. The sensor task is responsible for AVIRIS sensor meaintenance, laboratory calibration, and field operations. The AVIRIS data facility is responsible for data archiving, data calibration, quality monitoring and distribution. In this paper we describe recent improvements in these two primary AVIRIS tasks.
Airborne↗
Laying The Foundations for FAIR-ER Science: ISA And LSDA Data Submission Process in NASA’s Evolving Data Management Environment
The Life Sciences Data Archive (LSDA) archives data resulting from research on the effects of spaceflight on humans and the development of countermeasures to mitigate spaceflight hazards. Archivists work with researchers to ensure that unique and high value data products and their metadata are preserved and managed to support current and future research. Currently, LSDA is updating its procedures and data submission requirements in response to the evolving data preservation environment at NASA. LSDA is implementing best practices for research data management through the establishment of clear data submission guidelines, integration of the FAIR (Findability, Accessibility, Interoperability, Reusability) principles, and use of the ISA (Investigation, Study, Assay) research metadata framework for data discoverability and transparency into the data management processes. These changes directly impact LSDA’s requirements for research data submissions. The newly revised Research Data Submission Agreement (RDSA), formerly the Data Submission Agreement (DSA), introduces ISA-compatible metadata collection standards to LSDA’s process. Adherence to LSDA’s data submission guidelines enhances the FAIR-ness of the repository’s collections for future users. This presentation will discuss (1) how submission of research data and associated metadata are impacted by current data management policies, (2) benefits of the adoption of FAIR principles and the ISA metadata framework for retrospective studies utilizing existing LSDA datasets and historic data collections, and (3) the support LSDA will provide to researchers during this transition.
Data submission↗
A Robust, Low-Cost Virtual Archive for Science Data
Despite their expense tape silos are still often the only affordable option for petabytescale science data archives, particularly when other factors such as data reliability, floor space, power and cooling load are accounted for. However, the complexity, management software, hardware reliability and access latency of tape silos make online data storage ever more attractive. Drastic reductions in low-cost mass-market PC disk drivers help to make this more affordable (approx. 1$/GB), but are challenging to scale to the petabyte range and of questionable reliability for archival use, On the other hand, if much of the science archive could be "virtualized", i.e., produced on demand when requested by users, we would need store only a fraction of the data online, perhaps bringing an online-only system into in affordable range. Radiance data from the satellite-borne Moderate Resolution Imaging Spectroradiometer (MODIS) instrument provides a good opportunity for such a virtual archive: the raw data amount to 140 GB/day, but these are small relative to the 550 GB/day making up the radiance products. These data are routinely processed as inputs for geophysical parameter products and then archived on tape at the Goddard Earth Sciences Distributed Active Archive (GES DAAC) for distributing to users. Virtualizing them would be an immediate and signifcant reduction in the amount of data being stored in the tape archives and provide more customizable products. A prototype of such a virtual archive is being developed to prove the concept and develop ways of incorporating the robustness that a science data archive requires.
Lynnes, Christopher↗
Data management, archiving, visualization and analysis of space physics data
A series of programs for the visualization and analysis of space physics data has been developed at UCLA. In the course of those developments, a number of lessons have been learned regarding data management and data archiving, as well as data analysis. The issues now facing those wishing to develop such software, as well as the lessons learned, are reviewed. Modern media have eased many of the earlier problems of the physical volume required to store data, the speed of access, and the permanence of the records. However, the ultimate longevity of these media is still a question of debate. Finally, while software development has become easier, cost is still a limiting factor in developing visualization and analysis software.
Russell, C. T.↗
CDDIS: NASA's Archive of Space Geodesy Data and Products Supporting GGOS
The Crustal Dynamics Data Information System (CDDIS) supports data archiving and distribution activities for the space geodesy and geodynamics community. The main objectives of the system are to store space geodesy and geodynamics related data and products in a central archive, to maintain information about the archival of these data,to disseminate these data and information in a timely manner to a global scientific research community, and provide user based tools for the exploration and use of the archive. The CDDIS data system and its archive is a key component in several of the geometric services within the International Association of Geodesy (IAG) and its observing systemthe Global Geodetic Observing System (GGOS), including the IGS, the International DORIS Service (IDS), the International Laser Ranging Service (ILRS), the International VLBI Service for Geodesy and Astrometry (IVS), and the International Earth Rotation and Reference Systems Service (IERS). The CDDIS provides on-line access to over 17 Tbytes of dataand derived products in support of the IAG services and GGOS. The systems archive continues to grow and improve as new activities are supported and enhancements are implemented. Recently, the CDDIS has established a real-time streaming capability for GNSS data and products. Furthermore, enhancements to metadata describing the contents ofthe archive have been developed to facilitate data discovery. This poster will provide a review of the improvements in the system infrastructure that CDDIS has made over the past year for the geodetic community and describe future plans for the system.
CNS↗
Structure and kinematics of the broad-line regions in active galaxies from IUE variability data
IUE archival data are used here to investigate the structure nad kinematics of the broad-line regions (BLRs) in nine AGN. It is found that the centroid of the line-continuum cross-correlation functions (CCFs) can be determined with reasonable reliability. The errors in BLR size estimates from CCFs for irregularly sampled light curves are fairly well understood. BLRs are found to have small luminosity-weighted radii, and lines of high ionization tend to be emitted closer to the central source than lines of low ionization, especially for low-luminosity objects. The motion of the gas is gravity-dominated with both pure inflow and pure outflow of high-velocity gas being excluded at a high confidence level for certain geometries.
Koratkar, Anuradha P.↗
Cassini/Huygens Program Archive Plan for Science Data
The purpose of this document is to describe the Cassini/Huygens science data archive system which includes policy, roles and responsibilities, description of science and supplementary data products or data sets, metadata, documentation, software, and archive schedule and methods for archive transfer to the NASA Planetary Data System (PDS).
NASA Planetary Science Data System↗
Archiving of Planetary Ring Data
Stellar occultation data provide our only Earth-based means of probing planetary rings at kilometer spatial resolution. The occultation data archive at MIT contains original data and analysis products of stellar occultations by the ring systems of the planets Jupiter, Saturn, Uranus, and Neptune observed by members of the group (and other groups) from 1977 to the present. During this time period, several media have been used to record and store the original and processed data: (1) chart records; (2) printed output, (3) audio reel tape; (4) audio cassette tape; (5) 7-track, 1/2-inch computer tape; (6) 9-track, 1/2-inch computer tape at 800, 1600, and 6250 bpi; (7) NOVA disk platters (2.5 and 5.0 Mbyte); (8) write once optical disks; (9) punched cards; and (10) read-write optical disks. With the rapid change of computer technology over this time period, some of these media have become not only obsolete, but nearly extinct. In particular, it has become nearly impossible to find any facilities that can still read 800 bpi tapes, which contain the only copies of several important data sets for the ring system of Uranus. In particular, we have an extensive ring data collection that includes data sets for the following Uranian ring occultations: U0, U11, U12, U13, U14, U25, U17, and U36.
Elliot, James L.↗
The Open Data Repositorys Data Publisher
Data management and data publication are becoming increasingly important components of researcher's workflows. The complexity of managing data, publishing data online, and archiving data has not decreased significantly even as computing access and power has greatly increased. The Open Data Repository's Data Publisher software strives to make data archiving, management, and publication a standard part of a researcher's workflow using simple, web-based tools and commodity server hardware. The publication engine allows for uploading, searching, and display of data with graphing capabilities and downloadable files. Access is controlled through a robust permissions system that can control publication at the field level and can be granted to the general public or protected so that only registered users at various permission levels receive access. Data Publisher also allows researchers to subscribe to meta-data standards through a plugin system, embargo data publication at their discretion, and collaborate with other researchers through various levels of data sharing. As the software matures, semantic data standards will be implemented to facilitate machine reading of data and each database will provide a REST application programming interface for programmatic access. Additionally, a citation system will allow snapshots of any data set to be archived and cited for publication while the data itself can remain living and continuously evolve beyond the snapshot date. The software runs on a traditional LAMP (Linux, Apache, MySQL, PHP) server and is available on GitHub (http://github.com/opendatarepository) under a GPLv2 open source license. The goal of the Open Data Repository is to lower the cost and training barrier to entry so that any researcher can easily publish their data and ensure it is archived for posterity.
Astrobiology data↗